// Module included in the following assemblies:
//
// assembly-storage.adoc

[id='ref-persistent-storage-{context}']
= Persistent storage

[role="_abstract"]
Persistent data storage retains data in the event of system disruption.
For pods that use persistent data storage, data is persisted across pod failures and restarts.
Because of its permanent nature, persistent storage is recommended for production environments.

The following examples show common types of persistent volumes supported by Kubernetes:

* If your Kubernetes cluster runs on Amazon AWS, Kubernetes can provision Amazon EBS volumes
* If your Kubernetes cluster runs on Microsoft Azure, Kubernetes can provision Azure Disk Storage volumes
* If your Kubernetes cluster runs on Google Cloud, Kubernetes can provision Persistent Disk volumes
* If your Kubernetes cluster runs on bare metal, Kubernetes can provision local persistent volumes

To use persistent storage in Strimzi, you specify `persistent-claim` in the storage configuration of the `Kafka` or `ZooKeeper` resources.
If you are using node pools, you can also specify `persistent-claim` in the storage configuration of individual node pools.

You configure the resource so that pods use {K8sPersistentVolumeClaims} (PVCs) to make storage requests on persistent volumes (PVs).
PVs represent storage volumes that are created on demand and are independent of the pods that use them.
The PVC requests the amount of storage required when a pod is being created.
The underlying storage infrastructure of the PV does not need to be understood. 
If a PV matches the storage criteria, the PVC is bound to the PV.

You have two options for specifying the storage type:

`storage.type: persistent-claim`:: If you choose `persistent-claim` as the storage type, a single persistent storage volume is defined. 

`storage.type: jbod`:: When you select `jbod` as the storage type, you have the flexibility to define an array of persistent storage volumes using unique IDs. 

In a production environment, it is recommended to configure the following:

* For Kafka or node pools, set `storage.type` to `jbod` with one or more persistent volumes.
* For ZooKeeper, set `storage.type` as `persistent-claim` for a single persistent volume.

Persistent storage also has the following configuration options:

`id` (optional)::
A storage identification number. This option is mandatory for storage volumes defined in a JBOD storage declaration.
Default is `0`.

`size` (required)::
The size of the persistent volume claim, for example, "1000Gi".

`class` (optional)::
PVCs can request different types of persistent storage by specifying a {K8SStorageClass}.
Storage classes define storage profiles and dynamically provision PVs based on that profile.  
If a storage class is not specified, the storage class marked as default in the Kubernetes cluster is used.
Persistent storage options might include SAN storage types or {K8sLocalPersistentVolumes}.

`selector` (optional)::
Configuration to specify a specific PV.
Provides key:value pairs representing the labels of the volume selected.

`deleteClaim` (optional)::
Boolean value to specify whether the PVC is deleted when the cluster is uninstalled.
Default is `false`.

WARNING: Increasing the size of persistent volumes in an existing Strimzi cluster is only supported in Kubernetes versions that support persistent volume resizing. The persistent volume to be resized must use a storage class that supports volume expansion.
For other versions of Kubernetes and storage classes that do not support volume expansion, you must decide the necessary storage size before deploying the cluster.
Decreasing the size of existing persistent volumes is not possible.

.Example persistent storage configuration for Kafka and ZooKeeper
[source,yaml,subs="attributes+"]
----
apiVersion: {KafkaApiVersion}
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    storage:
      type: jbod
      volumes:
      - id: 0
        type: persistent-claim
        size: 100Gi
        deleteClaim: false
      - id: 1
        type: persistent-claim
        size: 100Gi
        deleteClaim: false
      - id: 2
        type: persistent-claim
        size: 100Gi
        deleteClaim: false
    # ...
  zookeeper:
    storage:
      type: persistent-claim
      size: 1000Gi
    # ...
----

.Example persistent storage configuration with specific storage class
[source,yaml,subs="attributes+"]
----
# ...
storage:
  type: persistent-claim
  size: 500Gi
  class: my-storage-class
# ...
----

Use a `selector` to specify a labeled persistent volume that provides certain features, such as an SSD.

.Example persistent storage configuration with selector
[source,yaml,subs="attributes+"]
----
# ...
storage:
  type: persistent-claim
  size: 1Gi
  selector:
    hdd-type: ssd
  deleteClaim: true
# ...
----

== Storage class overrides

WARNING: Storage class overrides are deprecated and will be removed in the future. As a replacement, use `KafkaNodePool` resources instead.

Instead of using the default storage class, you can specify a different storage class for one or more Kafka or ZooKeeper nodes.
This is useful, for example, when storage classes are restricted to different availability zones or data centers.
You can use the `overrides` field for this purpose.

In this example, the default storage class is named `my-storage-class`:

.Example storage configuration with class overrides
[source,yaml,subs="attributes+"]
----
apiVersion: {KafkaApiVersion}
kind: Kafka
metadata:
  labels:
    app: my-cluster
  name: my-cluster
  namespace: myproject
spec:
  # ...
  kafka:
    replicas: 3
    storage:
      type: jbod
      volumes:
      - id: 0
        type: persistent-claim
        size: 100Gi
        deleteClaim: false
        class: my-storage-class
        overrides:
        - broker: 0
          class: my-storage-class-zone-1a
        - broker: 1
          class: my-storage-class-zone-1b
        - broker: 2
          class: my-storage-class-zone-1c
      # ...
  # ...
  zookeeper:
    replicas: 3
    storage:
      deleteClaim: true
      size: 100Gi
      type: persistent-claim
      class: my-storage-class
      overrides:
        - broker: 0
          class: my-storage-class-zone-1a
        - broker: 1
          class: my-storage-class-zone-1b
        - broker: 2
          class: my-storage-class-zone-1c
  # ...
----

As a result of the configured `overrides` property, the volumes use the following storage classes:

* The persistent volumes of ZooKeeper node 0 use `my-storage-class-zone-1a`.
* The persistent volumes of ZooKeeper node 1 use `my-storage-class-zone-1b`.
* The persistent volumes of ZooKeeper node 2 use `my-storage-class-zone-1c`.
* The persistent volumes of Kafka broker 0 use `my-storage-class-zone-1a`.
* The persistent volumes of Kafka broker 1 use `my-storage-class-zone-1b`.
* The persistent volumes of Kafka broker 2 use `my-storage-class-zone-1c`.

The `overrides` property is currently used only to override the storage `class`. 
Overrides for other storage configuration properties is not currently supported.

=== Migrating from storage class overrides to node pools

Storage class overrides are deprecated and will be removed in the future.
If you are using storage class overrides, we encourage you to transition to using node pools instead.
To migrate the existing configuration, follow these steps:

1. Make sure you already use node pools resources.
   If not, you should xref:proc-migrating-clusters-node-pools-str[migrate the cluster to use node pools] first.
2. Create new xref:config-node-pools-str[node pools] with storage configuration using the desired storage class without using the overrides.
3. Move all partition replicas from the old broker using the storage class overrides.
   You can do this using xref:cruise-control-concepts-str[Cruise Control] or xref:assembly-reassign-tool-str[using the partition reassignment tool].
4. Delete the old node pool with the old brokers using the storage class overrides.

[id='ref-persistent-storage-pvc-{context}']
== PVC resources for persistent storage

When persistent storage is used, it creates PVCs with the following names:

`data-_cluster-name_-kafka-_idx_`::
PVC for the volume used for storing data for the Kafka broker pod `_idx_`.

`data-_cluster-name_-zookeeper-_idx_`::
PVC for the volume used for storing data for the ZooKeeper node pod `_idx_`.

== Mount path of Kafka log directories

The persistent volume is used by the Kafka brokers as log directories mounted into the following path:

[source,shell,subs="+quotes,attributes"]
----
/var/lib/kafka/data/kafka-log__IDX__
----

Where `_IDX_` is the Kafka broker pod index. For example `/var/lib/kafka/data/kafka-log0`.
