Skip to content
Percona Documentation

Check Storage-related objects

Storage-related objects worth to check in case of problems are the following ones:

It is important to remember that PVC is namespace-scoped, but PV and Storage Class are cluster-scoped.

Check the PVC

You can check all the PVC with the following command (use your namespace name instead of the <namespace> placeholder):

$ kubectl get pvc -n <namespace>
Expected output 
NAME                 STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
datadir-pxc-pxc-0    Bound    pvc-f3e7097f-accd-4f5d-9c9d-6f29b54a368b   24Gi       RWO            standard       42d
datadir-pxc-pxc-1    Bound    pvc-3ec336a8-69de-4cbc-aff8-700d41696447   24Gi       RWO            standard       42d
datadir-pxc-pxc-2    Bound    pvc-207e8a3e-1c83-4eae-b3f2-cf126f89ba9e   24Gi       RWO            standard       42d

The fields in the output of this command provide the following insights:

  • STATUS: shows the state of the PVC:
    • For normal working of an application, the status should be Bound.
    • If the status is not Bound, further investigation is required.
  • VOLUME: is the name of the Persistent Volume with which PVC is Bound to. Obviously, this field will be occupied only when a PVC is Bound.
  • CAPACITY: it is the size of the volume claimed.
  • STORAGECLASS: it indicates the Kubernetes storage class used for dynamic provisioning of Volume.
  • ACCESS MODES: Access mode indicates how Volume is used with the Pods. Access modes should have write permission if the application needs to write data, which is obviously true in case of databases.

Now you can check a specific PVC for more details using its name as follows:

$ kubectl get pvc datadir-pxc-pxc-0 -n <namespace> -oyaml # output stripped for brevity, name of PVC may vary
Expected output 
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  ...
  name: datadir-pxc-pxc-0
  namespace: default
  ...
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 25G
  storageClassName: standard
  volumeMode: Filesystem
  volumeName: pvc-f3e7097f-accd-4f5d-9c9d-6f29b54a368b
status:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 24Gi
  phase: Bound

Check the PV

It is important to remember that PV is a cluster-scoped Object. If you see any issues with attaching a Volume to a Pod, PV and PVC might be looked upon.

Check all the PV present in the Kubernetes cluster as follows:

$ kubectl get pv
Expected output 
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM                                                      STORAGECLASS   REASON   AGE
pvc-207e8a3e-1c83-4eae-b3f2-cf126f89ba9e   24Gi       RWO            Delete           Bound       default/datadir-pxc-pxc-2                                  standard                43d
pvc-3ec336a8-69de-4cbc-aff8-700d41696447   24Gi       RWO            Delete           Bound       default/datadir-pxc-pxc-1                                  standard                43d
pvc-f3e7097f-accd-4f5d-9c9d-6f29b54a368b   24Gi       RWO            Delete           Bound       default/datadir-pxc-pxc-0                                  standard                43d

Now you can check a specific PV for more details using its name as follows:

$ kubectl get pv pvc-f3e7097f-accd-4f5d-9c9d-6f29b54a368b -oyaml
Expected output 
apiVersion: v1
kind: PersistentVolume
metadata:
  ...
  name: pvc-f3e7097f-accd-4f5d-9c9d-6f29b54a368b
 ...
spec:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 24Gi
  claimRef:
    apiVersion: v1
    kind: PersistentVolumeClaim
    name: datadir-pxc-pxc-0
    namespace: default
    resourceVersion: "912868"
    uid: f3e7097f-accd-4f5d-9c9d-6f29b54a368b
  gcePersistentDisk:
    fsType: ext4
    pdName: pvc-f3e7097f-accd-4f5d-9c9d-6f29b54a368b
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: topology.kubernetes.io/zone
          operator: In
          values:
          - us-central1-a
        - key: topology.kubernetes.io/region
          operator: In
          values:
          - us-central1
  persistentVolumeReclaimPolicy: Delete
  storageClassName: standard
  volumeMode: Filesystem
status:
  phase: Bound

Fields to check if there are any issues in binding with PVC, are the claimRef and nodeAffinity.

The claimRef one indicates to which PVC this volume is bound to. This means that if by any chance PVC is deleted (e.g. by the appropriate finalizer), this section needs to be modified so that it can bind to a new PVC.

The spec.nodeAffinity field may influence the PV availability as well: for example, it can make Volume accessed in one availability zone only.

Check the StorageClass

StorageClass is also a cluster-scoped object, and it indicates what type of underlying storage is used for the Volumes.

You can set StorageClass in pxc.volumeSpec.persistentVolumeClaim.storageClassName, proxysql.volumeSpec.persistentVolumeClaim.storageClassName, and backup.storages.STORAGE-NAME.persistentVolumeClaim.storageClassName Custom Resource options.

The following command checks all the storage class present in the Kubernetes cluster, and allows to see which storage class is the default one:

$ kubectl get sc
Expected output 
NAME                 PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
premium-rwo          pd.csi.storage.gke.io   Delete          WaitForFirstConsumer   true                   44d
standard (default)   kubernetes.io/gce-pd    Delete          Immediate              true                   44d
standard-rwo         pd.csi.storage.gke.io   Delete          WaitForFirstConsumer   true                   44d

If some PVC does not refer any storage class explicitly, it means that the default storage class is used. Ensure there is only one default Storage class.

You can check a specific storage class as follows:

$ kubectl get sc standard -oyaml
Expected output 
allowVolumeExpansion: true
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
 annotations:
   storageclass.kubernetes.io/is-default-class: "true"
 creationTimestamp: "2022-10-09T06:28:03Z"
 labels:
   addonmanager.kubernetes.io/mode: EnsureExists
 name: standard
 resourceVersion: "906"
 uid: 933d37db-990b-4b2d-bf3a-dd091d0b00ae
parameters:
 type: pd-standard
provisioner: kubernetes.io/gce-pd
reclaimPolicy: Delete
volumeBindingMode: Immediate

Important things to observe here are the following ones:

  • Check if the provisioner and parameters are indicating the type of storage you intend to provision.
  • Check the volumeBindingMode especially if the storage cannot be accessed across availability zones. “WaitForFirstConsumer” volumeBindingMode ensures volume is provisioned only after a Pod requesting the Volume is created.
  • If you are going to rely on the Operator storage scaling functionality, ensure the storage class supports PVC expansion (it should have allowVolumeExpansion: true in the output of the above command).

Get expert help

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services. Join K8S Squad to benefit from early access to features and “ask me anything” sessions with the Experts.

Community Forum Get a Percona Expert Join K8S Squad

Last update: 2024-12-19