Initial troubleshooting

Percona Operator for PostgreSQL uses Custom Resources to manage options for the various components of the cluster.

• PerconaPGCluster Custom Resource with Percona PostgreSQL Cluster options (it has handy pg shortname also),

• PerconaPGBackup and PerconaPGRestore Custom Resources contain options for Percona XtraBackup used to backup Percona XtraDB Cluster and to restore it from backups (pg-backup and pg-restore shortnames are available for them).

The first thing you can check for the Custom Resource is to query it with kubectl get command:

$ kubectl get pg

Expected output

NAME       ENDPOINT                         STATUS   POSTGRES   PGBOUNCER   AGE
cluster1   cluster1-pgbouncer.default.svc   ready    3          3           30m

The Custom Resource should have Ready status.

Note

You can check which Percona’s Custom Resources are present and get some information about them as follows:

$ kubectl api-resources | grep -i percona

Expected output

perconapgbackups          pg-backup    pgv2.percona.com/v2            true         PerconaPGBackup
perconapgclusters         pg           pgv2.percona.com/v2            true         PerconaPGCluster
perconapgrestores         pg-restore   pgv2.percona.com/v2            true         PerconaPGRestore

Check the Pods

If Custom Resource is not getting Ready status, it makes sense to check individual Pods. You can do it as follows:

$ kubectl get pods

Expected output

NAME                                           READY   STATUS      RESTARTS   AGE
cluster1-backup-4vwt-p5d9j                     0/1     Completed   0          97m
cluster1-instance1-b5mr-0                      4/4     Running     0          99m
cluster1-instance1-b8p7-0                      4/4     Running     0          99m
cluster1-instance1-w7q2-0                      4/4     Running     0          99m
cluster1-pgbouncer-79bbf55c45-62xlk            2/2     Running     0          99m
cluster1-pgbouncer-79bbf55c45-9g4cb            2/2     Running     0          99m
cluster1-pgbouncer-79bbf55c45-9nrmd            2/2     Running     0          99m
cluster1-repo-host-0                           2/2     Running     0          99m
percona-postgresql-operator-79cd8586f5-2qzcs   1/1     Running     0          120m

The above command provides the following insights:

• READY indicates how many containers in the Pod are ready to serve the traffic. In the above example, cluster1-repo-host-0 container has all two containers ready (2/2). For an application to work properly, all containers of the Pod should be ready.
• STATUS indicates the current status of the Pod. The Pod should be in a Running state to confirm that the application is working as expected. You can find out other possible states in the official Kubernetes documentation .
• RESTARTS indicates how many times containers of Pod were restarted. This is impacted by the Container Restart Policy . In an ideal world, the restart count would be zero, meaning no issues from the beginning. If the restart count exceeds zero, it may be reasonable to check why it happens.
• AGE: Indicates how long the Pod is running. Any abnormality in this value needs to be checked.

You can find more details about a specific Pod using the kubectl describe pods <pod-name> command.

$ $ kubectl describe pods cluster1-instance1-b5mr-0

Expected output

...
Name:         cluster1-instance1-b5mr-0
Namespace:    default
...
Controlled By:  StatefulSet/cluster1-instance1-b5mr
Init Containers:
 postgres-startup:
...
Containers:
 database:
...
 pgbackrest:
...
   Restart Count:  0
   Liveness:   http-get https://:8008/liveness delay=3s timeout=5s period=10s #success=1 #failure=3
   Readiness:  http-get https://:8008/readiness delay=3s timeout=5s period=10s #success=1 #failure=3
   Environment:
...
   Mounts:
...
Volumes:
...
Events:
...

This gives a lot of information about containers, resources, container status and also events. So, describe output should be checked to see any abnormalities.

Get expert help

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services. Join K8S Squad to benefit from early access to features and “ask me anything” sessions with the Experts.

Community Forum Get a Percona Expert Join K8S Squad

---

Last update: 2024-04-17