Transport layer security (TLS)¶

The Percona Operator for PostgreSQL uses Transport Layer Security (TLS) cryptographic protocol for the following types of communication:

Internal - communication between PostgreSQL instances in the cluster
External - communication between the client application and the cluster

The internal certificate is also used as an authorization method for PostgreSQL Replica instances.

TLS security can be configured in following ways:

the Operator can generate long-term certificates automatically at cluster creation time,
you can generate certificates manually.

Note

Additionally, you can force your database cluster to use only encrypted channels for both internal and external communications. This effect is achieved by setting the tlsOnly Custom Resource option to true.

Allow the Operator to generate certificates automatically¶

The Operator is able to generate long-term certificates automatically and turn on encryption at cluster creation time, if there are no certificate secrets available. Just deploy your cluster as usual, with the kubectl apply -f deploy/cr.yaml command, and certificates will be generated.

Note

With the Operator versions before 2.5.0, autogenerated certificates for all database clusters were based on the same generated root CA. Starting from 2.5.0, the Operator creates root CA on per-cluster basis.

Check connectivity to the cluster¶

You can check TLS communication with use of the psql, the standard interactive terminal-based frontend to PostgreSQL. The following command will spawn a new pg-client container, which includes the needed command and can be used for the check (use your real cluster name instead of the <cluster-name> placeholder):

$ cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: pg-client
spec:
  replicas: 1
  selector:
    matchLabels:
      name: pg-client
  template:
    metadata:
      labels:
        name: pg-client
    spec:
      containers:
        - name: pg-client
          image: percona/percona-distribution-postgresql:17.5-2
          imagePullPolicy: Always
          command:
          - sleep
          args:
          - "100500"
          volumeMounts:
            - name: ca
              mountPath: "/tmp/tls"
      volumes:
      - name: root
        secret:
          secretName: <cluster_name>-cert-ca
          items:
          - key: root.crt
            path: root.crt
            mode: 0777
EOF

Now get shell access to the newly created container, and launch the PostgreSQL interactive terminal to check connectivity over the encrypted channel (please use real cluster-name, PostgreSQL user login and password):

$ kubectl exec -it deployment/pg-client -- bash -il
[postgres@pg-client /]$ PGSSLMODE=verify-ca PGSSLROOTCERT=/tmp/tls/ca.crt psql postgres://<postgresql-user>:<postgresql-password>@<cluster-name>-pgbouncer.<namespace>.svc.cluster.local

Now you should see the prompt of PostgreSQL interactive terminal:

$ psql (17.5.2)
Type "help" for help.
cluster1=>

Generate certificates manually¶

You can customize TLS for the Operator by providing your own TLS certificates. To do this, you must create two Kubernetes Secret objects before deploying your cluster:

One for external communication, later referenced by the spec.customTLSSecret field in the deploy/cr.yaml
One for internal communication (used for replication authentication), referenced by the spec.customReplicationTLSSecret field in the deploy/cr.yaml.

Each Secret must contain the following fields:

tls.crt (the TLS certificate)
tls.key (the TLS private key)
ca.crt (the Certificate Authority certificate)

Note that you cannot use only one custom set of certificates. If you provide a custom TLS Secret, you must also provide a custom replication TLS Secret, and both must contain the same ca.crt.

Provide pre-existing custom certificates¶

For example, you have files named ca.crt, my_tls.key, and my_tls.crt. Run the following command to create a custom TLS Secret named cluster1-tls:

$ kubectl create secret generic -n postgres-operator cluster1-tls \
  --from-file=ca.crt=ca.crt \
  --from-file=tls.key=my_tls.key \
  --from-file=tls.crt=my_tls.crt

In the same way, create the custom TLS replication Secret, for example replication1-tls.

Next, reference your Secrets in the deploy/cr.yaml Custom Resource manifest as follows:

add a Secret created for the external use to the secrets.customTLSSecret.name field
add a Secret created for internal communications to the secrets.customReplicationTLSSecret.name field

Here’s the sample configuration:

spec:
  ...
  secrets:
    customTLSSecret:
      name: cluster1-tls
    customReplicationTLSSecret:
      name: replication1-tls
  ...

Now you can create a cluster with your custom certificates:

$ kubectl apply -f deploy/cr.yaml

Provide a pre-existing custom root CA certificate to the Operator¶

You can also provide a custom root CA certificate to the Operator. In this case the Operator will not generate one itself, but will use the user-provided CA certificate. This can be useful if you would like to have several database clusters with certificates generated by the Operator based on the same root CA.

To make the Operator use a custom root certificate, create a separate secret with this certificate and specify this secret in the Custom Resource options before you deploy a cluster.

For example, if you have files named my_tls.key and my_tls.crt stored on your local machine, you could run the following command to create a Secret named cluster1-ca-cert in the postgres-operator namespace:

$ kubectl create secret generic -n postgres-operator cluster1-ca-cert \
  --from-file=tls.crt=my_tls.crt \
  --from-file=tls.key=my_tls.key

You also need to specify details about this secret in your deploy/cr.yaml manifest:

...
secrets:
  customRootCATLSSecret:
    name: cluster1-ca-cert
    items:
      - key: "tls.crt"
        path: "root.crt"
      - key: "tls.key"
        path: "root.key"

Now, you can create the cluster with the kubectl apply -f deploy/cr.yaml command. The Operator should use the root CA certificate you had provided.

Warning

This approach allows using root CA certificate auto-generated by the Operator for some other clusters, but it needs caution. If the cluster with auto-generated certificate has delete-ssl finalizer enabled, the certificate will be deleted at the cluster deletion event even if it was manually provided to some other cluster.

Generate custom certificates for the Operator yourself¶

Understand certificate requirements¶

To find out the certificates specifics needed for the Operator, view the certificates generated by the Operator automatically. For example, if you have a cluster deployed in some staging environment.

Here’s how to do it:

Check the secrets created by the Operator:

$ kubectl get secrets

Expected output

cluster1-cluster-ca-cert        Opaque   2      143m
cluster1-cluster-cert           Opaque   3      143m
cluster1-instance1-frdm-certs   Opaque   6      143m
cluster1-instance1-qcqk-certs   Opaque   6      143m
cluster1-instance1-wq55-certs   Opaque   6      143m
cluster1-pgbackrest             Opaque   5      143m
cluster1-pgbouncer              Opaque   6      143m
cluster1-pguser-cluster1        Opaque   12     143m
cluster1-replication-cert       Opaque   3      143m

The Secrets of interest are cluster1-cluster-cert for external communication and cluster1-replication-cert for internal communication.

You can examine the auto-generated CA certificate (ca.crt) as follows:

$ kubectl get secret/cluster1-cluster-cert -o jsonpath='{.data.ca\.crt}' | base64 --decode | openssl x509 -text -noout

Expected output

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            ec:f3:d6:f5:35:5c:97:0c:66:cc:90:ed:e6:4b:0a:07
        Signature Algorithm: ecdsa-with-SHA384
        Issuer: CN = postgres-operator-ca
        Validity
            Not Before: Dec 24 13:58:21 2023 GMT
            Not After : Dec 21 14:58:21 2033 GMT
        Subject: CN = postgres-operator-ca
        Subject Public Key Info:
        ...
    ...

You can check the auto-generated TLS certificate (tls.crt) in a similar way:

External communicationInternal communication

$ kubectl get secret/cluster1-cluster-cert -o jsonpath='{.data.tls\.crt}' | base64 --decode | openssl x509 -text -noout

Expected output

Certificate:
    Data:
       Version: 3 (0x2)
       Serial Number:
           43:ac:81:65:4e:c6:1b:15:db:ca:36:c4:16:96:79:1b
       Signature Algorithm: ecdsa-with-SHA384
       Issuer: CN=postgres-operator-ca
       Validity
           Not Before: Jul 22 08:15:42 2025 GMT
           Not After : Jul 22 09:15:42 2026 GMT
       Subject: CN=cluster1-primary.default.svc.cluster.local.
       Subject Public Key Info:
           Public Key Algorithm: id-ecPublicKey
               Public-Key: (256 bit)
               pub:
                   04:cd:06:b5:27:67:64:2b:a3:9e:84:e6:31:81:7f:
                   3f:a9:ae:c9:da:bd:b8:76:3e:f0:09:bd:b8:eb:03:
                   88:c2:d3:4b:2a:1f:e9:5b:97:cf:4e:7b:b3:12:2b:
                   47:ee:a6:24:fb:29:ae:01:74:e2:4c:5c:3e:f9:8d:
                   cb:ff:0a:62:8d
               ASN1 OID: prime256v1
               NIST CURVE: P-256
       X509v3 extensions:
           X509v3 Key Usage: critical
               Digital Signature, Key Encipherment
           X509v3 Basic Constraints: critical
               CA:FALSE
           X509v3 Authority Key Identifier:
               59:98:FE:88:1B:54:A0:7D:DD:20:A0:F6:29:08:05:C7:18:38:7C:92
           X509v3 Subject Alternative Name:
               DNS:cluster1-primary.default.svc.cluster.local., DNS:cluster1-primary.default.svc, DNS:cluster1-primary.default, DNS:cluster1-primary, DNS:cluster1-replicas.default.svc.cluster.local., DNS:cluster1-replicas.default.svc, DNS:cluster1-replicas.default, DNS:cluster1-replicas
    Signature Algorithm: ecdsa-with-SHA384
    ...

$ kubectl get secret/cluster1-replication-cert -o jsonpath='{.data.tls\.crt}' | base64 --decode | openssl x509 -text -noout

Expected output

Certificate:
     Data:
          Version: 3 (0x2)
          Serial Number:
              31:1b:1e:ca:06:e6:98:4d:7e:de:6d:1b:68:d8:53:0e
          Signature Algorithm: ecdsa-with-SHA384
          Issuer: CN=postgres-operator-ca
          Validity
              Not Before: Jul 22 08:15:42 2025 GMT
              Not After : Jul 22 09:15:42 2026 GMT
          Subject: CN=_crunchyrepl
          Subject Public Key Info:
              Public Key Algorithm: id-ecPublicKey
                  Public-Key: (256 bit)
                  pub:
                      04:b1:f7:9d:cd:33:0d:a5:19:a3:f2:fd:f6:b3:cd:
                      e1:a5:e4:19:11:ec:18:db:fe:9c:a8:7e:eb:d2:27:
                      59:d1:ef:3b:09:24:58:21:6a:54:60:30:1c:be:b0:
                      7a:39:c5:91:6f:01:ee:d1:0b:23:86:0c:16:cf:fc:
                      7d:7e:39:cb:0e
                  ASN1 OID: prime256v1
                  NIST CURVE: P-256
          X509v3 extensions:
              X509v3 Key Usage: critical
                  Digital Signature, Key Encipherment
              X509v3 Basic Constraints: critical
                  CA:FALSE
              X509v3 Authority Key Identifier:
                  59:98:FE:88:1B:54:A0:7D:DD:20:A0:F6:29:08:05:C7:18:38:7C:92
              X509v3 Subject Alternative Name:
                  DNS:_crunchyrepl
    Signature Algorithm: ecdsa-with-SHA384
    ...

Both secrets share the same ca.crt certificate but have different tls.crt certificates. The tls.crt in the Secret for external communications should have a Common Name (CN) setting that matches the primary Service name (CN = cluster1-primary.default.svc.cluster.local. in the above example). Similarly, the tls.crt in the Secret for internal communications should have a Common Name (CN) setting that matches the preset replication user: CN=_crunchyrepl.

Generate certificates¶

One of the options to create certificates yourself is to use CloudFlare PKI and TLS toolkit .

You must generate certificates twice: one set is for external communications, and another set is for internal ones!

Let’s say that your cluster name is cluster1 and the desired namespace is postgres-operator. The commands to generate certificates may look as follows:

Set cluster context

$ export CLUSTER_NAME=cluster1
$ export NAMESPACE=postgres-operator

Generate the root CA certificate:

$ cat <<EOF | cfssl gencert -initca - | cfssljson -bare ca
  {
    "CN": "*",
    "key": {
      "algo": "ecdsa",
      "size": 384
    }
  }
  EOF

Expected output

2025/07/22 18:44:00 [INFO] generating a new CA key and certificate from CSR
2025/07/22 18:44:00 [INFO] generate received request
2025/07/22 18:44:00 [INFO] received CSR
2025/07/22 18:44:00 [INFO] generating key: ecdsa-384
2025/07/22 18:44:00 [INFO] encoded CSR
2025/07/22 18:44:00 [INFO] signed certificate with serial number         558041563526770695468617559855840603242491856749

You should have the following files:

ca-key.pem – CA private key
ca.pem – CA certificate

Define the CA signing policy for certificates signed by the CA.

$ cat <<EOF > ca-config.json
  {
     "signing": {
       "default": {
          "expiry": "87600h",
          "usages": ["digital signature", "key encipherment", "content commitment"]
        }
     }
  }
  EOF

Explanation of the values:

expiry - sets the lifetime for the certificates
usages specifies what the certificate is valid for:
- digital signature: for signing data
- key encipherment: for secure key exchange
- content commitment: ensures data integrity

Generate the custom TLS certificates for external communication and sign them using the previously created CA certificate. These certificates have the Common Name (CN) cluster1-primary.postgres-operator.svc.cluster.local

$ cat <<EOF | cfssl gencert -ca=ca.pem  -ca-key=ca-key.pem -config=./ca-config.json - | cfssljson -bare server
  {
     "hosts": [
       "localhost",
       "${CLUSTER_NAME}-primary",
       "${CLUSTER_NAME}-primary.${NAMESPACE}",
       "${CLUSTER_NAME}-primary.${NAMESPACE}.svc.cluster.local",
       "${CLUSTER_NAME}-primary.${NAMESPACE}.svc",
       "${CLUSTER_NAME}-replicas.${NAMESPACE}.svc.cluster.local",
       "${CLUSTER_NAME}-replicas.${NAMESPACE}.svc",
       "${CLUSTER_NAME}-replicas.${NAMESPACE}",
       "${CLUSTER_NAME}-tls-replicas"
     ],
     "CN": "${CLUSTER_NAME}-primary.${NAMESPACE}.svc.cluster.local", 
     "key": {
       "algo": "ecdsa",
       "size": 384
     }
  }
EOF

You should have the following files as defined by the -bare server part of the command:

server.pem - the signed certificate
server-key.pem - the private key

Generate the custom TLS certificates for internal communication and sign them using the previously created CA certificate. These certificates have the Common Name (CN) _crunchyrepl.
```
$ cat <<EOF | cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=./ca-config.json - | cfssljson -bare replication
  {
    "CN": "_crunchyrepl",
    "key": {
      "algo": "ecdsa",
      "size": 384
    }
  }
  EOF
```
You should have the following files as defined by the -bare server part of the command:
- replication.pem - the signed certificate
- replication-key.pem - the private key

You can find more on generating certificates this way in official Kubernetes documentation .

Refer to the Provide pre-existing custom certificates section for the steps to create Secrets and configure the Operator. Replace the values with your files.

Check your certificates for expiration¶

$ kubectl get secrets

First, check the necessary secrets names (cluster1-cluster-cert and cluster1-replication-cert by default):

You will have the following response:

NAME                            TYPE     DATA   AGE
cluster1-cluster-cert           Opaque   3      11m
...
cluster1-replication-cert       Opaque   3      11m
...

Now use the following command to find out the certificates validity dates, substituting Secrets names if necessary:

$ {
  kubectl get secret/cluster1-replication-cert -o jsonpath='{.data.tls\.crt}' | base64 --decode | openssl x509 -noout -dates
  kubectl get secret/cluster1-cluster-cert -o jsonpath='{.data.ca\.crt}' | base64 --decode | openssl x509 -noout -dates
  }

The resulting output will be self-explanatory:

notBefore=Jun 28 10:20:19 2023 GMT
notAfter=Jun 27 11:20:19 2024 GMT
notBefore=Jun 28 10:20:18 2023 GMT
notAfter=Jun 25 11:20:18 2033 GMT

Update certificates¶

The Operator automatically updates the automatically-generated certificates to ensure your applications continue operation without communication issues. However, the Operator doesn’t update custom certificates. It is your responsibility to timely update them.

Update custom certificates¶

You can update only custom certificates for external and / or internal communication and keep the same root CA certificate.

You can update the contents of your existing Secrets referenced in the spec.customTLSSecret and/or spec.customReplicationTLSSecret fields in deploy/cr.yaml without changing their names. In this case, the Operator detects the updated certificate data and applies the changes to the running cluster without restarting it. Such update is called hot reload.

This example shows how you can do it. Let’s say you have the following certificates and Secrets:

server.pem / server-key.pem and the cluster1-cert Secret for external communication,
replica.pem / replica-key.pem and cluster1-replication-cert Secret for internal communication
ca.pem / ca-key.pem is the existing CA root certificate that you keep

Your cluster is deployed in the postgres-operator namespace.

Set the context for the cluster:
```
$ export NAMESPACE=postgres-operator
```

Create a YAML manifest for the cluster1-cert Secret. Run the following command to generate a YAML manifest (adjust file paths if needed):

$ kubectl create secret generic cluster1-cert \
   --from-file=tls.crt=server.pem \
   --from-file=tls.key=server-key.pem \
   --from-file=ca.crt=ca.pem \
   -n "$NAMESPACE" \
   --dry-run=client -o yaml > cluster1-cert.yaml

Create a YAML manifest for the cluster1-replication-cert Secret. Run the following command to generate a YAML manifest (adjust file paths if needed):

$ kubectl create secret generic cluster1-replication-cert \
   --from-file=tls.crt=replica.pem \
   --from-file=tls.key=replica-key.pem \
   --from-file=ca.crt=ca.pem \
   -n "$NAMESPACE" \
   --dry-run=client -o yaml > cluster1-replication-cert.yaml

Apply the manifests to update the Secrets:

$ kubectl apply -f cluster1-cert.yaml -f cluster1-replication-cert.yaml -n "$NAMESPACE"

If you create new Secrets with new names and values, update the spec.customTLSSecret and spec.customReplicationTLSSecret fields in the deploy/cr.yaml. When you apply the new configuration,this causes the Operator to restart the cluster.

Update a custom root CA certificate¶

Here’s what you need to know if you wish to update a custom root CA certificate:

If you change a root CA certificate, you must also change your custom TLS certificates for external and internal communications as these must be signed with the same root CA.
The new root CA and associated certs must be stored in new Secrets (not overwriting existing ones). This ensures rollback capability in case of misconfiguration or validation issues.
You must pause the cluster before applying changes. This prevents the Operator from restarting or reconfiguring Pods mid-update.

To update a custom root CA certificate, do the following:

Generate a new root CA certificate and key. For example, you have them in files named new-ca.pem and new-ca-key.pem.
Generate all dependent certificates for external and internal communication and sign them using the new root CA certificate. Check the Generate certificates manually section for the steps. For example, you end up with the following certificates:
- server.pem and server-key.pem for external communication
- replication.pem and replication-key.pem for internal communication

Create a new Secret object for the new root CA certificate and define the new CA certificate and key within. Let’s name it cluster1-ca-cert-new.

$ kubectl create secret generic -n postgres-operator cluster1-ca-cert-new \
  --from-file=ca.crt=new-ca.pem \
  --from-file=ca.key=new-ca-key.pem

Create new Secrets for external and internal communications, named cluster1-tls and cluster1-replication-tls respectively

$ kubectl create secret generic -n postgres-operator cluster1-tls \
  --from-file=ca.crt=ca.pem \
  --from-file=tls.key=server-key.pem \
  --from-file=tls.crt=server.pem

$ kubectl create secret generic -n postgres-operator cluster1-replication-tls \
  --from-file=ca.crt=ca.pem \
  --from-file=tls.key=replication-key.pem \
  --from-file=tls.crt=replication.pem

Pause the cluster to prevent the Operator to restart the Pods mid-update.

$ kubectl patch pg cluster1 \
  --type merge \
  --patch '{"spec": {"pause": true}}' \
  --namespace postgres-operator

Specify details about new custom certificates in the deploy/cr.yaml. Since this is a provisioned cluster, apply the patch as follows:

$ kubectl patch pg cluster1 \
    --type merge \
    --patch '{
        "spec": {
            "secrets": {
                "customRootCATLSSecret": {
                    "name": "cluster1-ca-cert-new",
                    "items": [
                        {
                            "key": "ca.crt",
                            "path": "root.crt"
                        },
                        {
                            "key": "ca.key",
                            "path": "root.key"
                        }
                    ]
                },
                "customTLSSecret": {
                    "name": "cluster1-tls"
                },
                "customReplicationTLSSecret": {
                    "name": "cluster1-replication-tls"
                }
            }
        }
    }' \
    --namespace postgres-operator

Unpause the cluster to resume the Operator control:

$ kubectl patch pg cluster1 \
  --type merge \
  --patch '{"spec": {"pause": false}}' \
  --namespace postgres-operator

Keep certificates after deleting the cluster¶

In case of cluster deletion, objects, created for SSL (Secret, certificate, and issuer) are not deleted by default.

If the user wants the cleanup of objects created for SSL, there is a finalizers.percona.com/delete-ssl Custom Resource option, which can be set in deploy/cr.yaml: if this finalizer is set, the Operator will delete Secret, certificate and issuer after the cluster deletion event.

Last update: 2025-08-14