Skip to content
logo
Percona Operator for PostgreSQL
Anti-affinity and tolerations
Initializing search
    percona/k8spg-docs
    percona/k8spg-docs
    • Welcome
      • System Requirements
      • Design and architecture
      • Comparison with other solutions
      • Install on Minikube
      • Install on Google Kubernetes Engine (GKE)
      • Install with Helm
      • Generic Kubernetes installation
      • Install on OpenShift
      • Application and system users
      • Changing PostgreSQL Options
      • Anti-affinity and tolerations
        • Affinity and anti-affinity
        • Simple approach - configure Node Affinity based on nodeLabel
          • Advanced approach - use standard Kubernetes constraints
          • Default Affinity rules
        • Tolerations
      • Transport Encryption (TLS/SSL)
      • Telemetry
      • Backup and restore
      • Upgrade Percona Distribution for PostgreSQL and the Operator
      • Horizontal and vertical scaling
      • Monitor with Percona Monitoring and Management (PMM)
      • Restart or pause the cluster
      • Deploy a standby cluster for Disaster Recovery
      • Provide Percona Operator for PostgreSQL single-namespace and multi-namespace deployment
      • Use PostgreSQL tablespaces with Percona Operator for PostgreSQL
      • Custom Resource options
      • Operator installation options
      • Percona certified images
      • Frequently Asked Questions
      • Release notes index
      • Percona Operator for PostgreSQL 1.3.0 (2022-08-04)
      • Percona Operator for PostgreSQL 1.2.0 (2022-04-06)
      • Percona Operator for PostgreSQL 1.1.0 (2021-12-07)
      • Percona Operator for PostgreSQL 1.0.0 (2021-10-07)
      • Percona Operator for PostgreSQL 0.2.0 (2021-08-12)
      • Percona Operator for PostgreSQL 0.1.0 (2021-05-10)

    • Affinity and anti-affinity
    • Simple approach - configure Node Affinity based on nodeLabel
      • Advanced approach - use standard Kubernetes constraints
      • Default Affinity rules
    • Tolerations

    Binding Percona Distribution for PostgreSQL components to Specific Kubernetes/OpenShift Nodes¶

    The operator does good job automatically assigning new Pods to nodes with sufficient resources to achieve balanced distribution across the cluster. Still there are situations when it is worth to ensure that pods will land on specific nodes: for example, to get speed advantages of the SSD equipped machine, or to reduce network costs choosing nodes in a same availability zone.

    Appropriate sections of the deploy/cr.yaml file (such as pgPrimary or pgReplicas) contain keys which can be used to do this, depending on what is the best for a particular situation.

    Affinity and anti-affinity¶

    Affinity makes Pod eligible (or not eligible - so called “anti-affinity”) to be scheduled on the node which already has Pods with specific labels, or has specific labels itself (so called “Node affinity”). Particularly, Pod anti-affinity is good to reduce costs making sure several Pods with intensive data exchange will occupy the same availability zone or even the same node - or, on the contrary, to make them land on different nodes or even different availability zones for the high availability and balancing purposes. Node affinity is useful to assign PostgreSQL instances to specific Kubernetes Nodes (ones with specific hardware, zone, etc.).

    Pod anti-affinity is controlled by the antiAffinityType option, which can be put into pgPrimary, pgBouncer, and backup sections of the deploy/cr.yaml configuration file. This option can be set to one of two values:

    • preferred Pod anti-affinity is a sort of a soft rule. It makes Kubernetes trying to schedule Pods matching the anti-affinity rules to different Nodes. If it is not possible, then one or more Pods are scheduled to the same Node. This variant is used by default.
    • required Pod anti-affinity is a sort of a hard rule. It forces Kubernetes to schedule each Pod matching the anti-affinity rules to different Nodes. If it is not possible, then a Pod will not be scheduled at all.

    Node affinity can be controlled by the pgPrimary.affinity.nodeAffinityType option in the deploy/cr.yaml configuration file. This option can be set to either preferred or required similarly to the antiAffinityType option.

    Simple approach - configure Node Affinity based on nodeLabel¶

    The Operator provides the pgPrimary.affinity.nodeLabel option, which should contains one or more key-value pairs. If the node is not labeled with each key-value pair and nodeAffinityType is set to required, the Pod will not be able to land on it.

    The following example forces Operator to lend Percona Distribution for PostgreSQL instances on the Nodes having the kubernetes.io/region: us-central1 label:

    affinity:
      nodeAffinityType: required
      nodeLabel:
        kubernetes.io/region: us-central1
    

    Advanced approach - use standard Kubernetes constraints¶

    Previous way can be used with no special knowledge of the Kubernetes way of assigning Pods to specific Nodes. Still in some cases more complex tuning may be needed. In this case pgPrimary.affinity.advanced option placed in the deploy/cr.yaml file turns off the effect of the nodeLabel and allows to use standard Kubernetes affinity constraints of any complexity:

    affinity:
       advanced:
         podAffinity:
           requiredDuringSchedulingIgnoredDuringExecution:
           - labelSelector:
               matchExpressions:
               - key: security
                 operator: In
                 values:
                 - S1
             topologyKey: failure-domain.beta.kubernetes.io/zone
         podAntiAffinity:
           preferredDuringSchedulingIgnoredDuringExecution:
           - weight: 100
             podAffinityTerm:
               labelSelector:
                 matchExpressions:
                 - key: security
                   operator: In
                   values:
                   - S2
               topologyKey: kubernetes.io/hostname
         nodeAffinity:
           requiredDuringSchedulingIgnoredDuringExecution:
             nodeSelectorTerms:
             - matchExpressions:
               - key: kubernetes.io/e2e-az-name
                 operator: In
                 values:
                 - e2e-az1
                 - e2e-az2
           preferredDuringSchedulingIgnoredDuringExecution:
           - weight: 1
             preference:
               matchExpressions:
               - key: another-node-label-key
                 operator: In
                 values:
                 - another-node-label-value
    

    You can see the explanation of these affinity options in Kubernetes documentation.

    Default Affinity rules¶

    The following anti-affinity rules are applied to all Percona Distribution for PostgreSQL Pods:

    affinity:
      podAntiAffinity:
        preferredDuringSchedulingIgnoredDuringExecution:
        - podAffinityTerm:
            labelSelector:
              matchExpressions:
              - key: vendor
                operator: In
                values:
                - crunchydata
              - key: pg-pod-anti-affinity
                operator: Exists
              - key: pg-cluster
                operator: In
                values:
                - cluster1
            topologyKey: kubernetes.io/hostname
          weight: 1
    

    You can see the explanation of these affinity options in Kubernetes documentation.

    Note

    Setting required anti-affinity type will result in placing all Pods on separate nodes, so default configuration will require 7 Kubernetes nodes to deploy the cluster with separate nodes assigned to one PostgreSQL primary, two PostgreSQL replica instances, three pgBouncer and one pgBackrest Pod.

    Tolerations¶

    Tolerations allow Pods having them to be able to land onto nodes with matching taints. Toleration is expressed as a key with and operator, which is either exists or equal (the latter variant also requires a value the key is equal to). Moreover, toleration should have a specified effect, which may be a self-explanatory NoSchedule, less strict PreferNoSchedule, or NoExecute. The last variant means that if a taint with NoExecute is assigned to node, then any Pod not tolerating this taint will be removed from the node, immediately or after the tolerationSeconds interval, like in the following example.

    You can use pgPrimary.tolerations key in the deploy/cr.yaml configuration file as follows:

    tolerations:
    - key: "node.alpha.kubernetes.io/unreachable"
      operator: "Exists"
      effect: "NoExecute"
      tolerationSeconds: 6000
    

    The Kubernetes Taints and Toleratins contains more examples on this topic.

    Contact Us

    For free technical help, visit the Percona Community Forum.

    To report bugs or submit feature requests, open a JIRA ticket.

    For paid support and managed or consulting services , contact Percona Sales.


    Last update: 2023-02-09
    Back to top
    Percona LLC and/or its affiliates, © 2009 - 2022
    Made with Material for MkDocs

    Cookie consent

    We use cookies to recognize your repeated visits and preferences, as well as to measure the effectiveness of our documentation and whether users find what they're searching for. With your consent, you're helping us to make our documentation better.