Skip to content
logo
Percona Operator for MySQL
Load Balancing with HAProxy
Initializing search
    percona/k8spxc-docs
    percona/k8spxc-docs
    • Welcome
      • System Requirements
      • Design and architecture
      • Comparison with other solutions
      • Install with Helm
      • Install with kubectl
      • Install on Minikube
      • Install on Google Kubernetes Engine (GKE)
      • Install on Amazon Elastic Kubernetes Service (AWS EKS)
      • Install on Microsoft Azure Kubernetes Service (AKS)
      • Install on OpenShift
      • Generic Kubernetes installation
      • Multi-cluster and multi-region deployment
      • Application and system users
      • Changing MySQL Options
      • Anti-affinity and tolerations
      • Labels and annotations
      • Local Storage support
      • Defining environment variables
      • Load Balancing with HAProxy
        • Passing custom configuration options to HAProxy
          • Edit the deploy/cr.yaml file
          • Use a ConfigMap
          • Use a Secret Object
        • Enabling the Proxy protocol
      • Load Balancing with ProxySQL
      • Transport Encryption (TLS/SSL)
      • Data at rest encryption
      • Telemetry
      • Backup and restore
      • Upgrade Database and Operator
      • Horizontal and vertical scaling
      • Monitor with Percona Monitoring and Management (PMM)
      • Add sidecar containers
      • Restart or pause the cluster
      • Crash recovery
      • Debug and troubleshoot
      • How to install Percona XtraDB Cluster in multi-namespace (cluster-wide) mode
      • How to upgrade Percona XtraDB Cluster manually
      • How to use private registry
      • Custom Resource options
      • Percona certified images
      • Operator API
      • Frequently Asked Questions
      • Old releases (documentation archive)
      • Release notes index
      • Percona Operator for MySQL based on Percona XtraDB Cluster 1.12.0 (2022-12-07)
      • Percona Operator for MySQL based on Percona XtraDB Cluster 1.11.0 (2022-06-03)
      • Percona Distribution for MySQL Operator 1.10.0 (2021-11-24)
      • Percona Distribution for MySQL Operator 1.9.0 (2021-08-09)
      • Percona Kubernetes Operator for Percona XtraDB Cluster 1.8.0 (2021-05-26)
      • Percona Kubernetes Operator for Percona XtraDB Cluster 1.7.0 (2021-02-02)
      • Percona Kubernetes Operator for Percona XtraDB Cluster 1.6.0 (2020-09-09)
      • Percona Kubernetes Operator for Percona XtraDB Cluster 1.5.0 (2020-07-21)
      • Percona Kubernetes Operator for Percona XtraDB Cluster 1.4.0 (2020-04-29)
      • Percona Kubernetes Operator for Percona XtraDB Cluster 1.3.0 (2020-01-06)
      • Percona Kubernetes Operator for Percona XtraDB Cluster 1.2.0 (2019-09-20)
      • Percona Kubernetes Operator for Percona XtraDB Cluster 1.1.0 (2019-07-15)
      • Percona Kubernetes Operator for Percona XtraDB Cluster 1.0.0 (2019-05-29)

    • Passing custom configuration options to HAProxy
      • Edit the deploy/cr.yaml file
      • Use a ConfigMap
      • Use a Secret Object
    • Enabling the Proxy protocol

    Configuring Load Balancing with HAProxy¶

    Percona Operator for MySQL based on Percona XtraDB Cluster provides a choice of two cluster components to provide load balancing and proxy service: you can use either HAProxy or ProxySQL. You can control which one to use, if any, by enabling or disabling via the haproxy.enabled and proxysql.enabled options in the deploy/cr.yaml configuration file.

    Use the following command to enable HAProxy:

    $ kubectl patch pxc cluster1 --type=merge --patch '{
      "spec": {
         "haproxy": {
            "enabled": true,
            "size": 3,
            "image": "percona/percona-xtradb-cluster-operator:1.12.0-haproxy" },
         "proxysql": { "enabled": false }
      }}'
    

    Note

    For obvious reasons the Operator will not allow the simultaneous enabling of both HAProxy and ProxySQL.

    The resulting HAPproxy setup will contain two services:

    • cluster1-haproxy service listening on ports 3306 (MySQL) and 3309 (the proxy protocol). This service is pointing to the number zero Percona XtraDB Cluster member (cluster1-pxc-0) by default when this member is available. If a zero member is not available, members are selected in descending order of their numbers (e.g. cluster1-pxc-2, then cluster1-pxc-1, etc.). This service can be used for both read and write load, or it can also be used just for write load (single writer mode) in setups with split write and read loads.

    • cluster1-haproxy-replicas listening on port 3306 (MySQL). This service selects Percona XtraDB Cluster members to serve queries following the Round Robin load balancing algorithm.

    When the cluster with HAProxy is upgraded, the following steps take place. First, reader members are upgraded one by one: the Operator waits until the upgraded Percona XtraDB Cluster member becomes synced, and then proceeds to upgrade the next member. When the upgrade is finished for all the readers, then the writer Percona XtraDB Cluster member is finally upgraded.

    Passing custom configuration options to HAProxy¶

    You can pass custom configuration to HAProxy in one of the following ways:

    • edit the deploy/cr.yaml file,
    • use a ConfigMap,
    • use a Secret object.

    Note

    If you specify a custom HAProxy configuration in this way, the Operator doesn’t provide its own HAProxy configuration file. That’s why you should specify either a full set of configuration options or nothing.

    Edit the deploy/cr.yaml file¶

    You can add options from the haproxy.cfg configuration file by editing haproxy.configuration key in the deploy/cr.yaml file. Here is an example:

    ...
    haproxy:
        enabled: true
        size: 3
        image: percona/percona-xtradb-cluster-operator:1.12.0-haproxy
        configuration: |
          global
            maxconn 2048
            external-check
            stats socket /var/run/haproxy.sock mode 600 expose-fd listeners level user
          defaults
            log global
            mode tcp
            retries 10
            timeout client 10000
            timeout connect 100500
            timeout server 10000
          frontend galera-in
            bind *:3309 accept-proxy
            bind *:3306
            mode tcp
            option clitcpka
            default_backend galera-nodes
          frontend galera-replica-in
            bind *:3309 accept-proxy
            bind *:3307
            mode tcp
            option clitcpka
            default_backend galera-replica-nodes
    

    Use a ConfigMap¶

    You can use a configmap and the cluster restart to reset configuration options. A configmap allows Kubernetes to pass or update configuration data inside a containerized application.

    Use the kubectl command to create the configmap from external resources, for more information see Configure a Pod to use a ConfigMap.

    For example, you define a haproxy.cfg configuration file with the following setting:

    global
      maxconn 2048
      external-check
      stats socket /var/run/haproxy.sock mode 600 expose-fd listeners level user
    defaults
      log global
      mode tcp
      retries 10
      timeout client 10000
      timeout connect 100500
      timeout server 10000
    frontend galera-in
      bind *:3309 accept-proxy
      bind *:3306
      mode tcp
      option clitcpka
      default_backend galera-nodes
    frontend galera-replica-in
      bind *:3309 accept-proxy
      bind *:3307
      mode tcp
      option clitcpka
      default_backend galera-replica-nodes
    

    You can create a configmap from the haproxy.cfg file with the kubectl create configmap command.

    You should use the combination of the cluster name with the -haproxy suffix as the naming convention for the configmap. To find the cluster name, you can use the following command:

    $ kubectl get pxc
    

    The syntax for kubectl create configmap command is:

    kubectl create configmap <cluster-name>-haproxy <resource-type=resource-name>
    

    The following example defines cluster1-haproxy as the configmap name and the haproxy.cfg file as the data source:

    $ kubectl create configmap cluster1-haproxy --from-file=haproxy.cfg
    

    To view the created configmap, use the following command:

    $ kubectl describe configmaps cluster1-haproxy
    

    Use a Secret Object¶

    The Operator can also store configuration options in Kubernetes Secrets. This can be useful if you need additional protection for some sensitive data.

    You should create a Secret object with a specific name, composed of your cluster name and the haproxy suffix.

    Note

    To find the cluster name, you can use the following command:

    $ kubectl get pxc
    

    Configuration options should be put inside a specific key inside of the data section. The name of this key is haproxy.cfg for ProxySQL Pods.

    Actual options should be encoded with Base64.

    For example, let’s define a haproxy.cfg configuration file and put there options we used in the previous example:

    global
      maxconn 2048
      external-check
      stats socket /var/run/haproxy.sock mode 600 expose-fd listeners level user
    defaults
      log global
      mode tcp
      retries 10
      timeout client 10000
      timeout connect 100500
      timeout server 10000
    frontend galera-in
      bind *:3309 accept-proxy
      bind *:3306
      mode tcp
      option clitcpka
      default_backend galera-nodes
    frontend galera-replica-in
      bind *:3309 accept-proxy
      bind *:3307
      mode tcp
      option clitcpka
      default_backend galera-replica-nodes
    

    You can get a Base64 encoded string from your options via the command line as follows:

    $ cat haproxy.cfg | base64 --wrap=0
    
    $ cat haproxy.cfg | base64
    

    Note

    Similarly, you can read the list of options from a Base64 encoded string:

    $ echo "IGdsb2JhbAogICBtYXhjb25uIDIwNDgKICAgZXh0ZXJuYWwtY2hlY2sKICAgc3RhdHMgc29ja2V0\
      IC92YXIvcnVuL2hhcHJveHkuc29jayBtb2RlIDYwMCBleHBvc2UtZmQgbGlzdGVuZXJzIGxldmVs\
      IHVzZXIKIGRlZmF1bHRzCiAgIGxvZyBnbG9iYWwKICAgbW9kZSB0Y3AKICAgcmV0cmllcyAxMAog\
      ICB0aW1lb3V0IGNsaWVudCAxMDAwMAogICB0aW1lb3V0IGNvbm5lY3QgMTAwNTAwCiAgIHRpbWVv\
      dXQgc2VydmVyIDEwMDAwCiBmcm9udGVuZCBnYWxlcmEtaW4KICAgYmluZCAqOjMzMDkgYWNjZXB0\
      LXByb3h5CiAgIGJpbmQgKjozMzA2CiAgIG1vZGUgdGNwCiAgIG9wdGlvbiBjbGl0Y3BrYQogICBk\
      ZWZhdWx0X2JhY2tlbmQgZ2FsZXJhLW5vZGVzCiBmcm9udGVuZCBnYWxlcmEtcmVwbGljYS1pbgog\
      ICBiaW5kICo6MzMwOSBhY2NlcHQtcHJveHkKICAgYmluZCAqOjMzMDcKICAgbW9kZSB0Y3AKICAg\
      b3B0aW9uIGNsaXRjcGthCiAgIGRlZmF1bHRfYmFja2VuZCBnYWxlcmEtcmVwbGljYS1ub2Rlcwo=" | base64 --decode
    

    Finally, use a yaml file to create the Secret object. For example, you can create a deploy/my-haproxy-secret.yaml file with the following contents:

    apiVersion: v1
    kind: Secret
    metadata:
      name: cluster1-haproxy
    data:
      my.cnf: "IGdsb2JhbAogICBtYXhjb25uIDIwNDgKICAgZXh0ZXJuYWwtY2hlY2sKICAgc3RhdHMgc29ja2V0\
         IC92YXIvcnVuL2hhcHJveHkuc29jayBtb2RlIDYwMCBleHBvc2UtZmQgbGlzdGVuZXJzIGxldmVs\
         IHVzZXIKIGRlZmF1bHRzCiAgIGxvZyBnbG9iYWwKICAgbW9kZSB0Y3AKICAgcmV0cmllcyAxMAog\
         ICB0aW1lb3V0IGNsaWVudCAxMDAwMAogICB0aW1lb3V0IGNvbm5lY3QgMTAwNTAwCiAgIHRpbWVv\
         dXQgc2VydmVyIDEwMDAwCiBmcm9udGVuZCBnYWxlcmEtaW4KICAgYmluZCAqOjMzMDkgYWNjZXB0\
         LXByb3h5CiAgIGJpbmQgKjozMzA2CiAgIG1vZGUgdGNwCiAgIG9wdGlvbiBjbGl0Y3BrYQogICBk\
         ZWZhdWx0X2JhY2tlbmQgZ2FsZXJhLW5vZGVzCiBmcm9udGVuZCBnYWxlcmEtcmVwbGljYS1pbgog\
         ICBiaW5kICo6MzMwOSBhY2NlcHQtcHJveHkKICAgYmluZCAqOjMzMDcKICAgbW9kZSB0Y3AKICAg\
         b3B0aW9uIGNsaXRjcGthCiAgIGRlZmF1bHRfYmFja2VuZCBnYWxlcmEtcmVwbGljYS1ub2Rlcwo="
    

    When ready, apply it with the following command:

    $ kubectl create -f deploy/my-haproxy-secret.yaml
    

    Note

    Do not forget to restart Percona XtraDB Cluster to ensure the cluster has updated the configuration.

    Enabling the Proxy protocol¶

    The Proxy protocol allows HAProxy to provide a real client address to Percona XtraDB Cluster.

    Note

    To use this feature, you should have a Percona XtraDB Cluster image version 8.0.21 or newer.

    Normally Proxy protocol is disabled, and Percona XtraDB Cluster sees the IP address of the proxying server (HAProxy) instead of the real client address. But there are scenarios when making real client IP-address visible for Percona XtraDB Cluster is important: e.g. it allows to have privilege grants based on client/application address, and significantly enhance auditing.

    You can enable Proxy protocol on Percona XtraDB Cluster by adding proxy_protocol_networks option to pxc.configuration key in the deploy/cr.yaml configuration file.

    Note

    Depending on the load balancer of your cloud provider, you may also need setting haproxy.externaltrafficpolicy option in deploy/cr.yaml.

    More information about Proxy protocol can be found in the official HAProxy documentation.

    Contact Us

    For free technical help, visit the Percona Community Forum.

    To report bugs or submit feature requests, open a JIRA ticket.

    For paid support and managed or consulting services , contact Percona Sales.


    Last update: 2023-02-09
    Back to top
    Percona LLC and/or its affiliates, © 2009 - 2022
    Made with Material for MkDocs

    Cookie consent

    We use cookies to recognize your repeated visits and preferences, as well as to measure the effectiveness of our documentation and whether users find what they're searching for. With your consent, you're helping us to make our documentation better.