Run with Docker Compose¶

For more information about using Docker, see the Docker Docs . Use the latest version of Docker. Versions provided through apt and yum may be outdated and cause errors.

Percona collects Telemetry data in the Percona packages and Docker images.

Review Get more help for ways that we can work with you.

This guide describes how to deploy a three-node Percona XtraDB Cluster (PXC) 8.4 with Docker Compose. You generate Secure Sockets Layer (SSL) certificates on the first node. You then copy the certificates to the other two nodes to enable secure communication.

Percona XtraDB Cluster with Docker

The following procedure is for evaluation and testing only. The MySQL certificates generated here are self-signed. For production, generate and store proper certificates. Also configure storage, secure networking, backup, and monitoring.

Single-host deployment limits availability

This guide runs all three nodes on a single host, which is not highly available. If the host reboots or the disk fails, the entire cluster fails and data can be lost. For production, run each of the three nodes on different physical machines or distinct failure domains. Use scheduling such as Docker Swarm or Kubernetes with anti-affinity to prevent a single failure from taking out the cluster. For production guidance, see High availability.

Prerequisites¶

The deployment requires the following:

Docker and Docker Compose installed (or Podman 4.1 or later and Podman Compose; see Appendix: Podman alternative)
At least 3 GB of memory per container. Percona XtraDB Cluster requires substantial memory. Without enough Random-Access Memory (RAM), the Out-of-Memory (OOM) killer can terminate the MySQL process. Termination commonly occurs during startup or during the State Snapshot Transfer (SST). The cluster can then fail or restart in a loop. Allocate enough free RAM for three nodes plus overhead (for example, at least 10 GB total). The example Compose file sets a 3 GB limit per container. If the host has more RAM, increase the limit accordingly.
Familiarity with Docker volumes and networks

Directory structure¶

Create a directory structure to organize configuration, certificate files, and the Docker Compose definition.

Run the following commands to create the directory structure:

mkdir -p pxc-cluster/{certs,conf.d,init}
cd pxc-cluster

Expected result: No output. The current directory is pxc-cluster with subdirectories certs, conf.d, and init.

After you run these commands, the pxc-cluster/ working directory contains the following layout:

pxc-cluster/
├── certs/
├── conf.d/
└── init/

The directory structure separates configuration files, Transport Layer Security (TLS) and SSL certificates, and initialization scripts.

Configuration files¶

Create conf.d/custom.cnf with minimal SSL settings:

[mysqld]
ssl-ca=/etc/mysql/certs/ca.pem
ssl-cert=/etc/mysql/certs/server-cert.pem
ssl-key=/etc/mysql/certs/server-key.pem

Configure password storage with Docker Secrets or .env.

A .env file is convenient but exposes secret values through docker inspect and process listings. Prefer Docker Secrets (or Podman secrets), which mount passwords as files instead of environment variables.

Choose one of the following options:
Option A: Docker Secrets (recommended)Option B: .env (quick local test only)
Create two secret files in the project root. Each file must contain only the password with no trailing newline. Generate strong passwords and write them in one step:
openssl rand -base64 24 | tr -d '\n' > mysql_root_password openssl rand -base64 24 | tr -d '\n' > xtrabackup_password chmod 600 mysql_root_password xtrabackup_password
Each file contains a single line with no trailing newline:
- mysql_root_password — the MySQL root password.
- xtrabackup_password — the XtraBackup replication password.
Add mysql_root_password and xtrabackup_password to your .gitignore. The Compose file in Docker Compose setup references these files through the secrets: block. The container reads passwords from the mounted files through MYSQL_ROOT_PASSWORD_FILE and XTRABACKUP_PASSWORD_FILE. The Percona XtraDB Cluster 8.4 image supports these _FILE variables. If your image does not support _FILE variables, use Option B.
Create a file named .env in the directory root with the following contents:
MYSQL_ROOT_PASSWORD=<password> XTRABACKUP_PASSWORD=<password>
Add .env to your .gitignore. In the Compose file, replace the secrets block and the _FILE variables with MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD} and XTRABACKUP_PASSWORD=${XTRABACKUP_PASSWORD}.

SSL certificate generation¶

For broader guidance on protecting cluster traffic, see Encrypt traffic.

Copy the SSL certificate generation script. The script uses OpenSSL to create a Certificate Authority (CA), generate a Certificate Signing Request (CSR), and sign the server certificate. Save the script as init/create-ssl-certs.sh:

#!/bin/bash
set -e
CERT_DIR=./certs
mkdir -p "$CERT_DIR"
cd "$CERT_DIR"
# Generate CA key and certificate
openssl genrsa -out ca-key.pem 2048
openssl req -new -x509 -nodes -days 3650 \
    -key ca-key.pem \
    -subj "/C=XX/ST=State/L=City/O=Organization/CN=RootCA" \
    -out ca.pem
# Generate server key and CSR
openssl req -newkey rsa:2048 -nodes \
    -keyout server-key.pem \
    -subj "/C=XX/ST=State/L=City/O=Organization/CN=pxc-node" \
    -out server-req.pem
# Sign server certificate with CA
openssl x509 -req -in server-req.pem -days 3650 \
    -CA ca.pem -CAkey ca-key.pem -set_serial 01 \
    -out server-cert.pem
# Restrict permissions
chmod 600 *.pem

Make the script executable:
```
chmod +x init/create-ssl-certs.sh
```
Expected result: No output.
Run the script to create the certs:
```
./init/create-ssl-certs.sh
```
Expected result: No output. The script creates ca-key.pem, ca.pem, server-key.pem, server-req.pem, and server-cert.pem in certs/.

The certificates expire in 10 years. For production environments, implement certificate rotation and expiration monitoring.

Warning

To regenerate certificates, delete the certs/ directory first, then run the script again. Running the script twice without removing existing files can fail or overwrite keys. Overwritten keys break trust across nodes already joined to the cluster.
Copy certificates to all nodes (only for multi-host).

All three nodes must use the same set of SSL certificates. The docker-compose.yml in this guide mounts the same ./certs directory for all three services (pxc1, pxc2, pxc3). For a single-host deployment, all nodes share the certs/ directory created in step 3.

For a multi-host deployment, copy the certificates to each machine. Each host must have its own certs/ directory (or equivalent path) for its Compose project. From node 1, run the following commands:
```
scp -r ./certs/ user@node2-host:/path/to/pxc-cluster/certs
scp -r ./certs/ user@node3-host:/path/to/pxc-cluster/certs
```
Expected result: Progress or confirmation for each file copied to each host. No output indicates success after the transfer completes.

Ensure each host’s Compose file mounts that host’s certificate directory. See Multi-host deployment for firewall, name resolution, and time synchronization.

Multi-host deployment¶

When you run each node on a separate machine, configure the following on every host in addition to copying certificates.

Firewall¶

Allow cluster and client traffic between hosts. Open the following ports on each host for the other hosts’ Internet Protocol (IP) addresses or subnet:

3306 (MySQL client)
4444 (Percona XtraBackup snapshot transfer)
4567 (Galera replication)
4568 (Galera incremental state transfer)

Name resolution and CLUSTER_JOIN¶

Containers on one host must reach containers on other hosts by hostname or IP address. The example Compose file uses CLUSTER_JOIN=pxc1, which works only when all containers run on the same host. The name pxc1 resolves only inside the host network where that container runs.

For a multi-host deployment, set CLUSTER_JOIN to the IP address or Fully Qualified Domain Name (FQDN). Use the address of the host that runs the bootstrap node. On the hosts that run pxc2 and pxc3, set one of the following:

CLUSTER_JOIN=<IP_OF_NODE_1>
CLUSTER_JOIN=<FQDN_OF_NODE_1>

Replace the placeholder with the actual IP address or FQDN of the machine where pxc1 runs. Add Domain Name System (DNS) records or /etc/hosts entries on each host. The value in CLUSTER_JOIN must resolve to the correct machine.

Time synchronization¶

Percona XtraDB Cluster and Galera require consistent time across nodes. Synchronize the clock on every host with a Network Time Protocol (NTP) implementation such as chrony or systemd-timesyncd. Clock skew between hosts can cause replication issues and cluster instability. Enable and run NTP before starting the cluster.

Docker Compose setup¶

Create docker-compose.yml in the project root. The following example uses Docker Secrets (Option A in Configuration files). If you chose the .env option (Option B), make these changes:

Remove the top-level secrets: block.
Remove each service’s secrets: list.
Replace the _FILE variables with MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD} and XTRABACKUP_PASSWORD=${XTRABACKUP_PASSWORD}.

Warning

If the host runs out of memory, the cluster can fail. Ensure the host has at least enough free RAM for three nodes plus overhead (for example, 10 GB total). The following example sets a 3 GB memory limit per container. The daemon does not start a container when the host cannot satisfy the limit. If the host has more RAM, increase the limit (for example, 4 GB per container for a 12 GB+ host).

The memory limit syntax depends on the Compose runtime. Docker Compose v2 (the docker compose plugin) honors the deploy.resources.limits.memory block in the following example. podman-compose and legacy docker-compose v1 ignore the deploy: block.

Memory limits with Podman Compose

If you run the stack with podman compose or podman-compose, replace the deploy.resources.limits.memory block on each service with mem_limit: 3g. The mem_limit: key sits at the service level.

The example sets the following environment variables on each service:

Variable	Required	Description	Example
`MYSQL_ROOT_PASSWORD_FILE`	Yes	Path inside the container to the file holding the MySQL root password. Required when using Docker Secrets (Option A).	`/run/secrets/mysql_root_password`
`XTRABACKUP_PASSWORD_FILE`	Yes	Path inside the container to the file holding the XtraBackup replication password. Required when using Docker Secrets (Option A).	`/run/secrets/xtrabackup_password`
`CLUSTER_NAME`	Yes	Logical name shared by all three nodes. Must be identical across pxc1, pxc2, and pxc3.	`pxc-cluster`
`CLUSTER_JOIN`	On join nodes only	Address of the bootstrap node. Omit on the bootstrap node (pxc1). On a single host, use the container name. On multiple hosts, use the IP address or FQDN.	`pxc1`
`MYSQL_ROOT_PASSWORD`	Option B only	MySQL root password value. Used in place of `MYSQL_ROOT_PASSWORD_FILE` when configuring with `.env`.	`${MYSQL_ROOT_PASSWORD}`
`XTRABACKUP_PASSWORD`	Option B only	XtraBackup replication password value. Used in place of `XTRABACKUP_PASSWORD_FILE` when configuring with `.env`.	`${XTRABACKUP_PASSWORD}`

secrets:
  mysql_root_password:
    file: ./mysql_root_password
  xtrabackup_password:
    file: ./xtrabackup_password
services:
  pxc1:
    image: percona/percona-xtradb-cluster:8.4
    container_name: pxc1
    environment:
      - MYSQL_ROOT_PASSWORD_FILE=/run/secrets/mysql_root_password
      - CLUSTER_NAME=pxc-cluster
      - XTRABACKUP_PASSWORD_FILE=/run/secrets/xtrabackup_password
    secrets:
      - mysql_root_password
      - xtrabackup_password
    volumes:
      - ./certs:/etc/mysql/certs:ro,Z
      - ./conf.d:/etc/percona-xtradb-cluster.conf.d:ro,Z
      - ./data-pxc1:/var/lib/mysql,Z
    networks:
      - pxcnet
    ports:
      - "3306:3306"
    command: ["--wsrep-new-cluster"]
    deploy:
      resources:
        limits:
          memory: 3G
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
      interval: 10s
      timeout: 5s
      retries: 5
  pxc2:
    image: percona/percona-xtradb-cluster:8.4
    container_name: pxc2
    environment:
      - MYSQL_ROOT_PASSWORD_FILE=/run/secrets/mysql_root_password
      - CLUSTER_NAME=pxc-cluster
      - CLUSTER_JOIN=pxc1
      - XTRABACKUP_PASSWORD_FILE=/run/secrets/xtrabackup_password
    secrets:
      - mysql_root_password
      - xtrabackup_password
    volumes:
      - ./certs:/etc/mysql/certs:ro,Z
      - ./conf.d:/etc/percona-xtradb-cluster.conf.d:ro,Z
      - ./data-pxc2:/var/lib/mysql,Z
    networks:
      - pxcnet
    depends_on:
      pxc1:
        condition: service_healthy
    deploy:
      resources:
        limits:
          memory: 3G
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
      interval: 10s
      timeout: 5s
      retries: 5
  pxc3:
    image: percona/percona-xtradb-cluster:8.4
    container_name: pxc3
    environment:
      - MYSQL_ROOT_PASSWORD_FILE=/run/secrets/mysql_root_password
      - CLUSTER_NAME=pxc-cluster
      - CLUSTER_JOIN=pxc1
      - XTRABACKUP_PASSWORD_FILE=/run/secrets/xtrabackup_password
    secrets:
      - mysql_root_password
      - xtrabackup_password
    volumes:
      - ./certs:/etc/mysql/certs:ro,Z
      - ./conf.d:/etc/percona-xtradb-cluster.conf.d:ro,Z
      - ./data-pxc3:/var/lib/mysql,Z
    networks:
      - pxcnet
    depends_on:
      pxc1:
        condition: service_healthy
    deploy:
      resources:
        limits:
          memory: 3G
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
      interval: 10s
      timeout: 5s
      retries: 5
networks:
  pxcnet:
    driver: bridge

Volume persistence: The example uses bind mounts (./data-pxc1, ./data-pxc2, ./data-pxc3). Data lives on the host and survives docker compose down. For production, consider named volumes. Named volumes let the runtime manage storage instead of tying data to a single host path. Without bind mounts or named volumes, database data is ephemeral and is lost when containers are removed.

SELinux and rootless: The example uses :Z (or :z) on all volume mounts. The runtime relabels them for the container. SELinux relabeling is required when SELinux is enabled. RHEL, CentOS Stream, Rocky, AlmaLinux, and Fedora enable SELinux by default. Rootless runtimes (Podman or Docker rootless) also commonly need relabeling. Without relabeling, the container cannot read certs/, conf.d/, or the data directories and fails at startup. On systems without SELinux, remove ,Z from the volume suffixes. For more on SELinux configuration, see SELinux.

Deployment¶

The Compose file sets depends_on: pxc1: condition: service_healthy for pxc2 and pxc3. The join nodes start only after pxc1 passes its healthcheck. This dependency prevents a race where the join nodes start before the bootstrap node is ready. Choose one of the following approaches:

Run docker compose up -d once. Compose starts pxc1, waits for the healthcheck to pass, then starts pxc2 and pxc3.
Start pxc1 first, then start the other nodes after pxc1 reports healthy. The following procedure shows this approach.
Start the bootstrap node to initialize the cluster:
```
docker compose up -d pxc1
```
Expected result: A line such as [+] Running 1/1 - Container pxc1 Started. The container pxc1 is running.

With Podman, use podman compose up -d pxc1 or podman-compose up -d pxc1.
Wait for pxc1 to report healthy before starting pxc2 and pxc3. The bootstrap node must accept connections, or the other nodes can fail to join. Use a wait loop:
```
until docker exec pxc1 mysqladmin ping -h localhost &>/dev/null; do
  echo "Waiting for pxc1..."
  sleep 2
done
```
Expected result: The loop prints Waiting for pxc1... until the node responds, then exits with no further output.
Start the remaining nodes:
```
docker compose up -d pxc2 pxc3
```
Expected result: Lines such as [+] Running 2/2 - Container pxc2 Started - Container pxc3 Started. Both containers are running.

With Podman, use podman compose up -d pxc2 pxc3 or podman-compose up -d pxc2 pxc3.

Validation¶

Check the status of each node from the host. Pass the root password according to your secret store:

Docker Secrets: -p$(cat mysql_root_password)
.env: -p${MYSQL_ROOT_PASSWORD}

With Podman, use podman exec instead of docker exec in every command.

Verify cluster size and primary status:

docker exec pxc1 mysql -uroot -p$(cat mysql_root_password) -e "SHOW STATUS LIKE 'wsrep_cluster_size';"
docker exec pxc2 mysql -uroot -p$(cat mysql_root_password) -e "SHOW STATUS LIKE 'wsrep_cluster_status';"

Expected result: wsrep_cluster_size returns a row with value 3. wsrep_cluster_status returns Primary. If the value is non-Primary, the node is isolated.

Verify additional cluster health indicators on any node (for example, pxc1):

docker exec pxc1 mysql -uroot -p$(cat mysql_root_password) -e "SHOW STATUS LIKE 'wsrep_ready';"
docker exec pxc1 mysql -uroot -p$(cat mysql_root_password) -e "SHOW STATUS LIKE 'wsrep_connected';"
docker exec pxc1 mysql -uroot -p$(cat mysql_root_password) -e "SHOW STATUS LIKE 'wsrep_local_state_comment';"

Expected result: wsrep_ready = ON, wsrep_connected = ON, wsrep_local_state_comment = Synced.

All three nodes are joined and synchronized when the listed health indicators show the expected values. To validate replication end-to-end, see Verify replication.

Backup¶

Back up data before major changes or shutdown. The following example dumps all databases from pxc1 to a file on the host. Use the root password from your secret file or .env:

docker exec pxc1 mysqldump -uroot -p$(cat mysql_root_password) --all-databases > backup.sql

Expected result: The file backup.sql is created in the current directory with SQL statements for all databases. If you use .env, substitute -p${MYSQL_ROOT_PASSWORD}. With Podman, use podman exec instead of docker exec.

Troubleshooting¶

View container logs for debugging:

docker compose logs -f pxc1
docker compose logs -f pxc2
docker compose logs -f pxc3

Each command streams that container’s logs (stdout and stderr). The -f option follows the log output. Omit -f for a one-off dump.

Expected result: Log lines from MySQL and PXC until you press Ctrl+C.

With Podman, use podman compose logs -f pxc1.

The following sections cover common issues.

Containers exit or fail to start¶

This issue typically appears at startup before the cluster forms:

Symptom: One or more containers exit immediately or fail to reach the healthy state.
Diagnosis: Check the container logs with docker compose logs pxc1 (and pxc2, pxc3).
Solution: Ensure the bootstrap node (pxc1) is healthy before starting pxc2 and pxc3.

Cluster size stays at 1¶

The bootstrap node started, but the join nodes failed to register:

Symptom: wsrep_cluster_size returns 1 instead of 3.
Diagnosis: The join nodes started before pxc1 became ready.
Solution: Start pxc1 first. Wait for the healthcheck to pass. Restart pxc2 and pxc3 after pxc1 is up.

Permission denied on certs or config¶

The container cannot read mounted host paths:

Symptom: The container fails to read certs/ or conf.d/.
Diagnosis: SELinux blocks the container from accessing host paths without relabeling.
Solution: Add :Z or :z to all volume mounts. See the SELinux note in Docker Compose setup and Appendix: Podman alternative.

Nodes cannot reach each other¶

Network configuration blocks communication between cluster members:

Symptom: Galera reports nodes as unreachable, or the cluster size remains 1.
Diagnosis: Network configuration prevents nodes from communicating.
Solution: On a single host, verify all containers are on the same network with docker network inspect pxc-cluster_pxcnet. On multiple hosts, see Multi-host deployment for firewall rules, name resolution, and NTP.

Cleanup and shutdown¶

To stop the cluster and remove containers (data in data-pxc1/, data-pxc2/, data-pxc3/ is preserved):

docker compose down

Expected result: Containers pxc1, pxc2, and pxc3 are stopped and removed. Project network is removed. Data in data-pxc1/, data-pxc2/, and data-pxc3/ remains on the host.

To stop and remove containers and volumes (the -v option deletes all database data):

docker compose down -v

Expected result: Containers and any named volumes are removed. Bind-mounted data directories (data-pxc1/, data-pxc2/, data-pxc3/) are not removed by this command.

Data loss

The Compose file in this guide uses bind mounts (./data-pxc1, ./data-pxc2, ./data-pxc3), not named volumes, so docker compose down -v removes only named volumes if present. To fully reset, remove the data directories manually after stopping the containers:

rm -rf data-pxc1 data-pxc2 data-pxc3

This operation deletes all database data permanently.

Appendix: Podman alternative¶

Podman runs the same container images as Docker and accepts the Compose file in this guide. Podman differs from Docker in the following defaults that affect this deployment:

Rootless containers run as a non-root user.
SELinux relabeling is enforced on bind mounts.
Networking uses netavark (or the older Container Network Interface (CNI)) instead of the Docker bridge driver.
Some Compose keys are honored by Docker Compose v2 but not by podman-compose (for example, deploy.resources.limits.memory).

This deployment is designed and primarily tested with Docker Compose. The same Compose file and workflow run with Podman after the adjustments described in the following sections. Use Podman 4.1 or later. The built-in podman compose subcommand requires 4.1 or newer. With older Podman, use the separate podman-compose tool. If you use Podman, verify that the cluster operates correctly before using it beyond testing or experimentation.

Use the following with Podman:

Prerequisites: Podman and Podman Compose. Install Podman Compose if you are not using built-in podman compose (for example, pip install podman-compose or your distribution’s package). For rootless Podman, ensure the user has enough resources (for example, sysctl user.max_user_namespaces and subuid/subgid ranges).
Commands: Replace docker compose with podman compose or podman-compose. Replace docker exec with podman exec. Examples:
- Start node 1: podman compose up -d pxc1 (or podman-compose up -d pxc1)
- Start other nodes: podman compose up -d pxc2 pxc3
- Validate: podman exec pxc1 mysql -uroot -p$(cat mysql_root_password) -e "SHOW STATUS LIKE 'wsrep_cluster_size';" (or -p${MYSQL_ROOT_PASSWORD} if using .env)
Directory structure: The same layout (pxc-cluster/ with certs/, conf.d/, and init/) works with Podman. Run podman compose from the project directory so the relative volume paths in the Compose file resolve correctly.
Compose file: The docker-compose.yml works as-is with Podman; the bridge network and volume mounts are supported. For rootless Podman, see the volume mount options in Handle rootless permissions for Podman.

Handle rootless permissions for Podman¶

In Docker, the daemon runs as root and can override file permissions. In Podman running as a normal user, the MySQL process inside the container runs as User Identifier (UID) 1001 or 999. That UID may lack permission to read host files. Ensure host files are readable by the container. On systems with SELinux, use the :Z or :z mount option to relabel the volume for the container. Use :Z for private relabeling and :z for shared relabeling. Use the following in your docker-compose.yml:

volumes:
  - ./certs:/etc/mysql/certs:ro,Z
  - ./conf.d:/etc/percona-xtradb-cluster.conf.d:ro,Z

Apply the same volume options to every service (pxc1, pxc2, pxc3). Without :Z or :z, the container may fail to read certs or config when running rootless on SELinux.

Compare `podman-compose` and `docker-compose`¶

The two tools differ in stability and behavior:

If you use podman-compose (the Python tool): podman-compose reads the directory structure and either the secret files or .env the same way as Docker.
If you use docker-compose with the Podman socket: This combination is often more stable for PXC. Set DOCKER_HOST to reference the Podman socket (for example, unix:///run/user/$(id -u)/podman/podman.sock) so docker compose connects to Podman.

Resolve Podman network issues¶

PXC uses specific ports for cluster communication (4567, 4568, 4444). Podman uses different networking (netavark or CNI). If nodes cannot find each other, try the following:

Keep using a named network in the Compose file (for example, pxcnet) so Podman can resolve container names (pxc1, pxc2, pxc3).
If problems persist, set network_mode: slirp4netns on the services, or run the stack rootful to use the default bridge.