Multi-data center setup for disaster recovery

Disaster can happen at any moment. To keep your services running smoothly, you can set up two Percona XtraDB Clusters in different locations (called “sites”). You then configure them to replicate data between each other. This makes sure both clusters have the same data and stay in sync. One site works as the primary site, and the other is a replica. It is usually in a standby mode.

If the primary site goes down, you need a way to move the workload to the backup site so that users won’t notice anything.

Once the primary site is fixed, you can move the services back to it.

This guide explains how to set up a disaster recovery system and transfer workloads between sites when something goes wrong.

If you’re dealing with a single node or the primary Pod going down inside one running cluster, see In-cluster failover instead.

Assumptions

• This guide is about two Percona XtraDB Clusters (PXC) set up with the Operator in Kubernetes. The clusters are in two separate sites which represent different Kubernetes environments.

  To differentiate the clusters, let’s name them:

  • cluster1 is the PXC on the primary site
  • cluster2 is the PXC on the replica site

• In this setup we deploy clusters in different namespaces in the same Kubernetes cluster. Each cluster requires its own Operator Deployment. By default, the Operator only watches its own namespace (WATCH_NAMESPACE), so a single Operator Deployment can’t manage both sites.

• The primary and replica sites must be identical. The easiest way to achieve this is to make a backup on the primary site and restore it on the replica.

• We assume your applications are already set up to automatically switch to the replica site B if the primary site goes down. Setting this up is not covered in this guide.

---

Last update: September 30, 2026
Created: May 19, 2025