Skip to content

Upgrade Percona XtraDB Cluster

The following documents contain details about relevant changes in the 8.0 series of MySQL and Percona Server for MySQL. Make sure you deal with any incompatible features and variables mentioned in these documents when upgrading to Percona XtraDB Cluster 8.0.

Important changes in Percona XtraDB Cluster 8.0

Traffic encryption is enabled by default

The pxc_encrypt_cluster_traffic variable, which enables traffic encryption, is set to ON by default in Percona XtraDB Cluster 8.0.

Unless you configure a node accordingly (each node in your cluster must use the same SSL certificates) or try to join a cluster running PXC 5.7 which unencrypted cluster traffic, the node will not be able to join resulting in an error.

The error message 
... [ERROR] ... [Galera] handshake with remote endpoint ...
This error is often caused by SSL issues. ...

See also

sections Encrypting PXC Traffic, Configuring Nodes for Write-Set Replication

Shut down the cluster and upgrade each node to PXC 8.0. It is important that you make backups before attempting an upgrade.

PXC strict mode is enabled by default

Percona XtraDB Cluster in 8.0 runs with PXC Strict Mode enabled by default. This will deny any unsupported operations and may halt the server if a strict mode validation fails. It is recommended to first start the node with the pxc_strict_mode variable set to PERMISSIVE in the MySQL configuration file.

All configuration settings are stored in the default MySQL configuration file:

  • Path on Debian and Ubuntu: /etc/mysql/mysql.conf.d/mysqld.cnf

  • Path on Red Hat and CentOS: /etc/my.cnf

After you check the log for any tech preview features or unsupported features and you have fixed any of the encountered incompatibilities, set the variable back to ENFORCING at run time:

mysql> SET pxc_strict_mode=ENFORCING;

Restarting the node with the updated configuration file also sets variable to ENFORCING.

The configuration file layout has changed in PXC 8.0

All configuration settings are stored in the default MySQL configuration file:

  • Path on Debian and Ubuntu: /etc/mysql/mysql.conf.d/mysqld.cnf

  • Path on Red Hat and CentOS: /etc/my.cnf

Before you start the upgrade, move your custom settings from /etc/mysql/percona-xtradb-cluster.conf.d/wsrep.cnf (on Debian and Ubuntu) or from /etc/percona-xtradb-cluster.conf.d/wsrep.cnf (on Red Hat and CentOS) to the new location accordingly.

Note

If you have moved your my.cnf file to a different location and added a symlink to /etc/my.cnf, the RPM package manager, when upgrading, can delete the symlink and put a default my.cnf file in /etc/.

caching_sha2_password is the default authentication plugin

In Percona XtraDB Cluster 8.0, the default authentication plugin is caching_sha2_password. The ProxySQL option –syncusers will not work if the Percona XtraDB Cluster user is created using caching_sha2_password. Use the mysql_native_password authentication plugin in these cases.

Be sure you are running on the latest 5.7 version before you upgrade to 8.0.

mysql_upgrade is part of SST

mysql_upgrade is now run automatically as part of SST. You do not have to run it manually when upgrading your system from an older version.

Major upgrade scenarios

Upgrading PXC from 5.7 to 8.0 may have slightly different strategies depending on the configuration and workload on your PXC cluster.

Note that the new default value of pxc-encrypt-cluster-traffic (set to ON versus OFF in PXC 5.7) requires additional care. You cannot join a 5.7 node to a PXC 8.0 cluster unless the node has traffic encryption enabled as the cluster may not have some nodes with traffic encryption enabled and some nodes with traffic encryption disabled. For more information, see Traffic encryption is enabled by default.

Scenario: No active parallel workload or with read-only workload

If there is no active parallel workload or the cluster has read-only workload while upgrading the nodes, complete the following procedure for each node in the cluster:

  1. Shutdown one of the node 5.7 cluster nodes.

  2. Remove 5.7 PXC packages without removing the data-directory.

  3. Install PXC 8.0 packages.

  4. Restart the mysqld service.

Important

Before upgrading, make sure your application can work with a reduced cluster size. If the cluster operates with an even number of nodes, the cluster may have split-brain.

This upgrade flow auto-detects the presence of the 5.7 data directory and trigger the upgrade as part of the node bootup process. The data directory is upgraded to be compatible with PXC 8.0. Then the node joins the cluster and enters synced state. The 3-node cluster is restored with 2 nodes running PXC 5.7 and 1 node running PXC 8.0.

Note

Since SST is not involved, SST based auto-upgrade flow is not started.

PXC 8.0 uses Galera 4 while PXC 5.7 uses Galera-3. The cluster will continue to use the protocol version 3 used in Galera 3 effectively limiting some of the functionality. With all nodes upgraded to version 8.0, protocol version 4 is applied.

Tip

The protocol version is stored in the protocol_version column of the wsrep_cluster table.

mysql> USE mysql;

mysql> SELECT protocol_version from wsrep_cluster;

The example of the output is the following:

+------------------+
| protocol_version |
+------------------+
|                4 |
+------------------+
1 row in set (0.00 sec)

As soon as the last 5.7 node shuts down, the configuration of the remaining two nodes is updated to use protocol version 4. A new upgraded node will then join using protocol version 4 and the whole cluster will maintain protocol version 4 enabling the support for additional Galera 4 facilities.

It may take longer to join the last upgraded node since it will invite IST to obtain the configuration changes.

Note

Starting from Galera 4, the configuration changes are cached to gcache and the configuration changes are donated as part of IST or SST to help build the certification queue on the JOINING node. As other nodes (say n2 and n3), already using protocol version 4, donate the configuration changes when the JOINER node is booted.

The situation was different for the previous and penultimate nodes since the donation of the configuration changes is not supported by protocol version 3 that they used.

With IST involved on joining the last node, the smart IST flow is triggered to take care of the upgrade even before MySQL starts to look at the data directory.

Important

It is not recommended to restart the last node without upgrading it.

Scenario: Upgrade from PXC 5.6 to PXC 8.0

First, upgrade PXC from 5.6 to the latest version of PXC 5.7. Then proceed with the upgrade using the procedure described in Scenario: No active parallel workload or with read-only workload.

Minor upgrade

To upgrade the cluster, follow these steps for each node:

  1. Make sure that all nodes are synchronized.

  2. Stop the mysql service:

    $ sudo service mysql stop

  3. Upgrade Percona XtraDB Cluster and Percona XtraBackup packages. For more information, see Installing Percona XtraDB Cluster.

  4. Back up grastate.dat, so that you can restore it if it is corrupted or zeroed out due to network issue.

  5. Now, start the cluster node with 8.0 packages installed, PXC will upgrade the data directory as needed - either as part of the startup process or a state transfer (IST/SST).

    In most cases, starting the mysql service should run the node with your previous configuration. For more information, see Adding Nodes to Cluster.

    $ sudo service mysql start

    Note

    On CentOS, the /etc/my.cnf configuration file is renamed to my.cnf.rpmsave. Make sure to rename it back before joining the upgraded node back to the cluster.

    PXC Strict Mode is enabled by default, which may result in denying any unsupported operations and may halt the server. For more information, see pxc-strict-mode is enabled by default.

    pxc-encrypt-cluster-traffic is enabled by default. You need to configure each node accordingly and avoid joining a cluster with unencrypted cluster traffic. For more information, see Traffic encryption is enabled by default.

  6. Repeat this procedure for the next node in the cluster until you upgrade all nodes.

Get expert help

If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.

Community Forum Get a Percona Expert

Last update: 2023-07-21