Skip to content
Percona Software for MySQL Documentation
Rate this page
☆ ☆ ☆ ☆ ☆
Thanks for your feedback
Thank you! The feedback has been submitted.

For help, click the link below to get free database assistance or contact our experts for personalized support.

Get help from Percona

Percona Server for MySQL

Documentation

8.4.11-11 (August 20, 2026)

Percona Server for MySQL 8.4 - Documentation

This documentation is for the latest release: Percona Server for MySQL 8.4.11-11 (Release Notes).

Percona Server for MySQL is a freely available, fully compatible, enhanced, and open source drop-in replacement for any MySQL database. It provides superior and optimized performance, greater scalability and availability, enhanced backups, and increased visibility and instrumentation.

Thousands of enterprises trust Percona Server for MySQL to provide better performance and concurrency for their most demanding workloads.

New to Percona?

Start here to choose your installation path.

  1. Are you building a High Availability (HA) Cluster?
  2. Yes → Use Percona Distribution for MySQL. For a multi-node HA cluster, Percona XtraDB Cluster (PXC) is the clustered database; PDPS includes the server or PXC plus tested versions of HAProxy, ProxySQL, Orchestrator, and XtraBackup.

  3. No → Go to step 2.

  4. Do you just need a drop-in replacement for MySQL?

  5. Yes → Use Percona Server for MySQL. It is lightweight and focuses on the core engine performance.

  6. Is your operating system (OS) very new?

  7. Check → Verify support on the Percona Software and Platform Lifecycle page.

  8. Are you deploying on Kubernetes (a platform for running containerized applications)?

  9. Yes → Use Percona Operator for MySQL (for Percona Server for MySQL) or Percona Operator for MySQL based on Percona XtraDB Cluster (PXC) (for a clustered, high-availability setup). Operators automate deployment, scaling, backups, and failover on Kubernetes.
  10. No → Not sure? Learn more about the differences to compare options.

Learn more about the differences →

For Monitoring and Management

Percona Monitoring and Management (PMM) monitors and provides actionable performance data for MySQL variants, including Percona Server for MySQL, Percona XtraDB Cluster, Oracle MySQL Community Edition, Oracle MySQL Enterprise Edition, and MariaDB. PMM captures metrics and data for the InnoDB, XtraDB, and MyRocks storage engines, and has specialized dashboards for specific engine details.

Get started with PMM .

Review Get help from Percona for ways that we can work with you.

Quickstart guide

Get started quickly with our Quickstart guide.

Quickstart guide

Installation guides

Find the best installation solution with our step-by-step installation instructions.

Installation instructions

Upgrade instructions

Upgrade your Percona Server for MySQL installation with our comprehensive upgrade guides.

Upgrade instructions

Audit Log Filter component

Learn about the Audit Log Filter component that allows you to monitor, log, and block a connection or query actively executed on the selected server.

Audit Log Filter

Get help from Percona

Our documentation is packed with information, but it can’t cover everything you need to know about Percona Server for MySQL. It also won’t cover every scenario you might come across. New to Percona? Start with choosing your path. Don’t be afraid to try things out and ask questions when you get stuck.

Percona’s Community Forum

Be a part of a space where you can tap into a wealth of knowledge from other database enthusiasts and experts who work with Percona’s software every day. While our service is entirely free, keep in mind that response times can vary depending on the complexity of the question. You are engaging with people who genuinely love solving database challenges.

We recommend visiting our Community Forum . It’s an excellent place for discussions, technical insights, and support around Percona database software. If you’re new and feeling a bit unsure, our FAQ and Guide for New Users ease you in.

If you have thoughts, feedback, or ideas, the community team would like to hear from you at Any ideas on how to make the forum better? . We’re always excited to connect and improve everyone’s experience.

Percona experts

Percona experts bring years of experience in tackling tough database performance issues and design challenges.

We understand your challenges when managing complex database environments. That’s why we offer various services to help you simplify your operations and achieve your goals.

Service Description
24/7 Expert Support Our dedicated team of database experts is available 24/7 to assist you with any database issues. We provide flexible support plans tailored to your specific needs.
Hands-On Database Management Our managed services team can take over the day-to-day management of your database infrastructure, freeing up your time to focus on other priorities.
Expert Consulting Our experienced consultants provide guidance on database topics like architecture design, migration planning, performance optimization, and security best practices.
Comprehensive Training Our training programs help your team develop skills to manage databases effectively, offering virtual and in-person courses.

We’re here to help you every step of the way. Whether you need a quick fix or a long-term partnership, we’re ready to provide our expertise and support.

What to do next

Choose your path: Percona Server for MySQL, Percona Distribution for MySQL, or Percona Operators

When setting up Percona repositories (online locations where installable software packages are stored), you must choose between two primary paths: Percona Server for MySQL (PS) or Percona Distribution for MySQL (PDPS). Both ship the same Percona Server for MySQL 8.4 database server; PDPS adds a curated set of surrounding components (backup, proxy, orchestration) tested together. They are designed for different operational scopes.

The labels ps-8.4 and pdps-8.4 are repository identifiers (short names for the repository you enable). You use them when enabling the Percona repository—for example, with the percona-release setup tool or in the configuration for your system’s package manager (the tool that installs software: apt on Debian/Ubuntu, yum or dnf on Red Hat–style systems). The number (8.4) matches the MySQL major version.

To read more about Percona Distribution for MySQL, see the Percona Distribution for MySQL 8.4 documentation.

Check Platform Support

Before choosing a repository, verify that your operating system (OS) version is supported for that specific product. Review the Percona Software and Platform Lifecycle page to confirm support for your platform and version. If your OS is not listed (for example, a very new Linux distribution such as a recently released Ubuntu or Debian), use a supported platform or wait until support is added.

Comparison Overview

Feature Percona Server for MySQL (ps-8.4) Percona Distribution for MySQL (pdps-8.4) Percona Operators
Primary Goal A performance-enhanced, drop-in replacement for MySQL Community Edition. A curated collection of components tested together as a complete enterprise stack. Kubernetes-native deployment and lifecycle management for Percona Server for MySQL or Percona XtraDB Cluster (PXC).
Included Components Database server, client, and essential plugins. Database server, Percona XtraBackup (backup), HAProxy and ProxySQL (traffic and query routing), and Orchestrator (replication management). Operator plus database pods; backup, proxy, and orchestration vary by Operator (PS or PXC).
Release Cycle Follows the MySQL Community release cadence. Follows a coordinated release cycle where all bundled components are validated for inter-compatibility. Follows Operator release cadence; deploys supported PS or PXC versions.
Ideal For Standalone instances or simple primary/replica setups (one server accepts writes; others hold read-only copies). High Availability (HA) clusters—systems that keep running even if one server or component fails—and mission-critical enterprise environments. Kubernetes and cloud-native environments; automated failover, scaling, and backups.

Percona XtraDB Cluster (PXC) is a high-availability clustered database based on MySQL; multiple nodes stay in sync so the cluster keeps running if one node fails. It is a different product from Percona Server for MySQL (which is a single-server database).

Which one should I install?

Need High Availability (HA) or a single supported stack (server, proxy, backup)? Choose Percona Distribution for MySQL (PDPS). Deploying on Kubernetes? See Percona Operators below. Otherwise, choose Percona Server for MySQL (PS).

Percona Server for MySQL (ps-8.4)

Choose this if any of the following conditions apply:

  • Wanting a minimal footprint focused solely on the database engine.

  • Managing your own infrastructure components (backups, proxies, or orchestration) independently.

  • Requiring the latest performance patches and features available in Percona Server but not needing a bundled ecosystem.

Percona Distribution for MySQL (pdps-8.4)

Choose this if any of the following conditions apply:

  • Deploying a High Availability (HA) environment (for example, using Percona XtraDB Cluster).

  • Wanting a curated, version-aligned stack where the server, proxy, and backup tools are validated for compatibility as a single unit.

  • Preferring a single repository entry-point that provides all the tools required for a full production lifecycle (Server + Backups + Management).

If you need only some components (for example, XtraBackup or ProxySQL) with Percona Server for MySQL, you can install those separately; PDPS is for users who want one supported, integrated stack.

Deploying on Kubernetes?

If you run your workloads on Kubernetes (a platform for running containerized applications), use Percona Operators instead of the repository-based installation. Operators automate deployment, scaling, backups, and failover on Kubernetes.

Next steps

Percona Server for MySQL

Percona Distribution for MySQL

Release notes

Percona Server for MySQL 8.4 release notes index

Percona Server for MySQL 8.4.11-11 (2026-08-20)

Get started with Quickstart Guide for Percona Server for MySQL.

Percona Server for MySQL 8.4.11-11 includes all the features and bug fixes available in the MySQL 8.4.11 Community Edition in addition to enterprise-grade features developed by Percona.

Release highlights

Percona Server for MySQL 8.4.11-11

Percona Server for MySQL 8.4.11-11 introduces the following new features and improvements:

  • Adds OpenID Connect (OIDC) authentication and authorization. Users can authenticate with Identity tokens issued by external Identity Providers (IDPs) instead of MySQL passwords. The OIDC plugin supports multiple IDPs, maps IDP groups to MySQL roles, supports proxy users based on group membership, and refreshes JSON Web Key Set (JWKS) signing keys at runtime. Find more information in OpenID Connect authentication and in Get started with OpenID Connect authentication.

  • Improves InnoDB performance for workloads limited by Least Recently Used (LRU) flush speed. The improvements reduce LRU list mutex contention, restore dedicated LRU manager threads, optimize LRU scanning, and allow single-page flushing to proceed while an LRU batch flush is running.

  • Improves InnoDB buffer pool initialization on NUMA-enabled systems by using multi-threaded memory allocation. The improvement reduces initialization time and can shorten server startup for instances with large buffer pools.

  • Improves InnoDB performance for highly concurrent range-select workloads by reducing BUF_BLOCK_MUTEX contention when multiple threads access the same buffer pool page. The improvement increases throughput for read workloads that repeatedly access the same hot pages.

  • Adds timestamps to the Group Communication System (GCS) debug trace file. The timestamps make large trace files easier to analyze and help correlate Group Replication communication events with other server activity.

MySQL 8.4.11

Improvements and bug fixes introduced by Oracle for MySQL 8.4.11 and included in Percona Server for MySQL are the following:

  • Fixed an issue that could cause an InnoDB deadlock during B-tree page merges while concurrent searches were running. (Bug #39129182)

  • Fixed an issue where stricter InnoDB row-size validation could reject or generate warnings for table definitions accepted by earlier MySQL LTS releases. (Bug #120323, Bug #39249507)

  • Fixed an issue that could produce incorrect values when adding an AUTO_INCREMENT column to an existing InnoDB table. (Bug #115136, Bug #37105825)

  • Fixed an issue that could return incorrect results when a scalar subquery and its outer query referenced the same Common Table Expression (CTE). (Bug #120403, Bug #39321676)

  • Fixed an issue that could cause some LEFT JOIN queries with OR conditions to perform full table scans instead of index range scans. (Bug #113288, Bug #36061036)

  • Fixed an issue that could prevent the server from starting on Oracle Linux 9 or Red Hat Enterprise Linux 9 when innodb_redo_log_encrypt=ON was configured. (Bug #39181231)

  • Fixed an issue that caused memory leaks in the statement_digest() and statement_digest_text() functions. (Bug #104115, Bug #33073320)

Find the complete list of bug fixes and changes in the MySQL 8.4.11 release notes .

New features

  • PS-10999: Adds OpenID Connect (OIDC) authentication and authorization. Users can authenticate with Identity tokens issued by external Identity Providers (IDPs) instead of MySQL passwords. The OIDC plugin supports multiple IDPs, maps IDP groups to MySQL roles, supports proxy users based on group membership, and refreshes JSON Web Key Set (JWKS) signing keys at runtime. Find more information in OpenID Connect authentication and in Get started with OpenID Connect authentication.

Improvements

  • PS-11444: Improves InnoDB performance for I/O-bound workloads by reducing contention on the buffer pool Least Recently Used (LRU) list mutex during physical page reads.

  • PS-11445: Improves InnoDB Least Recently Used (LRU) flushing performance by restoring dedicated LRU manager threads for each buffer pool. The LRU manager threads handle LRU flush batches, reducing the work performed by page cleaner threads.

  • PS-11446: Improves InnoDB Least Recently Used (LRU) flushing performance by correcting the LRU scan behavior so that scans continue through the old sublist instead of repeatedly restarting from the end of the LRU list.

  • PS-11447: Improves InnoDB flushing performance by allowing a single-page flush to run while a Least Recently Used (LRU) batch flush is in progress. The number of concurrent single-page flushes is limited to one per buffer pool instance.

  • PS-10595: Improves InnoDB buffer pool initialization on NUMA-enabled systems by using multi-threaded memory allocation, reducing startup time for instances with large buffer pools.

  • PS-11120: Improves InnoDB performance for highly concurrent range-select workloads by reducing BUF_BLOCK_MUTEX contention when multiple threads access the same buffer pool page.

  • PS-11216: Improves Group Replication debugging by adding timestamps to the Group Communication System (GCS) debug trace file, making it easier to correlate and analyze group communication events.

Bug Fixes

  • PS-11135: Fixed an issue where the Audit Log Filter returned an incorrect error message for filters with invalid JSON format, incorrectly suggesting that audit_log_filter.event_mode=FULL could resolve the problem. The error message now reports the actual filter format issue.

  • PS-11143: Fixed an issue where using the Thread Pool with Performance Schema could cause the server to exit unexpectedly with a SIGSEGV signal.

  • PS-11182: Fixed an issue where audit_log_read() required the current read sequence to be explicitly closed before initializing a new read sequence.

  • PS-11186: Fixed an issue where the max_array_length parameter in audit_log_read() did not limit the number of returned events when using FULL or REDUCED audit log event mode.

  • PS-11202: Fixed an issue where reading corrupted page-tracking files could cause the server to exit. The server now reports a warning and continues without page-tracking information.

  • PS-11204: Fixed an issue where granting the SET_USER_ID privilege on a Percona Server for MySQL 8.0 source could cause replication to a Percona Server for MySQL 8.4 replica to stop.

  • PS-11209: Fixed an issue where Common Table Expression (CTE) queries could fail with error 1146 (Table doesn't exist) when executed with high concurrency.

  • PS-11217: Fixed an issue that prevented the Group Communication System (GCS) debug trace file from being rotated or archived without restarting Group Replication.

  • PS-11273: Fixed an issue where the server could exit with signal SIGABRT during startup when authentication_policy was set to an invalid value.

  • PS-11297: Fixed an InnoDB performance regression caused by unnecessary record offset validation in non-debug builds.

Additional resources

Percona Server for MySQL 8.4.10-10 (2026-06-30)

Percona Server for MySQL 8.4.9-9 was not released; 8.4.10-10 is the next build in this series. This release includes enhancements and bug fixes from MySQL 8.4.9 and MySQL 8.4.10.

Get started with Quickstart Guide for Percona Server for MySQL.

Percona Server for MySQL 8.4.10-10 includes all the features and bug fixes available in the MySQL 8.4.10 Community Edition in addition to enterprise-grade features developed by Percona.

Release highlights

Percona Server for MySQL 8.4.10-10

Percona Server for MySQL 8.4.10-10 introduces the following new features and improvements:

  • Integrates the new Key Management Interoperability Protocol (KMIP) library into the key management component.

  • Enhances Thread Pool statistics by adding new status variables for queue monitoring and wait time analysis. Reports the number of requests waiting in normal and high-priority queues, the number of requests not yet entered into a queue, and aggregate queue wait time statistics, including average, minimum, maximum, and standard deviation wait times.

  • Adds JSONL (JSON Lines) output format for Audit Log Filter.

  • Increases the verbosity of the data dictionary upgrade process, making it easier to diagnose issues that occur during upgrade.

  • Logs SQL statements for the Audit Log Filter table_access class and the read and insert subclasses.

  • Flushes the audit log buffer on server shutdown when the ASYNCHRONOUS logging strategy is in use, preventing the loss of buffered events.

  • Reduces memory pressure in the Audit Log Filter component caused by VFS buffering.

  • Aligns the audit_log_filter.format=NEW output between the 8.0 plugin and the 8.4 component.

  • Optimizes performance for mem_root_deque.

MySQL 8.4.10

Improvements and bug fixes introduced by Oracle for MySQL 8.4.10 and included in Percona Server for MySQL are the following:

  • Connection attribute parsing could read a length-encoded size field before verifying that the complete field was present in the packet, leading to an out-of-bounds read. A size check is now performed before reading the field. (Bug #39116965)

  • A regression in row size estimation for ROW_FORMAT=COMPRESSED tables could cause CREATE TABLE to fail with Row size too large for tables that were accepted in earlier releases. (Bug #39129182, Bug #120323)

  • Under certain circumstances, when calculating the maximum possible index record size, an assertion failure could occur. (Bug #85060, Bug #25579578)

Find the complete list of bug fixes and changes in the MySQL 8.4.10 release notes .

Security updates

This release addresses the following security vulnerabilities:

  • CVE-2026-46850: A vulnerability in MySQL Shell (Shell for VS Code) allows a low-privileged attacker with network access via HTTP to compromise MySQL Shell, with potential scope change impact on additional products (CVSS 3.1 Base Score 9.9).

  • CVE-2026-46860: A vulnerability in MySQL Router allows an unauthenticated attacker with network access via HTTP to compromise MySQL Router (CVSS 3.1 Base Score 9.8).

  • CVE-2026-46861: A vulnerability in MySQL NDB Cluster (NDB Operator) allows a low-privileged attacker with network access via HTTP to access or modify critical data, with potential scope change impact (CVSS 3.1 Base Score 9.6).

  • CVE-2026-46862: A vulnerability in MySQL Router allows an unauthenticated attacker with network access via TLS to cause a hang or repeatable unexpected exit of MySQL Router (CVSS 3.1 Base Score 7.5).

  • CVE-2026-46863: A vulnerability in MySQL Server connection handling allows an unauthenticated attacker with network access via multiple protocols to cause a hang or repeatable unexpected exit of the server (CVSS 3.1 Base Score 7.5).

  • CVE-2026-46869: A vulnerability in MySQL Shell (Dump and Load) allows an unauthenticated attacker with network access to access critical data when user interaction is required (CVSS 3.1 Base Score 6.5).

  • CVE-2026-46870: A vulnerability in MySQL Shell (Shell for VS Code) allows a low-privileged attacker with network access to compromise MySQL Shell, with potential scope change impact (CVSS 3.1 Base Score 8.5).

  • CVE-2026-46871: A vulnerability in MySQL Shell (Shell for VS Code) allows a low-privileged attacker with network access via multiple protocols to access critical data (CVSS 3.1 Base Score 6.5).

New features

  • PS-10070: Integrated the new Key Management Interoperability Protocol (KMIP) library into the key management component.

  • PS-10083: Enhances Thread Pool statistics by adding new status variables for queue monitoring and wait time analysis. The new metrics report the number of requests waiting in normal and high-priority queues, the number of requests not yet entered into a queue, and aggregate queue wait time statistics, including average, minimum, maximum, and standard deviation wait times. These additions provide greater visibility into thread pool workload and queueing behavior.

  • PS-10989: Adds JSONL (JSON Lines) output format for Audit Log Filter.

Improvements

  • PS-8867: Increases the verbosity of the data dictionary upgrade process, making it easier to diagnose issues that occur during upgrade.

  • PS-9774: The Audit Log Filter table_access class and read and insert subclasses now log statements in the audit log.

  • PS-10312: The audit log buffer is now flushed on server shutdown when the ASYNCHRONOUS logging strategy is in use, preventing the loss of buffered events.

  • PS-10331: Reduces memory pressure in the Audit Log Filter component caused by VFS buffering.

  • PS-10339: Resolves an inconsistency in audit_log_filter.format=NEW between the 8.0 plugin and the 8.4 component.

  • PS-11161: Optimizes performance for mem_root_deque.

Bug Fixes

  • PS-8670: InnoDB could exit unexpectedly on startup while resuming an interrupted ALTER TABLESPACE encryption operation because progress was persisted before affected pages were flushed to disk in the target encryption state.

  • PS-9773: The Audit Log Filter audit_log_read() function always showed "null" instead of the bookmarked and current information.

  • PS-9791: The audit_log_filter component generated audit records that did not include complete user, host, and query information in a single event, making the logs difficult to analyze and process.

  • PS-10228: An audit log filter defined with no value did not log any events.

  • PS-10338: audit_log_filter_set_filter() returned OK for filters that referenced unknown field names instead of reporting the error.

  • PS-10348: The Audit Log Filter did not support filtering by integer fields.

  • PS-10351: Reduced the number of events emitted by the Audit Log Filter to improve performance.

  • PS-10378: With the MeCab plugin, a boolean-mode full-text query that used LIMIT still computed ranking scores for all matches, preventing the limit-based optimization from being applied.

  • PS-10435: The general_query.str event field in audit log filters could not be replaced.

  • PS-10448: A prepared INSERT statement failed to write rows across partitions.

  • PS-10481: Fixed the issue where the range optimizer could fall back to a full table scan instead of using an index range scan for IN() predicates on non-binary UTF-8 collations when oversized values were present. This behavior could negatively impact query performance. The optimizer now correctly handles oversized values and continues to use index range scans when appropriate.

  • PS-10589: Added support for building Percona Server with GCC 15, Clang 21, and Clang 22. Fixed compilation issues identified by newer Clang compiler versions.

  • PS-10593: The audit log plugin could exit unexpectedly with a segmentation fault when configured with the PERFORMANCE logging strategy and audit buffering enabled.

  • PS-10853: The audit_log_filter plugin exited unexpectedly because throwing calls were made from filesystem functions declared noexcept.

  • PS-10872: Corrected event subclass naming in the Audit Log Filter and aligned its JSON output with the upstream format.

  • PS-10873: The legacy audit log MTR test audit_log_charset failed in Percona Server for MySQL 8.0.

  • PS-10951: audit_log_filter_set_user() incorrectly updated active sessions instead of only new connections.

  • PS-10987: Improved Audit Log Filter message and connection events by suppressing nested general/status records in audit_log_filter.event_mode=REDUCED.

  • PS-11014: Audit Log Filter replace-field rules returned Parse 'function' arguments list must be an array when query_digest() was specified as a plain JSON string.

  • PS-11073: Tables with large rows that were originally created on earlier MySQL versions could not be created on MySQL 8.4.9.

  • PS-11241: The debug make_page_dirty command logged generic byte updates for compressed index pages, violating redo parser invariants during crash recovery.

  • PS-11242: Preallocated transaction IDs used by clone consistent snapshot could break the descending order of rw_trx_list and trigger a debug assertion during commit.

  • PS-11243: Crash recovery could fail while replaying an unfinished tablespace encryption operation when Global Transaction Identifier (GTID) mode and binary logging were enabled.

Builds and packaging

Percona Server for MySQL 8.4.10-10 adds support for Ubuntu 26.04.

  • Percona Server for MySQL releases include a mixture of PGO and non-PGO builds. Where Profile-Guided Optimization (PGO) is enabled, the compiler uses runtime profiling data from representative workloads to guide optimization, which can improve throughput and reduce latency compared with non-PGO builds.

  • See Profile-Guided Optimization (PGO) and non-PGO builds for benefits, considerations, and which build you receive for your platform.

Additional resources

Percona Server for MySQL 8.4.8-8 (2026-03-12)

Get started with Quickstart Guide for Percona Server for MySQL.

Percona Server for MySQL 8.4.8-8 includes all the features and bug fixes available in the MySQL 8.4.8 Community Edition in addition to enterprise-grade features developed by Percona.

Release highlights

MySQL 8.4.8

Improvements and bug fixes introduced by Oracle for MySQL 8.4.8 and included in Percona Server for MySQL are the following:

  • The warning associated with redo logging being disabled is no longer present, as the underlying condition that triggered the warning has been eliminated. (Bug #37645185)

  • A problem affecting the handling of large insert operations has been corrected, improving stability during bulk data loads. (Bug #38208188)

  • An error that could arise when running certain SQL statements has been resolved. (Bug #38573285)

  • Issues encountered when generating table definitions via SHOW CREATE TABLE have been fixed. (Bug #38448700)

  • Bug #38298692 was addressed as part of the same fix set as Bug #38448700, resolving related inconsistencies in table metadata handling. (Bug #38298692)

  • Performance regressions affecting queries that rely on regular expression matching have been corrected. (Bug #114056, Bug #36326728)

  • The bundled OpenSSL dependency has been updated, addressing the issue tracked under Bug #38632932. (Bug #38632932)

  • A concurrency flaw in InnoDB that could occur when executing SQL through the que_eval_sql interface has been removed. (Bug #118705, Bug #38310595)

  • A timing issue that allowed binary logs to be removed before persisted expiration settings were fully applied has been fixed. (Bug #38554467)

  • A fault affecting clustered environments, where multiple instances could lose connectivity under specific conditions, has been corrected. (Bug #38380392)

  • Several defects that prevented connections from closing properly when using the Thread Pool have been resolved. (Bug #38170188, Bug #36782728, Bug #38549372)

  • An issue that caused gaps in GTID sequences when the replica_skip_errors option was enabled has been fixed. (Bug #28590993)

Find the complete list of bug fixes and changes in the MySQL 8.4.8 release notes .

Bug Fixes

  • PS-9828: The Audit Log Filter could crash when audit_log_filter.file pointed to a non-existent directory.

  • PS-10229: The audit_log_filter_remove_filter failed after upgrading from Percona Server for MySQL 8.0 to 8.4.6 due to an index being renamed from filter_name to filtername. The issue resulted in the error: ERROR: Failed to remove filter from users table.

  • PS-10232: MySQL 8.0.40 and later could be about 2–3 times slower than 8.0.39 in a particular index scan scenario.

  • PS-10260: Expanded Fast Index Creation could fail with partitioned tables.

  • PS-10287: A table definition mismatch between MyRocks and the data dictionary after a crash could cause a crash or assertion failures.

  • PS-10332: MySQL terminated with signal SIGABRT when an invalid value was specified for audit-log-filter.compression.

  • PS-10345: The Audit Log Filter could crash when the definition passed to audit_log_filter_set_filter() contained invalid data replacement fields.

  • PS-10347: The AuditJsonHandler used by the audit_log_read UDF could return malformed JSON.

  • PS-10383: The MeCab plugin did not enforce phrase adjacency for quoted phrase search in BOOLEAN MODE.

  • PS-10387: The audit_log_read() function ignored max_array_length and had pagination issues.

  • PS-10446: Back-ported the fix for Bug #118512 (missing semi-join condition causes incorrect result) to 8.4.

  • PS-10447: Back-ported the fix for Bug #119606 (assertion failure in item_func.cc:3937) to 8.4.

  • PS-10545: MySQL Test Run (MTR) --only-big-test incorrectly skipped tests that included both big_test.inc and no_valgrind_without_big.inc.

  • PS-10591: On systems running OpenSSL 3.4 or later (for example, Debian Trixie and Oracle Linux 10), the use of SHAKE128 and SHAKE256 resulted in an assertion failure. Because SHAKE algorithms are extendable-output functions (XOFs), EVP_MD_CTX_size() returned 0, which triggered assert(digest_size > 0) in finalize(). The implementation was updated to properly handle XOF algorithms and to use EVP_DigestFinalXOF() instead of EVP_DigestFinal_ex() for finalizing XOF digests.

Additional resources

Percona Server for MySQL 8.4.7-7 (2025-12-22)

Get started with Quickstart Guide for Percona Server for MySQL.

Percona Server for MySQL 8.4.7-7 includes all the features and bug fixes available in the MySQL 8.4.7 Community Edition in addition to enterprise-grade features developed by Percona.

Release highlights

Percona Server for MySQL 8.4.7-7

  • Announcement – Discontinuation of the Percona PRO Program

Percona has consolidated its build offerings. The Pro builds are no longer required. All features from the Percona Server for MySQL 8.0.x Pro build series have been merged into the main open source Community release. FIPS-capable binaries are included in all builds starting with this release. The FIPS mode remains off by default and must be enabled via OS/OpenSSL configuration.

  • The audit log plugin has been reintroduced in Percona Server for MySQL 8.4. However, it is already marked as deprecated and is planned for removal in a future release. This deprecation is due to the availability of the audit log filter component, which is the recommended replacement. Users should migrate to this component, which provides equivalent functionality with enhanced flexibility, performance, and filtering capabilities, ensuring continued support for auditing and compliance requirements.

MySQL 8.4.7

Improvements and bug fixes introduced by Oracle for MySQL 8.4.7 and included in Percona Server for MySQL are the following:

  • Fixed an issue where parallel scan thread creation could fail, causing assertion failures when falling back to single-thread mode. (Bug #38325137)

  • Fixed an issue where virtual index rollback could fail on 32-bit builds of MySQL Server under certain circumstances. (Bug #38167527)

  • Fixed an issue where very large buffer pools could require excessive memory chunks per instance, potentially causing allocation failures. The allocation is now validated to ensure proper memory allocation. (Bug #37994397)

  • Fixed an issue where assertion failures could occur due to data size and bounds mismatches during DDL file operations. (Bug #37882398)

  • Fixed an issue related to modifying the internal Full-Text Search (FTS) configuration. (Bug #37792010)

  • Fixed an issue related to virtual indexes. (Bug #37602657)

  • Fixed an issue where confusing warning messages could appear when row sizes exceeded maximum allowed limits with innodb_strict_mode=OFF, such as during table selection or column dropping operations. Error messages have been improved for clarity. (Bug #37003342, Bug #36768046, Bug #36867372)

Find the complete list of bug fixes and changes in the MySQL 8.4.7 release notes .

Improvements

  • PS-8980: Percona Server for MySQL 8.4.7-7 adds the performance_schema.account_failed_login_lock_status table, which lists the user accounts that have been temporarily locked after reaching the limit for failed login attempts.

  • PS-9826: Added metrics collection for component_js_lang. Improved error handling by adding a safety check before accessing the query result vector.

Bug Fixes

  • PS-9220: During slow‑query‑log rotation the server attempts to write to the base file slow.log (which isn’t created), raises an error - errno 2 – No such file or directory, but the logs are written with numeric suffixes (for example, slow.log.000087) that obey the configured size and file‑count limits.

  • PS-9452: Under pool-of-threads with high_prio_mode=transactions, new login requests were routed to the normal queue. When active threads exceeded thread_pool_size * (thread_pool_oversubscribe + 1), the thread pool starved this queue. Because connection counts were incremented pre-queueing, retrying clients quickly exhausted max connections, causing a “Too many connections” error and locking out non-admin users.

  • PS-9815: Percona Server using the component_keyring_vault component exited when it attempted to load a component_keyring_vault.cnf file containing incorrect setting values.

  • PS-9834: The Audit Log Filter incorrectly deletes all rotated logs if the audit_log_filter.max_size is set > 2 GiB.

  • PS-9837: A server exit occurs on a replica that uses replica_parallel_workers > 0 and slave_preserve_commit_order=ON when many threads repeatedly delete rows and then insert new rows having the same secondary‑index values, creating duplicate index entries. Changing the workload to a single UPDATE (instead of a delete‑followed‑by‑insert) prevents the duplicate‑key situation and stops the replica from exiting.

  • PS-10113: Percona Server exited when a JavaScript routine attempted to allocate more memory than the internal heap limit allowed.

  • PS-10131: The audit_log_rotate() could be delayed up to one second because FileWriterBuffering::close() waited for a flush‑worker thread that was sleeping in mysql_cond_timedwait. The fix signals the flush‑worker immediately before the wait, ensuring prompt buffer flushing and preventing rotation delays.

  • PS-10283: A debug‑assertion failure (field.field_no < dict_index_get_n_fields(index)) that occurred during recovery of InnoDB tables using ROW_FORMAT=REDUNDANT. The fix added a conditional bypass of the index‑field validation when the server was in recovery mode, and the table was non‑compact (redundant), preventing the assertion from triggering and allowing the server to start and recover normally.

  • PS-10324: The audit log filter component was missing required fields for the table_access and general event classes.

MyRocks

  • PS-9680: Calling ALTER TABLE from multiple threads in parallel could trigger the error “Invalid argument: Global seqno is required, but disabled.”

  • PS-9838: The server’s estimate for how many rows fit in one index leaf (guess_rec_per_key()) uses two numbers: used_keyparts (the parts actually stored in the index) and user_defined_key_parts (the parts the user declared). If these two counts differ, the formula calculates the wrong capacity, leading to inaccurate index‑size estimates.

  • PS-9840: An issue during initialization could cause the process to stop midway when encountering a corrupted datadir. Because the system did not mark initialization as complete, the cleanup phase was skipped, leaving internal RocksDB structures unreleased and resulting in an assertion failure. The fix ensures that validation occurs only after initialization is marked as complete, guaranteeing that cleanup always runs and all resources are properly freed.

  • PS-9842: An assertion could occur due to an invalid iterator. After this fix, Rdb_iterator_base is always invalidated when an error occurs.

  • PS-10067: This fix resolves malformed reads and improves the stability of index iteration. The update fixed Rdb_convert_to_record_key_decoder::skip() so that it now advances both the key‑slice reader and the unpack‑information reader. By consuming the per‑field metadata during iteration with Rdb_key_field_iterator::next(), the function correctly handles index fields created via make_unpack_unknown_varlength().

  • PS-10075: An issue could cause a checksum mismatch when reading records. In some cases, the reader would skip per-field metadata but still attempt to read the checksum, resulting in an invalid position. This fix ensures that the checksum is only verified when the reader is correctly positioned, preventing false checksum errors.

  • PS-10210: Added enforcement for the ranges for the following MyRocks variables: rocksdb_block_size now rejects sizes larger than 4 GiB, rocksdb_keep_log_file_num must be greater than 0, and the max value of rocksdb_max_file_opening_threads is set to 262,144. Additionally, the rocksdb_debug_cardinality_multiplier parameter was constrained to valid ranges, eliminating the crashes that occurred when it was set to 0.

  • PS-10227: An unexpected server exit occurred when rocksdb_table_stats_skip_system_cf was enabled. Previously, enabling this option left the system column‑family pointer null during RocksDB recovery, causing an assertion failure in Rdb_cf_manager::get_cf() and aborting mysqld with signal 6. The fix introduced proper initialization and safety checks for the system column family, allowing the server to start normally while still skipping system‑CF statistics.

Packaging and build notes

  • Percona Server for MySQL 8.4 adds support for Debian 13.

  • Percona Server for MySQL 8.4 has ended support for Ubuntu 20.04.

Known limitation

On releases that still shipped Debian 11 packages (through 8.4.8-8), MySQL Shell is not available as a prebuilt package for Debian 11 (Bullseye) in the Percona APT repository. Recent MySQL Shell releases are built with GNU Compiler Collection (GCC) 11 or later, while Debian 11 ships with GCC 10, making these binaries incompatible.

Additional resources

Percona Server for MySQL 8.4.6-6 (2025-09-08)

Get started with Quickstart Guide for Percona Server for MySQL.

Percona Server for MySQL 8.4.6-6 includes all the features and bug fixes available in the MySQL 8.4.6 Community Edition in addition to enterprise-grade features developed by Percona.

Release highlights

Percona Server for MySQL 8.4.6-6

Implements Link-Time Optimization (LTO) to generate more optimized release binaries, resulting in improved performance for specific workloads.

MySQL 8.4.6

Improvements and bug fixes introduced by Oracle for MySQL 8.4.6 and included in Percona Server for MySQL are the following:

  • Fixed an issue where rebuilding a primary key with duplicate entries could cause the server to stop unexpectedly. (Bug #37822992)

  • Fixed an issue related to dropping columns that were part of an index. (Bug #37726881)

  • Fixed an issue with indexing spatial datatype columns. (Bug #36682518)

  • Fixed an issue where creating a secondary index on a VARCHAR column could exceed configured memory limits, with the amount allocated being directly related to the value of innodb_ddl_buffer_size, leading to errors such as ERROR 1136 (21S01): Column count doesn’t match value count at row 1. (Bug #37233273)

Find the complete list of bug fixes and changes in the MySQL 8.4.6 release notes.

Bug Fixes

  • PS-9159: MySQL server exited when running SELECT GLOBAL_TEMPORARY_TABLES during an active Data Definition Language (DDL) operation.

  • PS-9666: MyRocks exited when inserting into a table with unique_checks=OFF and Time to Live (TTL) enabled.

  • PS-9784: Fix for gcc-15 compilation issues.

  • PS-9823: The mysql_migrate_keyring component didn’t work with Percona Server components.

  • PS-9836: Fix for performance regression when using audit_log_filter vs audit_log plugins.

  • PS-9861: Fixed an issue preventing installation of the audit_log_filter plugin when the component_keyring_kmip plugin, which integrates with Fortanix Data Security Manager (DSM) via the Key Management Interoperability Protocol (KMIP), was enabled.

  • PS-9916: Added logging of an ERROR message when a source server is stopped while waiting for acknowledgments (ACKs).

  • PS-9956: Percona Server exited using Group Replication with the audit_log_filter component enabled.

Build & packaging notes

  • The official packages were built with the WITH_LTO=ON flag to enable the Link-Time Optimization (LTO) feature.

  • Percona Server for MySQL 8.4.6.6 supports Red Hat Enterprise Linux 10.

Additional resources

Percona Server for MySQL 8.4.5-5 (2025-05-29)

Get started with Quickstart Guide for Percona Server for MySQL.

Percona Server for MySQL 8.4.5-5 includes all the features and bug fixes available in the MySQL 8.4.5 Community Edition in addition to enterprise-grade features developed by Percona.

Release highlights

Percona Server for MySQL 8.4.5-5

  • Updates the C++ level of the KMPI library to enhance error handling capabilities.

  • Improves optimizer behavior by restoring correct handling of const tables in test_quick_select(). A MySQL Upstream refactor (commit 9a13c1c) removed the QEP_TAB dependency, causing get_quick_record_count() to no longer pass const table information. This could lead to suboptimal range scan boundaries. The applied patch resolves the issue by explicitly passing const_tables to test_quick_select(), ensuring consistent behavior with the pre-refactor logic.

The latest MyRocks storage engine incorporates code based on RocksDB version 9.3.1. Percona has applied minor modifications to the original RocksDB codebase. Check the list of modifications at https://github.com/percona/rocksdb/.

This release adds the following changes to the list of MyRocks variables.

Adds new MyRocks variables

Changes default values of MyRocks variables

Deprecates MyRocks variable

Removes MyRocks variables

MySQL 8.4.5

Improvements and bug fixes introduced by Oracle for MySQL 8.4.5 and included in Percona Server for MySQL are the following:

  • Fixed an issue where CHECK TABLE sometimes incorrectly reported that spatial indexes were corrupted. (Bug #37286473)

  • Fixed an issue in InnoDB redo log recovery to improve data safety after a crash. (Bug #37061960)

  • Fixed an issue where reading index_id values could lead to incorrect behavior with indexes. (Bug #36993445, Bug #37709706)

  • Fixed a bug related to the lower_case_table_names setting that caused inconsistent behavior with table names on different systems. (Bug #32288105)

  • Fixed a bug where mysqldump did not properly escape certain special characters in its output. (Bug #37540722, Bug #37709163)

  • The fprintf_string() function in mysqldump did not use the correct quote character for escaping strings. (Bug #37607195)

Find the complete list of bug fixes and changes in the MySQL 8.4.5 release notes.

Improvements

  • PS-9561: Updates the C++ level of the KMPI library to enhance error handling capabilities.

  • PS-9810: Updates the list of MyRocks variables. You can find the list of variables in MyRocks server variables.

Bug Fixes

  • PS-9390: In some cases, using JSON_TABLE inside an IN or EXISTS subquery caused incorrect results. This happened when the subquery referred to a table from the main query, and a semijoin optimization was applied. Percona merged the fix from MySQL.

  • PS-9609: The audit_log_filter plugin could not be installed when Percona Server was using component_keyring_kmip.

  • PS-9628: The binlog_encryption did not work with component_keyring_kmip.

  • PS-9703: In-place ALTER TABLE operations that internally rebuilt tables sometimes resulted in lost rows if a concurrent purge happened.

  • PS-9719: When binlog_transaction_dependency_tracking was changed during a high-load workload, MySQL encountered a segmentation fault.

  • PS-9723: MySQL server exited in xpl::Ssl_context::~Ssl_context() under heavy load of ALTER INSTANCE RELOAD TLS queries.

  • PS-9753: Applied an optimizer patch from Enhanced MySQL to restore correct handling of const tables in test_quick_select().

  • PS-9764: Added clang-20 to Azure Pipelines and fixed clang-20 compilation issues.

  • PS-9777: The binlog_utils_udf plugin did not handle binlog.index entries the same as the Percona Server code did.

  • PS-9780: The maximum size of audit_log_filter rule was increased from 1024 characters to 16 000 characters.

  • PS-9661: The encryption of system tablespaces using component_keyring_kmip failed.

Additional resources

Percona Server for MySQL 8.4.4-4 (2025-03-18)

Get started with Quickstart Guide for Percona Server for MySQL.

Percona Server for MySQL 8.4.4-4 includes all the features and bug fixes available in the MySQL 8.4.4 Community Edition in addition to enterprise-grade features developed by Percona.

Release highlights

Percona Server for MySQL 8.4.4-4

  • Improves the Data masking performance by introducing an internal term cache. The new cache speeds up lookups for gen_blocklist() and gen_dictionary() functions by storing dictionary data in memory. However, if the dictionary table is modified directly (outside of the proper functions), the cache may become out of sync. To fix this, use the new masking_dictionaries_flush() function.

    Changes also affect row-based replication: dictionary changes on the source server are replicated, but the term cache on the replica doesn’t update immediately. To address this, a new system variable, component_masking_functions.dictionaries_flush_interval_seconds, can be set to automatically refresh the cache at specified intervals, helping replicas stay in sync.

    Find more detailed information in the Data masking overview and in the Data masking component functions.

  • Improves the behavior of audit_log_filter_set_user to support wildcards in the hostname.

MySQL 8.4.4

Improvements and bug fixes introduced by Oracle for MySQL 8.4.4 and included in Percona Server for MySQL are the following:

  • Fixed an assertion in debug builds where certain IO buffer serializations caused system hangs. (Bug #37139618)

  • Resolved a failure when dropping the primary key and adding a new AUTO_INCREMENT column as the primary key in descending order using the INPLACE algorithm resulted in failure. (Bug #36658450)

  • Fixed incorrect results, including missing rows, in queries that used a descending primary key with the index_merge optimization. (Bug #106207, Bug #33767814)

  • Addressed a replication channel issue where MySQL failed to stop the channel properly when large transactions were being processed, and STOP REPLICA was requested. This issue also prevented graceful server shutdown, requiring process termination or system restart. (Bug #115966, Bug #37008345)

Find the complete list of bug fixes and changes in the MySQL 8.4.4 release notes.

Improvements

Bug fixes

  • PS-9391: The replication broke with the error HA_ERR_KEY_NOT_FOUND when the slave_rows_search_algorithms were set to INDEX_SCAN,HASH_SCAN.

  • PS-9416: The error messages from the Key Management Interoperability Protocol (KMIP) component were not descriptive.

  • PS-9509: Percona Server stopped tracking the global_connection_memory when using thread_handling='pool-of-threads'.

  • PS-9537: When building a new component that used mysql_command_xxx services (such as mysql_command_factory, mysql_command_query, etc.), it was impossible to reuse the same connection to run multiple queries. This issue was observed with SELECT queries, but it may also apply to INSERT, UPDATE, and DELETE operations.

  • PS-9542: Added Clang-19 to Azure pipelines, and fixed the clang-19 compilation issues.

  • PS-9551: When building a new component that used mysql_command_xxx services (such as mysql_command_factory, mysql_command_query, etc.), a server exit was encountered when setting the MYSQL_COMMAND_LOCAL_THD_HANDLE option.

  • PS-9611: An assertion failure occurred during server shutdown: !is_set() || m_can_overwrite_status.

  • PS-9612: Percona Server build failed if more than 128 threads were available. Percona merged the fix from MariaDB.

  • PS-9654: There was an incorrect usage of setup_component_customized.inc in the MySQL Test Runner (MTR) tests.

  • PS-9033: The audit_log_filter plugin did not register remote accesses.

  • PS-9464: Some queries that used hash antijoins returned incorrect results when the hash table did not fit in the join buffer and spilled to the disk. (The query triggering the issue specified LEFT JOIN, which was transformed internally from a left outer join to an antijoin.) Percona merged the fix from MySQL (Bug #116334, Bug #37161583).

  • PS-9614: The Pool-of-Threads timer thread failed to start if mysqld was started with --daemonize.

  • PS-9668: The server exited when executing LOCK TABLES FOR BACKUP after audit logs were enabled.

Additional resources

Percona Server for MySQL 8.4.3-3 (2024-12-18)

Get started with Quickstart Guide for Percona Server for MySQL.

Percona Server for MySQL 8.4.3-3 includes all the features and bug fixes available in the MySQL 8.4.3 Community Edition in addition to enterprise-grade features developed by Percona.

Release highlights

Improvements and bug fixes introduced by Oracle for MySQL 8.4.3 and included in Percona Server for MySQL are the following:

  • The query SELECT * FROM sys.innodb_lock_waits; now fetches only two locks per wait, instead of scanning all locks twice, improving performance under heavy load. Additionally, primary keys have been added to DATA_LOCKS and DATA_LOCK_WAITS. (Bug #100537, Bug #31763497)

  • Changes in MySQL 8.0.33 caused performance degradation for queries using joins on InnoDB tables due to refactoring of functions that were previously inline.

  • The server crashed when it tried to update columns altered with NULL as the default value using the INSTANT algorithm.

  • The server could crash during DELETE or UPDATE operations if a column was dropped using the INSTANT algorithm.

  • Importing a table created under a different sql_mode sometimes led to schema mismatches, risking data corruption in secondary indexes. The fix now includes integrity checks on the imported tablespace.

  • Rebuilding tables with secondary indexes required more file I/O operations compared to MySQL 8.0.26, which slowed down query performance.

Find the complete list of bug fixes and changes in the MySQL 8.4.3 release notes.

Bug fixes

  • PS-9382: After an upgrade, the telemetry daemon ran continuously. The telemetry daemon was manually stopped and the service was disabled. Adding percona_telemetry_disable=1 to the configuration file and restarting MySQL led to the server becoming unresponsive and required a forced termination.

  • PS-9453: The percona_telemetry tool caused a long wait on COND_thd_list if the root user is absent.

Additional resources

Percona Server for MySQL 8.4.2-2 (2024-11-04)

Get started with Quickstart Guide for Percona Server for MySQL.

Percona Server for MySQL 8.4.2-2 includes all the features and bug fixes available in the MySQL 8.4 Community Edition in addition to enterprise-grade features developed by Percona.

Release highlights

Improvements and bug fixes introduced by Oracle for MySQL 8.4.1 and 8.4.2 and included in Percona Server for MySQL are the following:

  • MySQL stopped unexpectedly during an UPDATE after an ALTER TABLE operation.

  • Shutting down the server after an XA START with an empty XA transaction caused it to stop unexpectedly.

  • Shutting down the replication applier or binlog applier during an empty XA transaction caused the system to stop unexpectedly.

  • The result from a spatial index with a column containing a spatial reference identifier (SRID) was empty. Using FORCE INDEX to scan this index caused an assertion error.

  • In some cases, after creating more than 8000 tables, the server failed to restart.

  • Startup tablespace file scanning performance was improved.

Find the complete list of bug fixes and changes in the MySQL 8.4.1 Release Notes and MySQL 8.4.2 Release Notes.

Bug fixes

  • PS-8057: slow_query_log_file does not match the filename defined in my.cnf.

  • PS-9144: Missing rows after running a null ALTER with ALGORTITHM=INPLACE.

  • PS-9214: An ALTER table online results in a “duplicate key” error on the primary key (only index).

  • PS-9306: The following MySQL versions unexpectedly exit if the database has more than 10K tables:

    • 8.0.38

    • 8.4.1

    • 9.0.0

  • PS-9314: Using a JSON_TABLE in Percona Server for MySQL 8.0.36 causes a signal 11 error.

  • PS-9286: The KMIP component left keys in a pre-active state.

  • PS-9384: A race condition between dict_stats_thread and the cost model initialization cause sporadic exits in Jenkins on start up.

Additional resources

Percona Server for MySQL 8.4.1

Due to a critical fix, MySQL Community Server 8.4.2 was released shortly (22 days later) after MySQL Community Server 8.4.1. Percona has skipped the release of Percona Server for MySQL 8.4.1. Percona Server for MySQL 8.4.2-2 contains all bug fixes and contents from MySQL Community Server 8.4.1 and MySQL Community Server 8.4.2.

Percona Server for MySQL 8.4.0-1 (2024-08-28)

Get started with Quickstart Guide for Percona Server for MySQL.

Percona Server for MySQL 8.4.0-1 includes all the features and bug fixes available in the MySQL 8.4 Community Edition in addition to enterprise-grade features developed by Percona.

Release highlights

In MySQL 8.0, the release model changed to include new features in patch releases, allowing MySQL to introduce new features more frequently. However, this approach was complex for projects and applications needing only critical patches with minimal changes.

MySQL then moved to a versioning model with two options: Innovation releases and Long-Term Support (LTS) releases. Both types are production-ready.

Innovation releases offer access to the latest features, making them ideal for dynamic environments with strong automated testing and continuous integration.

Changes were made to each Innovation release that are now included in the 8.4 (LTS) release.

  • Percona Server for MySQL 8.1.0-1 (2023-11-27)

  • Percona Server for MySQL 8.2.0-1 (2024-02-05)

  • Percona Server for MySQL 8.3.0-1 (2024-04-16)

LTS releases are more suitable for stable, established environments where minimal changes are needed. These releases include only essential fixes, reducing the risk of changes in the database software’s behavior.

This 8.4.0-1 release is the first 8.4 LTS series.

Improvements and bug fixes introduced by Oracle for MySQL 8.4 and included in Percona Server for MySQL are the following:

  • The MySQL native password has been deprecated and is no longer loaded by default. However, it can be loaded if needed.

  • The clone plugin allows cloning between different point releases within the same series. You only must match the major and minor version numbers for cloning.

  • GTIDs (Global Transaction Identifiers) can now handle groups of transactions, which helps speed up processing.

  • mysqldump can now create output for older versions of MySQL.

  • Automatic updates for histograms. When enabled, the histogram updates automatically whenever ANALYZE TABLE is run on the table. InnoDB’s automatic recalculation of persistent statistics also updates the histogram when automatic updates are enabled.

  • Adds a new privilege called FLUSH_PRIVILEGES. This privilege explicitly allows the use of FLUSH PRIVILEGES statements. Unlike the RELOAD privilege, FLUSH_PRIVILEGES only applies to FLUSH PRIVILEGES statements.

  • The terms “MASTER” and “SLAVE” in replication commands are being replaced with “SOURCE” and “REPLICA”. This change is part of an ongoing effort to use more inclusive language.

  • Removed the the mysqlpump utility.

  • Removed the mysql_upgrade utility.

  • The default values for specific InnoDB server system variables have changed. See What is new in MySQL 8.4 since 8.0 for details.

Find the complete list of bug fixes and changes in the MySQL 8.4 Release Notes.’

New features

  • PS-9233: Adds the UUID_VX component which provides a set of functions for generating and working with various versions of the Universally Unique Identifier (UUID).

Improvements

  • PS-9302: Changed underlying internal data structure used by the binlog transaction dependency tracking in WRITESET mode (MySQL 8.4 removed the COMMIT_ORDER mode). Instead of std::map (an RB-tree) we now use std::unordered_map (a hash) which gives much better performance for lookup operations. This change showed an up to 17% Queries per second (QPS) increase in the oltp_inlist_update workload.

Bug fixes

  • PS-9092: There were data inconsistencies during a high rate of page split/merge.

  • PS-9121: MySQL exited when InnoDB failed to update a spatial index.

  • PS-9151: Percona server 8.0 build failed on CentOS 7 with -DWITH_SSL=openssl11.

  • PS-9219: While converting the charset collation in a table, MySQL converted the date and time data types columns in the .ibd file. However, the collation_id in the .ibd file did not align with that of the data dictionary.

  • PS-9155: The server exited during the execution of the complicated query with 9 CTEs.

  • PS-9235: Keyring vault failed to work with binlog_rotate_encryption_master_key_at_startup.

Deprecation

  • PS-8963: The SEQUENCE_TABLE() function is deprecated and may be removed in a future release. We recommend that you use PERCONA_SEQUENCE_TABLE() instead. To maintain compatibility with existing third-party software, SEQUENCE_TABLE is no longer a reserved term and can be used as a regular identifier. Find more information in PERCONA_SEQUENCE_TABLE(n) function

Packaging notes

Percona Server for MySQL 8.4.0-1 is compatible with Ubuntu 24.04.

Additional resources

Features

Adaptive network buffers

To find the buffer size of the current connection, use the network_buffer_length status variable. Add SHOW GLOBAL to review the cumulative buffer sizes for all connections. This variable can help to estimate the maximum size of the network buffer’s overhead.

Network buffers grow towards the max_allowed_packet size and do not shrink until the connection is terminated. For example, if the connections are selected at random from the pool, an occasional big query eventually increases the buffers of all connections. The combination of max_allowed packet set to a value between 64MB and 128MB and the connection number between 256 and 1024 can create a large memory overhead.

Percona Server for MySQL implemented the net_buffer_shrink_interval variable to solve this issue. The default value is 0 (zero). If you set the value higher than 0, Percona Server records the network buffer’s maximum use size for the number of seconds set by net_buffer_shrink_interval. When the next interval starts, the network buffer is set to the recorded size. This action removes spikes in the buffer size.

You can achieve similar results by disconnecting and reconnecting the TCP connections, but this solution is a heavier process. This process disconnects and reconnects connections with small buffers.

net_buffer_shrink_interval

Option Description
Command-line: –net-buffer-shrink-interval=#
Scope: Global
Dynamic: Yes
Data type: integer
Default value: 0

The interval is measured in seconds. The default value is 0, which disables the functionality. The minimum value is 0, and the maximum value is 31536000.

Audit Log Filter

Audit Log Filter overview

The Audit Log Filter component audits activity on the server you configure by recording events to a log. Filtering rules control which events are written. Events that match a rule’s log condition are recorded; everything else is skipped and never reaches the log.

With the component enabled, the server can capture who connected, which statements ran, and which schemas sessions touched — subject to those filter rules.

Architecture

The following diagram shows how the Audit Log Filter component sits between the server core, your filter rules, and audit output.

Audit Log Filter component architecture

Why the filter component

The component replaces the legacy audit log plugin. The design centers on three goals that the plugin could not meet:

  • Change rules without restarting the server. Filter definitions and account assignments live in mysql system tables. Updates via audit_log_filter_set_filter() and audit_log_filter_set_user() take effect on new sessions (and reloads) without bouncing the server. The legacy plugin required audit_log_* system-variable changes that often meant a restart or plugin reinstall to adjust scope.

  • Scope audits per account, not globally. Each row in mysql.audit_log_user binds a user or user-pattern to a named filter, so admin@% and app@% can be audited differently on the same instance. The legacy plugin offered only global include/exclude lists (audit_log_include_accounts / audit_log_exclude_accounts) on top of a single policy preset.

  • Express rules as data, not as knob settings. Filter definitions are JSON documents. A rule can match an event, a class, a subclass, or a specific field value; combine matches with and / or / not; call predefined variables and functions; use log conditions to write or skip each event; and redact statement text in place with print / replace. The legacy plugin had policy presets (LOGINS, QUERIES, ALL, …) with far coarser control and no per-event skip logic or redaction.

A secondary benefit falls out of the design: events that a filter skips never reach the formatter or the writer, so the cost of “audit everything, then throw most of it away” is avoided at the source. Combined with per-account scoping, this lets operators raise audit detail on sensitive accounts without paying for verbose logging elsewhere.

See Write audit_log_filter definitions for the JSON grammar, Block statements with an audit log filter for abort, and Redact audit log fields for print / replace.

Audit data flow

The following diagram traces audit events from the server through filtering and formatting into the log (file, syslog, or another handler).

Audit Log Filter data flow

Set audit_log_filter.format at startup. Audit Log Filter file format overview compares NEW (the default in Percona Server 8.4), OLD, JSON, and JSONL.

audit_log_filter.event_mode selects which event classes the component audits (default REDUCED; FULL enables every class the component supports, including beyond the four core classes). That reference lists full class sets, validation rules, and behavior on older releases.

The component stores filter definitions and account assignments in the mysql system database. Set audit_log_filter.database at startup to use a different database.

You need AUDIT_ADMIN to administer the Audit Log Filter component.

Privileges

Define the privilege at server startup.

AUDIT_ADMIN

The server defines AUDIT_ADMIN so grantees can configure the component.

AUDIT_ABORT_EXEMPT

A filter may include an abort rule that blocks matching statements. To bypass those aborts, an account needs both SYSTEM_USER and the global AUDIT_ABORT_EXEMPT privilege (granted on *.*).

Only that pair lets matching statements run when a filter would otherwise abort them. The component still writes those statements to the audit log.

Audit Log Filter tables

The Audit Log Filter component uses mysql system tables on InnoDB. They hold account assignments and filter JSON. Point the component at another database with audit_log_filter.database when the server starts.

The audit_log_filter table stores named filter definitions:

Column name Description
NAME Filter name
FILTER JSON filter definition linked to that name

The audit_log_user table maps accounts to filters:

Column name Description
USER MySQL account user part
HOST MySQL account host part
FILTERNAME Name of the assigned filter

Filter storage hierarchy

The following diagram shows how rows in audit_log_filter map to audit_log_user, including the default % account.

Audit Log Filter storage hierarchy

On connect, the component loads one filter definition from the matching USER/HOST row in mysql.audit_log_user, or falls back to the default account (%). A concrete account (for example admin@localhost) overrides a % assignment. Rules inside that filter’s JSON (for example log conditions that test user.str or host.str with field items) apply after load — they narrow which events get written, but do not act as a second assigned filter. See Filter the Audit Log Filter logs for assignment order and wildcards, and Write audit_log_filter definitions for the JSON grammar that log conditions use.

Additional reading

Install the audit log filter

Installation script

Run audit_log_filter_linux_install.sql from the server share directory. The script creates the audit tables, then installs the component.

Prerequisites

plugin_dir locates the component library; set it at startup if the default path is wrong.

Database selection

The script picks the target database in this order:

  • If the component is already loaded, the script uses the database name from the audit_log_filter.database variable

  • If the component is not loaded, but you pass the -D db_name option to the mysql client when running the script, the script uses the specified db_name

  • If the component is not loaded and no -D option is provided, you must specify the mysql database when running the script

You can also point the component at another database with audit_log_filter.database. The name must be non-NULL, at most 64 characters, and valid—otherwise the server cannot open the audit log filter tables.

Install the component

By default, the script runs against the mysql database. Use either approach:

  • Option 1: Run the script from the command line with the -D mysql option:

    mysql -u root -p -D mysql < /path/to/mysql/share/audit_log_filter_linux_install.sql

  • Option 2: Connect to mysql database and run the script interactively:

    use mysql;
source /path/to/mysql/share/audit_log_filter_linux_install.sql;

    Replace /path/to/mysql/share/ with the actual path to your server installation’s share directory.

Verify the audit tables exist

Confirm the audit tables exist:

show tables in mysql like 'aud%';

A clean install lists two rows for audit_log_filter and audit_log_user. Other tables matching aud% in the mysql schema can also appear in the output; the install is successful as long as both audit_log_filter and audit_log_user are present.

Expected output on a clean install 
+------------------------+
| Tables_in_mysql (aud%) |
+------------------------+
| audit_log_filter       |
| audit_log_user         |
+------------------------+

Alternative: INSTALL COMPONENT method

INSTALL COMPONENT loads the binary only—it does not create tables, so filter UDFs fail until you run the install script.

INSTALL COMPONENT 'file://component_audit_log_filter';

The URN must include the component_ prefix. file://audit_log_filter fails to load the component.

Verify the component is registered

Confirm the component row exists:

select * from mysql.component;

Look for a row whose component_urn is file://component_audit_log_filter. Other components (for example component_percona_telemetry) can appear in the same result; the row count and column values depend on which other components the instance has loaded.

Example output 
+--------------+--------------------+------------------------------------+
| component_id | component_group_id | component_urn                      |
+--------------+--------------------+------------------------------------+
|            1 |                  1 | file://component_percona_telemetry |
|            2 |                  2 | file://component_audit_log_filter  |
+--------------+--------------------+------------------------------------+

Detect missing tables

Exercise a filter UDF to confirm the audit tables exist. The call fails when the component loads without them:

SELECT audit_log_filter_set_filter('log_all', '{"filter": {"log": true}}');
Expected output when tables are missing 
+---------------------------------------------------------------------+
| audit_log_filter_set_filter('log_all', '{"filter": {"log": true}}') |
+---------------------------------------------------------------------+
| ERROR: Failed to check filtering rule name existence                |
+---------------------------------------------------------------------+
1 row in set (0.00 sec)

Note

This error indicates that the component is loaded without the tables. Run audit_log_filter_linux_install.sql first.

Fix missing tables

If the component is installed but tables are missing, run the install script against mysql:

mysql -u root -p -D mysql < /path/to/mysql/share/audit_log_filter_linux_install.sql

Or interactively:

use mysql;
source /path/to/mysql/share/audit_log_filter_linux_install.sql;

The script adds the tables and does not reinstall the component.

Post-install verification

The following check applies to both install paths. Run it after the install script completes or after recovering from a missing-tables state.

Define a catch-all filter and bind it to the default account pattern. Both calls are required. audit_log_filter_set_filter() stores the filter definition in mysql.audit_log_filter. audit_log_filter_set_user() assigns it to a login pattern in mysql.audit_log_user. A filter that is defined but unassigned never reaches a session.

SELECT audit_log_filter_set_filter('log_all', '{"filter": {"log": true}}');
SELECT audit_log_filter_set_user('%', 'log_all');
SELECT audit_log_filter_flush();
Expected output 
+---------------------------------------------------------------------+
| audit_log_filter_set_filter('log_all', '{"filter": {"log": true}}') |
+---------------------------------------------------------------------+
| OK                                                                  |
+---------------------------------------------------------------------+
+-------------------------------------+
| audit_log_filter_set_user('%', 'log_all') |
+-------------------------------------+
| OK                                  |
+-------------------------------------+
+--------------------------+
| audit_log_filter_flush() |
+--------------------------+
| OK                       |
+--------------------------+

The % account pattern is the default-row fallback the component uses when no more specific row in mysql.audit_log_user matches the session account. For per-account assignment, replace % with a user_name@host_name pattern. For details, see audit_log_filter_set_user() and Which audit_log_user row applies.

Additional information

If you are replacing an existing audit plugin (the legacy audit_log plugin or the transitional audit_log_filter plugin) with the component, see Migrate to the audit log filter component for the variable mapping, policy translation, and a worked example. The general plugin-to-component framing is in Upgrade from plugins to components.

Additional reading

Review Get help from Percona for ways that we can work with you.

Migrate to the audit log filter component

Percona Server for MySQL 8.4 replaces two legacy audit sources — the audit_log plugin and the transitional audit_log_filter plugin — with component_audit_log_filter. This page covers migration from either source: it maps plugin configuration to the component’s system variables and filter JSON, and walks through a safe cutover.

Before you start, read:

Note

The component and the plugin use different system variables and a different on-disk format. Do not enable both at the same time, and do not set audit_log_* plugin variables on a server running the component. See the deprecation notice on the plugin page.

Which source are you migrating from?

Two legacy sources exist. The target is the same (component_audit_log_filter), but the starting point differs:

Source Recommended path
audit_log plugin (pre-8.4 installs, still available in 8.4 as a deprecated plugin) Upgrade to 8.4, install the component, translate audit_log_* variables to filter JSON, validate in parallel, then uninstall the plugin.
audit_log_filter plugin (transitional, 8.0 and early 8.4 builds) Upgrade to 8.4 first, then transition to the component per Upgrade from plugins to components → Transition after upgrade.

The detailed mapping in the following sections targets the audit_log plugin, because its configuration model differs most from the component. The plugin model uses global audit_log_* variables and policy presets. If you are migrating from the audit_log_filter plugin, the filter JSON you already wrote continues to work unchanged. Your migration reduces to a shorter path:

  1. Upgrade the server to 8.4.
  2. Uninstall the plugin.
  3. Run the component install script (see Install the audit log filter) — this creates mysql.audit_log_filter and mysql.audit_log_user and registers the component.
  4. Re-apply each filter with audit_log_filter_set_filter() and re-assign accounts with audit_log_filter_set_user(). If you exported the plugin’s filter/user tables, those rows can be re-inserted directly.
  5. Move non-filter settings (file path, format, rotation, syslog) from audit_log_filter_* plugin variables to the audit_log_filter.* component variables listed in the variable mapping.

See also Upgrade from plugins to components → Transition after upgrade.

What changes, at a glance

  • Configuration moves from global audit_log_* system variables into JSON filter definitions stored in mysql.audit_log_filter, plus per-account assignments in mysql.audit_log_user.
  • Scope moves from a single global include/exclude list to per-account assignments — admin@% and app@% can use different filters on the same server.
  • Rule changes no longer require a restart or plugin reinstall: update the JSON with audit_log_filter_set_filter() and the next session picks it up.
  • Log format names shift: the plugin’s OLD / NEW / JSON / CSV become the component’s OLD / NEW / JSON / JSONL. The component does not produce CSV output.
  • The audit_log_handler = SYSLOG path becomes audit_log_filter.handler = SYSLOG with the same syslog sub-variables renamed.

Migration steps

  1. Inventory the current configuration. On the 8.0 server, or on a plugin-enabled 8.4 server, capture the legacy settings:
SHOW VARIABLES LIKE 'audit_log_%';

Save the output. You will translate each non-default value into either a component variable, a filter JSON rule, or an audit_log_filter_set_user() call.

  1. Upgrade the server to 8.4 by following Upgrade procedures. The audit_log plugin remains loadable in 8.4, so the old log keeps flowing during the transition.

  2. Install the component per Install the audit log filter. The install script creates mysql.audit_log_filter and mysql.audit_log_user, then runs INSTALL COMPONENT.

  3. Translate configuration using the two mapping tables in the following sections. Apply non-filter settings (file path, format, rotation, syslog) as component variables. Apply scope settings (policy, include/exclude lists) as a filter definition plus user assignments.

  4. Run in parallel. This step is optional but recommended. With both the plugin and the component loaded, verify that the events you care about appear in the component’s log. Compare record types, SQL text, and redactions.

  5. Cut over. Uninstall the plugin with UNINSTALL PLUGIN audit_log;, remove audit_log_* entries from my.cnf, and leave the component as the sole audit writer. If you also had the transitional audit_log_filter plugin loaded, uninstall it as well.

  6. Verify. Log in as a subject account, execute a representative statement, and read the new log with audit_log_read() or by opening the file.

Option and variable mapping

The following maps plugin system variables (left) to component system variables (right). Anything not listed has no direct equivalent because it is subsumed by the filter JSON grammar.

Plugin variable Component equivalent Notes
audit_log_file audit_log_filter.file Default file name and data-directory placement differ; see Log file naming.
audit_log_format (OLD/NEW/JSON/CSV) audit_log_filter.format (OLD/NEW/JSON/JSONL) CSV is not supported by the component. For line-delimited ingest, use JSONL (see JSON and JSONL).
audit_log_strategy audit_log_filter.strategy Same ASYNCHRONOUS / PERFORMANCE / SEMISYNCHRONOUS / SYNCHRONOUS trade-offs.
audit_log_buffer_size audit_log_filter.buffer_size Applies to ASYNCHRONOUS / PERFORMANCE.
audit_log_rotate_on_size audit_log_filter.rotate_on_size Size-based rotation.
audit_log_rotations audit_log_filter.prune_seconds or audit_log_filter.max_size The component does not keep a fixed file count. Choose age-based pruning (prune_seconds) or total-size pruning (max_size); the two options are mutually exclusive. Convert “keep N files” to an approximate age or total-size budget (for example, rotate_on_size × N for max_size).
audit_log_flush audit_log_rotate() Manual rotation uses a UDF call rather than a variable toggle.
audit_log_handler (FILE / SYSLOG) audit_log_filter.handler Same two values.
audit_log_syslog_ident audit_log_filter.syslog_tag Renamed.
audit_log_syslog_facility audit_log_filter.syslog_facility —
audit_log_syslog_priority audit_log_filter.syslog_priority —
audit_log_policy (filter JSON) Translate to class selection — see Translating audit_log_policy to filter JSON.
audit_log_include_accounts / audit_log_exclude_accounts (filter JSON + audit_log_filter_set_user()) Per-account assignment replaces global lists — see Translating include/exclude lists.
audit_log_include_commands / audit_log_exclude_commands (filter JSON) Use log conditions that test general_sql_command.str.
audit_log_include_databases / audit_log_exclude_databases (filter JSON) Use log conditions that test table_database.str in table_access.

Component-only features that have no plugin counterpart include block-on-match (abort), field redaction (print / replace), predefined variables and functions inside conditions, and dynamic filter swapping (activate / ref). See Write filter definitions and its sub-pages.

Translating audit_log_policy to filter JSON

audit_log_policy gated what the plugin recorded. The component expresses the same four choices as filter definitions:

Plugin policy Equivalent filter definition
ALL Log everything the component sees (respects audit_log_filter.event_mode): { "filter": { "log": true } }
LOGINS { "filter": { "class": { "name": "connection" } } }
QUERIES { "filter": { "class": [ { "name": "general" }, { "name": "table_access" } ] } }
NONE Either do not assign a filter to the account, or bind an empty filter: { "filter": {} }

Install any of these with audit_log_filter_set_filter() and assign them with audit_log_filter_set_user(). For example:

SELECT audit_log_filter_set_filter('log_all', '{ "filter": { "log": true } }');
SELECT audit_log_filter_set_user('%',        'log_all');

Translating include/exclude lists

The plugin filtered globally. The component filters per account (via audit_log_user) and per event (via log conditions that compare event fields). Most migrations combine both.

Accounts

Plugin:

SET GLOBAL audit_log_include_accounts = 'app@%,admin@localhost';

Component — assign a logging filter to the two accounts and leave the default % with no filter (or an empty filter):

SELECT audit_log_filter_set_filter('log_all', '{ "filter": { "log": true } }');
SELECT audit_log_filter_set_user('app@%',          'log_all');
SELECT audit_log_filter_set_user('admin@localhost','log_all');

Plugin exclude list, inverted:

SET GLOBAL audit_log_exclude_accounts = 'monitor@%';

Component — keep the default % assigned to a logging filter and assign monitor@% to an empty filter (or leave it unassigned if % has no filter):

SELECT audit_log_filter_set_filter('no_log', '{ "filter": {} }');
SELECT audit_log_filter_set_user('monitor@%', 'no_log');

Commands

Plugin:

SET GLOBAL audit_log_include_commands = 'select,insert,update,delete';

Component — narrow the general class with a log condition that tests general_sql_command.str:

SELECT audit_log_filter_set_filter('log_dml', '{
  "filter": {
    "class": {
      "name": "general",
      "log": {
        "field": {
          "name": "general_sql_command.str",
          "value": ["select", "insert", "update", "delete"]
        }
      }
    }
  }
}');

For exclusion, wrap the condition in not.

Databases

Plugin:

SET GLOBAL audit_log_include_databases = 'app,reports';

Component — narrow table_access by table_database.str:

SELECT audit_log_filter_set_filter('log_app_reports', '{
  "filter": {
    "class": {
      "name": "table_access",
      "log": {
        "field": {
          "name": "table_database.str",
          "value": ["app", "reports"]
        }
      }
    }
  }
}');

For a complete grammar reference (field names, logical operators, variables, functions), see Write filter definitions and Definition fields reference.

Worked example

Starting plugin configuration in my.cnf:

[mysqld]
plugin-load-add          = audit_log.so
audit_log_format         = JSON
audit_log_policy         = ALL
audit_log_include_accounts = app@%,admin@localhost
audit_log_exclude_commands = set_option
audit_log_rotate_on_size = 104857600
audit_log_rotations      = 10

After upgrading to 8.4 and running audit_log_filter_linux_install.sql, replace it with:

[mysqld]
audit_log_filter.format         = JSON
audit_log_filter.rotate_on_size = 104857600
audit_log_filter.max_size       = 1048576000

The plugin’s audit_log_rotations = 10 with audit_log_rotate_on_size = 104857600 (100 MiB) maps to a 1 GiB total-size cap (10 × 104857600). Age-based retention with audit_log_filter.prune_seconds is an alternative; do not set both max_size and prune_seconds — the component accepts only one at a time.

Then define the filter and assign accounts:

SELECT audit_log_filter_set_filter('log_all_except_set_option', '{
  "filter": {
    "class": [
      { "name": "connection" },
      { "name": "table_access" },
      {
        "name": "general",
        "log": {
          "not": {
            "field": { "name": "general_sql_command.str", "value": "set_option" }
          }
        }
      }
    ]
  }
}');

SELECT audit_log_filter_set_user('app@%',           'log_all_except_set_option');
SELECT audit_log_filter_set_user('admin@localhost', 'log_all_except_set_option');

Accounts that do not match app@% or admin@localhost — and that have no explicit assignment to the default % filter — are not audited, which reproduces the plugin’s include-list behavior.

Cutover and verification

  1. Confirm the component is live:

    SELECT * FROM mysql.component WHERE component_urn = 'file://component_audit_log_filter';

  2. Log in as a subject account, run a representative statement, and read the new log:

    SELECT audit_log_read('{}');

    See Read log files.

  3. Uninstall the legacy plugin and clear its settings from my.cnf:

    UNINSTALL PLUGIN audit_log;

  4. Rotate once so new writes go to a fresh file under the component’s naming scheme:

    SELECT audit_log_rotate();

  5. For any pre-existing plugin log files you want to keep, archive them. The component does not ingest them, but you can run filter_audit_log_filter_files to post-process component logs.

Known caveats

  • Log content changes slightly even when you keep the same format. Some statements that the 8.0 plugin did not record appear in 8.4 logs because the server sends additional events. The plugin page calls out the SELECT $$ example. Percona does not plan to backport format changes to match 8.0 output.
  • The component stores state in mysql.audit_log_filter and mysql.audit_log_user. Back up those tables before making bulk changes, and include them in your normal mysql database backups.
  • Lifecycle events (server_startup, server_shutdown, and the audit class itself) are not valid filter targets — see Definition fields reference.
  • audit_log_filter.event_mode defaults to REDUCED. If you relied on plugin logging for classes beyond connection, general, table_access, and message, set event_mode = FULL in my.cnf before you start the server. Changing the value on a running server creates a window where in-flight events are evaluated inconsistently against the new mode, and audit output during that window cannot be reconciled after the fact. If a restart is not possible, follow the SET GLOBAL with audit_log_filter_flush() inside a maintenance window. For the full caveat, see audit_log_filter.event_mode.

Additional reading

Audit Log Filter quickstart

Use the following steps when the Audit Log Filter component and tables are already installed. For installation, see install the audit log filter. You need AUDIT_ADMIN. Changing audit_log_filter.disable at runtime also requires SYSTEM_VARIABLES_ADMIN.

Quickstart

  1. Optional: run SHOW GLOBAL STATUS LIKE 'audit_log_filter_events_written'; and note the counter.
  2. Run these five statements in order:
SET GLOBAL audit_log_filter.disable = false;
SELECT audit_log_filter_set_filter('log_all', '{ "filter": { "log": true } }');
SELECT audit_log_filter_set_user('%', 'log_all');
SELECT audit_log_filter_flush();
SELECT 1;
  1. Run SHOW GLOBAL STATUS LIKE 'audit_log_filter_events_written'; again. The counter should increase. If it does not, verify the component loaded, tables exist, and audit_log_filter.disable is false.

For JSON or JSONL audit_log_filter.format, read events with audit_log_read() (Reading Audit Log Filter files). With the default NEW format in Percona Server 8.4, inspect the file named by audit_log_filter.file on the server.

Next steps

After the catch-all log_all rule works, replace it with a tighter JSON filter (by user, database, table, or event class). Call audit_log_filter_set_filter(), audit_log_filter_set_user(), and audit_log_filter_flush() so sessions pick up changes. Author rules in Write audit_log_filter definitions, compare assignment to JSON rules in Filter the Audit Log Filter logs, and validate names in Audit Log Filter definition fields.

If you switch to JSON or JSONL, read Audit Log Filter file format overview and Reading Audit Log Filter files. For production, plan path, rotation, and retention with audit_log_filter.file, audit_log_rotate(), and Manage the Audit Log Filter files.

Also see: Audit Log Filter overview · Install the audit log filter · Disable Audit Log Filter logging · Audit log filter functions, options, and variables (full UDF and variable reference)

Log file formats

Audit Log Filter file format overview

On each auditable event, the component appends a record to the log. After startup, the first record describes the server and startup options; later records cover connections, disconnections, executed SQL, and more.

Which server actions become audit records depends on audit_log_filter.event_mode, not on the file format. See that variable for REDUCED versus FULL and for releases before it existed.

If LOAD_DATA references files, the component does not log file contents.

Set format with audit_log_filter.format at startup. Options:

Format Type Command Description
XML (new style) audit_log_filter.format=NEW New XML layout (default in Percona Server 8.4)
XML (old style) audit_log_filter.format=OLD Legacy XML layout. Deprecated and may be removed in a later version.
JSON audit_log_filter.format=JSON One top-level JSON array of events
JSONL audit_log_filter.format=JSONL Added in 8.4.9-9. One compact JSON object per line inside a wrapping array (see the JSON/JSONL topic).

By default, new-style XML logs are neither compressed nor encrypted.

When you change audit_log_filter.format, rename audit_log_filter.file as well—for example use audit.json or audit.jsonl for JSON or JSONL. Reusing one base name obscures format changes across rotated files.

Additional reading

Audit Log Filter format - XML (new style)

Percona Server for MySQL 8.4.9-9 tightens the NEW XML formatter:

  • 1-based <RECORD_ID> values (not 0-based)

  • Self-closing empty tags (<OS_LOGIN/>), not empty pairs

  • Single-space indent per level (was two)

  • Stable element order for easier parsers and upgrades

  • Richer startup (<NAME>Audit</NAME>): adds <VERSION>, <STARTUP_OPTIONS>, <OS_VERSION>, <MYSQL_VERSION>

  • Message events: <MAP> / <ELEMENT> / <KEY> / <VALUE> replace <MESSAGE_ATTRIBUTES>; records also carry <USER>, <OS_LOGIN>, <HOST>, <IP>, <STATUS>, <STATUS_CODE>

  • Consistent <COMMAND_CLASS> spellings (for example create_table, set_option)

  • audit_log_filter.event_mode decides which events appear: REDUCED (default) keeps core classes; FULL adds lifecycle-heavy record types (see that page for lists and pre-8.4.9-9 behavior).

On builds before 8.4.9-9, spot-check your own logs—output may differ. Code reference: components/audit_log_filter/log_record_formatter/new.cc, base.cc.

With audit_log_filter.format=NEW, the component writes UTF-8 XML.

Root <AUDIT> wraps <AUDIT_RECORD> elements—one event each.

Element order matches the legacy audit plugin style; parse by name, not column position.

Timestamps use the server local zone in YYYY-MM-DDTHH:MM:SS form without a UTC suffix.

Example — REDUCED mode (default)

With audit_log_filter.event_mode=REDUCED (default), the formatter emits the primary record per action. The following sample mixes record types. Actual fields depend on filters and configuration.

<?xml version="1.0" encoding="utf-8"?>
<AUDIT>
 <AUDIT_RECORD>
  <TIMESTAMP>2026-03-27T20:00:59</TIMESTAMP>
  <RECORD_ID>1_2026-03-27T20:00:59</RECORD_ID>
  <NAME>Audit</NAME>
  <SERVER_ID>1</SERVER_ID>
  <VERSION>1</VERSION>
  <STARTUP_OPTIONS>/usr/sbin/mysqld --defaults-file=/etc/my.cnf --basedir=/usr --port=3306 --socket=/var/run/mysqld/mysqld.sock --datadir=/var/lib/mysql</STARTUP_OPTIONS>
  <OS_VERSION>x86_64-Linux</OS_VERSION>
  <MYSQL_VERSION>8.4.9-9</MYSQL_VERSION>
 </AUDIT_RECORD>
 <AUDIT_RECORD>
  <TIMESTAMP>2026-03-27T20:00:59</TIMESTAMP>
  <RECORD_ID>2_2026-03-27T20:00:59</RECORD_ID>
  <NAME>Connect</NAME>
  <CONNECTION_ID>20</CONNECTION_ID>
  <STATUS>0</STATUS>
  <STATUS_CODE>0</STATUS_CODE>
  <USER>root</USER>
  <OS_LOGIN/>
  <HOST>localhost</HOST>
  <IP>127.0.0.1</IP>
  <COMMAND_CLASS>connect</COMMAND_CLASS>
  <CONNECTION_TYPE>TCP/IP</CONNECTION_TYPE>
  <PRIV_USER>root</PRIV_USER>
  <PROXY_USER/>
  <DB/>
 </AUDIT_RECORD>
 <AUDIT_RECORD>
  <TIMESTAMP>2026-03-27T20:00:59</TIMESTAMP>
  <RECORD_ID>28_2026-03-27T20:00:59</RECORD_ID>
  <NAME>TableRead</NAME>
  <CONNECTION_ID>20</CONNECTION_ID>
  <USER>root[root] @ localhost [127.0.0.1]</USER>
  <OS_LOGIN/>
  <HOST>localhost</HOST>
  <IP>127.0.0.1</IP>
  <COMMAND_CLASS>select</COMMAND_CLASS>
  <SQLTEXT>SELECT * FROM t_access</SQLTEXT>
  <DB>test</DB>
  <TABLE>t_access</TABLE>
 </AUDIT_RECORD>
 <AUDIT_RECORD>
  <TIMESTAMP>2026-03-27T20:00:59</TIMESTAMP>
  <RECORD_ID>29_2026-03-27T20:00:59</RECORD_ID>
  <NAME>Query</NAME>
  <CONNECTION_ID>20</CONNECTION_ID>
  <STATUS>0</STATUS>
  <STATUS_CODE>0</STATUS_CODE>
  <USER>root[root] @ localhost [127.0.0.1]</USER>
  <OS_LOGIN/>
  <HOST>localhost</HOST>
  <IP>127.0.0.1</IP>
  <COMMAND_CLASS>select</COMMAND_CLASS>
  <SQLTEXT>SELECT * FROM t_access</SQLTEXT>
 </AUDIT_RECORD>
 <AUDIT_RECORD>
  <TIMESTAMP>2026-03-27T20:00:59</TIMESTAMP>
  <RECORD_ID>45_2026-03-27T20:00:59</RECORD_ID>
  <NAME>Message</NAME>
  <CONNECTION_ID>20</CONNECTION_ID>
  <STATUS>0</STATUS>
  <STATUS_CODE>0</STATUS_CODE>
  <USER>root[root] @ localhost [127.0.0.1]</USER>
  <OS_LOGIN/>
  <HOST>localhost</HOST>
  <IP>127.0.0.1</IP>
  <COMMAND_CLASS>internal</COMMAND_CLASS>
  <COMPONENT>test_audit_api_message</COMPONENT>
  <PRODUCER>test_audit_api_message</PRODUCER>
  <MESSAGE>test_audit_api_message_internal</MESSAGE>
  <MAP>
   <ELEMENT>
    <KEY>my_numeric_key</KEY>
    <VALUE>-9223372036854775808</VALUE>
   </ELEMENT>
  </MAP>
 </AUDIT_RECORD>
 <AUDIT_RECORD>
  <TIMESTAMP>2026-03-27T20:00:59</TIMESTAMP>
  <RECORD_ID>88_2026-03-27T20:00:59</RECORD_ID>
  <NAME>Connect</NAME>
  <CONNECTION_ID>22</CONNECTION_ID>
  <STATUS>0</STATUS>
  <STATUS_CODE>0</STATUS_CODE>
  <USER>root</USER>
  <OS_LOGIN/>
  <HOST>localhost</HOST>
  <IP>127.0.0.1</IP>
  <COMMAND_CLASS>connect</COMMAND_CLASS>
  <CONNECTION_TYPE>SSL</CONNECTION_TYPE>
  <CONNECTION_ATTRIBUTES>
   <ATTRIBUTE>
    <NAME>_pid</NAME>
    <VALUE>764350</VALUE>
   </ATTRIBUTE>
   <ATTRIBUTE>
    <NAME>_platform</NAME>
    <VALUE>x86_64</VALUE>
   </ATTRIBUTE>
   <ATTRIBUTE>
    <NAME>_os</NAME>
    <VALUE>Linux</VALUE>
   </ATTRIBUTE>
   <ATTRIBUTE>
    <NAME>_client_name</NAME>
    <VALUE>libmysql</VALUE>
   </ATTRIBUTE>
   <ATTRIBUTE>
    <NAME>_client_version</NAME>
    <VALUE>8.4.9-9</VALUE>
   </ATTRIBUTE>
   <ATTRIBUTE>
    <NAME>program_name</NAME>
    <VALUE>mysqladmin</VALUE>
   </ATTRIBUTE>
  </CONNECTION_ATTRIBUTES>
  <PRIV_USER>root</PRIV_USER>
  <PROXY_USER/>
  <DB/>
 </AUDIT_RECORD>
 <AUDIT_RECORD>
  <TIMESTAMP>2026-03-27T20:00:59</TIMESTAMP>
  <RECORD_ID>90_2026-03-27T20:00:59</RECORD_ID>
  <NAME>Quit</NAME>
  <CONNECTION_ID>22</CONNECTION_ID>
  <STATUS>0</STATUS>
  <STATUS_CODE>0</STATUS_CODE>
  <USER>root</USER>
  <OS_LOGIN/>
  <HOST>localhost</HOST>
  <IP>127.0.0.1</IP>
  <COMMAND_CLASS>connect</COMMAND_CLASS>
  <CONNECTION_TYPE>SSL</CONNECTION_TYPE>
 </AUDIT_RECORD>
 <AUDIT_RECORD>
  <TIMESTAMP>2026-03-27T20:01:00</TIMESTAMP>
  <RECORD_ID>91_2026-03-27T20:01:00</RECORD_ID>
  <NAME>NoAudit</NAME>
  <SERVER_ID>1</SERVER_ID>
 </AUDIT_RECORD>
</AUDIT>

Example — FULL mode (additional events)

With audit_log_filter.event_mode=FULL, the formatter adds lifecycle records that wrap each action. The following fragment supplements the REDUCED example.

 <!-- Pre-authentication (before credentials are verified) -->
 <AUDIT_RECORD>
  <TIMESTAMP>2026-04-03T13:19:21</TIMESTAMP>
  <RECORD_ID>2_2026-04-03T13:19:21</RECORD_ID>
  <NAME>Pre Authenticate</NAME>
  <CONNECTION_ID>20</CONNECTION_ID>
  <STATUS>0</STATUS>
  <STATUS_CODE>0</STATUS_CODE>
  <USER/>
  <OS_LOGIN/>
  <HOST>localhost</HOST>
  <IP>127.0.0.1</IP>
  <COMMAND_CLASS>connect</COMMAND_CLASS>
  <CONNECTION_TYPE>TCP/IP</CONNECTION_TYPE>
 </AUDIT_RECORD>

 <!-- Command lifecycle: wraps every client command -->
 <AUDIT_RECORD>
  <NAME>Command Start</NAME>
  <RECORD_ID>5_2026-04-03T13:19:21</RECORD_ID>
  <TIMESTAMP>2026-04-03T13:19:21</TIMESTAMP>
  <STATUS>0</STATUS>
  <CONNECTION_ID>20</CONNECTION_ID>
  <COMMAND_CLASS>Query</COMMAND_CLASS>
 </AUDIT_RECORD>

 <!-- Parse events: before and after SQL parsing -->
 <AUDIT_RECORD>
  <NAME>Preparse</NAME>
  <RECORD_ID>6_2026-04-03T13:19:21</RECORD_ID>
  <TIMESTAMP>2026-04-03T13:19:21</TIMESTAMP>
  <COMMAND_CLASS>Parse</COMMAND_CLASS>
  <CONNECTION_ID>20</CONNECTION_ID>
  <FLAGS>0</FLAGS>
  <SQLTEXT>INSERT INTO t_access VALUES (1, 'inserted')</SQLTEXT>
  <REWRITTEN_QUERY></REWRITTEN_QUERY>
 </AUDIT_RECORD>
 <AUDIT_RECORD>
  <NAME>Postparse</NAME>
  <RECORD_ID>7_2026-04-03T13:19:21</RECORD_ID>
  <TIMESTAMP>2026-04-03T13:19:21</TIMESTAMP>
  <COMMAND_CLASS>Parse</COMMAND_CLASS>
  <CONNECTION_ID>20</CONNECTION_ID>
  <FLAGS>0</FLAGS>
  <SQLTEXT>INSERT INTO t_access VALUES (1, 'inserted')</SQLTEXT>
  <REWRITTEN_QUERY></REWRITTEN_QUERY>
 </AUDIT_RECORD>

 <!-- Query lifecycle: brackets a single SQL statement execution -->
 <AUDIT_RECORD>
  <NAME>Query Start</NAME>
  <RECORD_ID>124_2026-04-03T13:19:21</RECORD_ID>
  <TIMESTAMP>2026-04-03T13:19:21</TIMESTAMP>
  <STATUS>0</STATUS>
  <CONNECTION_ID>20</CONNECTION_ID>
  <COMMAND_CLASS>insert</COMMAND_CLASS>
  <SQLTEXT>INSERT INTO t_access VALUES (1, 'inserted')</SQLTEXT>
 </AUDIT_RECORD>
 <AUDIT_RECORD>
  <NAME>Query Status End</NAME>
  <RECORD_ID>126_2026-04-03T13:19:21</RECORD_ID>
  <TIMESTAMP>2026-04-03T13:19:21</TIMESTAMP>
  <STATUS>0</STATUS>
  <CONNECTION_ID>20</CONNECTION_ID>
  <COMMAND_CLASS>insert</COMMAND_CLASS>
  <SQLTEXT>INSERT INTO t_access VALUES (1, 'inserted')</SQLTEXT>
 </AUDIT_RECORD>

 <AUDIT_RECORD>
  <NAME>Command End</NAME>
  <RECORD_ID>129_2026-04-03T13:19:21</RECORD_ID>
  <TIMESTAMP>2026-04-03T13:19:21</TIMESTAMP>
  <STATUS>0</STATUS>
  <CONNECTION_ID>20</CONNECTION_ID>
  <COMMAND_CLASS>Query</COMMAND_CLASS>
 </AUDIT_RECORD>

 <!-- Stored-program execution -->
 <AUDIT_RECORD>
  <NAME>Execute</NAME>
  <RECORD_ID>91_2026-04-03T13:19:21</RECORD_ID>
  <TIMESTAMP>2026-04-03T13:19:21</TIMESTAMP>
  <COMMAND_CLASS>Stored Program</COMMAND_CLASS>
  <CONNECTION_ID>20</CONNECTION_ID>
  <DB>test</DB>
  <STORED_PROGRAM>trigger_nested_query</STORED_PROGRAM>
 </AUDIT_RECORD>

 <!-- Nested query inside a stored procedure -->
 <AUDIT_RECORD>
  <NAME>Query Nested Start</NAME>
  <RECORD_ID>92_2026-04-03T13:19:21</RECORD_ID>
  <TIMESTAMP>2026-04-03T13:19:21</TIMESTAMP>
  <STATUS>0</STATUS>
  <CONNECTION_ID>20</CONNECTION_ID>
  <COMMAND_CLASS>do</COMMAND_CLASS>
  <SQLTEXT>DO (SELECT 'nested query from stored procedure')</SQLTEXT>
 </AUDIT_RECORD>
 <AUDIT_RECORD>
  <NAME>Query Nested Status End</NAME>
  <RECORD_ID>93_2026-04-03T13:19:21</RECORD_ID>
  <TIMESTAMP>2026-04-03T13:19:21</TIMESTAMP>
  <STATUS>0</STATUS>
  <CONNECTION_ID>20</CONNECTION_ID>
  <COMMAND_CLASS>do</COMMAND_CLASS>
  <SQLTEXT>DO (SELECT 'nested query from stored procedure')</SQLTEXT>
 </AUDIT_RECORD>

 <!-- Global variable access -->
 <AUDIT_RECORD>
  <NAME>Variable Get</NAME>
  <RECORD_ID>284_2026-04-03T13:19:21</RECORD_ID>
  <TIMESTAMP>2026-04-03T13:19:21</TIMESTAMP>
  <COMMAND_CLASS>select</COMMAND_CLASS>
  <CONNECTION_ID>20</CONNECTION_ID>
  <VARIABLE_NAME>sort_buffer_size</VARIABLE_NAME>
  <VARIABLE_VALUE>262144</VARIABLE_VALUE>
 </AUDIT_RECORD>
 <AUDIT_RECORD>
  <NAME>Variable Set</NAME>
  <RECORD_ID>306_2026-04-03T13:19:21</RECORD_ID>
  <TIMESTAMP>2026-04-03T13:19:21</TIMESTAMP>
  <COMMAND_CLASS>set_option</COMMAND_CLASS>
  <CONNECTION_ID>20</CONNECTION_ID>
  <VARIABLE_NAME>sort_buffer_size</VARIABLE_NAME>
  <VARIABLE_VALUE>1048576</VARIABLE_VALUE>
 </AUDIT_RECORD>

 <!-- Authentication subclass events -->
 <AUDIT_RECORD>
  <NAME>Auth Credential Change</NAME>
  <RECORD_ID>337_2026-04-03T13:19:21</RECORD_ID>
  <TIMESTAMP>2026-04-03T13:19:21</TIMESTAMP>
  <COMMAND_CLASS>Authentication</COMMAND_CLASS>
  <CONNECTION_ID>20</CONNECTION_ID>
  <STATUS>0</STATUS>
  <USER>audit_tmp_user</USER>
  <HOST>localhost</HOST>
 </AUDIT_RECORD>
 <AUDIT_RECORD>
  <NAME>Auth Authid Rename</NAME>
  <RECORD_ID>368_2026-04-03T13:19:21</RECORD_ID>
  <TIMESTAMP>2026-04-03T13:19:21</TIMESTAMP>
  <COMMAND_CLASS>Authentication</COMMAND_CLASS>
  <CONNECTION_ID>20</CONNECTION_ID>
  <STATUS>0</STATUS>
  <USER>audit_tmp_user</USER>
  <HOST>localhost</HOST>
 </AUDIT_RECORD>
 <AUDIT_RECORD>
  <NAME>Auth Authid Drop</NAME>
  <RECORD_ID>399_2026-04-03T13:19:21</RECORD_ID>
  <TIMESTAMP>2026-04-03T13:19:21</TIMESTAMP>
  <COMMAND_CLASS>Authentication</COMMAND_CLASS>
  <CONNECTION_ID>20</CONNECTION_ID>
  <STATUS>0</STATUS>
  <USER>audit_renamed</USER>
  <HOST>localhost</HOST>
 </AUDIT_RECORD>
 <AUDIT_RECORD>
  <NAME>Auth Flush</NAME>
  <RECORD_ID>430_2026-04-03T13:19:21</RECORD_ID>
  <TIMESTAMP>2026-04-03T13:19:21</TIMESTAMP>
  <COMMAND_CLASS>Authentication</COMMAND_CLASS>
  <CONNECTION_ID>20</CONNECTION_ID>
  <STATUS>0</STATUS>
  <USER></USER>
  <HOST></HOST>
 </AUDIT_RECORD>

Record descriptions

Query (<NAME>Query</NAME>): carries <STATUS>, <STATUS_CODE>, <CONNECTION_ID>, <COMMAND_CLASS> (SQL command), usually <SQLTEXT>, plus <HOST>, <IP>, <USER>, <OS_LOGIN>.

Table access (TableRead, TableInsert, TableUpdate, TableDelete): <DB>, <TABLE>, <COMMAND_CLASS>, <SQLTEXT>, <HOST>, <IP>, <USER>, <OS_LOGIN>—no <STATUS> / <STATUS_CODE>.

Connection (Connect, Quit): <COMMAND_CLASS> = connect, <CONNECTION_TYPE>; Connect adds <PRIV_USER>, <PROXY_USER>, <DB>.

With client connection attributes, CONNECTION_ATTRIBUTES lists one ATTRIBUTE (NAME, VALUE) each.

Command Start / Command End (FULL): wrap each client command (Query, Ping, Quit); <COMMAND_CLASS> holds the COM name; include <STATUS>, <CONNECTION_ID>—omit host/user/IP context.

Query Start / Query Status End and Query Nested Start / Query Nested Status End (FULL): bracket one SQL execution; carry <STATUS>, <CONNECTION_ID>, <COMMAND_CLASS>, <SQLTEXT>—omit host/user/IP.

Preparse / Postparse (FULL): before/after parse; <COMMAND_CLASS> = Parse, plus <FLAGS>, <SQLTEXT>, <REWRITTEN_QUERY>.

Execute (FULL): stored-program call; <COMMAND_CLASS> = Stored Program, <DB>, <STORED_PROGRAM>.

Variable Get / Variable Set (FULL): global variable access; <COMMAND_CLASS>, <VARIABLE_NAME>, <VARIABLE_VALUE>.

Authentication subclasses (FULL): Pre Authenticate, Auth Credential Change, Auth Authid Rename, Auth Authid Drop, Auth Flush; <COMMAND_CLASS> is Authentication (or connect for pre-auth); include <STATUS>, <USER>, <HOST>.

Empty values use self-closing tags (<TAG/>) for USER, OS_LOGIN, HOST, IP, COMMAND_CLASS, PRIV_USER, PROXY_USER, DB.

<RECORD_ID> starts at 1. Indent one space per nesting level under <AUDIT> (records at 1 space, children +1, and so on).

Mandatory elements

The following elements appear on every <AUDIT_RECORD> in the NEW XML format:

Element Description
<NAME> Event subclass string. REDUCED mode: Audit, Connect, Query, Ping, TableRead, TableInsert, TableUpdate, TableDelete, Message, Quit, NoAudit. FULL mode adds: Pre Authenticate, Command Start, Command End, Preparse, Postparse, Query Start, Query Status End, Query Nested Start, Query Nested Status End, Execute, Variable Get, Variable Set, Auth Credential Change, Auth Authid Rename, Auth Authid Drop, Auth Flush.
<RECORD_ID> Sequence number and timestamp (see audit_log_filter file handling); format SEQ_TIMESTAMP where the timestamp part matches the formatter’s timestamp string.
<TIMESTAMP> Local date and time for the event.

Optional elements (by record category)

Many elements appear only for specific event classes. The following table lists elements that the NEW XML formatter (Percona Server for MySQL 8.4.9-9 onward) uses for at least one event type. The table does not promise that every field appears in every record.

Element Description
<COMMAND_CLASS> Meaning depends on the record: connection events (Connect, Quit) use connect; table-access and query events use the SQL command name (for example select, insert, create_table, set_option); message events use the message type (internal, user); command lifecycle events use the COM name (Query, Ping, Quit); parse events use Parse; stored-program events use Stored Program; authentication events use Authentication. Empty (<COMMAND_CLASS/>) on Ping records.
<CONNECTION_ID> Client connection ID.
<CONNECTION_ATTRIBUTES> Nested ATTRIBUTE elements, each with NAME and VALUE. Omitted when the record carries no attributes.
<CONNECTION_TYPE> Connection security / transport (for example TCP/IP, SSL, Socket).
<STATUS> Status code (0 for success, non-zero for failure). Present on Query, Connect, Quit, Ping, Message, and authentication records. Also on Command Start/End, Query Start/Status End in FULL mode. Not present on table-access records.
<STATUS_CODE> High-level status (0 for success, 1 for failure). Present alongside <STATUS> on Query, Connect, Quit, Ping, and Message records.
<SQLTEXT> Statement or digest text when the event carries SQL text.
<HOST>, <IP>, <USER> Client context. Present on Query, Connect, Quit, Ping, table-access, Message, Pre Authenticate, and authentication records. Not emitted on Command Start/End, Query Start/Status End, Preparse/Postparse, Execute, or Variable records.
<OS_LOGIN> External user from authentication (external_user). Present on Query, Connect, Quit, Ping, table-access, Message, and Pre Authenticate records.
<PRIV_USER>, <PROXY_USER>, <DB> Included on Connect records.
<SERVER_ID> On Audit and NoAudit records.
<VERSION> Audit log format version. On Audit (startup) records.
<STARTUP_OPTIONS> Server command-line arguments as a single string. On Audit (startup) records.
<OS_VERSION> Operating system and architecture. On Audit (startup) records.
<MYSQL_VERSION> Server version string. On Audit (startup) records.
<DB>, <TABLE> Database and table name on table-access records (TableRead, TableInsert, TableUpdate, TableDelete). <DB> also appears on Execute (stored program) records.
<VARIABLE_NAME>, <VARIABLE_VALUE> Global variable audit events (Variable Get, Variable Set). FULL mode only.
<STORED_PROGRAM> Stored program name on Execute events (<DB> also appears). FULL mode only.
<FLAGS>, <REWRITTEN_QUERY> Parse events (Preparse, Postparse). <SQLTEXT> also appears. FULL mode only.
<COMPONENT>, <PRODUCER>, <MESSAGE>, <MAP> Message events. <MAP> contains <ELEMENT> children, each with a <KEY> and <VALUE>. Message events also include <USER>, <OS_LOGIN>, <HOST>, <IP>, <STATUS>, and <STATUS_CODE> fields.

The component XML-escapes characters such as <, >, &, and " in element text. Server-side limits may truncate very long values.

Additional reading

Audit Log Filter format - XML (old style)

Deprecation notice

The OLD XML format (audit_log_filter.format=OLD) is deprecated and may be removed in a later version. Use XML (new style), JSON, or JSONL instead.

OLD XML wraps records in <AUDIT> … </AUDIT>. Each event is one <AUDIT_RECORD> (attribute-style XML).

Attribute order varies. Every record includes the attributes listed under Required attributes. Other attributes are optional by event type.

<?xml version="1.0" encoding="utf-8"?>
<AUDIT>
  <AUDIT_RECORD
    NAME="Audit"
    RECORD_ID="0_2023-03-29T11:15:52"
    TIMESTAMP="2023-03-29T11:15:52"
    SERVER_ID="1"/>
  <AUDIT_RECORD
    NAME="Command Start"
    RECORD_ID="1_2023-03-29T11:15:53"
    TIMESTAMP="2023-03-29T11:15:53"
    STATUS="0"
    CONNECTION_ID="1"
    COMMAND_CLASS="query"/>
  <AUDIT_RECORD
    NAME="Query"
    RECORD_ID="2_2023-03-29T11:15:53"
    TIMESTAMP="2023-03-29T11:15:53"
    COMMAND_CLASS="create_table"
    CONNECTION_ID="11"
    HOST="localhost"
    IP=""
    USER="root[root] @ localhost []"
    OS_LOGIN=""
    SQLTEXT="CREATE TABLE t1 (c1 INT)"
    STATUS="0"/>
  <AUDIT_RECORD
    NAME="Query Start"
    RECORD_ID="3_2023-03-29T11:15:53"
    TIMESTAMP="2023-03-29T11:15:53"
    STATUS="0"
    CONNECTION_ID="11"
    COMMAND_CLASS="create_table"
    SQLTEXT="CREATE TABLE t1 (c1 INT)"/>
  <AUDIT_RECORD
    NAME="Query Status End"
    RECORD_ID="4_2023-03-29T11:15:53"
    TIMESTAMP="2023-03-29T11:15:53"
    STATUS="0"
    CONNECTION_ID="11"
    COMMAND_CLASS="create_table"
    SQLTEXT="CREATE TABLE t1 (c1 INT)"/>
  <AUDIT_RECORD
    NAME="Query"
    RECORD_ID="5_2023-03-29T11:15:53"
    TIMESTAMP="2023-03-29T11:15:53"
    COMMAND_CLASS="create_table"
    CONNECTION_ID="11"
    HOST="localhost"
    IP=""
    USER="root[root] @ localhost []"
    OS_LOGIN=""
    SQLTEXT="CREATE TABLE t1 (c1 INT)"
    STATUS="0"/>
  <AUDIT_RECORD
    NAME="Command End"
    RECORD_ID="6_2023-03-29T11:15:53"
    TIMESTAMP="2023-03-29T11:15:53"
    STATUS="0"
    CONNECTION_ID="1"
    COMMAND_CLASS="query"/>
</AUDIT>

Required attributes

Attribute Description
NAME Action that produced the record
RECORD_ID Sequence number and timestamp; sequence resets when the component opens the log file
TIMESTAMP Event date and time

Optional attributes

Attribute Description
COMMAND_CLASS Action or command class
CONNECTION_ID Client connection ID
CONNECTION_TYPE Connection security / transport
DB Database name
HOST Client host name
IP Client IP address
OS_LOGIN External auth user (for example LDAP); empty for built-in auth
PRIV_USER Privilege check user (may differ from USER)
PROXY_USER Proxy user if used; otherwise empty
SERVER_ID Server ID
SQLTEXT SQL text
STATUS 0 success, non-zero error
TABLE Table name
USER Client user (may differ from PRIV_USER)

Additional reading

Audit Log Filter format - JSON and JSONL

JSON and JSONL emit the same key-value pairs per event. Required keys appear in every record; optional keys depend on event type and settings. Key order is not guaranteed; long values may be truncated.

Only file layout differs:

Format File structure Set with
JSON One top-level JSON array. Each event is a pretty-printed JSON object spanning multiple lines. audit_log_filter.format=JSON
JSONL One top-level JSON array. Each event is a single compact JSON object on its own line, separated by commas. audit_log_filter.format=JSONL

JSONL arrived in Percona Server for MySQL 8.4.9-9. Unlike plain JSON Lines, Percona JSONL keeps a wrapping JSON array and commas between lines, so the file stays valid JSON while remaining line-friendly for grep, jq, wc -l, streams, and aggregators.

Compression and encryption behave like JSON. audit_log_read() and audit_log_read_bookmark() read both formats.

JSON and JSONL alone expose some statistics (for example query timing and size)—use them to flag outliers in workload analysis.

Version changes

Percona Server for MySQL 8.4.9-9

  • Component startup and shutdown events include event, connection_id, account, login, and a startup_data object. The startup_data object holds server_id, os_version, mysql_version, and args (command-line arguments). Earlier releases exposed only server_id at the top level for those records.

  • Lifecycle event values changed from the internal names audit / noaudit to startup / shutdown.

  • On connection events, connection_attributes are nested inside the connection_data object.

  • Message events: the message_attributes key is replaced by map; message events also include account and login.

Attributes

Field sets match between JSON and JSONL; only wrapping differs. See the preceding table for details.

Every event object includes at least:

  • timestamp
  • id
  • class
  • event

Other common keys:

Name Description
account Database account for the event
connection_data Client connection details. From 8.4.9-9, connection_attributes nest here on connection events.
connection_id Client connection ID
general_data Statement or command when class is general
id Event ID
login How the client attached to the server
map 8.4.9-9+ Message payload (replaces message_attributes). Message events also carry account and login.
query_statistics Optional metrics for outlier detection
shutdown_data Component shutdown
startup_data Component startup; from 8.4.9-9 includes server_id, os_version, mysql_version, args
table_access_data Table access details
time UNIX timestamp (integer) when present
timestamp UTC time YYYY-MM-DD hh:mm:ss

JSON example

The following shows four event types recorded in REDUCED event mode: startup, connection, table access, and general status.

[
  {
    "timestamp": "2026-04-03 10:43:52",
    "id": 0,
    "class": "audit",
    "event": "startup",
    "connection_id": 12,
    "account": { "user": "root", "host": "localhost" },
    "login": { "user": "root", "os": "", "ip": "", "proxy": "" },
    "startup_data": {
      "server_id": 1,
      "os_version": "x86_64-Linux",
      "mysql_version": "8.4.9-9",
      "args": [
        "/usr/sbin/mysqld",
        "--defaults-file=/etc/my.cnf",
        "--basedir=/usr",
        "--user=mysql",
        "--datadir=/var/lib/mysql",
        "--socket=/var/run/mysqld/mysqld.sock",
        "--port=3306"
      ]
    }
  },
  {
    "timestamp": "2026-04-03 10:43:53",
    "id": 1,
    "class": "connection",
    "event": "connect",
    "connection_id": 39,
    "account": { "user": "root", "host": "localhost" },
    "login": { "user": "root", "os": "", "ip": "", "proxy": "" },
    "connection_data": {
      "connection_type": "socket",
      "status": 0,
      "db": "test",
      "connection_attributes": {
        "_pid": "824388",
        "_platform": "x86_64",
        "_client_version": "8.0.45",
        "_os": "Linux",
        "_client_name": "libmysql"
      }
    }
  },
  {
    "timestamp": "2026-04-03 10:43:53",
    "id": 9,
    "class": "table_access",
    "event": "read",
    "connection_id": 40,
    "account": { "user": "root", "host": "localhost" },
    "login": { "user": "root", "os": "", "ip": "", "proxy": "" },
    "table_access_data": {
      "db": "test",
      "table": "sbtest2",
      "query": "SELECT c FROM sbtest2 WHERE id BETWEEN 83000 AND 83099",
      "sql_command": "select"
    }
  },
  {
    "timestamp": "2026-04-03 10:43:53",
    "id": 11,
    "class": "general",
    "event": "status",
    "connection_id": 40,
    "account": { "user": "root", "host": "localhost" },
    "login": { "user": "root", "os": "", "ip": "", "proxy": "" },
    "general_data": {
      "command": "Query",
      "sql_command": "select",
      "query": "SELECT c FROM sbtest2 WHERE id BETWEEN 83000 AND 83099",
      "status": 0
    }
  }
]

JSONL example

In the JSONL format each event is a single compact JSON object on its own line, separated by commas inside a wrapping JSON array. The same events from the preceding JSON example look like this:

[
{"timestamp":"2026-04-03 10:43:52","id":0,"class":"audit","event":"startup","connection_id":12,"account":{"user":"root","host":"localhost"},"login":{"user":"root","os":"","ip":"","proxy":""},"startup_data":{"server_id":1,"os_version":"x86_64-Linux","mysql_version":"8.4.9-9","args":["/usr/sbin/mysqld","--defaults-file=/etc/my.cnf","--basedir=/usr","--user=mysql","--datadir=/var/lib/mysql","--socket=/var/run/mysqld/mysqld.sock","--port=3306"]}},
{"timestamp":"2026-04-03 10:43:53","id":1,"class":"connection","event":"connect","connection_id":39,"account":{"user":"root","host":"localhost"},"login":{"user":"root","os":"","ip":"","proxy":""},"connection_data":{"connection_type":"socket","status":0,"db":"test","connection_attributes":{"_pid":"824388","_platform":"x86_64","_client_version":"8.0.45","_os":"Linux","_client_name":"libmysql"}}},
{"timestamp":"2026-04-03 10:43:53","id":9,"class":"table_access","event":"read","connection_id":40,"account":{"user":"root","host":"localhost"},"login":{"user":"root","os":"","ip":"","proxy":""},"table_access_data":{"db":"test","table":"sbtest2","query":"SELECT c FROM sbtest2 WHERE id BETWEEN 83000 AND 83099","sql_command":"select"}},
{"timestamp":"2026-04-03 10:43:53","id":11,"class":"general","event":"status","connection_id":40,"account":{"user":"root","host":"localhost"},"login":{"user":"root","os":"","ip":"","proxy":""},"general_data":{"command":"Query","sql_command":"select","query":"SELECT c FROM sbtest2 WHERE id BETWEEN 83000 AND 83099","status":0}}
]

Additional reading

Define filters

Write audit_log_filter definitions

Audit log filters are JSON documents you pass to audit_log_filter_set_filter().

Every filter nests rules under a root filter object. For authoritative class, event, and field names that validation accepts, see Audit Log Filter definition fields.

Benefit Description
Smaller logs Target only the events you need—less disk, simpler retention.
Lighter I/O Fewer bytes per rotation window; less audit overhead on busy hosts.
Sharper security signal Emphasize sensitive tables, account changes, and chosen DML/DDL instead of noise.
Faster investigations Analysts skim fewer lines when irrelevant classes stay out of the file.
Compliance fit Capture the evidence frameworks ask for without logging everything.
Lower resource burn Trim CPU, memory, and disk spent on unwanted audit volume.

Basic structure

A filter is a JSON document whose root is a single filter object. filter contains an optional global log flag, an optional id (used for dynamic filter swapping), and one or more class rules. Each class rule can carry its own log condition and an optional event list; each event block can carry log, abort, print, or a nested filter. Conditions inside log / abort / print are built from field, variable, function, and the and / or / not logical operators.

The tree below summarizes where each key legally nests:

filter                                   # root object
├── log                                  # optional boolean; global default
├── id                                   # optional; referenced by activate / ref
└── class                                # one object or an array of class rules
    ├── name: <class>                    # "connection" | "general" | "table_access" | "message"
    ├── log                              # optional; boolean OR a condition
    ├── print                            # optional; field-replacement rule (redaction)
    └── event                            # one object or an array of event rules
        ├── name: <subclass>             # e.g. "connect", "status", "update", "insert", …
        ├── log                          # optional; boolean OR a condition
        ├── abort                        # optional; boolean OR a condition (blocks execution)
        ├── print                        # optional; field-replacement rule (redaction)
        └── filter                       # optional; subfilter for dynamic swapping

condition := {
  field    : { name, value }             # compare an event field
  variable : { name, value }             # compare a predefined server variable
  function : { name, args? }             # call a predefined function
  and | or : [ condition, … ]            # combine sub-conditions
  not      : condition                   # invert a sub-condition
}

Keep this shape in mind as you read the rest of the page: every subsequent example hangs something off one of these nodes.

A minimal skeleton:

{
  "filter": {
    "class": [
      {
        "name": "class_type",
        "event": [
          { "name": "event_subclass" }
        ]
      }
    ]
  }
}

Replace class_type with a real class name (connection, general, table_access, message) and event_subclass with one of its subclasses. Add a log condition under the class or the event to narrow which matching events are actually written.

Practical example

The following filter logs only connection events whose user.str is admin or developer and whose host.str is 192.168.0.1:

{
  "filter": {
    "class": [
      {
        "name": "connection",
        "log": {
          "and": [
            { "field": { "name": "user.str", "value": ["admin", "developer"] } },
            { "field": { "name": "host.str", "value": "192.168.0.1" } }
          ]
        }
      }
    ]
  }
}

Unpacking the shape:

  • "class" holds an array of class rules.
  • "name": "connection" selects the connection class.
  • "log" under the class block is a condition; the class logs only the events that satisfy it.
  • The condition is an and over two field comparisons. user.str and host.str are fields carried by every connection event (see Audit Log Filter definition fields). user.str’s value is an array, so the comparison is true when the field matches any value in the array.

Rule-level conditions like this one narrow which events are logged — they do not determine which filter a session uses. Session-to-filter binding is separate, done with audit_log_filter_set_user() and stored in mysql.audit_log_user. A session only reaches this JSON after the server has chosen this filter for it.

Log all events

Toggle global logging with the top-level log flag:

{
  "filter": { "log": true }
}

"log": true logs everything; "log": false logs nothing.

An empty filter object also logs everything:

{
  "filter": { }
}

That is equivalent to "log": true.

Behavior summary:

Option Details
Explicit log Honors true / false at that level.
No log, no class/event rules Defaults to on (log all).
Class or event rules present Each block can carry its own log override.

Log specific event classes

To limit logging to one class, set class.name, for example connection:

{
  "filter": {
    "class": { "name": "connection" }
  }
}

{ "name": "connection" } under filter.class audits connection events (connect, disconnect, change user—subject to event_mode and optional event narrowing).

The next example turns default logging off, then turns connection logging back on inside the class block:

{
  "filter": {
    "log": false,
    "class": { 
      "log": true, 
      "name": "connection" 
    }
  }
}

Log multiple classes or events

Log several classes either as separate objects in an array or as one array of names—equivalent when you do not need per-class options yet.

Per-class object list

Use a list of class objects when you will add per-class keys (user, event, …) later.

{
  "filter": {
    "class": [
      { "name": "connection" },
      { "name": "general" },
      { "name": "table_access" }
    ]
  }
}

Compact name array

Combine class names in one array when rules stay uniform across the listed classes:

{
  "filter": {
    "class": [
      { "name": [ "connection", "general", "table_access" ] }
    ]
  }
}
Prefer this compact form only when no per-class overrides are needed.

List of event and subclass options

Typical class and subclass pairs for authoring (subject to audit_log_filter.event_mode); validate exact spellings in Audit Log Filter definition fields.

Class name Event subclass Details
connection connect New sessions (success or failure)
connection change_user CHANGE USER
connection disconnect Session end
general status Query / command completion (success vs failure)
general log Statement log events (FULL mode)
general error Statement error events (FULL mode)
general result Statement result events (FULL mode)
table_access read Reads (SELECT, INSERT … SELECT, …)
table_access delete Deletes / truncate-style operations
table_access insert Inserts / REPLACE
table_access update Updates
message internal Internal audit API messages
message user User-emitted messages (audit_api_message_emit_udf())
connection pre_authenticate Pre-authentication handshake (FULL mode)
command start, end Client command wrapper (FULL mode)
parse preparse, postparse Before and after SQL parsing (FULL mode)
query start, status_end SQL execution bracket (FULL mode)
query nested_start, nested_status_end Nested SQL inside stored programs (FULL mode)
stored_program execute Stored-program invocation (FULL mode)
global_variable get, set Global variable read/write (FULL mode)
authentication flush, authid_create, credential_change, authid_rename, authid_drop Account and privilege changes (FULL mode)

Mix and match subclasses under event to mirror your threat model.

Event classes, log output, and SQL commands

Filter JSON uses lowercase class and event names (query / start). Log output uses different spellings depending on format:

  • NEW and OLD XML — <NAME> carries a display string (for example Query Start, TableRead, Log).
  • JSON and JSONL — class and event keys use the same lowercase names as filter definitions (for example "class": "query", "event": "start").

The SQL statement type appears separately as COMMAND_CLASS in XML or as sql_command inside event-specific objects in JSON. Filter on statement type with general_sql_command.str (REDUCED general/status records and FULL-mode general events) or sql_command_id (query class in FULL mode). The authoritative list of valid sql_command strings is the com_status_vars array in sql/mysqld.cc in the Percona Server source tree.

audit_log_filter.event_mode controls how many classes are emitted:

  • REDUCED (default from 8.4.9-9) — connection, general/status, table_access, and message only. DDL, DCL, TCL, and most administrative statements appear as a single general/status record (XML <NAME>Query</NAME>) or as table_access when the server reports table touches.
  • FULL — adds command, parse, query, stored_program, global_variable, and authentication. A single client statement can produce a sequence of ten or more records.

Lifecycle records (audit / server_startup / server_shutdown) may appear in the log at startup or shutdown but are not valid filter-definition targets. See Audit Log Filter definition fields.

SQL command values by category

The following tables list common COMMAND_CLASS / sql_command values grouped by SQL category. The list is representative, not exhaustive. For the complete set, see com_status_vars in sql/mysqld.cc.

DDL (Data Definition Language)

sql_command Example statement
create_db CREATE DATABASE test1
drop_db DROP DATABASE test1
alter_db ALTER DATABASE …
create_table CREATE TABLE t1 (…)
alter_table ALTER TABLE t1 …
drop_table DROP TABLE t1
truncate TRUNCATE TABLE t1
rename_table RENAME TABLE …
create_index CREATE INDEX …
drop_index DROP INDEX …
create_view CREATE VIEW …
drop_view DROP VIEW …
create_trigger CREATE TRIGGER …
drop_trigger DROP TRIGGER …
create_procedure CREATE PROCEDURE …
drop_procedure DROP PROCEDURE …
create_function CREATE FUNCTION …
drop_function DROP FUNCTION …
create_event CREATE EVENT …
drop_event DROP EVENT …
alter_tablespace CREATE TABLESPACE …, DROP TABLESPACE …, ALTER TABLESPACE …
create_compression_dictionary CREATE COMPRESSION_DICTIONARY …
drop_compression_dictionary DROP COMPRESSION_DICTIONARY …

DML (Data Manipulation Language)

sql_command Example statement
select SELECT …, SELECT DATABASE()
insert INSERT INTO …
insert_select INSERT INTO … SELECT …
update UPDATE …
delete DELETE FROM …
replace REPLACE INTO …
load LOAD DATA …
call_procedure CALL proc()
do DO …
ha_open, ha_read, ha_close HANDLER …

DCL (Data Control Language)

sql_command Example statement
grant GRANT …
grant_roles GRANT role …
revoke REVOKE …
revoke_all REVOKE ALL …
revoke_roles REVOKE role …
create_user CREATE USER …
drop_user DROP USER …
alter_user ALTER USER …
rename_user RENAME USER …
create_role CREATE ROLE …
drop_role DROP ROLE …
set_password SET PASSWORD …
set_role SET ROLE …

TCL (Transaction Control Language)

sql_command Example statement
begin START TRANSACTION, BEGIN
commit COMMIT
rollback ROLLBACK
savepoint SAVEPOINT …
release_savepoint RELEASE SAVEPOINT …
rollback_to_savepoint ROLLBACK TO SAVEPOINT …
xa_start XA START …
xa_end XA END …
xa_prepare XA PREPARE …
xa_commit XA COMMIT …
xa_rollback XA ROLLBACK …

Administration, replication, and session

sql_command Example statement
change_db USE db_name (also logged as client command Init DB in FULL mode)
set_option SET GLOBAL …, SET SESSION …
show_databases SHOW DATABASES
show_tables SHOW TABLES
show_fields DESCRIBE tbl, DESC tbl, SHOW COLUMNS …, SHOW FIELDS …
show_variables SHOW VARIABLES
show_status SHOW STATUS
show_grants SHOW GRANTS
flush FLUSH …
reset RESET …
kill KILL …
shutdown SHUTDOWN
restart RESTART
prepare_sql PREPARE …
execute_sql EXECUTE …
dealloc_sql DEALLOCATE PREPARE …
lock_tables LOCK TABLES …
unlock_tables UNLOCK TABLES
install_plugin INSTALL PLUGIN …
uninstall_plugin UNINSTALL PLUGIN …
install_component INSTALL COMPONENT …
uninstall_component UNINSTALL COMPONENT …
replica_start START REPLICA
replica_stop STOP REPLICA
binlog BINLOG …
purge PURGE BINARY LOGS …
optimize OPTIMIZE TABLE …
analyze ANALYZE TABLE …
check CHECK TABLE …
repair REPAIR TABLE …

Client-level COM names (not sql_command) appear on command class records in FULL mode. Common values include Query, Init DB (for USE), Ping, and Quit.

Typical event sequences

The tables below show which classes and events fire for common operations. Sequences assume audit_log_filter.event_mode=FULL and a filter that logs everything ({"filter": {"log": true}}). With REDUCED mode, only the rows marked REDUCED apply; other rows are skipped at runtime.

Column key:

  • Filter class / event — names to use in filter JSON.
  • Log <NAME> — value in NEW XML <NAME> (JSON event uses the filter event name).
  • sql_command — value in COMMAND_CLASS or general_sql_command.str when present.

CREATE DATABASE test1

Order Filter class / event Log <NAME> (NEW XML) sql_command or COM
1 command / start Command Start Query
2 parse / preparse Preparse Parse
3 parse / postparse Postparse Parse
4 general / log Log General
5 query / start Query Start create_db
6 query / status_end Query Status End create_db
7 general / result Result General
8 general / status Status or Query create_db (REDUCED: this is the only query-phase record)
9 command / end Command End Query

USE test1 followed by SELECT DATABASE()

USE and SELECT are separate client commands. Each command produces its own command / start … command / end wrapper.

USE test1

Order Filter class / event Log <NAME> sql_command or COM
1 command / start Command Start Init DB
2 general / log Log General
3 general / result Result General
4 general / status Status or Query change_db (REDUCED)
5 command / end Command End Init DB

SELECT DATABASE()

Order Filter class / event Log <NAME> sql_command or COM
1 command / start Command Start Query
2 parse / preparse Preparse Parse
3 parse / postparse Postparse Parse
4 general / log Log General
5 query / start Query Start select
6 query / status_end Query Status End select
7 general / result Result General
8 general / status Status or Query select (REDUCED)
9 command / end Command End Query

CREATE TABLE sbtest1 (…)

Same pattern as CREATE DATABASE, with create_table as the sql_command on query / start and query / status_end. table_access records are not emitted for DDL that does not read or write table data through the table-access API.

DROP TABLE sbtest1

Same FULL-mode sequence as CREATE TABLE, with drop_table as the sql_command.

ALTER TABLE …

Same FULL-mode sequence as CREATE DATABASE, with alter_table as the sql_command on query / start and query / status_end.

CREATE INDEX / DROP INDEX

Same FULL-mode sequence as CREATE TABLE. Index DDL uses create_index or drop_index as the sql_command:

Order Filter class / event Log <NAME> sql_command or COM
1 query / start Query Start create_index or drop_index
2 query / status_end Query Status End same as row 1

DESCRIBE tbl / DESC tbl

DESCRIBE and DESC are logged as show_fields, not a separate describe command name:

Order Filter class / event Log <NAME> sql_command or COM
1 query / start Query Start show_fields
2 query / status_end Query Status End show_fields

CREATE TABLESPACE / DROP TABLESPACE

Both statements use alter_tablespace as the sql_command on the query records. The command, parse, and general wrapper matches other COM Query statements.

Order Filter class / event Log <NAME> sql_command or COM
1 query / start Query Start alter_tablespace
2 query / status_end Query Status End alter_tablespace

CREATE COMPRESSION_DICTIONARY / DROP COMPRESSION_DICTIONARY

Order Filter class / event Log <NAME> sql_command or COM
1 query / start Query Start create_compression_dictionary or drop_compression_dictionary
2 query / status_end Query Status End same as row 1

DROP DATABASE test2

Same FULL-mode sequence as CREATE DATABASE, with drop_db as the sql_command. After the database is dropped, the server may run an implicit SELECT DATABASE(); that statement produces a second command wrapper with select as the sql_command.

SAVEPOINT save1

TCL statements routed through COM Query produce the same command, parse, and general wrapper as CREATE DATABASE. The query phase carries the TCL-specific sql_command:

Order Filter class / event Log <NAME> sql_command or COM
1 query / start Query Start savepoint
2 query / status_end Query Status End savepoint

In REDUCED mode, only general / status with general_sql_command.str = savepoint is emitted.

RELEASE SAVEPOINT and ROLLBACK TO SAVEPOINT use release_savepoint and rollback_to_savepoint respectively on the query records.

ROLLBACK

Same wrapper as other COM Query statements. The query phase uses rollback as the sql_command:

Order Filter class / event Log <NAME> sql_command or COM
1 query / start Query Start rollback
2 query / status_end Query Status End rollback

In REDUCED mode, only general / status with general_sql_command.str = rollback is emitted.

START TRANSACTION

Order Filter class / event Log <NAME> sql_command or COM
1 query / start Query Start begin
2 query / status_end Query Status End begin

COMMIT

Order Filter class / event Log <NAME> sql_command or COM
1 query / start Query Start commit
2 query / status_end Query Status End commit

SET @@SESSION.…

Session and global variable assignments use set_option on the query records. In FULL mode, global_variable / set records may also appear for some SET forms.

Order Filter class / event Log <NAME> sql_command or COM
1 query / start Query Start set_option
2 query / status_end Query Status End set_option

OPTIMIZE TABLE

Order Filter class / event Log <NAME> sql_command or COM
1 query / start Query Start optimize
2 query / status_end Query Status End optimize

ANALYZE TABLE, CHECK TABLE, and REPAIR TABLE use analyze, check, and repair respectively.

CREATE USER

Order Filter class / event Log <NAME> sql_command or COM
1 query / start Query Start create_user
2 query / status_end Query Status End create_user

In FULL mode, authentication / authid_create may also appear.

GRANT

Order Filter class / event Log <NAME> sql_command or COM
1 query / start Query Start grant
2 query / status_end Query Status End grant

The SQLTEXT on Query Status End may show normalized identifier quoting (for example backticks around schema and table names) that differs from the Query Start record.

DROP USER

DROP USER emits an authentication record between the query bracket records:

Order Filter class / event Log <NAME> sql_command or COM
1 query / start Query Start drop_user
2 authentication / authid_drop Auth Authid Drop Authentication
3 query / status_end Query Status End drop_user

INSERT INTO …

DML that touches a table adds a table_access record between the query bracket records. This sequence appears in both FULL and REDUCED modes for the table_access row.

Order Filter class / event Log <NAME> sql_command or COM
1 query / start Query Start insert
2 table_access / insert TableInsert insert (REDUCED)
3 query / status_end Query Status End insert

UPDATE and DELETE follow the same pattern with TableUpdate / update and TableDelete / delete. SELECT that reads tables emits TableRead / read instead.

Unless noted otherwise, the command, parse, and general wrapper for the examples in this section matches CREATE DATABASE.

Server startup

At component initialization the log receives lifecycle records that cannot be targeted in filter JSON:

Log <NAME> (NEW XML) JSON class / event Notes
Audit audit / startup Component startup; includes SERVER_ID, STARTUP_OPTIONS, OS_VERSION, MYSQL_VERSION in XML
Auth Flush authentication / flush Privilege cache flush during startup (FULL mode)

MySQL server_startup tracking events are not written to the audit log by this component.

SHUTDOWN

Order Filter class / event Log <NAME> sql_command or COM
1 query / start Query Start shutdown
2 general / log Log General
3 query / status_end Query Status End shutdown

Component shutdown writes an audit / shutdown record (XML <NAME>NoAudit</NAME>). MySQL server_shutdown tracking events are not written.

Filtering by statement type

To log only DDL against a schema, combine class rules with general_sql_command.str conditions:

{
  "filter": {
    "class": {
      "name": "general",
      "event": {
        "name": "status",
        "log": {
          "or": [
            { "field": { "name": "general_sql_command.str", "value": "create_db" } },
            { "field": { "name": "general_sql_command.str", "value": "drop_db" } },
            { "field": { "name": "general_sql_command.str", "value": "create_table" } },
            { "field": { "name": "general_sql_command.str", "value": "drop_table" } },
            { "field": { "name": "general_sql_command.str", "value": "alter_table" } }
          ]
        }
      }
    }
  }
}

In FULL mode, add parallel rules under the query class using sql_command_id if you need the bracketing query / start and query / status_end records instead of (or in addition to) the general / status summary.

For format-specific field layouts, see Audit Log Filter format - XML (new style) and Audit Log Filter format - JSON and JSONL.

Filter patterns

Inclusive patterns list what to log; exclusive patterns drop noisy work from an otherwise-on stream. Pick one approach per filter — combining them in a single rule set is harder to reason about than two separate filters bound to different account patterns.

Inclusive filters

Inclusive rules list what to log—pair classes with user, database, table, event, status, and similar fields until the stream matches your policy.

When to use inclusive filters

Spell out:

  • Which classes matter

  • Which accounts or objects to watch

  • Which subclasses or outcomes qualify

Typical uses: compliance evidence, privileged-user monitoring, schema-change tracking, incident response.

Tighter filters mean less noise and less I/O—always stage-test rules before production.

Inclusive filter example

Administrators often watch destructive table_access work—this skeleton logs update and delete only:

{
  "filter": {
    "class": [
      {
        "name": "table_access",
        "event": [
          {
            "name": ["update", "delete"],
            "log": {
              "and": [
                { "field": { "name": "table_database.str", "value": "app_db" } },
                { "field": { "name": "table_name.str", "value": "sensitive_tbl" } }
              ]
            }
          }
        ]
      }
    ]
  }
}

The rule matches update and delete table_access events, and the nested log condition narrows logging to events whose table_database.str equals app_db and table_name.str equals sensitive_tbl. read and insert subclasses are not named, so they skip the rule entirely.

  • "class" — begins the class rule block.
  • "name": "table_access" — limits to table-access events (subclasses: read, insert, update, delete).
  • "event" — selects the update and delete subclasses; other subclasses skip this rule.
  • "log" — a per-event condition. true logs every match; a field / and / or / not structure logs only matches that satisfy the condition.
  • "field" — compares an event field against a value. table_database.str and table_name.str are fields carried by every table_access event (see Audit Log Filter definition fields for the full list).

To widen or narrow the rule, change the field values, add or branches for multiple tables, or drop either field check to scope by database only or by table name only.

Log only UPDATE and DELETE on a specific table

To log update and delete against a single table — or a short list of tables inside one database — combine table_database.str with an or over table_name.str values:

{
  "filter": {
    "class": [
      {
        "name": "table_access",
        "event": [
          {
            "name": ["update", "delete"],
            "log": {
              "and": [
                { "field": { "name": "table_database.str", "value": "app_db" } },
                {
                  "or": [
                    { "field": { "name": "table_name.str", "value": "sensitive_tbl" } },
                    { "field": { "name": "table_name.str", "value": "audit_events" } }
                  ]
                }
              ]
            }
          }
        ]
      }
    ]
  }
}

To scope to one database only, omit the inner or and keep the table_database.str check. Note that table_access events do not carry a user field, so filtering by account must be done at assignment time with audit_log_filter_set_user().

Exclusive filters

Exclusive (or negated) rules drop noisy work. There are two idioms:

  • Set "log": true at the filter level, then set "log": false on a specific class or event to suppress just that slice.
  • Use a "not" logical operator inside a log condition to drop matches that satisfy an expression.

Suppress one class while logging everything else

{
  "filter": {
    "log": true,
    "class": { "name": "general", "log": false }
  }
}

This logs every event except general class events. log: true at the top enables logging globally, then the class-level log: false carves out one class.

Invert a field match with not

Wrap a condition in not inside a log item to drop the events that match it. Because inversion tests a field on the current event, use a field the class actually carries (for example, table_database.str on table_access; user.str on connection; general_user.str on general). The example below suppresses table_access events against the internal mysql schema:

{
  "filter": {
    "class": [
      {
        "name": "table_access",
        "log": {
          "not": {
            "field": { "name": "table_database.str", "value": "mysql" }
          }
        }
      }
    ]
  }
}

This logs table_access events except those touching the mysql system schema. For a user-based suppression, apply the not against a field that exists on the class you’re filtering — user.str on connection, general_user.str on general — or scope the filter to specific accounts at assignment time with audit_log_filter_set_user().

Exclusive filter example

This filter logs everything except general events, and within connection events drops connect/disconnect subclasses (keeping only change_user):

{
  "filter": {
    "log": true,
    "class": [
      {
        "name": "connection",
        "event": [
          { "name": "connect", "log": false },
          { "name": "disconnect", "log": false }
        ]
      },
      { "name": "general", "log": false }
    ]
  }
}
  • The top-level "log": true turns everything on by default.
  • The connection block’s event list silences the two noisy subclasses but leaves change_user on.
  • The second class entry turns off the entire general class.

Field-type rules (JSON numbers versus strings, connection_type symbolic constants such as "::tcp/ip", and which types are accepted for each field) are described under audit_log_filter_set_filter() and in Audit Log Filter definition fields.

Advanced filter constructs

The sections so far cover logging on or off at class/event granularity. The filter language also supports per-event conditions, execution blocking, references to server variables and functions, field-value replacement (redaction), and dynamic filter swapping.

Validate exact field names and types against Audit Log Filter definition fields before deploying any of these to production.

Test event field values

Inside an event block, a log item can carry a field comparison. The rule logs the event only when the field equals the given value.

{
  "filter": {
    "class": {
      "name": "general",
      "event": {
        "name": "status",
        "log": {
          "field": { "name": "general_command.str", "value": "Query" }
        }
      }
    }
  }
}

The above logs general/status events only when general_command.str equals Query (dropping Execute, Quit, and Change user). String fields take string values; integer fields (such as status on connection, or general_error_code on general) take JSON numbers.

Fields available on each class are listed in Audit Log Filter definition fields. Examples:

  • connection: status, user.str, host.str, ip.str, database.str, connection_type

  • general: general_error_code, general_user.str, general_command.str, general_query.str, general_sql_command.str, general_host.str, general_ip.str

  • table_access: query.str, table_database.str, table_name.str

Combine conditions with logical operators

and, or, and not let you build compound conditions. They take an array (for and / or) or a single sub-condition (for not) and can be nested arbitrarily.

{
  "filter": {
    "class": {
      "name": "general",
      "event": {
        "name": "status",
        "log": {
          "or": [
            {
              "and": [
                { "field": { "name": "general_command.str",    "value": "Query" } },
                { "field": { "name": "general_command.length", "value": 5 } }
              ]
            },
            {
              "and": [
                { "field": { "name": "general_command.str",    "value": "Execute" } },
                { "field": { "name": "general_command.length", "value": 7 } }
              ]
            }
          ]
        }
      }
    }
  }
}

This logs general/status events whose general_command is either Query (length 5) or Execute (length 7). Use not to invert any sub-expression — for example, { "not": { "field": { "name": "user.str", "value": "healthcheck" } } } matches everything except the healthcheck account (on a class that actually carries user.str).

Block execution with abort

An event block can carry an abort item that prevents matching statements from executing. The same block cannot carry both log and abort; use separate event blocks when you need both behaviors on the same class. Because blocking is a policy-enforcement capability — not observability — it is covered on its own page: Block statements with an audit log filter.

Reference predefined variables

A log or abort condition can test a predefined variable with a variable item. The condition is true when the variable equals the given value.

{
  "filter": {
    "class": {
      "name": "general",
      "event": {
        "name": "status",
        "log": {
          "variable": {
            "name": "audit_log_connection_policy_value",
            "value": "::none"
          }
        }
      }
    }
  }
}

Predefined variables mirror the legacy-mode audit_log_*_policy system variables so operators can re-tune an active filter by changing a server variable rather than rewriting the JSON:

  • audit_log_connection_policy_value — 0/"::none", 1/"::errors", 2/"::all".
  • audit_log_policy_value — 0/"::none", 1/"::logins", 2/"::all", 3/"::queries".
  • audit_log_statement_policy_value — 0/"::none", 1/"::errors", 2/"::all".

Symbolic constants ("::none", "::all", …) are case-sensitive strings and are interchangeable with the numeric form.

Reference predefined functions

Use a function item to call a built-in inside a log, abort, print, or replace condition. The name field holds the function name without parentheses. The args field is an array of arguments. Omit args when the function takes none.

The Percona Server audit log filter component implements two predefined functions: string_find and query_digest. Other names from the upstream MySQL Enterprise Audit reference, such as find_in_include_list, find_in_exclude_list, and debug_sleep, are not recognized in this component. Calls to those names fail validation.

The following filter logs general/status events only when the executed statement contains the substring password:

{
  "filter": {
    "class": {
      "name": "general",
      "event": {
        "name": "status",
        "log": {
          "function": {
            "name": "string_find",
            "args": [
              { "string": { "field": "general_query.str" } },
              { "string": { "string": "password" } }
            ]
          }
        }
      }
    }
  }
}

Each element of the args array is a typed wrapper object with exactly one member. The member name is the argument type — string is the only argument type currently used. The member value is an object with exactly one member whose name identifies the value source. Two sources are accepted:

  • field — read the named event field at evaluation time, for example { "string": { "field": "general_query.str" } }.

  • string — supply a literal string, for example { "string": { "string": "password" } }.

Bare-string array elements (for example "password") and bare-field elements ({ "field": "general_query.str" } without the outer string wrapper) are rejected by filter validation.

Available functions:

  • string_find(text, substr) returns true when substr appears inside text. The comparison is case-sensitive. Both arguments are strings. Wrap each argument in the typed-wrapper format shown in the preceding example. This function helps match keywords inside general_query.str or query.str without forcing a digest comparison.

  • query_digest([str]) has two shapes that depend on the argument:

    • No argument: the function returns the normalized statement digest of the current event’s SQL text. Use this shape inside a replace clause to substitute literal SQL with the digest. For details, see Redact audit log fields.

    • One string argument: the function returns true when the supplied digest equals the current event’s digest. Use this shape inside a log, abort, or print condition. Wrap the call in not to invert the match. Use this form to act on specific statement patterns. For example, block any statement whose digest matches SELECT ?.

    Starting in Percona Server for MySQL 8.4.9-9, query_digest accepts a string shorthand for its single argument: "args": "SELECT ?". The shorthand is equivalent to the verbose form "args": [{ "string": { "string": "SELECT ?" } }]. Earlier releases require the verbose form. The shorthand is specific to query_digest — string_find requires the verbose form for both arguments.

A condition that calls a function evaluates to true when the function returns a truthy value. Truthy values include non-empty strings and Boolean true. Combine function calls with and, or, and not to build richer rules. For example, log only when string_find matches and the user is not on an allow list.

Replace event field values (redaction)

A print / replace item inside a class or event block rewrites statement text on one of four class/field pairs (general/general_query.str, table_access/query.str, query/query.str, parse/query.str; the last two require FULL mode) as a query_digest before the event is logged, which keeps literal values out of the audit stream. This is a compliance/PII capability covered on its own page: Redact audit log fields.

Replace a filter dynamically

A filter can swap itself for a different rule set mid-session. Nest a filter block inside an event and give the outer filter an id; use activate to trigger the swap and ref to point back at the original.

{
  "filter": {
    "id": "main",
    "class": {
      "name": "table_access",
      "event": {
        "name": ["update", "delete"],
        "log": false,
        "filter": {
          "class": {
            "name": "general",
            "event": { "name": "status",
                       "filter": { "ref": "main" } }
          },
          "activate": {
            "or": [
              { "field": { "name": "table_name.str", "value": "temp_1" } },
              { "field": { "name": "table_name.str", "value": "temp_2" } }
            ]
          }
        }
      }
    }
  }
}

How it behaves:

  • main waits for table_access update or delete events. log: false means those events are not logged directly.
  • When one of those events touches temp_1 or temp_2, the inner filter activates.
  • The inner filter waits for the next general/status event (typically end of statement), logs it, then the nested ref: main restores the outer filter.

Net effect: you log one general/status entry per statement that touched temp_1 or temp_2, instead of many table_access rows. A single UPDATE temp_1, temp_3 SET ... emits one log entry rather than one per row touched.

activate is only valid inside a subfilter — using it on the top-level filter raises an error. id values are scoped to the filter definition only; they are unrelated to the audit_log_filter_id system variable.

Best practices

  1. Start wide, then narrow — begin with a noisy catch-all in staging, then peel away classes you do not need.
  2. Test combinations — overlapping rules and assignments surprise people; validate on a clone.
  3. Plan retention — pair filters with rotation, pruning, and disk budgets (Manage the Audit Log Filter files).
  4. Watch overhead — granular auditing costs CPU and I/O; ramp detail gradually and watch latency plus audit_log_filter_* status counters.

Worked example: financial tracking filter

This page authors the JSON. To deploy a finished filter, store it with audit_log_filter_set_filter(), bind it to accounts with audit_log_filter_set_user(), and reload sessions with audit_log_filter_flush(). The Audit Log Filter quickstart and Install the audit log filter cover the deploy sequence.

The following worked example threads JSON authoring, deployment, and verification end-to-end.

This filter logs DML on two tables in financial_db and all connection events. Account scoping is applied at assignment time, since table_access events do not carry a user field.

-- Create the filter
SELECT audit_log_filter_set_filter('financial_tracking', '{
  "filter": {
    "class": [
      {
        "name": "table_access",
        "event": [
          {
            "name": ["insert", "update", "delete"],
            "log": {
              "and": [
                { "field": { "name": "table_database.str", "value": "financial_db" } },
                {
                  "or": [
                    { "field": { "name": "table_name.str", "value": "accounts" } },
                    { "field": { "name": "table_name.str", "value": "transactions" } }
                  ]
                }
              ]
            }
          }
        ]
      },
      {
        "name": "connection",
        "event": [
          { "name": "connect" },
          { "name": "disconnect" }
        ]
      }
    ]
  }
}');

Field-value typing (JSON numbers for integer fields such as status, strings for *.str fields, and "::tcp/ip"-style symbolic constants for connection_type) is covered under audit_log_filter_set_filter().

Assign the filter to just the accounts that should be audited instead of to %:

-- Assign to specific accounts; other users keep their existing filter (or none).
SELECT audit_log_filter_set_user('admin@%',        'financial_tracking');
SELECT audit_log_filter_set_user('finance_team@%', 'financial_tracking');

The filter records:

  • insert / update / delete on financial_db.accounts and financial_db.transactions for the two assigned accounts.

  • connect / disconnect events for the same accounts.

  • Nothing for other users (they are not bound to this filter) and nothing outside the declared schema/tables unless you extend the JSON.

Inspect metadata directly:

-- Check created filters
SELECT * FROM mysql.audit_log_filter;

-- Check user assignments
SELECT * FROM mysql.audit_log_user;

Tail the audit file (default under the data directory) to confirm events stream as expected.

Additional reading

Block statements with an audit log filter

The Audit Log Filter can do more than observe. An abort item inside an event block prevents matching statements from executing. Blocked statements are recorded in the server error log (not the audit log), and the client sees the following error:

ERROR 1045 (28000): Statement was aborted by an audit log filter

Use blocking to enforce policy at the SQL layer — for example, to stop writes against a sensitive table from any application path, independently of GRANTs.

For background on how filters are authored, see Write audit_log_filter definitions. This page assumes you already have a working filter JSON and want to add blocking.

Warning

You can lock yourself out. If an abort rule matches a statement your own account runs, the statement fails. The AUDIT_ABORT_EXEMPT privilege bypasses abort conditions and lets you recover from a misconfiguration. The component grants AUDIT_ABORT_EXEMPT automatically to SYSTEM_USER accounts. Keep at least one such account available.

Syntax

abort lives inside an event block and can be:

  • A Boolean — true aborts every event the event block matches; false is a no-op.
  • A condition — built from field, and, or, not, variable, or function items (same grammar as a log condition). The event is aborted when the condition evaluates to true.
"event": {
  "name": [ "qualifying subclass names" ],
  "abort": condition
}

For the field grammar used in conditions, see Test event field values and Combine conditions with logical operators on the authoring page.

Example 1: Block all DML on any table

This filter blocks insert, update, and delete statements against every table:

{
  "filter": {
    "class": {
      "name": "table_access",
      "event": {
        "name": ["insert", "update", "delete"],
        "abort": true
      }
    }
  }
}

Use this kind of rule on a standby or read-only replica to enforce immutability at the SQL surface, then assign it with audit_log_filter_set_user().

Example 2: Block DML on a specific table

Restrict the block to a single table by comparing table_database.str and table_name.str:

{
  "filter": {
    "class": {
      "name": "table_access",
      "event": {
        "name": ["insert", "update", "delete"],
        "abort": {
          "and": [
            { "field": { "name": "table_database.str", "value": "finances" } },
            { "field": { "name": "table_name.str", "value": "bank_account" } }
          ]
        }
      }
    }
  }
}

Every attempt to write to finances.bank_account — regardless of which application issued it — fails with the audit abort error. The server error log records the block; the audit log file does not.

Example 3: Block a list of tables

Use or to enumerate several target tables inside the same and:

{
  "filter": {
    "class": {
      "name": "table_access",
      "event": {
        "name": ["insert", "update", "delete"],
        "abort": {
          "and": [
            { "field": { "name": "table_database.str", "value": "finances" } },
            {
              "or": [
                { "field": { "name": "table_name.str", "value": "bank_account" } },
                { "field": { "name": "table_name.str", "value": "ledger" } },
                { "field": { "name": "table_name.str", "value": "transactions" } }
              ]
            }
          ]
        }
      }
    }
  }
}

Combine blocking with logging

Blocking and logging are independent. The same event block cannot carry both log and abort; use separate event blocks when you need both behaviors on the same class. The example below logs read access to bank_account and blocks insert, update, and delete against it:

{
  "filter": {
    "class": {
      "name": "table_access",
      "event": [
        {
          "name": "read",
          "log": {
            "field": { "name": "table_name.str", "value": "bank_account" }
          }
        },
        {
          "name": ["insert", "update", "delete"],
          "abort": {
            "field": { "name": "table_name.str", "value": "bank_account" }
          }
        }
      ]
    }
  }
}

Recovery

If an abort rule prevents administrators from running essential statements:

  1. Connect with an account that holds AUDIT_ABORT_EXEMPT (typically a SYSTEM_USER account).
  2. Replace the filter with a safer definition using audit_log_filter_set_filter().
  3. Reassign accounts with audit_log_filter_set_user() if needed.

Test abort rules on a clone before applying them to production.

See also

Redact audit log fields

Audit events that carry SQL statement text can leak sensitive values into the audit log. Examples include credentials, personal data, and secret literals. The Audit Log Filter can rewrite that text as a statement digest before the event is written. The log then records the statement shape without capturing the literal values.

This page covers the print / replace mechanism. For base filter authoring, see Write audit_log_filter definitions.

What can be replaced

Four class and field pairs support replacement. They use two distinct field names (general_query.str and query.str), and only the query_digest function may supply the replacement value:

Event class Replaceable field Notes
general general_query.str
table_access query.str
query query.str Requires audit_log_filter.event_mode = FULL
parse query.str Requires audit_log_filter.event_mode = FULL

Replacement happens during filtering. The choice of literal text or digest applies regardless of which log format the writer produces later, including XML, JSON, and JSONL.

Shape

A print item goes inside a class or event block:

"print": {
  "field": {
    "name": "field_name",
    "print": condition,
    "replace": replacement_value
  }
}

  • name — the replaceable field. Use a name from the preceding table.

  • print — a condition. When the condition evaluates to true, the field is kept. When the condition evaluates to false, the field is replaced. Set "print": false to replace unconditionally.

  • replace — the replacement value, specified as a function item. The component permits only query_digest with no arguments.

The conditional form lets you mix redacted and literal statements in the same filter.

Examples

Redact every general event

Replace statement text in every general event with its digest:

{
  "filter": {
    "class": {
      "name": "general",
      "print": {
        "field": {
          "name": "general_query.str",
          "print": false,
          "replace": {
            "function": { "name": "query_digest" }
          }
        }
      }
    }
  }
}

Redact both statement-carrying classes

general and table_access both carry statement text. Combine them into one filter:

{
  "filter": {
    "class": [
      {
        "name": "general",
        "print": {
          "field": {
            "name": "general_query.str",
            "print": false,
            "replace": { "function": { "name": "query_digest" } }
          }
        }
      },
      {
        "name": "table_access",
        "print": {
          "field": {
            "name": "query.str",
            "print": false,
            "replace": { "function": { "name": "query_digest" } }
          }
        }
      }
    ]
  }
}

The resulting audit stream contains only digests — no literal SQL text — which is a common baseline for PCI/PII environments.

Redact only specific events

Scope replacement to a subset of events by nesting print inside an event block. This filter redacts query.str on insert and update table_access events but leaves read and delete alone:

{
  "filter": {
    "class": {
      "name": "table_access",
      "event": {
        "name": ["insert", "update"],
        "print": {
          "field": {
            "name": "query.str",
            "print": false,
            "replace": { "function": { "name": "query_digest" } }
          }
        }
      }
    }
  }
}

Redact account-management statements

query_digest with an argument is a Boolean comparator — useful in a log condition to decide whether to log an event based on its digest. Combined with field checks, you can log (and redact) only specific statement types. This filter fires on general/status events for account-management DDL and replaces the literal statement with its digest:

{
  "filter": {
    "class": {
      "name": "general",
      "event": {
        "name": "status",
        "print": {
          "field": {
            "name": "general_query.str",
            "print": false,
            "replace": {
              "function": { "name": "query_digest" }
            }
          }
        },
        "log": {
          "or": [
            { "field": { "name": "general_sql_command.str", "value": "alter_user" } },
            { "field": { "name": "general_sql_command.str", "value": "alter_user_default_role" } },
            { "field": { "name": "general_sql_command.str", "value": "create_role" } },
            { "field": { "name": "general_sql_command.str", "value": "create_user" } }
          ]
        }
      }
    }
  }
}

For the full set of general_sql_command.str values, see Test event field values.

Conditional redaction

To keep literal text for most statements and redact only specific ones, or to do the reverse, use query_digest as a comparator inside print. With an argument, the function returns true when the current statement digest equals the argument.

Keep the literal text when the digest matches SELECT ?; replace otherwise:

"print": {
  "field": {
    "name": "general_query.str",
    "print": {
      "function": {
        "name": "query_digest",
        "args": "SELECT ?"
      }
    },
    "replace": {
      "function": { "name": "query_digest" }
    }
  }
}

Invert with not — redact only the matching statements, keep literal text for everything else:

"print": {
  "field": {
    "name": "general_query.str",
    "print": {
      "not": {
        "function": {
          "name": "query_digest",
          "args": "SELECT ?"
        }
      }
    },
    "replace": {
      "function": { "name": "query_digest" }
    }
  }
}

Additional reading

Audit Log Filter definition fields

Canonical class, event, and field names for audit_log_filter_set_filter() validation—aligned with Percona Server source filter_definition_fields.md.

For SQL command values (create_db, select, grant, and so on), typical event sequences for DDL/DCL/TCL operations, and the mapping between filter names and log output, see Event classes, log output, and SQL commands.

audit_log_filter.event_mode decides which sections apply. The following notes contrast REDUCED and FULL.

Notes

  • Names here are filter-definition names—they may differ from JSON log output keys.

  • Field type reflects the current server validator (get_event_field_value_type()).

  • Some numeric-looking fields validate as string when the server leaves them untyped.

  • Only documented class names pass validation.

  • When audit_log_filter.event_mode is REDUCED (the default from 8.4.9-9 onward), only these events are tracked and accepted by filter-definition validation:

    • general: status

    • connection: connect, disconnect, change_user

    • table_access: read, insert, update, delete

    • message: internal, user

    In REDUCED mode, class names that exist only for extended auditing are rejected entirely. Those classes are global_variable, command, query, stored_program, authentication, and parse. Subclass names not in the preceding list are also rejected during filter validation. Examples of rejected subclasses include general/log and connection/pre_authenticate. At runtime, events outside the REDUCED set are silently skipped. With FULL, those six classes and their subclasses are valid filter targets.

  • Lifecycle-related records with class names audit, server_startup, and server_shutdown are not valid filter-definition targets. Startup and shutdown lifecycle events are ignored by the audit log filter if they are received.

  • For connection.connection_type, the validator accepts numeric values 0–5 and the pseudo-constants ::undefined, ::tcp/ip, ::socket, ::named_pipe, ::ssl, and ::shared_memory.

general

Supported events: log, error, result, status
REDUCED mode: only status

Field name Field type Description
general_error_code integer Event error code.
general_thread_id unsigned integer Event thread ID. Acts as an alias of general_connection_id.
general_connection_id unsigned integer Event connection ID.
general_user.str string User name recorded for the general event.
general_user.length unsigned integer User name length.
general_command.str string General command text, for example Query.
general_command.length unsigned integer General command text length.
general_query.str string SQL statement text associated with the event.
general_query.length unsigned integer SQL statement text length.
general_host.str string Client host name.
general_host.length unsigned integer Client host name length.
general_sql_command.str string SQL command name associated with the statement, for example select.
general_sql_command.length unsigned integer SQL command name length.
general_external_user.str string External user or OS login associated with the event.
general_external_user.length unsigned integer External user or OS login length.
general_ip.str string Client IP address.
general_ip.length unsigned integer Client IP address length.

connection

Supported events: connect, disconnect, change_user, pre_authenticate
REDUCED mode: connect, disconnect, change_user

Field name Field type Description
status integer Current connection event status.
connection_id unsigned integer Connection ID.
user.str string User name of this connection.
user.length unsigned integer User name length.
priv_user.str string Privileged user name.
priv_user.length unsigned integer Privileged user name length.
external_user.str string External user name or OS login.
external_user.length unsigned integer External user name length.
proxy_user.str string Proxy user used for the connection.
proxy_user.length unsigned integer Proxy user name length.
host.str string Connection host name.
host.length unsigned integer Connection host name length.
ip.str string Connection IP address.
ip.length unsigned integer Connection IP address length.
database.str string Default database specified at connection time.
database.length unsigned integer Default database name length.
connection_type integer Connection type code. Use a numeric value from 0 to 5 or one of the pseudo-constants in the following table.

connection_type values:

Value Meaning
0 or ::undefined Undefined
1 or ::tcp/ip TCP/IP
2 or ::socket Socket
3 or ::named_pipe Named pipe
4 or ::ssl TCP/IP with encryption
5 or ::shared_memory Shared memory

table_access

Supported events: read, insert, update, delete
REDUCED mode: all events

Field name Field type Description
connection_id unsigned integer Event connection ID.
sql_command_id integer SQL command ID.
query.str string SQL statement text.
query.length unsigned integer SQL statement text length.
table_database.str string Database name associated with event.
table_database.length unsigned integer Database name length.
table_name.str string Table name associated with event.
table_name.length unsigned integer Table name length.

global_variable (FULL mode only)

Supported events: get, set

Field name Field type Description
connection_id string Event connection ID.
variable_name.str string Variable name.
variable_name.length string Variable name length.
variable_value.str string Variable value.
variable_value.length string Variable value length.

command (FULL mode only)

Supported events: start, end

Field name Field type Description
status string Command event status code.
connection_id string Event connection ID.
command.str string Command text.
command.length string Command text length.

query (FULL mode only)

Supported events: start, nested_start, status_end, nested_status_end

Field name Field type Description
status string Query event status code.
connection_id string Event connection ID.
sql_command_id string SQL command string associated with the query event. The field name is retained as sql_command_id for compatibility.
query.str string SQL query text.
query.length string SQL query text length.
query_charset string SQL query character set name.

stored_program (FULL mode only)

Supported events: execute

Field name Field type Description
connection_id string Event connection ID.
database.str string Database where the stored program is defined.
database.length string Database name length.
name.str string Stored program name.
name.length string Stored program name length.

authentication (FULL mode only)

Supported events: flush, authid_create, credential_change, authid_rename, authid_drop

Field name Field type Description
status string Authentication event status.
connection_id string Event connection ID.
user.str string User name.
user.length string User name length.
host.str string Host name.
host.length string Host name length.

message

Supported events: internal, user
REDUCED mode: all events

Field name Field type Description
connection_id string Event connection ID.
component.str string Component name.
component.length string Component name length.
producer.str string Message producer name.
producer.length string Message producer name length.
message.str string Message text.
message.length string Message text length.

parse (FULL mode only)

Supported events: preparse, postparse

Field name Field type Description
connection_id string Event connection ID.
flags string Parse rewrite flags value.
query.str string Original SQL query text.
query.length string Original SQL query text length.
rewritten_query.str string Rewritten SQL query text.
rewritten_query.length string Rewritten SQL query text length.

Additional reading

Operate

Audit Log Filter security

The Audit Log Filter component writes audit files. Restrict the log directory to trusted operators and ensure the server can write there.

Logs are plaintext by default and may hold credentials, SQL text, and other sensitive data.

The default file under the data directory is audit_filter.log. Override location with audit_log_filter.file at startup.

Starting in Percona Server for MySQL 8.4.9-9, if the parent directory is missing the component errors and the server starts without Audit Log Filter active.

Rotation leaves multiple files on disk—protect every generation.

Additional reading

Audit Log Filter compression and encryption

Compression

Enable compression for any format with audit_log_filter.compression at server startup.

Allowed values:

  • NONE — default, no compression
  • GZIP — GNU zip compression

With both compression and encryption enabled, the component compresses first, then encrypts. To recover a file manually, decrypt, then decompress.

Encryption

Encrypt any audit log format. The component generates the first password; you may rotate to custom passwords afterward. Passwords live in the keyring—enable a keyring first.

Set audit_log_filter.encryption at startup. Values:

  • NONE — default, no encryption
  • AES — AES-256-CBC

AES uses a 256-bit key.

Encryption-related functions:

Function name Description
audit_log_encryption_password_set() Stores a password in the keyring. With encryption on, also rotates the log: renames the current file and starts a new file encrypted with the new password.
audit_log_encryption_password_get() With no argument, returns the active password. With a keyring ID, returns that archived or current password by ID.

audit_log_filter.password_history_keep_days controls how long archived passwords stay available. When non-zero, calling audit_log_encryption_password_set() can expire older keyring entries.

On startup with encryption enabled, if no audit password exists the component generates one and stores it. Call audit_log_encryption_password_get() to inspect it.

Manually uncompressing and decrypting audit log filter files

Decrypt with OpenSSL, for example:

openssl enc -d -aes-256-cbc -pass pass:password
    -iter iterations -md sha256
    -in audit.timestamp.log.pwd_id.enc
    -out audit.timestamp.log

You need the password and iteration count from audit_log_encryption_password_get().

That function returns JSON, for example for file audit.20190415T151322.log.20190414T223342-2.enc with keyring ID audit-log-20190414T223342-2:

SELECT audit_log_encryption_password_get('audit-log-20190414T223342-2');
Expected output 
{"password":"{randomly-generated-alphanumeric-string}","iterations":568977}

Additional reading

Reading Audit Log Filter files

Audit Log Filter exposes a SQL API to read audit files in JSON or JSONL only. Layout and the JSONL option are covered in Audit Log Filter format - JSON and JSONL and Audit Log Filter file format overview. Set audit_log_filter.format accordingly; audit_log_filter.file defines the path, base name, and suffix used to locate files.

If a file no longer matches that pattern, readers ignore it.

Reader functions

Two functions read JSON or JSONL audit files:

Read commands

A session holds at most one active read context. Pick one of the following commands to open, advance, or close it. For the full argument reference, see audit_log_read().

Resume from a bookmark

Start a read at the position returned by audit_log_read_bookmark():

SELECT audit_log_read(audit_log_read_bookmark());

Start at a timestamp

Start a read at an explicit timestamp. When the timestamp omits a time part, the component assumes 00:00:00:

SELECT audit_log_read('{"start": {"timestamp": "2026-05-20 12:28:10"}}');
SELECT audit_log_read('{"start": {"timestamp": "2026-05-20"}}');

Address one specific event

Pass a bookmark literal with timestamp and id and no start envelope:

SELECT audit_log_read('{"timestamp": "2026-05-20 12:28:10", "id": 1561422}');

Limit the events per call

Cap how many events a single call returns by adding max_array_length to any positioning form:

SELECT audit_log_read('{"start": {"timestamp": "2026-05-20 12:28:10"}, "max_array_length": 3}');

Continue from the current cursor

After a read sequence is open, continue advancing without supplying a new position:

SELECT audit_log_read();

Close the active sequence

Release the reader cursor before opening a new sequence at a different position:

SELECT audit_log_read('null');

A read sequence also ends when the session ends. A single call cannot combine the start envelope with a top-level timestamp or id. To reposition while a sequence is active, close it first with 'null'.

Additional reading

Filter the Audit Log Filter logs

Rule-based filtering includes or excludes events using these attributes:

  • User account
  • Audit event class
  • Audit event subclass
  • Audit event fields (for example, COMMAND_CLASS or STATUS)

Which classes and subclasses you can target depends on audit_log_filter.event_mode. Validated names appear in Audit Log Filter definition fields.

Define multiple filters and assign any filter to multiple accounts, or register a default filter for accounts without a specific row. You define filters through SQL function calls.

After you define a filter, the server persists it in mysql system tables.

Audit Log Filter functions

The six management UDFs in the following table — audit_log_filter_flush(), audit_log_filter_set_filter(), audit_log_filter_remove_filter(), audit_log_filter_set_user(), audit_log_filter_remove_user(), and audit_log_rotate() — require the AUDIT_ADMIN privilege.

The reader UDFs (audit_log_read(), audit_log_read_bookmark()), the session-id helper (audit_log_session_filter_id()), and the keyring helpers (audit_log_encryption_password_get(), audit_log_encryption_password_set()) do not require AUDIT_ADMIN. The keyring helpers require an initialized keyring component or plugin.

These functions drive rule-based filtering:

Function Description Example
audit_log_filter_flush() Flush filter tables and reload definitions into the component SELECT audit_log_filter_flush();
audit_log_filter_set_filter() Create or replace a named filter SELECT audit_log_filter_set_filter('log_connections', '{ "filter": {} }');
audit_log_filter_remove_filter() Drop a named filter SELECT audit_log_filter_remove_filter('filter-name');
audit_log_filter_set_user() Bind a filter to a user account SELECT audit_log_filter_set_user('user-name@localhost', 'filter-name');
audit_log_filter_remove_user() Clear filter bindings for a user account SELECT audit_log_filter_remove_user('user-name@localhost');

Through SQL, you define, inspect, and change audit log filters; definitions live in the mysql system database.

audit_log_session_filter_id() returns the active audit log filter ID for the current session.

Filter definitions are JSON values.

Reloading rules from tables, persistence after audit_log_filter_set_filter(), and post-flush session behavior are covered under audit_log_filter_flush() and Persistence and refreshing in Audit log filter functions, options, and variables.

Filter modification lifecycle

The following diagram shows how filter changes persist, reload into the component, and reach sessions, including when audit_log_filter_flush() runs.

Audit Log Filter modification lifecycle

Constraints

Enable the component_audit_log_filter component and ensure audit tables exist before calling audit log filter functions. The account must hold the required privileges.

Filter definition validation

Introduced in Percona Server for MySQL 8.4.9-9.

The server validates filter definitions at parse time and rejects invalid input with a clear error:

  • Unknown field names (e.g., "WRONG.str")

  • Invalid class or event subclass names

  • Empty arrays (e.g., "class": [])

  • Unknown JSON keys (e.g., "classes" instead of "class", "events" instead of "event", "names" instead of "name", "logs" instead of "log")

  • print rules that reference invalid fields for any class in a multi-class array

  • Mismatched field types (e.g., negative values for unsigned fields, integers where only strings are allowed)

An empty filter object {} is equivalent to {"filter": {"log": true}} and logs every event. To log nothing, use {"filter": {"log": false}}.

Behavior before 8.4.9-9

Before 8.4.9-9, the parser silently ignored unknown keys. Misspelling a structural key caused the parser to skip that entire subtree and fall back to the default. For example, the filter {"filter": {"classes": [...]}} (note classes instead of class) was parsed as {"filter": {}}, which logs every event. No error was returned, and the filter appeared to succeed. The same applied to events instead of event, names instead of name, and logs instead of log. Upgrade to 8.4.9-9 or later to catch these mistakes at parse time.

Parse-phase subclass names must use preparse and postparse.

audit_log_filter.event_mode controls which classes new definitions may use; filters saved under FULL can still load under REDUCED with some classes skipped (see that variable).

Using the audit log filter functions

Assignment vs rules inside the JSON

Two mechanisms apply, in order:

  1. Assignment (mysql.audit_log_user) — Each session uses one named filter from the USER and HOST columns (the same user_name@host_name form you pass to audit_log_filter_set_user()). The server loads that filter’s JSON from mysql.audit_log_filter. Assignments do not merge: a session never evaluates two separate filter definitions at once.

  2. Rules inside the assigned JSON — log conditions on a class or event block narrow which events match within the already-selected filter. Conditions compare event fields (such as user.str, host.str, table_database.str, table_name.str, status) with field items, and combine them with and / or / not. They are not a second user- or host-level assignment row. For example, bind one filter to 'app'@'%' and still include a log condition under a connection rule so only connection events from chosen client hosts are logged.

See Test event field values and Combine conditions with logical operators for the grammar used in rule-level conditions.

Which audit_log_user row applies

On connect, the component selects a row in mysql.audit_log_user whose USER and HOST match the session account. Literal user@host pairs match when they equal the session identity. Starting in Percona Server for MySQL 8.4.4, wildcard characters (% and _) are allowed in the host portion of the assignment string (see audit_log_filter_set_user()); pattern matching matches the behavior when you create the row through that function.

Keep assignments non-overlapping when you use wildcards. If several rows could match one connection, precedence is not specified here—prefer explicit literal user@host rows (or one clear pattern plus a '%' default) and confirm behavior on a test server.

With no matching row, the component uses the default assignment: the account registered with audit_log_filter_set_user() using % as the user name (see audit_log_filter_set_user()).

If neither a matching row nor a default exists, the component skips event processing for that connection.

A specific account row overrides the default: if both admin@localhost and % have filters, admin from localhost uses the admin@localhost filter, not the default.

You can bind filters to named accounts or remove those bindings.

To clear a binding, unassign the filter or assign a different one. How sessions refresh when assignments change is described under audit_log_filter_set_user() and audit_log_filter_remove_filter().

set_filter options and available filters

JSON layout (filter, class, nested rules, examples) is in Write audit_log_filter definitions. The authoritative list of class names, event subclass names, and per-class field names that audit_log_filter_set_filter() accepts appears in Audit Log Filter definition fields. This section lists the keys that may appear at each level of a filter definition.

Filter-level keys

Key Role
log Boolean global default. true enables logging everywhere not turned off by a more specific rule; false requires per-class/per-event log overrides to write anything.
class One class block or an array of class blocks. Each block sets "name" to an event class (general, connection, table_access, message; plus global_variable, command, query, stored_program, authentication, parse when event_mode=FULL).
id Optional filter identifier. Referenced by activate / ref when one filter swaps itself for another mid-session (see Replace a filter dynamically).

Class-block keys

Key Role
name The class name. Use one of the values listed in the preceding section.
log Optional. Boolean, or a condition that narrows matches by event-field values.
event Optional. One event block or an array. Each block names one subclass and can carry its own log, abort, print, or nested filter. Subclasses depend on the class. For table_access: read, insert, update, delete. For connection: connect, disconnect, change_user, plus pre_authenticate in FULL. See the event and subclass table and definition fields.
print Optional. Field-replacement rule. The component limits replacement to four class/field pairs (general/general_query.str, table_access/query.str, query/query.str, parse/query.str) with the query_digest function. See Redact audit log fields.

Event-block keys

Key Role
name Subclass name. May be a string (one subclass) or an array (several).
log Optional. Boolean, or a condition over event fields.
abort Optional. Boolean or condition that blocks execution of matching statements — see Block statements with an audit log filter.
print Optional. Field-replacement rule scoped to this event — see Redact audit log fields.
filter Optional. Nested subfilter used with activate / ref for dynamic filter swapping — see Replace a filter dynamically.

Conditions

log and abort accept a Boolean (true / false) or a condition built from these items:

Item Role
field { "name": "<event-field>", "value": <value> }. Compares an event field such as user.str, host.str, table_database.str, table_name.str, status, general_command.str, or general_sql_command.str. Per-class fields are listed in Audit Log Filter definition fields.
variable { "name": "<server-variable>", "value": <value> }. Compares a predefined variable such as audit_log_connection_policy_value.
function { "name": "<function>", "args": [...] }. Calls a built-in. The Percona implementation supports two functions: string_find and query_digest — see Reference predefined functions.
and Array of sub-conditions; all must be true.
or Array of sub-conditions; at least one must be true.
not A single sub-condition; true when the inner evaluates to false.

Field-value typing (JSON numbers for integer fields such as status, strings for *.str fields, and "::tcp/ip"-style symbolic constants for connection_type) is covered under audit_log_filter_set_filter() and in the connection section of Audit Log Filter definition fields.

Examples

Start from a single-class filter that logs every connection event the component sees:

SELECT audit_log_filter_set_filter('log_connection', '{
  "filter": {
    "class": { "name": "connection" }
  }
}');

Narrow to a single subclass by nesting event inside the class block (not as a sibling of class):

SELECT audit_log_filter_set_filter('log_connect_only', '{
  "filter": {
    "class": {
      "name": "connection",
      "event": { "name": "connect" }
    }
  }
}');

Narrow further by adding a log condition that tests event fields — here, log connect events from only two accounts, originating from one host:

SELECT audit_log_filter_set_filter('log_admin_connect', '{
  "filter": {
    "class": {
      "name": "connection",
      "event": {
        "name": "connect",
        "log": {
          "and": [
            { "field": { "name": "user.str", "value": ["admin", "developer"] } },
            { "field": { "name": "host.str", "value": "10.0.0.5" } }
          ]
        }
      }
    }
  }
}');

For full authoring detail, see:

Additional reading

Manage the Audit Log Filter files

Audit log files can fill disks and grow without bound unless you cap them.

Use rotation to rename the active file and start a fresh one with the original name—manually with a UDF or automatically by size.

With automatic rotation enabled, prune old files by age, total size, or both.

Manual log rotation

audit_log_filter.rotate_on_size defaults to 1 GB. Set it to 0 to disable automatic rotation; you must rotate by hand.

Run SELECT audit_log_rotate(); to rotate immediately. Requires AUDIT_ADMIN.

Starting in Percona Server for MySQL 8.4.9-9, rotation finishes immediately in ASYNCHRONOUS and PERFORMANCE strategies instead of waiting on the background flush thread. Several rotations in the same second add a numeric suffix (for example audit_filter.20250401T120000-1.log) so files are not overwritten.

Pruning runs when audit_log_filter.max_size or audit_log_filter.prune_seconds is greater than zero. The server prunes immediately when you set either variable, and again on rotation (automatic when rotate_on_size > 0, or manual via audit_log_rotate()). There is no background timer — with automatic rotation disabled, schedule manual rotations or rely on the prune that runs when you set the variable.

After rotation, delete archived files you no longer need yourself. audit_log_read() can read renamed files only if their names still match the active naming pattern.

Additional reading

Audit Log Filter file naming conventions

Name qualities

An audit log path has:

  • Optional directory prefix

  • Base file name

  • Optional suffix from compression or encryption

Compression or encryption append suffixes:

  • Compression adds .gz

  • Encryption adds .pwd_id.enc

pwd_id identifies the keyring entry for that password; the component stores keys in the keyring.

With both features, both suffixes appear (for example on audit_filter.log).

Example names:

Default name Enabled feature
audit_filter.log No compression or encryption
audit_filter.log.gz Compression
audit_filter.log.pwd_id.enc Encryption
audit_filter.log.gz.pwd_id.enc Compression and encryption

Encryption ID format

Each pwd_id contains:

  • UTC creation time as YYYYMMDDThhmmss

  • A sequence starting at 1, incremented when several passwords share one timestamp

Examples:

20230417T082215-1
20230301T061400-1
20230301T061400-2

Example encrypted file names:

audit_filter.log.20230417T082215-1.enc
audit_filter.log.20230301T061400-1.enc
audit_filter.log.20230301T061400-2.enc

The password with the highest sequence for a given timestamp is current.

Rotation sequence suffix

Starting in Percona Server for MySQL 8.4.9-9, multiple rotations in the same second append -N so the server never overwrites a prior file:

audit_filter.20250401T120000.log      -- first rotation at 12:00:00
audit_filter.20250401T120000-1.log    -- second rotation at 12:00:00

Update parsers to accept the optional -N suffix.

Renaming operations

At startup, if the target path already has a file, the component renames it and opens a new empty file.

At shutdown, the component renames the active log file.

Additional reading

Reference

Audit log filter functions, options, and variables

Reference for audit log filter functions and options / variables.

Audit log filter functions

Available UDFs:

Function name
audit_log_encryption_password_get(keyring_id)
audit_log_encryption_password_set(new_password)
audit_log_filter_flush()
audit_log_read()
audit_log_read_bookmark()
audit_log_session_filter_id()
audit_log_filter_remove_filter(filter_name)
audit_log_filter_remove_user(user_name)
audit_log_rotate()
audit_log_filter_set_filter(filter_name, definition)
audit_log_filter_set_user(user_name, filter_name)

audit_log_encryption_password_get(keyring_id)

Returns the audit encryption password (and iteration metadata) from the enabled keyring. Without a working keyring, the call errors.

Parameters

keyring_id — Omit to fetch the active password. Pass a keyring ID to read a specific archived or current entry.

Returns

JSON with password and iterations for the requested keyring entry.

Example

SELECT audit_log_encryption_password_get();
Expected output 
+---------------------------------------------+
| audit_log_encryption_password_get()         |
+---------------------------------------------+
| {"password":"passw0rd","iterations":5689}   |
+---------------------------------------------+

audit_log_encryption_password_set(new_password)

Sets a new audit encryption password in the keyring (and may rotate the log file when encryption is active—see Audit Log Filter compression and encryption).

Parameters

new_password — String up to 766 bytes.

Returns

OK on success; an error string on failure.

Example

SELECT audit_log_encryption_password_set('passw0rd');
Expected output 
+-----------------------------------------------------+
| audit_log_encryption_password_set('passw0rd')       |
+-----------------------------------------------------+
| OK                                                  |
+-----------------------------------------------------+

audit_log_filter_flush()

Reloads filter JSON and account rows from mysql.audit_log_filter / mysql.audit_log_user into the component so memory matches disk.

Table edits alone—including direct DML and audit_log_filter_set_filter()—do not refresh every open session. Call audit_log_filter_flush() when all sessions must see new rules (details under Persistence and refreshing on audit_log_filter_set_filter()).

From Percona Server for MySQL 8.4.9-9 onward, a flush detaches existing sessions until they reconnect or run CHANGE_USER; new connections pick up the reloaded registry immediately. If you cannot tolerate a gap, reconnect clients after flushing.

If audit_log_filter_flush() fails, the call returns an error message in the result string and the previously loaded filter rules and user assignments stay in effect. The audit log keeps writing under the prior configuration until the next successful flush replaces it. Retry the call after correcting the underlying cause so that the new on-disk rules take effect.

This function requires the AUDIT_ADMIN privilege.

Parameters

None.

Returns

This function returns either an OK for success or an error message for failure.

Example

SELECT audit_log_filter_flush();
Expected output 
+--------------------------+
| audit_log_filter_flush() |
+--------------------------+
| OK                       |
+--------------------------+

audit_log_read()

Reads JSON or JSONL audit files and returns events as a JSON array string. Other formats error.

Parameters

The function accepts a single JSON argument that selects a starting point. Four forms are valid:

  • Empty or omitted argument. The function continues from the current read cursor in this session. Calling audit_log_read() with no argument is equivalent to passing '{}'. Both forms require a read context that an earlier call already established. Without a context, the call returns the Wrong argument format error.

  • start envelope. The function begins a new read sequence at the position described by an inner bookmark. The bookmark accepts timestamp and id. A timestamp-only start is legal. When the timestamp has no time part, the component assumes 00:00:00.

  • Bookmark literal. Pass the JSON returned by audit_log_read_bookmark() directly, with no start envelope. The bookmark form requires both timestamp and id. Passing one without the other returns the Wrong argument format error. Passing a non-string timestamp or a non-integer id returns the bad bookmark format error.

  • JSON null. The argument 'null' closes the active read sequence in this session. Use this form to release the reader cursor before opening a new sequence with a different start or bookmark. The function returns the string OK.

Position arguments are mutually exclusive. A single call cannot combine start with a top-level timestamp or id. A call cannot supply a new start or bookmark while a read context is already active in the session. To reposition, close the sequence with 'null' and then issue a new start or bookmark literal.

The optional max_array_length key caps how many events the call returns. Use it to page through dense periods or to limit response size. The key is valid in any form that supplies a position.

Seed reads with audit_log_read_bookmark() when you want to resume from the tail of the log.

Returns

JSON array text, JSON NULL when no more events are available, the literal string OK when the call closes the active read sequence with the 'null' argument, or an error. From Percona Server 8.4.8-8 onward, output is strictly valid JSON, honors max_array_length, and resumes cleanly from bookmarks.

Examples

Continue from the bookmark returned by audit_log_read_bookmark():

SELECT audit_log_read(audit_log_read_bookmark());
Expected output 
+------------------------------------------------------------------------------+
| audit_log_read(audit_log_read_bookmark())                                   |
+------------------------------------------------------------------------------+
| [{"timestamp": "2023-06-02 09:43:25", "id": 10, "class": "connection"}]        |
+------------------------------------------------------------------------------+

Start a new read sequence at an explicit timestamp:

SELECT audit_log_read('{"start": {"timestamp": "2026-05-20 12:28:10"}}');

Start a new read sequence at the beginning of a date. The component assumes a 00:00:00 time part when the timestamp omits one:

SELECT audit_log_read('{"start": {"timestamp": "2026-05-20"}}');

Cap the number of events returned in one call by combining start with max_array_length:

SELECT audit_log_read('{"start": {"timestamp": "2026-05-20 12:28:10"}, "max_array_length": 3}');

Address a single specific event by passing a bookmark literal with timestamp and id and no start envelope:

SELECT audit_log_read('{"timestamp": "2026-05-20 12:28:10", "id": 1561422}');

Close the active read sequence so that the next call can open a new one at a different position:

SELECT audit_log_read('null');

audit_log_read_bookmark()

Returns a JSON bookmark for the latest event (JSON / JSONL only—other formats error).

Pass the bookmark into audit_log_read() to begin there.

SELECT audit_log_read(audit_log_read_bookmark());

Parameters

None.

Returns

This function returns a JSON string containing a bookmark for success or NULL and an error for failure.

Example

SELECT audit_log_read_bookmark();
Expected output 
+----------------------------------------------------+
| audit_log_read_bookmark()                          |
+----------------------------------------------------+
| {"timestamp": "2023-06-02 09:43:25", "id": 10}     |
+----------------------------------------------------+

audit_log_session_filter_id()

Returns the active filter ID for this session, or 0 when no filter applies.

audit_log_filter_remove_filter(filter_name)

Drops a filter definition and clears mysql.audit_log_user rows that pointed at it.

From Percona Server for MySQL 8.4.9-9, only sessions using that filter detach; others keep logging.

This function requires the AUDIT_ADMIN privilege.

Parameters

filter_name - a selected filter name as a string.

Returns

This function returns either an OK for success or an error message for failure.

If the filter name does not exist, no error is generated.

Example

SELECT audit_log_filter_remove_filter('filter-name');
Expected output 
+------------------------------------------------+
| audit_log_filter_remove_filter('filter-name')  |
+------------------------------------------------+
| OK                                             |
+------------------------------------------------+

audit_log_filter_remove_user(user_name)

Removes the mysql.audit_log_user row for that account pattern.

Open sessions keep their cached filter until reconnect/CHANGE_USER; new sessions fall back to the % default or stop auditing if none exists.

Passing user_name = '%' clears the default assignment.

This function requires the AUDIT_ADMIN privilege.

Parameters

user_name - a selected user name in either the user_name@host_name format or %.

Returns

This function returns either an OK for success or an error message for failure.

If the user_name has no filter assigned, no error is generated.

Example

SELECT audit_log_filter_remove_user('user-name@localhost');
Expected output 
+------------------------------------------------------+
| audit_log_filter_remove_user('user-name@localhost')  |
+------------------------------------------------------+
| OK                                                   |
+------------------------------------------------------+

audit_log_rotate()

Rotates the active audit file immediately and returns the archived name.

From 8.4.9-9, colliding timestamps add a -N suffix so rotations never overwrite each other.

This function requires the AUDIT_ADMIN privilege.

Parameters

None.

Returns

This function returns the renamed file name.

Example

SELECT audit_log_rotate();

audit_log_filter_set_filter(filter_name, definition)

Writes JSON for filter_name to mysql.audit_log_filter (create or update). Each stored revision gets a new filter ID.

From Percona Server for MySQL 8.4.9-9, validation runs at parse time—bad fields, unknown classes/subclasses, empty arrays, stray JSON keys, or broken print rules abort the call with a detailed error, for example:

ERROR: Incorrect rule definition: Unknown field name "WRONG.str" for class "general"

Before 8.4.9-9, the parser silently ignored unknown keys. Misspelling a structural key (for example classes instead of class) caused the subtree to be skipped, and the filter fell back to default behavior (log everything) with no error. Upgrade to 8.4.9-9 or later to catch these mistakes. For details, see Filter definition validation.

In REDUCED event mode, a filter definition that references disabled event classes or subclasses fails the same way (error returned, filter not stored).

This function requires the AUDIT_ADMIN privilege.

Persistence and refreshing

audit_log_filter_set_filter() persists JSON but does not patch in-memory state for every open session. Run audit_log_filter_flush() to reload all definitions into the component.

After 8.4.9-9 flushes, existing sessions detach until reconnect/CHANGE_USER; brand-new sessions pick up changes immediately.

Parameters

  • filter_name - a selected filter name as a string.

  • definition - Defines the definition as a JSON value.

Returns

This function returns either an OK for success or an error message for failure.

Example

SET @filter = '{ "filter": { "log": true } }';
SELECT audit_log_filter_set_filter('filter-name', @filter);
Expected output 
+-------------------------------------------------------------+
| audit_log_filter_set_filter('filter-name', @filter)  |
+-------------------------------------------------------------+
| OK                                                          |
+-------------------------------------------------------------+

audit_log_filter_set_user(user_name, filter_name)

Binds filter_name to a login pattern in mysql.audit_log_user.

From Percona Server for MySQL 8.4.4, host wildcards (%, _) work in the host portion ('usr1@%', 'usr2%172.16.10.%', 'usr3@%.mycorp.com', …).

This UDF controls which named filter loads for a session. JSON user / host keys inside the filter still narrow events after load—they are not a second assignment row. See Assignment vs rules inside the JSON.

Avoid overlapping patterns unless you accept ambiguous matches—prefer literals or one pattern plus %.

One active mapping per account row; this call replaces any previous mapping.

From 8.4.9-9, open sessions keep the old mapping until reconnect/CHANGE_USER; flush if you must realign everyone immediately.

The special user % is the default row used when no literal match exists; specific user@host rows always beat %.

This function requires the AUDIT_ADMIN privilege.

Parameters

  • user_name - a selected user name in either the user_name@host_name format or %.

  • filter_name - a selected filter name as a string.

Returns

This function returns either an OK for success or an error message for failure.

Example

SELECT audit_log_filter_set_user('user-name@localhost', 'filter-name');
Expected output 
+-------------------------------------------------------------------+
| audit_log_filter_set_user('user-name@localhost', 'filter-name')  |
+-------------------------------------------------------------------+
| OK                                                                |
+-------------------------------------------------------------------+

Audit log filter options and variables

Audit Log Filter component uses the SQL form audit_log_filter.<option> (for example audit_log_filter.file).

Command-line and option file conventions

Use the same option spelling as the Command-line field in each variable’s reference table: the long name uses hyphens in the audit-log-filter prefix and keeps a dot before the option (for example --audit-log-filter.file). Read-only options require a server restart; use SET GLOBAL only where that variable is documented as dynamic.

Variable index

These are system variables (audit_log_filter.<option>). Each entry below the index lists its command-line name and whether it is dynamic.

Name
audit_log_filter.buffer_size
audit_log_filter.compression
audit_log_filter.database
audit_log_filter.direct_io
audit_log_filter.disable
audit_log_filter.encryption
audit_log_filter.event_mode
audit_log_filter.file
audit_log_filter.format
audit_log_filter.format_unix_timestamp
audit_log_filter.handler
audit_log_filter.key_derivation_iterations_count_mean
audit_log_filter.max_size
audit_log_filter.password_history_keep_days
audit_log_filter.prune_seconds
audit_log_filter.read_buffer_size
audit_log_filter.rotate_on_size
audit_log_filter.strategy
audit_log_filter.syslog_facility
audit_log_filter.syslog_priority
audit_log_filter.syslog_tag

Variables

audit_log_filter.buffer_size

Option name Description
Command-line –audit-log-filter.buffer-size
Dynamic No
Scope Global
Data type Integer
Default 1048576
Minimum value 4096
Maximum value 18446744073709547520
Units bytes
Block size 4096

Read-only size of the asynchronous audit buffer (multiples of 4096 bytes). Events queue here before hitting disk. Requires restart to change.

The component allocates one buffer for its lifetime.

Example

my.cnf (restart required):

[mysqld]
audit-log-filter.buffer-size=2097152

audit_log_filter.compression

Option name Description
Command-line –audit-log-filter.compression
Dynamic No
Scope Global
Data type Enumeration
Default NONE
Valid values NONE or GZIP

Read-only compression mode: NONE (default) or GZIP. Requires restart.

Example

my.cnf (restart required):

[mysqld]
audit-log-filter.compression=GZIP

audit_log_filter.database

Option name Description
Command-line –audit-log-filter.database
Dynamic No
Scope Global
Data type String
Default mysql

Read-only database hosting audit_log_filter / audit_log_user. Must be non-NULL, ≤ 64 characters, and valid—otherwise the component cannot start. Restart to change.

Example

my.cnf (restart required):

[mysqld]
audit-log-filter.database=mysql

audit_log_filter.direct_io

Option name Description
Command-line –audit-log-filter.direct-io
Dynamic No
Scope Global
Data type Boolean
Default OFF

Introduced in Percona Server for MySQL 8.4.9-9.

This variable is tech preview and may be removed in a future release.

This read-only variable opens the audit log file with O_DIRECT on Linux, bypassing the OS page cache. This variable requires a server restart to change.

When enabled, audit log writes bypass the Linux page cache, reducing memory pressure on busy servers with high audit log throughput.

Writes use a 4 KB aligned staging buffer internally.

If the file system does not support O_DIRECT or a direct write fails at runtime, the component gracefully falls back to buffered I/O with a warning.

Whether O_DIRECT works depends on the file system that backs the audit log path. Local ext4 and XFS volumes usually support direct I/O; tmpfs usually does not. Confirm support for the audit log directory before enabling audit_log_filter.direct_io.

Example

my.cnf (restart required):

[mysqld]
audit-log-filter.direct-io=ON

audit_log_filter.disable

Option name Description
Command-line –audit-log-filter.disable
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

When ON, stops audit output for all sessions.

Runtime changes need SYSTEM_VARIABLES_ADMIN and AUDIT_ADMIN.

Example

Runtime:

SET GLOBAL audit_log_filter.disable = ON;

Persist across restarts (my.cnf):

[mysqld]
audit-log-filter.disable=ON

audit_log_filter.encryption

Option name Description
Command-line –audit-log-filter.encryption
Dynamic No
Scope Global
Data type Enumeration
Default NONE
Valid values NONE or AES

This read-only variable defines the encryption type for the audit log filter file. This variable requires a server restart to change. The values can be either of the following:

  • NONE - the default value, no encryption

  • AES

Example

my.cnf (restart required):

[mysqld]
audit-log-filter.encryption=AES

audit_log_filter.event_mode

Option name Description
Command-line –audit-log-filter.event-mode
Dynamic Yes
Scope Global
Data type Enumeration
Default REDUCED
Available values REDUCED, FULL

Introduced in Percona Server for MySQL 8.4.9-9.

This variable controls which event classes and subclasses are processed by the audit log filter component.

REDUCED (default) — limits processing to the four classes general, connection, table_access, and message. Only the subclasses in the following list are enabled. The extended classes are not processed in this mode.

  • general/status

  • connection/connect, connection/disconnect, connection/change_user

  • table_access/*

  • message/*

The following event classes exist in the server but are disabled in REDUCED mode (they are available only in FULL mode):

  • global_variable

  • command

  • query

  • stored_program

  • authentication

  • parse

The audit log filter disables the following subclasses in REDUCED mode:

  • general/log

  • general/error

  • general/result

  • connection/pre_authenticate

In REDUCED mode, calling a stored procedure logs the outer CALL statement but not the individual SQL statements executed inside the procedure body. The general/status event generated by the internal Quit command is also suppressed in REDUCED mode.

FULL — all event classes and subclasses are processed, including global_variable, command, query, stored_program, authentication, and parse on top of the REDUCED set. Through Percona Server for MySQL 8.4.8-8 there was no audit_log_filter.event_mode setting; the component always processed the full class set (equivalent to FULL). The variable was introduced in 8.4.9-9 with default REDUCED.

Runtime mode switch

Set event_mode in my.cnf and restart the server. Changing the value on a running server produces inconsistent audit output and is not recommended.

Changing event_mode at runtime invalidates the cached audit rules and bumps the filter generation. Each session recompiles its filter against the new mode on the next event the session emits.

The switch creates a window where in-flight events can be evaluated inconsistently. The new mode value becomes visible before the audit rule cache is invalidated. The cache invalidation does not atomically refresh every session. The behavior applies to switches in either direction, between REDUCED and FULL.

During this window, events may be:

  • Filtered against the previous mode’s compiled rules even after the new mode is in effect.

  • Logged inconsistently across sessions, because one session has already picked up the new filter generation while another session still applies the previous compiled cache.

If a restart is not possible, narrow the window by following the SET GLOBAL change with audit_log_filter_flush() inside a maintenance window with minimal auditable traffic. The flush detaches existing sessions and forces each session to recompile against the new mode on reconnect or CHANGE_USER. Audit output during the switch window cannot be reconciled after the fact.

Example

Set in my.cnf and restart the server. This is the recommended form:

[mysqld]
audit-log-filter.event-mode=FULL

The runtime form is shown for reference only. See the runtime mode switch caveats before using it:

SET GLOBAL audit_log_filter.event_mode = 'FULL';
SELECT @@GLOBAL.audit_log_filter.event_mode;

In REDUCED mode, if you call audit_log_filter_set_filter() with a new filter definition that references disabled event classes or subclasses, the call fails: the server returns a descriptive error and does not store the filter. However, persisted filters created under FULL mode that reference disabled classes will still load after a restart or audit_log_filter_flush() — the disabled classes are silently skipped with a warning.

audit_log_filter.file

Option name Description
Command-line –audit-log-filter.file
Dynamic No
Scope Global
Data type String
Default audit_filter.log

This read-only variable defines the filename of the audit log filter file. The component writes events to this file. This variable requires a server restart to change.

The filename can be either of the following:

  • a relative path name - the component looks for this file in the data directory

  • a full path name - the component uses the given value

If you use a full path name, ensure the directory exists and is accessible only to users who need to view the log and the server. If the parent directory does not exist, the component reports an error and the server starts without the audit log filter component active.

For more information, see Naming conventions

Example

Relative to the data directory (my.cnf, restart required):

[mysqld]
audit-log-filter.file=audit_filter.log

Absolute path:

[mysqld]
audit-log-filter.file=/var/log/mysql/audit_filter.log

audit_log_filter.format

Option name Description
Command-line –audit-log-filter.format
Dynamic No
Scope Global
Data type Enumeration
Default NEW
Available values OLD, NEW, JSON, JSONL

This read-only variable defines the audit log filter file format. This variable requires a server restart to change.

The available values are the following:

Example

my.cnf (restart required):

[mysqld]
audit-log-filter.format=JSON

audit_log_filter.format_unix_timestamp

Option name Description
Command-line –audit-log-filter.format-unix-timestamp
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

Introduced in Percona Server for MySQL 8.4.9-9, this option is supported for JSON-format and JSONL-format files.

Enabling this option adds a time field to JSON-format and JSONL-format files. The integer represents the UNIX timestamp value and indicates the date and time when the audit event was generated. Changing the value causes a file rotation because all records must either have or do not have the time field. This option requires the AUDIT_ADMIN and SYSTEM_VARIABLES_ADMIN privileges.

This option does nothing when used with other format types.

Example

Runtime (JSON-format or JSONL-format only; causes rotation when toggled):

SET GLOBAL audit_log_filter.format_unix_timestamp = ON;

Persist across restarts (my.cnf):

[mysqld]
audit-log-filter.format-unix-timestamp=ON

audit_log_filter.handler

Option name Description
Command-line –audit-log-filter.handler
Dynamic No
Scope Global
Data type String
Default FILE

This read-only variable defines where the component writes the audit log filter file. This variable requires a server restart to change. The following values are available:

  • FILE - component writes the log to a location specified in audit_log_filter.file

  • SYSLOG - component writes to the syslog

Example

Write to a file under the data directory (my.cnf, restart required):

[mysqld]
audit-log-filter.handler=FILE
audit-log-filter.file=audit_filter.log

Write to syslog (use with audit_log_filter.syslog_tag and related options):

[mysqld]
audit-log-filter.handler=SYSLOG
audit-log-filter.syslog-tag=myapp-audit

audit_log_filter.key_derivation_iterations_count_mean

Option name Description
Command-line –audit-log-filter.key-derivation-iterations-count-mean
Dynamic Yes
Scope Global
Data type Integer
Default 600000
Minimum value 1000
Maximum value 1000000

Defines the mean value of iterations used by the password-based derivation routine while calculating the encryption key and iv values. A random number represents the actual iteration count and deviates no more than 10% from this value.

Example

Runtime:

SET GLOBAL audit_log_filter.key_derivation_iterations_count_mean = 120000;

audit_log_filter.max_size

Option name Description
Command-line –audit-log-filter.max-size
Dynamic Yes
Scope Global
Data type Integer
Default 1073741824
Minimum value 0
Maximum value 18446744073709551615
Unit bytes
Block size 4096

This variable defines the maximum combined size of all audit log files before pruning occurs.

The default value is 1073741824 (1 GiB).

Behavior: * A limit of 0 disables size-based pruning.

  • With a positive limit, pruning runs when the combined size of all audit log files exceeds that limit.

  • The server rounds each non-zero limit down to the nearest multiple of 4096 bytes (block size).

  • Any limit below 4096 is treated as 0 (disabled).

Recommendation: When both audit_log_filter.rotate_on_size and audit_log_filter.max_size are greater than 0, set audit_log_filter.max_size to at least seven times the audit_log_filter.rotate_on_size value.

Pruning requirements: Configure audit_log_filter.max_size for size-based pruning or audit_log_filter.prune_seconds for age-based pruning. The two options are mutually exclusive. Pruning runs when you set either variable, when automatic rotation occurs, or when you call audit_log_rotate().

Example

Combined size cap for pruning (example: 10 GiB), with rotation and age pruning configured elsewhere:

SET GLOBAL audit_log_filter.max_size = 10737418240;

audit_log_filter.password_history_keep_days

Option name Description
Command-line –audit-log-filter.password-history-keep-days
Dynamic Yes
Scope Global
Data type Integer
Default 0
Minimum value 0
Maximum value 18446744073709551615
Unit days

Defines when passwords may be removed and measured in days.

Encrypted log files have passwords stored in the keyring. The component also stores a password history. A password does not expire, despite being past the value, in case the password is used for rotated audit logs. The operation of creating a password also archives the previous password.

The default value is 0 (zero). This value disables the expiration of passwords. Passwords are retained forever.

If the component starts and encryption is enabled, the component checks for an audit log filter encryption password. If a password is not found, the component generates a random password and stores it in the keyring. To read the active password (or iteration metadata), call audit_log_encryption_password_get().

Call audit_log_encryption_password_set(new_password) to set a specific password.

Example

Retain archived encryption passwords for 30 days:

SET GLOBAL audit_log_filter.password_history_keep_days = 30;

audit_log_filter.prune_seconds

Option name Description
Command-line –audit-log-filter.prune-seconds
Dynamic Yes
Scope Global
Data type Integer
Default 0
Minimum value 0
Maximum value 18446744073709551615
Unit seconds

Defines when the audit log filter file is pruned. This pruning is based on the age of the file. The value is measured in seconds.

A value of 0 (zero) is the default and disables pruning. The maximum value is 18446744073709551615.

A value greater than 0 enables pruning. An audit log filter file can be pruned after this value.

To enable log pruning, set audit_log_filter.max_size for size-based pruning or audit_log_filter.prune_seconds for age-based pruning. The two options are mutually exclusive — setting one to a positive value clears the other. Pruning runs when you set either variable, when automatic rotation occurs (rotate_on_size > 0), or when you call audit_log_rotate().

Example

Prune files older than seven days (604800 seconds), with rotation enabled:

SET GLOBAL audit_log_filter.prune_seconds = 604800;

audit_log_filter.read_buffer_size

Option name Description
Command-line –audit-log-filter.read-buffer-size
Dynamic Yes
Scope Global
Data type Integer
Unit Bytes
Default 32768
Minimum value 32768
Maximum value 4294967295

Introduced in Percona Server for MySQL 8.4.9-9, this option is supported for JSON-format and JSONL-format files.

The size of the buffer for reading from the audit log filter file. audit_log_read() reads only from this buffer size.

Example

Runtime:

SET GLOBAL audit_log_filter.read_buffer_size = 65536;

audit_log_filter.rotate_on_size

Option name Description
Command-line –audit-log-filter.rotate-on-size
Dynamic Yes
Scope Global
Data type Integer
Default 1073741824

Performs an automatic log file rotation based on the size. The default value is 1073741824. If the value is greater than 0, when the log file size exceeds the value, the component renames the current file and opens a new log file using the original name.

If you set the value to less than 4096, the component does not automatically rotate the log files. You can rotate the log files manually using audit_log_rotate(). If the value is not a multiple of 4096, the component truncates the value to the nearest multiple.

Example

Rotate when the file reaches 512 MiB:

SET GLOBAL audit_log_filter.rotate_on_size = 536870912;

audit_log_filter.strategy

Option name Description
Command-line –audit-log-filter.strategy
Dynamic No
Scope Global
Data type Enumeration
Default ASYNCHRONOUS

This read-only variable defines the Audit Log filter component’s logging method. This variable requires a server restart to change. The valid values are the following:

Values Description
ASYNCHRONOUS Waits for free outer buffer space
PERFORMANCE If the outer buffer does not have enough space, drops the entire event atomically (the event is either fully written or fully dropped, keeping the log output well-formed)
SEMISYNCHRONOUS Operating system permits caching
SYNCHRONOUS Each request calls fsync() to flush the audit event to durable storage before the audited statement returns to the client. Expect higher write latency compared to SEMISYNCHRONOUS.

The following diagram summarizes buffering, dropping behavior, and durability characteristics for each strategy value.

Audit Log Filter logging strategies

Performance trade-offs

Switch from the default ASYNCHRONOUS to SYNCHRONOUS only when durability is more important than throughput and latency—for example strict compliance or forensic environments where every audit event must be on durable storage before the client sees the audited statement complete.

The main risk of ASYNCHRONOUS is a crash or power-loss window: the newest audit events can still be in memory and may be lost if they have not yet been flushed or synced to disk. That is not the same as dropping under load: ASYNCHRONOUS waits for buffer space and does not discard events when the buffer is full. The mode that can drop whole events when the outer buffer lacks space is PERFORMANCE.

SYNCHRONOUS behavior change in 8.4.9-9

Through Percona Server for MySQL 8.4.8-8, the SYNCHRONOUS value did not call fsync() per event. The sync_on_write flag was set but never honored, so the strategy behaved as SEMISYNCHRONOUS. From 8.4.9-9 onward, SYNCHRONOUS issues an fsync() after every audit event write, matching the documented contract.

The fix affects only configurations that already set strategy=SYNCHRONOUS. Such configurations see new write latency on every audited statement after the upgrade. Expect higher CPU on the audit write path and higher tail latency on the audited workload. Plan capacity before upgrading durability-critical instances, or switch to SEMISYNCHRONOUS if the prior (lower-latency) behavior is acceptable for the compliance posture.

Compressed file writers also issue Z_SYNC_FLUSH before each fsync() so that compressed bytes reach disk along with the underlying records.

Example

my.cnf (restart required):

[mysqld]
audit-log-filter.strategy=SYNCHRONOUS

audit_log_filter.syslog_tag

Option Description
Command-line –audit-log-filter.syslog-tag=
Dynamic No
Scope Global
Data type String
Default audit-filter

This read-only variable specifies the syslog tag value. This variable requires a server restart to change.

Example

my.cnf (restart required; use with audit-log-filter.handler=SYSLOG):

[mysqld]
audit-log-filter.syslog-tag=myapp-audit

audit_log_filter.syslog_facility

Option name Description
Command-line –audit-log-filter.syslog-facility
Dynamic No
Scope Global
Data type Enumeration
Default LOG_USER

This read-only variable specifies the syslog facility value. This variable requires a server restart to change. The option has the same meaning as the appropriate parameter described in the syslog(3) manual .

The component validates the value against a fixed enum. The following names are accepted:

LOG_USER, LOG_AUTHPRIV, LOG_CRON, LOG_DAEMON, LOG_FTP, LOG_KERN, LOG_LPR, LOG_MAIL, LOG_NEWS, LOG_SYSLOG, LOG_AUTH, LOG_UUCP, LOG_LOCAL0, LOG_LOCAL1, LOG_LOCAL2, LOG_LOCAL3, LOG_LOCAL4, LOG_LOCAL5, LOG_LOCAL6, LOG_LOCAL7.

LOG_SECURITY is accepted on platforms whose syslog.h defines LOG_SECURITY, such as FreeBSD. On other platforms, including most Linux distributions, the component rejects LOG_SECURITY.

Example

my.cnf (restart required):

[mysqld]
audit-log-filter.syslog-facility=LOG_USER

audit_log_filter.syslog_priority

Option name Description
Command-line –audit-log-filter.syslog-priority
Dynamic No
Scope Global
Data type Enumeration
Default LOG_INFO

This read-only variable defines the priority value for the syslog. This variable requires a server restart to change. The option has the same meaning as the appropriate parameter described in the syslog(3) manual .

The component validates the value against a fixed enum. The following names are accepted:

LOG_INFO, LOG_ALERT, LOG_CRIT, LOG_ERR, LOG_WARNING, LOG_NOTICE, LOG_EMERG, LOG_DEBUG.

Example

my.cnf (restart required):

[mysqld]
audit-log-filter.syslog-priority=LOG_INFO

Audit log filter status variables

Counters and gauges for audit filter activity. These are not configuration: you cannot SET them; they only change as the server processes audit events. For how they differ from system variables (naming, SHOW commands, and purpose), see the table at the start of Audit log filter options and variables. For SHOW GLOBAL STATUS examples, see Command-line and option file conventions. These names use underscores only (audit_log_filter_<name>). Names and counters are registered in Percona Server 8.4 as SHOW_VAR status_vars[] in sys_vars.cc; behavior is documented on the SysVars helpers in sys_vars.h. Buffering and drop behavior for file logging is implemented in file_writer_buffering.cc.

Name Description Example
audit_log_filter_current_size Current size in bytes of the active audit log file. Resets when the log is rotated. SHOW GLOBAL STATUS LIKE 'audit_log_filter_current_size';
audit_log_filter_direct_writes Number of times data was written synchronously while bypassing the write buffer, for example when a record is larger than the buffer under the ASYNCHRONOUS strategy. SHOW GLOBAL STATUS LIKE 'audit_log_filter_direct_writes';
audit_log_filter_event_max_drop_size Size in bytes of the largest event dropped when the PERFORMANCE strategy cannot buffer it (buffer full or oversized record with drop-on-full). SHOW GLOBAL STATUS LIKE 'audit_log_filter_event_max_drop_size';
audit_log_filter_events Number of audit events handled by the audit log filter component. SHOW GLOBAL STATUS LIKE 'audit_log_filter_events';
audit_log_filter_events_filtered Number of audit events that were filtered out (no log line written). SHOW GLOBAL STATUS LIKE 'audit_log_filter_events_filtered';
audit_log_filter_events_lost Number of audit events not written, including PERFORMANCE-mode drops and failures from the log writer. SHOW GLOBAL STATUS LIKE 'audit_log_filter_events_lost';
audit_log_filter_events_written Number of audit events written to the audit log. SHOW GLOBAL STATUS LIKE 'audit_log_filter_events_written';
audit_log_filter_total_size Total size in bytes of data written to audit log files; increases even when a log file is rotated. SHOW GLOBAL STATUS LIKE 'audit_log_filter_total_size';
audit_log_filter_write_waits Number of times an event waited for space in the audit buffer under the ASYNCHRONOUS strategy. SHOW GLOBAL STATUS LIKE 'audit_log_filter_write_waits';

To read the current value in SQL as a single row:

SELECT variable_value
FROM performance_schema.global_status
WHERE variable_name = 'audit_log_filter_events_written';

Additional reading

Audit Log Filter restrictions

General restrictions

Event coverage

  • The component logs SQL statements only—not NoSQL APIs such as the Memcached interface.

  • Event coverage follows audit_log_filter.event_mode. In REDUCED mode, stored programs log the outer CALL, not each statement inside the body. The component never logs file contents referenced by statements such as LOAD DATA.

  • Set audit_log_filter.event_mode in my.cnf and restart the server. Changing the value on a running server creates a window where in-flight events are evaluated inconsistently, in either direction between REDUCED and FULL. Audit output during the switch cannot be reconciled after the fact. If a restart is not possible, follow the SET GLOBAL change with audit_log_filter_flush() inside a maintenance window to narrow the window. For details, see audit_log_filter.event_mode.

Cluster and replication

  • On a cluster, install Audit Log Filter on every node that runs SQL.

  • Aggregate audit data from every node yourself—the component does not centralize it.

  • Maintain a separate rule set per server. An uninitialized replica writes no audit rows for rules it lacks. Configure replication so you do not blindly overwrite replica-local filter tables with source changes unless that is intended.

    By default, replication can replicate mysql.audit_log_filter and mysql.audit_log_user from source to replica. Use replication filters or channels to exclude those tables when replicas should keep local rules.

    Replicated table rows alone do not refresh in-memory filter state. Restart, reload the component, or run audit_log_filter_flush() to apply table changes to the running component.

Filter validation

  • Starting in Percona Server for MySQL 8.4.9-9, numeric fields in rules may use integer or string forms, and connection_type accepts symbolic constants—see audit_log_filter_set_filter() and Audit Log Filter definition fields.

  • Before 8.4.9-9, the parser silently ignored unknown JSON keys. Misspelling a structural key (for example classes instead of class, events instead of event, names instead of name, logs instead of log) caused the subtree to be skipped, and the filter fell back to default behavior (log everything) with no error. Upgrade to 8.4.9-9 or later to catch these mistakes at parse time.

Synchronizing audit log filters between a source and a replica

Replicate the filter tables and run audit_log_filter_flush() on the replica so replicated rows take effect in the component. Post-flush session behavior is documented under audit_log_filter_flush().

Procedure: scheduled event to flush filters on the replica

  1. Install Percona Server for MySQL on the source with Audit Log Filter.

  2. Clone a replica from a source backup.

  3. On the replica, create a one-minute event:

    USE mysql;
CREATE EVENT auditflush
    ON SCHEDULE EVERY 1 MINUTE
    COMMENT 'Flush audit log filters every minute'
    DO
      SELECT audit_log_filter_flush();

    The event applies replicated filter changes shortly after replication commits them. For session details after flush (8.4.9-9+), see audit_log_filter_flush().

  4. On the source, define a filter, assign it to users (UDFs or direct table edits), and call audit_log_filter_flush() on the source when needed.

  5. After a minute, confirm the replica sees the filter; run queries as that user and verify the audit file.

If nothing appears, wait for the next event tick or run SELECT audit_log_filter_flush(); on the replica immediately; confirm replication applied the rows and the event is ENABLED. Persistent gaps usually mean a replication issue—investigate the replica’s replication health.

If the component is missing on either host, install it on source and replica independently.

Limitation

Direct INSERT/UPDATE/DELETE on the source without audit_log_filter_flush() there still replicate to the replica; the scheduled event flushes on the replica, so replica filters update while the source in-memory state stays stale until someone flushes there.

Additional reading

Disable Audit Log Filter logging

The audit_log_filter.disable system variable turns audit logging on or off for all connections:

Value Effect
audit_log_filter.disable = true Stops audit logging.
audit_log_filter.disable = false Enables audit logging.

Set it in an option file, on the command line, or at runtime:

SET GLOBAL audit_log_filter.disable = true;

Privileges required

Runtime changes require both:

  • AUDIT_ADMIN
  • SYSTEM_VARIABLES_ADMIN

Additional reading

Uninstall Audit Log Filter

To remove the Audit Log Filter component from the server, run:

UNINSTALL COMPONENT 'file://component_audit_log_filter';

What this does:

  • UNINSTALL COMPONENT — Drops a previously installed component from the server.
  • file://component_audit_log_filter — URN for the Audit Log Filter component, which applies your rules and decides which audit events are recorded.

Additional reading

Managing binary log disk space

Controlling binary log disk usage can be difficult because binary log sizes vary. The database writes each transaction in full to a single binary log file and cannot split a write across multiple files. This requirement can lead to large log files, especially when transactions are large.

binlog_space_limit

Attribute Description
Uses the command line Yes
Uses the configuration file Yes
Scope Global
Dynamic No
Variable type ULONG_MAX
Default value 0 (unlimited)
Maximum value - 64-bit platform 18446744073709547520

This variable sets an upper limit on the total size of all binary logs in bytes. When the combined size exceeds this limit, the server automatically purges the oldest binary logs until the total size falls below the limit or only the active log remains.

A default value of 0 disables this feature. In this case, the server does not enforce a size limit and continues to write binary logs until the binary logs exhaust the available disk space.

Example

Set the binlog_space_limit to 50 GB in the my.cnf file:

[mysqld]
...
binlog_space_limit = 50G
...

Extended SELECT INTO OUTFILE/DUMPFILE

Percona Server for MySQL extends the SELECT INTO ... OUTFILE and SELECT INTO DUMPFILE commands to add support for UNIX sockets and named pipes. Before this was implemented the database would return an error for such files.

This feature allows using LOAD DATA LOCAL INFILE in combination with SELECT INTO OUTFILE to quickly load multiple partitions across the network or in other setups, without having to use an intermediate file that wastes space and I/O.

Expanded fast index creation

Percona has implemented several changes related to MySQL’s fast index creation feature. Fast index creation was implemented in MySQL as a way to speed up the process of adding or dropping indexes on tables with many rows.

This feature implements a session variable that enables extended fast index creation. Besides optimizing DDL directly, expand_fast_index_creation may also optimize index access for subsequent DML statements because using it results in much less fragmented indexes.

The mysqldump command

A new option, --innodb-optimize-keys, was implemented in mysqldump. It changes the way InnoDB tables are dumped, so that secondary and foreign keys are created after loading the data, thus taking advantage of fast index creation. More specifically:

  • KEY, UNIQUE KEY, and CONSTRAINT clauses are omitted from CREATE TABLE statements corresponding to InnoDB tables.

  • An additional ALTER TABLE is issued after dumping the data, in order to create the previously omitted keys.

ALTER TABLE

When ALTER TABLE requires a table copy, secondary keys are now dropped and recreated later, after copying the data. The following restrictions apply:

  • Only non-unique keys can be involved in this optimization.

  • If the table contains foreign keys, or a foreign key is being added as a part of the current ALTER TABLE statement, the optimization is disabled for all keys.

  • If the table is partitioned, the optimization is disabled for all keys.

OPTIMIZE TABLE

Internally, OPTIMIZE TABLE is mapped to ALTER TABLE ... ENGINE=innodb for InnoDB tables. As a consequence, it now also benefits from fast index creation, with the same restrictions as for ALTER TABLE.

Caveats

InnoDB fast index creation uses temporary files in tmpdir for all indexes being created. So make sure you have enough tmpdir space when using expand_fast_index_creation. It is a session variable, so you can temporarily switch it off if you are short on tmpdir space and/or don’t want this optimization to be used for a specific table.

There’s also a number of cases when this optimization is not applicable:

  • UNIQUE indexes in ALTER TABLE are ignored to enforce uniqueness where necessary when copying the data to a temporary table;

  • ALTER TABLE and OPTIMIZE TABLE always process tables containing foreign keys as if expand_fast_index_creation is OFF to avoid dropping keys that are part of a FOREIGN KEY constraint;

  • ALTER TABLE and OPTIMIZE TABLE always process partitioned tables as if expand_fast_index_creation is OFF;

  • mysqldump –innodb-optimize-keys ignores foreign keys because InnoDB requires a full table rebuild on foreign key changes. So adding them back with a separate ALTER TABLE after restoring the data from a dump would actually make the restore slower;

  • mysqldump –innodb-optimize-keys ignores indexes on AUTO_INCREMENT columns, because they must be indexed, so it is impossible to temporarily drop the corresponding index;

  • mysqldump –innodb-optimize-keys ignores the first UNIQUE index on non-nullable columns when the table has no PRIMARY KEY defined, because in this case InnoDB picks such an index as the clustered one.

System variables

expand_fast_index_creation

Option Description
Command Line: Yes
Config file No
Scope: Local/Global
Dynamic: Yes
Data type Boolean
Default value ON/OFF

See also

Improved InnoDB fast index creation

Thinking about running OPTIMIZE on your InnoDB Table? Stop!

FIPS compliance

The Federal Information Processing Standards (FIPS) are a set of regulations mandated by the United States government that ensure the security of computer systems for non-military government agencies and their contractors. These standards specify how to perform cryptographic operations, such as encryption, hashing, and digital signatures. FIPS mode is a mode of operation that enforces these standards and rejects any non-compliant algorithms or parameters.

Percona Server for MySQL can run in FIPS mode if a FIPS-enabled OpenSSL library and FIPS Object Module are available at runtime or if compiled using a FIPS-validated version of OpenSSL. FIPS mode remains off by default and must be enabled via OS/OpenSSL configuration.

Version changes

Introduced in Percona Server for MySQL Pro build for 8.4.0-1. You can also receive this functionality by building Percona Server for MySQL from source code.

Percona Server for MySQL 8.4.0-5 includes the FIPS-capable binaries in all builds. No Pro build is required for FIPS compatibility.

Prerequisites

To prepare Percona Server for MySQL for FIPS certification, do the following:

  • Check that your operating system includes FIPS pre-approved OpenSSL library in version 3.0.x or higher. The following distributions includes FIPS pre-approved OpenSSL library in version 3.0.x or higher:

    • RedHat Enterprise Linux 9 and derivatives

    • Oracle Linux 9

    The following distributions also includes OpenSSL library in version 3.0.x but do not have FIPS-approved crypto provider installed by default (you can build the crypto provider from the source for testing):

    • Debian 12

    • Ubuntu 22.04 Pro (the OpenSSL FIPS 140-3 certification is under implementation)

      Note

      If you enable FIPS on Ubuntu Pro with $ sudo pro enable fips-updates and then disable FIPS with $ sudo pro disable fips-updates, Percona Server for MySQL may stop operating properly. For example, if you disable FIPS on Ubuntu Pro with $ sudo pro disable fips-updates and enable the FIPS mode on Percona Server with ssl-fips-mode=ON, Percona Server may not load the SSL certificate.

The FIPS mode variables

Percona Server for MySQL uses the same variables and values as MySQL. Percona Server for MySQL enables control of FIPS mode on the server side and the client side:

  • The ssl_fips_mode system variable shows whether the server operates in FIPS mode. This variable is disabled by default.

    The ssl_fips_mode system variable has these values:

    • 0 - disables FIPS mode
    • 1 - enables FIPS mode. The exact behavior of the enabled FIPS mode depends on the OpenSSL version. The server only specifies the FIPS value to OpenSSL.
    • 2 - enables strict FIPS mode. This value provides more restrictions than the 1 value. The exact behavior of the strict FIPS mode depends on the OpenSSL version. The server only specifies the FIPS value to OpenSSL.

  • The --ssl-fips-mode client/server option controls whether a given client operates in FIPS mode. This setting does not change the server setting. This option is disabled by default.

    The --ssl-fips-mode client/server option has these values:

    • OFF - disables FIPS mode
    • ON - enables FIPS mode. The exact behavior of the enabled FIPS mode depends on the OpenSSL version. The server only specifies the FIPS value to OpenSSL.
    • STRICT - enables strict FIPS mode. This value provides more restrictions than the ON value. The exact behavior of the strict FIPS mode depends on the OpenSSL version. The server only specifies the FIPS value to OpenSSL.

    The server operation in FIPS mode does not depend on which crypto module (regular or FIPS-approved) is set as the default in the OpenSSL configuration file. The server always respects the value of --ssl-fips-mode server command line option (OFF, ON, or STRICT). The ssl_fips_mode global system variable is read-only and cannot be changed at runtime.

Enable the FIPS mode

To enable the FIPS mode, pass --ssl-fips-mode=ON or --ssl-fips-mode=STRICT to mysqld as a command line argument or add ssl-fips-mode=ON or --ssl-fips-mode=STRICT to the configuration file. Ignore the warning that the --ssl-fips-mode client/server option is deprecated.

Check that FIPS mode is enabled

To ensure that the FIPS mode is enabled, do the following:

  • Pass --log-error-verbosity=3 to mysqld as a command line argument or add log-error-verbosity=3 to the configuration file.

  • Check that the error log contains the following message:

    A FIPS-approved version of the OpenSSL cryptographic library has been detected in the operating system with a properly configured FIPS module available for loading. Percona Server for MySQL will load this module and run in FIPS mode.

JS language support

js_lang stored procedure and function overview

This feature is a tech preview. Before using this feature in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.

You can use stored procedures and functions written in JS with a MySQL-compatible database. This feature helps you manage complex data processing tasks. This method improves performance. This method lets developers run complex operations faster. If you know JS, you can use your existing skills. Using stored procedures and functions reduces the work done by client applications. Using stored procedures and functions also improves overall system performance. Using stored procedures and functions helps developers process data faster. Using stored procedures and functions also makes maintenance and scaling easier. This approach is a good solution for developers who know JS.

Benefit Description
Familiarity Developers who already know JS can use their existing skills.
Efficiency JS can run complex data processing tasks more efficiently.
Performance Stored procedures and functions in JS improve database performance. They reduce the work done by client applications.
Reusability You can write code once in stored procedures and functions. Then you can use that code in multiple applications.
Scalability Using stored procedures and functions makes database operations easier to maintain and scale.
Simplified Development Process This feature makes development easier for developers who know JS.
Integration with Client Applications Stored procedures and functions work well with client applications. You need less additional processing.
Optimization This feature improves overall system performance through efficient data processing.

Limitations

The JS procedure parameters cannot be JS reserved words and must be legal JS identifiers .

Our implementation offers the same level of JS support as the V8 engine inside the context of a database engine. You can check out the details at v8.dev/docs and tc39.es/ecma262 . Developers have access to standard operators, data types, objects (such as Math), and functions defined in the ECMAScript standard. However, objects and modules specific to Node.NS or DOM, which are only available in browsers, are not accessible.

In a typical database environment, direct access to external files (like reading or writing files on the server’s file system) is restricted. Our implementation adheres to a trusted external routine language policy, ensuring routines cannot perform operations beyond what is normally possible for database users. Consequently, file or network I/O operations are not supported within our routines.

Our system supports asynchronous JS code, but it does not work well for database routines. Since everything runs on the same thread and there is nothing to wait for asynchronously, using asynchronous code is unnecessary and not recommended.

We always run JS code in strict mode, and developers cannot disable or change this setting.

Type conversions

SQL and JS use different data types, so the js_lang component converts values when passing SQL parameters to JS and back. SQL NULL values are converted to JS null values, and JS null and undefined values are always mapped to SQL NULL. When converting to JS strings, data is automatically converted from the SQL parameter’s character set to utf8mb4.

For detailed information about type conversions, including complete conversion tables and rules, see js_lang type conversions.

System variables

The js_lang component provides the following system variables for configuring JS routine execution:

Variable name Description Default
js_lang.max_mem_size Maximum memory size (soft limit) for JS routines 8 MB
js_lang.max_mem_size_hard_limit_factor Hard limit factor for memory allocation 0 (disabled)

The js_lang.max_mem_size variable sets a soft limit on memory usage per JS environment. The js_lang.max_mem_size_hard_limit_factor variable (settable only at start-up) allows you to override V8’s internal hard memory limit, though this is not recommended for most users.

These variables help prevent runaway scripts from consuming excessive memory or CPU time. For detailed information about each variable, including configuration options, examples, and memory limit behavior, see js_lang component system variables.

Status variables

The js_lang component provides global status variables for monitoring JS routine execution:

  • Memory usage: js_lang_total_heap_size, js_lang_peak_total_heap_size, js_lang_used_heap_size, js_lang_peak_used_heap_size, js_lang_external_memory_size, js_lang_peak_external_memory_size
  • Contexts: js_lang_contexts, js_lang_peak_contexts
  • Call count: js_lang_stored_program_call_count

Note

Status variable values are approximate as they are refreshed at Isolate creation/destruction and GC time, not on each operation.

User-defined functions

The js_lang component includes a set of User-Defined Functions (UDFs) that retrieve and clear information about the last JS error that occurred in the current connection for the current user. This information updates each time a JS error occurs for the current connection and user. Successful execution of JS code does not change this state.

The following UDFs are helpful for debugging JS code:

  • JS_GET_LAST_ERROR(): Returns the error message for the last JS error that occurred in the current connection for the current user.

  • JS_GET_LAST_ERROR_INFO(): Returns extended information about the last JS error that occurred in the current connection for the current user. In addition to the error message, it tries to provide the exact line and column where the problem occurred, as well as the stack trace if available.

  • JS_CLEAR_LAST_ERROR(): Resets the information about the last JS error for the current connection and user, as if no error had occurred.

  • JS_GET_MEMORY_USAGE_JSON(): Returns information about memory usage by the JS environment (isolate) for the current user and connection pair, as well as total memory usage by all JS environments in the server, in the form of a JSON object. The returned object includes local (per-environment) and global (aggregated) memory statistics with heap sizes, external memory, and context counts. If there is no JS environment for the current user-connection pair, the local member is null.

For more information about using these functions for troubleshooting, see Troubleshoot js_lang procedures and functions.

Console API

The js_lang component provides support for the JS Console API as described at console.spec.whatwg.org . The Console API provides debugging support including logging, timers, and counters. Users can use methods of the JS console object to write messages to the console log, and then inspect this log using UDFs.

Our implementation supports logging calls (assert(), clear(), debug(), error(), info(), log(), warn()), counting (count(), countReset()), grouping (group(), groupCollapsed(), groupEnd()), and timing (time(), timeLog(), timeEnd()). Each user-connection pair has its own separate console log instance, with size limits controlled by the js_lang.max_console_log_size variable.

For detailed information about the Console API, including supported calls, format specifiers, log severity levels, and UDFs for accessing console logs, see js_lang Console API.

Further reading

Install js_lang component

This feature is a tech preview. Before using this feature in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.

The plugin_dir system variable specifies where the component library is located. If you need to, you should set the plugin_dir variable when you start the server.

To install the js_lang component, you need to run the following command:

INSTALL COMPONENT 'file://component_js_lang';

If you decide to uninstall the component, you may have to restart the server before you can reinstall it.

When you install the component_js_lang, it gives you a new global privilege called CREATE_JS_ROUTINE. This privilege allows you to create JS routines within the database.

For more details, check out INSTALL COMPONENT.

Further reading

js_lang component system variables

This feature is a tech preview. Before using this feature in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.

The following sections describe the system variables available in the js_lang component.

js_lang system variables

The following js_lang system variables are available:

Variable name Description
js_lang.max_mem_size Maximum memory size for JS routines
js_lang.max_mem_size_hard_limit_factor Maximum memory size hard limit factor

The following variables are described in detail:

js_lang.max_mem_size

Options Description
Command-line –js-lang.max-mem-size=value
Dynamic Yes
Scope Global
Data type Numeric
Default 8388608 (8 MB)
Minimum value 3145728 (3 MB)
Maximum value 1073741824 (1 GB)
Block size 1024 bytes

This variable sets the soft memory allocation limit for JS routines. The component configures V8 with this limit. Since every JS execution (or session, depending on the isolation mode) creates a V8 environment, this limit prevents a single poorly written script or a loop from consuming all the server’s physical memory.

Values are rounded down to the nearest multiple of 1024 bytes (block size). V8 requires a minimum heap size to start, which is typically around 10 MB.

V8 behavior with the soft limit:

The soft limit is a threshold that triggers V8’s garbage collection (GC) process. When memory usage approaches the soft limit configured by this variable, V8 starts incremental marking in the background to identify objects that can be freed. This background GC runs without pausing JS execution. If the soft limit is reached or exceeded and GC cannot free enough memory, V8 Isolate Termination is triggered, stopping JS execution and returning a “Memory limit exceeded” error to the MySQL client.

Relationship to hard limit:

The hard limit is calculated as js_lang.max_mem_size * js_lang.max_mem_size_hard_limit_factor and is configured separately using the js_lang.max_mem_size_hard_limit_factor variable. When the hard limit is reached, V8 performs a “last resort” garbage collection, stopping all execution to attempt to free every possible byte of memory. If this final GC cannot free enough memory, V8 triggers out-of-memory (OOM) handling, which terminates the mysqld process and causes a server exit. The hard limit is disabled by default (factor = 0) because it causes a server exit.

This variable is dynamic and can be changed at runtime using the SET statement. The updated value is applied to new V8 isolate contexts created after the change. Sessions that are already running may not be affected until the next time a context is initialized.

An example of setting the variable:

SET GLOBAL js_lang.max_mem_size = 16777216;

This sets the limit to 16 MB. To make the change persistent across server restarts, add this setting to your configuration file:

[mysqld]
js_lang.max_mem_size = 16777216

js_lang.max_mem_size_hard_limit_factor

Options Description
Command-line –js-lang.max-mem-size-hard-limit-factor=value
Dynamic No
Scope Global
Data type Numeric
Default 0
Minimum value 0
Maximum value 1024

This variable controls the hard limit for JS routine memory allocation. When set to a non-zero value, the component calculates the hard limit as js_lang.max_mem_size * js_lang.max_mem_size_hard_limit_factor in bytes.

When a JS session’s memory usage reaches the hard limit threshold, V8 performs a “last resort” garbage collection, stopping all execution to attempt to free every possible byte of memory. If this final GC cannot free enough memory, V8 triggers out-of-memory (OOM) handling, which terminates the mysqld process and causes a server exit. This is different from the soft limit behavior, where V8 Isolate Termination stops only the JS execution and returns an error to the client.

When set to 0 (the default), the component does not set an explicit V8 memory limit. Instead, V8 uses the default limit, which is typically greater than 1 GB). The default V8 limt avoids a server exit from an abrupt out-of-memory (OOM) scenario.

Warning

Allocating more than 1 GB of memory is not safe, as this allocation can exceed V8’s default hard limit and cause the server to exit.

A non-zero value enforces the V8 hard limit. This setting ensures that the hard memory limit will never be exceeded, at the price of process abort. Note that large single allocations that exceed this limit will cause server exit.

You should be aware that:

  • V8 may abort the process if a single allocation attempt exceeds the limit and garbage collection cannot free enough memory.

  • Changing this variable requires a server restart

In earlier versions, the component set the V8 memory limit to js_lang.max_mem_size * 4 bytes by default. This setting worked well for detecting memory limit violations with small allocations, but failed when a single huge allocation exceeded the limit. Attempting to allocate more than js_lang.max_mem_size * 4 bytes in one operation could cause the server to crash because V8 aborts the process when an allocation exceeds its memory limit and garbage collection cannot help.

Note

This variable relates to the internal heap limit for JS routines and works in conjunction with js_lang.max_mem_size. For more information about memory limits and troubleshooting, see Troubleshoot js_lang procedures and functions.

Further reading

js_lang privileges

This feature is a tech preview. Before using this feature in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.

Privileges control what users can do. You use them to give specific permissions to different users. This ability helps you keep your data secure by only allowing authorized users to access and change information in the database.

Privileges

To create routines within a database, you must be granted the CREATE_JS_ROUTINE privilege and the standard CREATE ROUTINE privilege.

GRANT CREATE_JS_ROUTINE ON *.* TO user1@localhost;

If a user is granted the ability to create routines and holds the CREATE_JS_ROUTINE privilege, they are capable of creating stored functions and procedures using JS.

However, it is important to note that at this time, the creation of JS triggers or events is not supported.

Further reading

js_lang stored function or procedure

This feature is a tech preview. Before using this feature in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.

Once the component’s installed, you can write stored functions and procedures in JavaScript. The syntax looks like this:

CREATE
    [DEFINER = user]
    FUNCTION [IF NOT EXISTS] sp_name ([func_parameter[,...]])
    RETURNS type
    LANGUAGE JS [other-func-characteristic ...] AS js_routine_body

CREATE
    [DEFINER = user]
    PROCEDURE [IF NOT EXISTS] sp_name ([proc_parameter[,...]])
    LANGUAGE JS [other-proc-characteristic ...] AS js_routine_body

routine_body:
        text_string_literal | dollar_quoted_string

Use the LANGUAGE JS clause when creating a routine.

CREATE FUNCTION f1(n INT) RETURNS INT LANGUAGE JS AS $$
    return n*42;
$$

CREATE PROCEDURE p1(a INT, b INT, OUT r INT) LANGUAGE JS AS $$
  r = a * b;
$$

You can modify or delete stored programs in JS by using the standard ALTER PROCEDURE/FUNCTION and DROP PROCEDURE/FUNCTION statements. These statements do not require the CREATE_JS_ROUTINE privilege.

Further reading

js_lang type conversions

This feature is a tech preview. Before using this feature in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.

SQL and JS use different data types, so the js_lang component converts values when passing SQL parameters to JS and back. This document describes how these conversions work.

Convert SQL data types to JS

SQL NULL values are converted to JS null values.

SQL type JS return type Notes
BOOLEAN, TINYINT, SHORTINT, MEDIUMINT, INT Number
BIGINT Number or BigInt Number for values [-2^53-1, 2^53-1], BigInt otherwise
DECIMAL String
FLOAT, DOUBLE Number
BIT(k) Number or BigInt Number for k ≤ 53, BigInt for k > 53
TIME, DATE, TIMESTAMP, DATETIME String
YEAR Number
CHAR, VARCHAR, TINYTEXT, TEXT, MEDIUMTEXT, LONGTEXT String Fails if length exceeds 2^29 - 24
BINARY, VARBINARY, TINYBLOB, BLOB, MEDIUMBLOB, LONGBLOB DataView
ENUM, SET String
GEOMETRY and spatial types DataView
JSON Object

When the data converts to a JS string, it automatically changes from the SQL parameter’s character set to utf8mb4, which JS uses.

Convert JS data types to SQL

The system uses the target SQL data type to determine how to convert each value. It typically converts a JS value into a basic type—such as a string, integer, or double—based on the specified SQL type. Once converted, the system stores the result in the corresponding SQL parameter or return value.

If a value exceeds allowed limits or uses an unsupported format, the conversion fails and triggers an error. During this process, the system automatically converts JS strings from the utf8mb4 encoding to the character set defined by the SQL parameter.

The system always maps JS null and undefined values to SQL NULL, regardless of the target SQL type.

JS to SQL type conversion rules

Target SQL Data Type Conversion Rules Explanation Example
BOOLEAN, TINYINT, SHORTINT, MEDIUMINT, INT, BIGINT (Version 8.4.5) - Numbers: stored as integers
- Booleans: true → 1, false → 0
- BigInts: stored as integers when possible
- Other types: converted to strings first
(Version 8.4.4) - JS Integers/Numbers: integers stored as-is, BigInts attempted as integers, others as strings.)		 Preserves native numeric forms where possible; other values default to string representation 42 → 42
3.14 → "3.14"
true → "1"
DECIMAL - All values converted to strings
- Booleans: converted to 0/1, then stored as doubles		 Supports precision formatting; special handling ensures Booleans fit numeric context 123.45 → "123.45"
true → 1.0
FLOAT, DOUBLE - Numbers: stored as doubles
- (Version 8.4.5) - Booleans: converted to 0/1, then stored as doubles
- Others: converted to strings		 Treats numeric and Boolean inputs consistently using floating-point representation 3.14 → 3.14
true → 1.0
"3.14" → "3.14"
BIT Converted to SQL BIT type Only binary-compatible values allowed 1 → BIT(1)
TIME, DATE, TIMESTAMP, DATETIME All values converted to strings Usually expects ISO date formats or equivalents Date() → "2024-01-30"
CHAR, VARCHAR, TEXT, etc. All values converted to strings
Charset conversion from utf8mb4 if needed		 Supports text types with encoding fallback "hello" → "hello"
BINARY, VARBINARY, BLOB, etc. - ArrayBuffer/View: stored directly
- Others: converted to strings		 Binary data must be explicitly wrapped; others fallback to string buffer → binary
SET - Numbers: stored as integers/doubles
- BigInts: stored as integers
- Others: converted to strings with charset conversion if needed		 Tries native storage before falling back to strings 1 → 1
"value" → "value"
GEOMETRY - Valid ArrayBuffer/View: stored as binary
- Others: cause an error		 Enforces format rules to maintain spatial integrity valid buffer → GEOMETRY
JSON Converted using JSON.stringify() Converts objects or arrays to serialized strings {key: "value"} → "{"key":"value"}"

Further reading

js_lang Console API

This feature is a tech preview. Before using this feature in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.

The js_lang component provides support for the JS Console API as described at console.spec.whatwg.org . The Console API provides rudimentary support for debugging including logging, timers, and counters. Users can use methods of the JS console object to write messages to the console log, and then later inspect this log using UDFs.

Note

This document does not go into details describing the behavior of each console call. Refer to the Console API specification for detailed information. You can also use developer.mozilla.org/en-US/docs/Web/API/console as a more convenient reference.

Supported logging calls

Our implementation supports the following logging calls described in the Console API specification:

  • assert()
  • clear()
  • debug()
  • error()
  • info()
  • log()
  • warn()

Supported counting, grouping, and timing calls

We also support calls implementing functionality for:

  • Counting: count(), countReset()
  • Grouping: group(), groupCollapsed(), groupEnd()
  • Timing: time(), timeLog(), timeEnd()

Unsupported calls

We do not support trace(), table(), dir(), and dirxml() calls from the specification. They can be called but do nothing.

Log severity levels

Our implementation supports Error, Warning, Info, and Debug log severity levels. The console.log() and console.timeLog() calls use Info log severity.

Substitution and format specifiers

For those logging calls which support substitution/format specifiers per specification, we support the following substitution/format specifiers: %s, %i, %d, %f, %o, %O, %c, %%.

String conversion

Unless the specification says otherwise (for example, when substitution/format specifiers are used), when we convert a JS value to a string in order to add it to a console log message, we try to use a representation that is useful for debugging purposes. However, for Object values, the current result of such conversion is rather brief and far from optimal (see Node.js or browsers for example).

Console log instances

Each user in each connection gets access to its own separate instance of the console log.

Console log size limits

The number of console log messages that are kept for each user-connection pair is limited by the global dynamic js_lang.max_console_log_size variable. Once the console instance for the pair reaches this limit, we start to discard the oldest messages from the console log when new messages are added.

Accessing console logs

The following UDFs are available for accessing and managing console logs:

  • JS_GET_CONSOLE_LOG(): Returns a simple and plain representation of the console log for the current user and connection pair. We only show the text of the log messages in this representation (for example, no timestamps or level of log messages is shown). Each log message gets its own line and is indented according to its grouping.

  • JS_GET_CONSOLE_LOG_JSON(): Returns a JSON representation of the console log for the current user and connection pair. Each log entry is represented by an object in a JSON array returned. This object contains all the available information about the log entry, such as the log entry timestamp, its log level, whether it belongs to a group (in which case we provide the hierarchy of the groups it belongs to in the form of an array), and whether it is a group header. This representation can be used for advanced processing of console log output such as filtering by severity or providing an interactive version of the log output.

  • JS_CLEAR_CONSOLE_LOG(): Empties the console log for the current user and connection pair. It also returns the number of log entries it has removed. Note that unlike the similar console.clear() call, this UDF does not reset the current group stack for the console instance.

Further reading

Troubleshoot js_lang procedures and functions

This feature is a tech preview. Before using this feature in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.

The component includes a set of User-Defined Functions (UDFs) that retrieve and clear information about the last JS error that occurred in the current connection for the current user. This information updates each time a JS error occurs for the current connection and user. Successful execution of JS code does not change this state.

The following UDFs are helpful for debugging JS code.

  • JS_GET_LAST_ERROR(): Returns the error message for the last JS error that occurred in the current connection for the current user.

  • JS_GET_LAST_ERROR_INFO(): Returns extended information about the last JS error that occurred in the current connection for the current user. In addition to the error message, it tries to provide the exact line and column where the problem occurred, as well as the stack trace if available.

  • JS_CLEAR_LAST_ERROR(): Resets the information about the last JS error for the current connection and user, as if no error had occurred.

Terminating JS routine execution

You can terminate the execution of a JS routine in the following ways:

  1. You can kill a connection or statement that executes a JS routine using KILL or KILL QUERY, and the execution is aborted without much delay. For example, you can use KILL QUERY to abort a JS routine executing a long or infinite loop.

  2. Exceeding the MAX_EXECUTION_TIME timeout (if present) for a statement that executes a JS routine aborts execution without much delay. For example, this option can be used to limit the execution time of a JS routine performing a long computation.

Further reading

Uninstall the js_lang component

This feature is a tech preview. Before using this feature in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.

The uninstall works only when no connections are using JavaScript stored programs. If there are connections, the procedure fails with an error.

To remove the component, run the following:

UNINSTALL COMPONENT 'file://component_js_lang';

Further reading

Kill idle transaction

Database servers face a constant challenge: managing resources efficiently while maintaining system stability. The kill idle transactions timeout option is a strategic tool to address this challenge. The server automatically stops any transaction that remains inactive for longer than this limit. This action prevents forgotten or stuck transactions from slowing down your database by blocking critical cleanup processes.

The option has the following benefits:

  • Automatically terminates long-running, inactive database connections

  • Prevents resource hogging by abandoned or forgotten transactions

  • Frees up database connection pools

  • Reduces unnecessary memory consumption

  • Protects against potential connection leaks

  • Prevents unnecessary server load from dormant sessions

You must also consider the following:

  • May interrupt legitimate long-running queries

  • Requires careful configuration to avoid disrupting critical processes

  • Can cause unexpected application behavior if timeout is too aggressive

  • Might create additional overhead in monitoring and logging

  • Requires precise tuning for different application requirements

We recommend that starting with a conservative timeout setting and reviewing the logs frequently to track terminated transactions.

This feature works with all types of database storage that support transactions.

Best practices

Consider these recommended practices when configuring the idle transaction timeout:

  • Starting with a higher timeout value: Begin by setting the kill_idle_transaction to a larger value, such as 600 seconds (10 minutes). This initial higher value provides a buffer and reduces the risk of prematurely terminating legitimate, long-running transactions while you observe your application’s behavior under normal load.

  • Monitoring logs for premature terminations: After implementing the idle transaction timeout, actively monitor the MySQL error logs for messages indicating that transactions are being killed. If you observe “Killed idle transaction” messages frequently for transactions that should still be active, it suggests that your timeout value is too aggressive and needs adjustment.

  • Testing in a staging environment: Before applying any changes to the kill_idle_transaction setting in your production environment, thoroughly test the configuration in a staging or development environment that closely mirrors your production setup. This testing allows you to identify and resolve any unintended consequences, such as premature transaction terminations, without impacting your live application and data.

Determine the idle transaction threshold

When setting up a database, you must decide how long to let inactive transactions sit before ending them. This decision affects the database’s performance.

Items to consider Description
How your database is used Look at how long transactions usually take and how often they happen. If most transactions finish quickly, you should end idle ones sooner.
How many things happen at once Count how many transactions your system handles simultaneously. You may need to end idle transactions faster to free up space for new ones.
How it affects speed Monitor how idle transactions change your database’s speed. If they noticeably slow down the database, ending these transaction can help keep everything running smoothly.
What your business needs What’s important for your work. Some transactions may need more time.

InnoDB purge

The InnoDB purge process removes outdated row versions (undo logs) from the system. When a transaction modifies data, InnoDB keeps old row versions for rollback and to support transactions running with multi-version concurrency control (MVCC). Once these versions are no longer needed, the purge process deletes them to free up space and improve performance.

Blocking the InnoDB purge can lead to increased disk space usage and potential performance degradation. This feature helps prevent issues such as:

Benefit Description
Limiting idle transactions Kills any idle transaction after a specified threshold, ensuring transactions don’t remain idle for too long.
Preventing mistakes Users can’t accidentally block the InnoDB purge by leaving transactions idle.
Improving performance Keeping the purge process running smoothly helps maintain optimal database performance.

System variables

kill_idle_transaction

Option Description
Config file Yes
Scope: Global
Dynamic: Yes
Data type Integer
Default value 0 (disabled)
Unit Seconds

If set to a non-zero value, the server kills any idle transaction after it stays idle for this number of seconds.

Examples

The SET GLOBAL kill_idle_transaction = 300; command configures the server to automatically end any idle transaction that has lasted for 300 seconds (5 minutes). This command immediately takes effect for the current and new server sessions. An idle transaction holds resources, potentially preventing other operations from proceeding. This setting helps to release these resources if a transaction is unintentionally left open.

SET GLOBAL kill_idle_transaction = 300;

The [mysqld] section in the my.cnf configuration file allows you to set server-wide options that persist across server restarts. Adding the line kill_idle_transaction = 300 under [mysqld] makes the idle transaction timeout of 300 seconds the default setting for the server. This setting ensures that the server automatically terminates idle transactions after 5 minutes every time it starts.

You must restart the server for changes in the my.cnf file to take effect. This configuration prevents long-held idle transactions from consuming resources over extended periods.

[mysqld]
kill_idle_transaction = 300

Monitor terminated transactions

If the kill_idle_transaction setting is active and idle transactions have been terminated, this command will output any lines from the error log that contain the “Killed idle transaction” message. Each matching line typically includes a timestamp and details about the terminated transaction, such as its ID and the duration it was idle.

If the kill_idle_transaction setting is active and idle transactions have been terminated, this command will output any lines from the error log that contain the “Killed idle transaction” message. Each matching line typically includes a timestamp and details about the terminated transaction, such as its ID and the duration it was idle.

The command produces no output if the server has not terminated any idle transactions since the last log rotation or server start. Regularly checking this log helps you verify that the idle transaction option is working as expected and provides insights into transaction management within your server.

$ grep "Killed idle transaction" /var/log/mysql/error.log

PERCONA_SEQUENCE_TABLE(n) function

Using the PERCONA_SEQUENCE_TABLE() function provides the following:

Benefit Description
Generates Sequences Acts as an inline table-valued function that generates a sequence of numbers.
Table-Valued Function Unlike traditional scalar functions, PERCONA_SEQUENCE_TABLE() returns a virtual table with a single column named value containing the generated sequence.
Simpler Syntax Simplifies queries that need to generate predictable sequences of numbers.
Flexibility Allows dynamic definition of sequences within queries, offering more control compared to pre-defined tables for sequences.
Predefined Sequence Does not manage sequences like Oracle or PostgreSQL; instead, it allows definition and generation of sequences within a SELECT statement.
Customization Enables customization of starting value, increment/decrement amount, and number of values to generate.

Version update

Percona Server for MySQL 8.4 deprecated SEQUENCE_TABLE(), and Percona may remove this function in a future release. We recommend that you use PERCONA_SEQUENCE_TABLE() instead.

To maintain compatibility with existing third-party software, SEQUENCE_TABLE is no longer a reserved term and can be used as a regular identifier.

Table functions

The function is an inline table-valued function. This function creates a temporary table with multiple rows. You can use this function within a single SELECT statement. Oracle MySQL Server only has the JSON_TABLE table function. The Percona Server for MySQL has the JSON_TABLE and PERCONA_SEQUENCE_TABLE() table functions. A single SELECT statement generates a multi-row result set. In contrast, a scalar function (like EXP(x) or LOWER(str) always returns a single value of a specific data type.

Syntax

As with any derived tables , a table function requires an alias in the SELECT statement.

The result set is a single column with the predefined column name value of type BIGINT UNSIGNED. You can reference the value column in SELECT statements. The following statements are valid. Using n as the number of generated values, the following is the basic syntax:

PERCONA_SEQUENCE_TABLE(n) [AS] alias

SELECT … FROM PERCONA_SEQUENCE_TABLE(n) [AS] alias

PERCONA_SEQUENCE_TABLE(n) [AS] alias

SELECT * FROM PERCONA_SEQUENCE_TABLE(n) AS tt;
SELECT <expr(value)> FROM PERCONA_SEQUENCE_TABLE(n) AS tt;

The first number in the series, the initial term, is defined as 0, and the series ends with a value less than n.

Basic sequence generation

In this example, the following statement generates a sequence:

SELECT * FROM PERCONA_SEQUENCE_TABLE(3) AS tt;
Expected output 
+-------+
| value |
+-------+
|     0 |
|     1 |
|     2 |
+-------+

Start with a specific value

You can define the initial value using the WHERE clause. The following example starts the sequence with 4.

SELECT value AS result \
       FROM \
            (SELECT seq AS value
             FROM PERCONA_SEQUENCE_TABLE(8)) AS tt \
       WHERE value >= 4;
Expected output 
+--------+
| result |
+--------+
|      4 |
|      5 |
|      6 |
|      7 |
+--------+

Filter even numbers

Consecutive terms increase or decrease by a common difference. The default common difference value is 1. However, it is possible to filter the results using the WHERE clause to simulate common differences greater than 1.

The following example prints only even numbers from the 0..7 range:

SELECT value AS result \
       FROM PERCONA_SEQUENCE_TABLE(8) AS tt \
       WHERE value % 2 = 0;
Expected output 
+--------+
| result |
+--------+
|      0 |
|      2 |
|      4 |
|      6 |
+--------+

Generate random numbers

The following is an example of using the function to populate a table with a set of random numbers:

SELECT FLOOR(RAND() * 100) AS result \
       FROM PERCONA_SEQUENCE_TABLE(4) AS tt;

The output could be the following:

Expected output 
+--------+
| result |
+--------+
|     24 |
|     56 |
|     70 |
|     25 |
+--------+

Generate random strings

You can populate a table with a set of pseudo-random strings with the following statement:

SELECT MD5(value) AS result \
       FROM PERCONA_SEQUENCE_TABLE(4) AS tt;
Expected output 
+----------------------------------+
| result                           |
+----------------------------------+
| f17d9c990f40f8ac215f2ecdfd7d0451 |
| 2e5751b7cfd7f053cd29e946fb2649a4 |
| b026324c6904b2a9cb4b88d6d61c81d1 |
| 26ab0db90d72e28ad0ba1e22ee510510 |
+----------------------------------+

Add a sequence to a table

You can add the sequence as a column to a new table or an existing table, as shown in this example:

CREATE TABLE t1 AS SELECT * FROM PERCONA_SEQUENCE_TABLE(4) AS tt;

SELECT * FROM t1;
Expected output 
+-------+
| value |
+-------+
|     0 |
|     1 |
|     2 |
|     3 |
+-------+

Sequences are helpful for various purposes, such as populating tables and generating test data.

The ProcFS plugin

The ProcFS plugin provides access to the Linux performance counters by running SQL queries against a Percona Server for MySQL 8.4.

You may be unable to capture operating system metrics in certain environments, such as Cloud installations or MySQL-as-a-Service installations. These metrics are essential for complete system performance monitoring.

The plugin does the following:

  • Reads selected files from the /proc file system and the /sys file system.

  • Populates the file names and their content as rows in the INFORMATION_SCHEMA.PROCFS view.

The system variable procfs_files_spec provides access to the /proc and the /sys files and directories. This variable cannot be changed at run time, preventing a compromised account from giving itself greater access to those file systems.

Install the PLUGIN manually

We recommend installing the plugin as part of the package. If needed, you can install this plugin manually. Copy the procfs.so file to the mysql plugin installation directory and execute the following command:

INSTALL PLUGIN procfs SONAME 'procfs.so';

Access privileges required

Only users with the ACCESS_PROCFS dynamic privilege can access the INFORMATION_SCHEMA.PROCFS view. During the plugin startup, this dynamic privilege is registered with the server.

After the plugin installation, grant a user access to the INFORMATION_SCHEMA.PROCFS view by executing the following command:

GRANT ACCESS_PROCFS ON *.* TO 'user'@'host';

Important

An SELinux policy or an AppArmor profile may prevent access to file locations needed by the ProcFS plugin, such as the ‘/proc/sys/fs/file-nr’ directory or any sub-directories or files under ‘/proc/irq/’. Either edit the policy or profile to ensure that the plugin has the necessary access. If the policy and profile do not allow access, the plugin may may have unexpected behavior.

For more information, see Working with SELinux and Working with AppArmor.

Using the ProcFS plugin

Authorized users can obtain information from individual files by specifying the exact file name within a WHERE clause. Files that are not included are ignored and considered not to exist.

All files that match the procfs_files_spec are opened, read, stored in memory, and, finally, returned to the client. It is critical to add a WHERE clause to return only specific files to limit the impact of the plugin on the server’s performance. A failure to use a WHERE clause can lead to lengthy query response times, high load, and high memory usage on the server. The WHERE clause can contain either an equality operator, the LIKE operator, or the IN operator. The LIKE operator limits file globbing. You can write file access patterns in the glob(7) style , such as /sys/block/sd[a-z]/stat;/proc/version\*

The following example returns the proc/version:

SELECT * FROM INFORMATION_SCHEMA.PROCFS WHERE FILE = '/proc/version';

Tables

PROCFS

The schema definition of the INFORMATION_SCHEMA.PROCFS view is:

CREATE TEMPORARY TABLE `PROCFS` (
`FILE` varchar(1024) NOT NULL DEFAULT '',
`CONTENTS` longtext NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Status variables provide the basic metrics:

Name Description
procfs_access_violations The number of attempted queries by users without the ACCESS_PROCFS privilege.
procfs_queries The number of queries made against the procfs view.
procfs_files_read The number of files read to provide content
procfs_bytes_read The number of bytes read to provide content

Variable

procfs_files_spec

Option Description
Scope: Global
Dynamic: Yes
Read, Write, or Read-Only: Read-Only

The default value for procfs_files_spec is: /proc/cpuinfo;/proc/irq//;/proc/loadavg/proc/net/dev;/proc/net/sockstat;/proc/net/sockstat_rhe4;/proc/net/tcpstat;/proc/self/net/netstat;/proc/self/stat;/proc/self/io;/proc/self/numa_maps/proc/softirqs;/proc/spl/kstat/zfs/arcstats;/proc/stat;/proc/sys/fs/file-nr;/proc/version;/proc/vmstat

Enables access to the /proc and /sys directories and files. This variable is global, read only, and is set by using either the mysqld command line or by editing my.cnf.

Limitations

The following limitations are:

  • Only first 60k of /proc/ /sys/ files are returned

  • The file name size is limited to 1k

  • The plugin cannot read files if path does not start from /proc or /sys

  • Complex WHERE conditions may force the plugin to read all configured files.

Uninstall plugin

The following statement removes the procfs plugin.

UNINSTALL PLUGIN procfs;

Support for PROXY protocol

The proxy protocol allows an intermediate proxying server speaking proxy protocol (ie. HAProxy) between the server and the ultimate client (i.e. mysql client etc) to provide the source client address to the server, which normally would only see the proxying server address instead.

As the proxy protocol amounts to spoofing the client address, it is disabled by default, and can be enabled on per-host or per-network basis for the trusted source addresses where trusted proxy servers are known to run. Unproxied connections are not allowed from these source addresses.

Note

Ensure that proper firewall access control lists (ACL) are in place when this feature is enabled.

Proxying is supported only for TCP over IPv4 and IPv6 connections. The UNIX socket connections can not be proxied and do not fall under the effect of using the asterisk symbol (*).

You cannot have a proxied IP address that is 127.0.0.1 or ::1, even if the IP address is in the proxy_protocol_networks.

System variables

proxy_protocol_networks

Option Description
Command-line Yes
Config file Yes
Scope Global
Dynamic No
Default (empty string)

This variable is a global-only, read-only variable, which is either an asterisk symbol(*), or a list of comma-separated IPv4 and IPv6 network and host addresses. For security reasons we do not recommend using an asterisk symbol for the IP address. This symbol causes the server to accept the proxy protocol from any host. Network addresses are specified in CIDR notation, i.e. 192.168.0.0/24. To prevent source host spoofing, the setting of this variable must be as restrictive as possible to include only trusted proxy hosts.

SEQUENCE_TABLE(n) function

Using SEQUENCE_TABLE() function provides the following:

Benefit Description
Generates Sequences Acts as an inline table-valued function that generates a sequence of numbers.
Table-Valued Function Unlike traditional scalar functions, SEQUENCE_TABLE() returns a virtual table with a single column named value containing the generated sequence.
Simpler Syntax Simplifies queries that need to generate predictable sequences of numbers.
Flexibility Allows dynamic definition of sequences within queries, offering more control compared to pre-defined tables for sequences.
Predefined Sequence Does not manage sequences like Oracle or PostgreSQL; instead, it allows definition and generation of sequences within a SELECT statement.
Customization Enables customization of starting value, increment/decrement amount, and number of values to generate.

Version update

Percona Server for MySQL 8.4 deprecated SEQUENCE_TABLE(), and this function can be removed in a future release. We recommend that you use PERCONA_SEQUENCE_TABLE() instead.

To maintain compatibility with existing third-party software, SEQUENCE_TABLE is no longer a reserved term and can be used as a regular identifier.

Table functions

The function is an inline table-valued function. This function creates a temporary table with multiple rows. You can use this function within a single SELECT statement. Oracle MySQL Server only has the JSON_TABLE table function. The Percona Server for MySQL has the JSON_TABLE and SEQUENCE_TABLE() table functions. A single SELECT statement generates a multi-row result set. In contrast, a scalar function (like EXP(x) or LOWER(str) always returns a single value of a specific data type.

Syntax

As with any derived tables , a table function requires an alias in the SELECT statement.

The result set is a single column with the predefined column name value of type BIGINT UNSIGNED. You can reference the value column in SELECT statements. The following statements are valid. Using n as the number of generated values, the following is the basic syntax:

  • SEQUENCE_TABLE(n) [AS] alias
SELECT … FROM SEQUENCE_TABLE(n) [AS] alias

SEQUENCE_TABLE(n) [AS] alias

SELECT * FROM SEQUENCE_TABLE(n) AS tt;
SELECT <expr(value)> FROM SEQUENCE_TABLE(n) AS tt;

The first number in the series, the initial term, is defined as 0, and the series ends with a value less than n.

Example usage

Using SEQUENCE_TABLE():

SELECT * FROM SEQUENCE_TABLE(5)) AS sequence_data;

Using PERCONA_SEQUENCE_TABLE():

SELECT * FROM PERCONA_SEQUENCE_TABLE(5)) AS sequence_data;

Basic sequence generation

In this example, the following statement generates a sequence:

SELECT * FROM SEQUENCE_TABLE(3) AS tt;
Expected output 
+-------+
| value |
+-------+
|     0 |
|     1 |
|     2 |
+-------+

Start with a specific value

You can define the initial value using the WHERE clause. The following example starts the sequence with 4.

SELECT value AS result FROM SEQUENCE_TABLE(8) AS tt WHERE value >= 4;
Expected output 
+--------+
| result |
+--------+
|      4 |
|      5 |
|      6 |
|      7 |
+--------+

Filter even numbers

Consecutive terms increase or decrease by a common difference. The default common difference value is 1. However, it is possible to filter the results using the WHERE clause to simulate common differences greater than 1.

The following example prints only even numbers from the 0..7 range:

SELECT value AS result FROM SEQUENCE_TABLE(8) AS tt WHERE value % 2 = 0;
Expected output 
+--------+
| result |
+--------+
|      0 |
|      2 |
|      4 |
|      6 |
+--------+

Generate random numbers

The following is an example of using the function to populate a table with a set of random numbers:

SELECT FLOOR(RAND() * 100) AS result FROM SEQUENCE_TABLE(4) AS tt;

The output could be the following:

Expected output 
+--------+
| result |
+--------+
|     24 |
|     56 |
|     70 |
|     25 |
+--------+

Generate random strings

You can populate a table with a set of pseudo-random strings with the following statement:

SELECT MD5(value) AS result FROM SEQUENCE_TABLE(4) AS tt;
Expected output 
+----------------------------------+
| result                           |
+----------------------------------+
| f17d9c990f40f8ac215f2ecdfd7d0451 |
| 2e5751b7cfd7f053cd29e946fb2649a4 |
| b026324c6904b2a9cb4b88d6d61c81d1 |
| 26ab0db90d72e28ad0ba1e22ee510510 |
+----------------------------------+

Add a sequence to a table

You can add the sequence as a column to a new table or an existing table, as shown in this example:

CREATE TABLE t1 AS SELECT * FROM SEQUENCE_TABLE(4) AS tt;

SELECT * FROM t1;
Expected output 
+-------+
| value |
+-------+
|     0 |
|     1 |
|     2 |
|     3 |
+-------+

Sequences are helpful for various purposes, such as populating tables and generating test data.

Slow query log rotation and expiration

Percona has implemented two new variables, max_slowlog_size and max_slowlog_files to provide users with ability to control the slow query log disk usage. These variables have the same behavior as the max_binlog_size variable and the max_binlog_files variable used for controlling the binary log.

max_slowlog_size

Option Description
Command-line Yes
Config file Yes
Scope Global
Dynamic Yes
Data type numeric
Default 0 (unlimited)
Range 0 - 1073741824

The max_slowlog_size variable controls when the server rotates the slow query log file based on size.

By default, the value is set to 0, which means the server does not automatically rotate the slow query log file.

The block size is 4096 bytes. If you set a value that is not a multiple of 4096, the server rounds it down to the nearest multiple of 4096. For example, setting max_slowlog_size to any value less than 4096 will effectively set the value to 0.

If you set a limit for this size and enable this feature, the server will rename the slow query log file to slow_query_log_file.000001 once it reaches the specified size.

max_slowlog_files

Option Description
Command-line Yes
Config file Yes
Scope Global
Dynamic Yes
Data type numeric
Default 0 (unlimited)
Range 0 - 102400

This variable limits the total amount of slow query log files and is used with max_slowlog_size.

The server creates and adds slow query logs until reaching the range’s upper value. When the upper value is reached, the server creates a new slow query log file with a higher sequence number and deletes the log file with the lowest sequence number maintaining the total amount defined in the range.

Thread pool

Thread pooling can improve performance and scalability for MySQL databases. This technique reuses a fixed number of threads to handle multiple client connections and execute statements. It reduces the overhead of creating and destroying threads and avoids the contention and context switching that can occur when there are too many threads.

If you have fewer than 20,000 connections, using the thread pool does not provide significant benefits. It’s better to keep thread pooling disabled and use the default method.

The default method, called one-thread-per-connection, creates a new thread for each client that connects to the MySQL server. This thread manages all queries and responses for that connection until it’s closed. This approach works well for a moderate number of connections, but it can become inefficient as the number of connections increases.

MySQL supports thread pooling through the thread pool plugin, which replaces the default one-thread-per-connection model. When a statement arrives, the thread group either begins executing it immediately or queues it for later execution in a round-robin fashion. The high-priority queue consists of several thread groups, each managing client connections. Each thread group has a listener thread that listens for incoming statements from the connections assigned to the group. The thread pool exposes several system variables that can be used to configure its operation, such as thread_pool_size, thread_pool_algorithm, thread_pool_stall_limit, and others.

The thread pool plugin consists of several thread groups, each of which manages a set of client connections. As connections are established, the thread pool assigns them to thread groups using the round-robin method. This method assigns threads fairly and efficiently. Here’s how it works:

  1. The thread pool starts with a set number of thread groups.

  2. When a new task arrives, the pool needs to assign it to a group.

  3. It does this by going through the groups in order, one by one.

  4. Let’s say you have four thread groups. The assignment would work like this:

  5. Task 1 goes to Group 1
  6. Task 2 goes to Group 2
  7. Task 3 goes to Group 3
  8. Task 4 goes to Group 4

  9. Task 5 goes back to Group 1

  10. This pattern continues, always moving to the next group and starting over when it reaches the end.

  11. Each group handles its assigned tasks using its available threads.

This round-robin approach spreads work evenly across all groups. It prevents any single group from getting overloaded while others sit idle. This method helps maintain balanced performance across the system.

MySQL executes statements using one thread per client connection. When the number of connections increases past a specific point, performance degrades. This feature introduces a dynamic thread pool, which enables the server to maintain top performance even with a large number of client connections. The server decreases the number of threads using the thread pool and reduces the context switching and hot lock contentions. The thread pool is most effective with OLTP workloads (relatively short CPU-bound queries).

Set the thread pool variable thread_handling to pool-of-threads by adding the following line to my.cnf:

thread_handling=pool-of-threads

Although the default values for the thread pool should provide good performance, additional tuning should be performed with the dynamic system variables. The goal is to minimize the number of open transactions on the server. Short-running transactions commit faster and deallocate server resources and locks.

Due to the following differences, this implementation is not compatible with upstream:

  • Built into the server, upstream implements the thread pool as a plugin

  • Does not minimize the number of concurrent transactions

Priority Queue:

A queue that assigns a priority to each data element and processes them according to their priority. The data element with the highest priority is served first, regardless of its order in the queue. A priority queue can be implemented using an array, a linked list, a heap, or a binary search tree. It can also be ascending or descending, meaning that the highest priority is either the smallest or the largest value.

Priority connection scheduling

The thread pool limits the number of concurrently running queries. The number of open transactions may remain high. Connections with already-started transactions are added to the end of the queue. A high number of open transactions has implications for the currently running queries. The thread_pool_high_prio_tickets variable controls the high-priority queue policy and assigns tickets to each new connection.

The thread pool adds the connection to the high-priority queue and decrements the ticket if the connection has the following attributes:

  • Has an open transaction

  • Has a non-zero number of high-priority tickets

Otherwise, the variable adds the connection to the normal-priority queue with the initial value.

Each time, the thread pool checks the high-priority queue for the next connection. When the high-priority queue is empty, the thread pool picks connections from the normal-priority queue. The default behavior is to put events from already started transactions into the high-priority queue.

If the value equals 0, all connections are put into the normal-priority queue. If the value exceeds zero, each connection could be put into a high-priority queue.

The thread_pool_high_prio_mode variable prioritizes all statements for a connection or assigns connections to the normal-priority queue. To implement this new thread_pool_high_prio_mode variable

Normal-priority queue throttling

Thread pool performance can degrade, or even deadlock, under high concurrency when a thread group becomes oversubscribed — that is, when the number of active worker threads in the group reaches thread_pool_oversubscribe — and most of those threads are simultaneously waiting on locks held by a transaction whose connection has not yet been picked up by the thread pool.

The oversubscribe limit does not account for threads that have marked themselves inactive while waiting. As a result, the total number of threads in the pool (both active and waiting) continues to grow until the pool reaches thread_pool_max_threads. If the connection executing the transaction that holds the lock enters the thread pool before thread_pool_max_threads is reached, the pool ends up running a large number of threads concurrently (proportional to thread_pool_max_threads), resulting in suboptimal performance. Otherwise, the pool deadlocks, because no more threads can be created to process the blocking transactions and release their locks.

To prevent this scenario, the thread pool throttles the normal-priority queue when the total number of worker threads (both active and waiting) reaches the thread_pool_oversubscribe limit. While the pool is throttled, new transactions are not started and no new threads are created; the pool processes only queued events from already-started transactions, allowing their locks to be released so the pool can drain.

Handling long network waits

Specific workloads (large result sets, BLOBs, slow clients) can wait longer on network I/O (socket reads and writes). Whenever the server waits, this should be communicated to the thread pool so it can start a new query by either waking a waiting thread or sometimes creating a new one.

System variables

thread_handling

Option Description
Command-line Yes
Config file Yes
Scope Global
Dynamic No
Data type String
Default one-thread-per-connection

Defines how the server dispatches statements from client connections to execution threads. The default, one-thread-per-connection, dedicates a single thread to each connection for the connection’s entire lifetime. To activate the thread pool, set thread_handling to pool-of-threads in my.cnf and restart the server; all connections are then processed through the shared pool controlled by thread_pool_size and the other thread_pool_* variables. The no-threads value serves all connections from a single thread and is intended for debugging only. Because thread_handling is not dynamic, changing the value requires a server restart. As a rule of thumb, the thread pool benefits workloads with very high connection counts (typically 20,000 or more) and short CPU-bound queries; for smaller connection counts, one-thread-per-connection usually performs better.

Value Description
one-thread-per-connection One thread handles all requests for a connection.
pool-of-threads A thread pool handles requests for all connections.
no-threads A single thread serves all connections (debugging mode only).

thread_pool_high_prio_mode

This variable provides more fine-grained control over high-priority scheduling globally or per connection.

The following values are allowed:

Value Description
transactions Default. Only statements from already started transactions may go into the high-priority queue, depending on the number of high-priority tickets currently available in the connection (see thread_pool_high_prio_tickets).
statements All individual statements go into the high-priority queue, regardless of transactional state or available tickets. Use the statements value to prioritize AUTOCOMMIT transactions or administrative statements. Setting thread_pool_high_prio_mode to statements globally effectively disables high-priority scheduling, because all connections use the high-priority queue.
none Disables the priority queue for a connection. Useful for connections that are insensitive to execution latency, such as monitoring. Setting thread_pool_high_prio_mode to none globally effectively disables high-priority scheduling, because all connections use the normal-priority queue.

thread_pool_high_prio_tickets

Option Description
Command-line: Yes
Config file: Yes
Scope: Global, Session
Dynamic: Yes
Data type: Numeric
Default value: 4294967295

This variable controls the high-priority queue policy. Assigns the selected number of tickets to each new connection to enter the high-priority queue. Setting this variable to 0 disables the high-priority queue.

thread_pool_idle_timeout

Option Description
Command-line: Yes
Config file: Yes
Scope: Global
Dynamic: Yes
Data type: Numeric
Default value: 60 (seconds)

Defines the number of seconds a worker thread that has no work remains in the thread pool before exiting. When an idle thread’s wait exceeds thread_pool_idle_timeout, the thread is removed from the pool, freeing memory and OS resources; if load returns, a new worker thread is created (subject to thread_pool_oversubscribe and thread_pool_max_threads). A lower value reclaims resources faster under bursty or intermittent workloads but increases thread-creation overhead when activity resumes; a higher value keeps threads warm and reduces creation churn at the cost of additional idle memory. Observe Threadpool_idle_threads and Threadpool_threads to see how many threads are being retained in the pool at the current setting.

thread_pool_max_threads

Option Description
Command-line: Yes
Config file: Yes
Scope: Global
Dynamic: Yes
Data type: Numeric
Default value: 100000

This variable can limit the maximum number of threads in the pool. When the limit is reached, the server does not create new threads.

thread_pool_oversubscribe

Option Description
Command-line: Yes
Config file: Yes
Scope: Global
Dynamic: Yes
Data type: Numeric
Default value: 3

Defines the maximum number of worker threads that may be active simultaneously within a single thread group. When a worker thread in a group blocks (on I/O, a lock, or is otherwise stalled — see thread_pool_stall_limit), the group may create additional worker threads up to the thread_pool_oversubscribe limit so that queued requests can continue to be processed. A higher value improves responsiveness under workloads that frequently block, at the cost of more context switching and contention; a lower value keeps concurrency tighter to the CPU, but values below the default of 3 can cause frequent thread sleep and wake-up cycles. When the total number of worker threads in a group (both active and waiting) reaches the thread_pool_oversubscribe limit, the thread pool throttles the normal-priority queue (see Normal-priority queue throttling) to prevent deadlocks and runaway thread creation. Tune alongside thread_pool_size and observe Threadpool_threads together with the queue-wait metrics to evaluate the effect.

thread_pool_size

Option Description
Command-line: Yes
Config file: Yes
Scope: Global
Dynamic: Yes
Data type: Numeric
Default value: Number of processors

Defines the number of thread groups in the thread pool. Each thread group maintains its own high-priority and normal-priority queues and handles the connections assigned to the group, using one active worker thread at a time; additional worker threads may be created within the group up to the thread_pool_oversubscribe limit. New connections are distributed across the thread groups in round-robin order, so thread_pool_size effectively sets the target concurrency of the thread pool. As a starting point, use a value equal to the number of available CPU cores and tune from there based on observed Threadpool_idle_threads, Threadpool_threads, and the queue-wait metrics Threadpool_average_queue_wait_us and Threadpool_average_hp_queue_wait_us.

thread_pool_stall_limit

Option Description
Command-line: Yes
Config file: Yes
Scope: Global
Dynamic: No
Data type: Numeric
Default value: 500 (ms)

Defines the number of milliseconds before a running thread is considered stalled. When this limit is reached, the thread pool will wake up or create another thread. This variable prevents a long-running query from monopolizing the pool.

Status variables

Threadpool_average_hp_queue_wait_us

Option Description
Scope: Global
Data type: String

This status variable reports aggregated wait-time statistics, in microseconds, for requests waiting in the high-priority queue. The value is a formatted string, for example:

avg: 0.000, min: 0.000, max: 0.000, dev: 0.000, cnt: 0

Each sample is a single wait-time measurement, in microseconds, taken when a request is dequeued by a worker thread. Every dequeued request contributes exactly one sample. The cnt field shows the number of samples collected since server start; avg, min, max, and dev (standard deviation) are computed across all samples. Requests that the listener thread picks up immediately, without waiting, are included as zero-valued samples.

Threadpool_average_queue_wait_us

Option Description
Scope: Global
Data type: String

This status variable reports aggregated wait-time statistics, in microseconds, for requests waiting in the normal-priority queue. The value is a formatted string, for example:

avg: 590.000, min: 470.000, max: 736.000, dev: 110.266, cnt: 5

Each sample is a single wait-time measurement, in microseconds, taken when a request is dequeued by a worker thread. Every dequeued request contributes exactly one sample. The cnt field shows the number of samples collected since server start; avg, min, max, and dev (standard deviation) are computed across all samples. Requests that the listener thread picks up immediately, without waiting, are included as zero-valued samples.

Threadpool_idle_threads

Option Description
Scope: Global
Data type: Numeric

This status variable shows the number of idle threads in the pool — worker threads that are part of the pool but are not currently executing a request. Together with Threadpool_threads, Threadpool_idle_threads indicates how busy the pool is: when the value is close to Threadpool_threads, the pool is underutilized; when the value is near zero, the pool is saturated. Threads that remain idle longer than thread_pool_idle_timeout seconds may exit the pool.

Threadpool_requests_starved_in_queue

Option Description
Scope: Global
Data type: Numeric

This status variable shows the number of requests in the normal-priority queue that are being starved by requests in the high-priority queue. A non-zero value indicates that worker threads are busy processing high-priority traffic and normal-priority requests are not being picked up, which can be used as an indicator of thread pool saturation.

Threadpool_requests_waiting_in_hp_queue

Option Description
Scope: Global
Data type: Numeric

This status variable shows the number of requests currently waiting in the thread pool’s high-priority queue. Which requests enter the high-priority queue is governed by thread_pool_high_prio_mode and thread_pool_high_prio_tickets. A sustained or growing value indicates that high-priority traffic is arriving faster than worker threads can drain the queue. Observe Threadpool_requests_waiting_in_hp_queue alongside Threadpool_average_hp_queue_wait_us to gauge the effect on latency, and alongside Threadpool_requests_starved_in_queue to detect when high-priority traffic is starving the normal-priority queue.

Threadpool_requests_waiting_in_queue

Option Description
Scope: Global
Data type: Numeric

This status variable shows the number of requests currently waiting in the thread pool’s normal-priority queue. Requests enter the normal-priority queue when they do not meet the criteria for the high-priority queue set by thread_pool_high_prio_mode and thread_pool_high_prio_tickets. A sustained or growing value indicates either that normal-priority traffic is arriving faster than worker threads can drain the queue, or that high-priority traffic is monopolizing the worker threads. Observe Threadpool_requests_waiting_in_queue alongside Threadpool_average_queue_wait_us to gauge the effect on latency, and alongside Threadpool_requests_starved_in_queue to detect starvation by the high-priority queue.

Threadpool_threads

Option Description
Scope: Global
Data type: Numeric

This status variable shows the total number of worker threads currently in the pool, including both idle threads and threads actively executing requests. The pool grows dynamically, up to thread_pool_max_threads, as load increases and threads stall (see thread_pool_stall_limit); threads that remain idle longer than thread_pool_idle_timeout seconds may exit. The target concurrency of the pool is set by thread_pool_size and thread_pool_oversubscribe. Compare Threadpool_threads with Threadpool_idle_threads to determine how many threads are actively running requests: busy threads = Threadpool_threads − Threadpool_idle_threads.

Trigger updates

In MySQL, the system efficiently handles multiple client queries to the same table by opening separate table instances for each query. This prevents delays and conflicts. The use of a “Table Cache” speeds up access by reducing the need to repeatedly open and close tables, improving overall performance.

The table_open_cache system variable controls the number of tables MySQL can keep open simultaneously across all threads. By increasing this setting, MySQL can handle more open files, although this requires more file descriptors. Despite a soft limit, MySQL can temporarily exceed it if queries demand more open tables. Upon query completion, MySQL automatically manages the cache by closing the least recently used tables.

The table_open_cache_instances system variable controls the number of open table cache instances in MySQL. By splitting the open tables cache into smaller segments (table_open_cache divided by table_open_cache_instances), sessions can access only one instance at a time for DML operations, reducing contention and improving performance when many sessions are running. For systems with 16 or more CPU cores, a value of 8 or 16 is recommended. However, if many large triggers are causing high memory usage, setting this variable to 1 can help limit memory consumption.

When a table with triggers is opened in the Table Cache, it also reads the trigger definitions and links the open table instance to its specific trigger instances. When a connection executes a Data Manipulation Language (DML) statement that activates a trigger, that connection uses its own instance of the trigger body for that particular table instance. This method of caching both the open table instances and their associated trigger bodies can unexpectedly use a significant amount of memory.

Percona Server for MySQL has the following abilities:

  • Avoid using table instances with fully-loaded and parsed triggers by read-only queries

  • Show trigger CREATE statements even if the statement is unparseable

The additional system variable reduces the Table Cache memory consumption on the server when tables that contain trigger definitions also are part of a significant read-only workload.

System variables

table_open_cache_triggers

Option Description
Command-line --table-open-cache-triggers
Dynamic Yes
Scope Global
Data type Integer
Default 524288
Minimum value 1
Maximum value 524288

This variable sets a soft limit on the maximum number of open tables in the Table Cache, which holds fully loaded triggers. By default, this value is set to the maximum to prevent any changes in behavior for existing users. If the number of open table instances with fully loaded triggers exceeds this limit, the system removes the least recently used unused table instances. You can set this value as a start-up option or change it dynamically while the system runs.

Status variables

The following status variables are available:

Variable name Description
table_open_cache_triggers_hits A hit means the statement required an open table instance with fully-loaded triggers and was able to get one from the table_open_cache.
table_open_cache_triggers_misses A miss means the statement requiring an open table instance with fully-loaded triggers was not found one in the table_open_cache. The statement may find a table instance without fully-loaded triggers and finalized their loading for it.
table_open_cache_triggers_overflows An overflow indicates the number of unused table instances with triggers that were expelled from the table_open_cache due to the table_open_cache_triggers soft limit. This variable may demonstrate that the table_open_cache_triggers value should be increased.

SHOW CREATE TRIGGER statment changes

The SHOW CREATE TRIGGER statement displays the SQL command that created a trigger, including definitions that may no longer be understandable. For example, if a trigger was created before a server upgrade that changed the trigger syntax, this statement will still show its definition.

Additional resources

For more information, see How MySQL opens and closes tables .

Percona Toolkit UDFs

The Percona Toolkit component adds user-defined functions (UDFs) that provide fast 64-bit hashing. These UDFs are primarily about speed and data integrity. They are implemented in C++ to perform the hashing operations much faster than standard SQL or stored procedures. With datasets often in the multi-terabyte range, they address several critical scenarios.

Prerequisites

Before using these functions, add the Percona Repository and install the component.

Why use these functions

Use case Context Benefit
High-speed data checksums (pt-table-checksum) pt-table-checksum must hash every row to verify replicas. MySQL’s built-in MD5() or SHA1() is CPU-intensive and slow for billions of rows. When you install fnv_64 or murmur_hash, the checksum process becomes significantly faster; pt-table-checksum can finish in hours instead of days.
Efficient data sharding and partitioning When you shard data across multiple servers, you need a consistent way to map a key (for example, a user ID) to a specific shard. A common pattern is murmur_hash(user_id) % 10 to distribute data across 10 shards. MurmurHash has a low collision rate and uniform distribution, so shards stay balanced and one server does not become a hotspot.
Change Data Capture (CDC) and auditing When you sync data to an external warehouse (for example, Snowflake or BigQuery) without a full binary log parser, you can store a hash of each row in a row_hash column and recompute on the next sync to detect changed rows. fnv_64 is very lightweight, so you can run this in an INSERT or UPDATE trigger with negligible impact on transaction latency.
Fast lookups for large string columns For tables with very long strings (for example, URLs or JSON) that you need to index or check for uniqueness, indexing the full string is inefficient. Index fnv_64(string) instead; you get a 64-bit integer index that is smaller, uses the buffer pool efficiently, and supports fast equality lookups.

Functions provided

The component registers these functions (one load registers all of them):

Function Description
fnv_64() 64-bit FNV hash. Simple, fast.
fnv1a_64() 64-bit FNV-1a variant. Often better distribution than FNV on sequential input.
murmur_hash() 64-bit MurmurHash. Strong avalanche; good for fewer collisions on diverse data.

Each function takes a string or numeric value and returns an unsigned 64-bit integer.

Technical comparison: FNV vs Murmur

Aspect fnv_64 / fnv1a_64 murmur_hash()
Speed Very fast; minimal operations per byte. Very fast; often comparable or slightly better on long strings.
Collision behavior Good for general use. FNV can cluster more on sequential or similar inputs; FNV-1a usually improves that. Better distribution and avalanche (small input change → large output change). Prefer for large datasets or when collision rate matters.
When to use General-purpose hashing, simple checksums, shard keys when collision risk is low. Distributed systems, large tables, or when you need the lowest practical collision rate.

Benchmark with your own data and workload when you choose; performance depends on key length, data patterns, and hardware.

Installation

The UDFs are shipped as a MySQL component (not as separate plugin files). After you install the Percona Server package, load the component. Full repository and package steps are in Install Percona Toolkit UDFs (Optional). Then run:

INSTALL COMPONENT 'file://component_percona_udf';
Expected output 
Query OK, 0 rows affected (0.01 sec)

Examples

Hash a single value (for example, for a lookup key or checksum):

SELECT fnv_64('test_string');
SELECT murmur_hash(12345);
Expected output 
+----------------------+
| fnv_64('test_string') |
+----------------------+
|   13528473474361592478 |
+----------------------+

+-------------------+
| murmur_hash(12345) |
+-------------------+
|  12345678901234567 |
+-------------------+

Use in queries—for example, to derive a shard or partition key from user_id:

SELECT user_id, murmur_hash(user_id) AS shard_key FROM users;
Expected output 
+---------+-------------------+
| user_id | shard_key         |
+---------+-------------------+
|       1 | 12345678901234567 |
|       2 | 98765432109876543 |
|       3 | 55555555555555555 |
+---------+-------------------+

Checksum a column to compare two tables (same expression on both sides gives the same result when data matches):

SELECT SUM(murmur_hash(col1)) AS checksum FROM replica_db.my_table;
-- Compare with the same expression on the source table.
Expected output 
+---------------------+
| checksum            |
+---------------------+
| 1234567890123456789 |
+---------------------+

Use in WHERE or joins when you need a stable numeric fingerprint:

SELECT * FROM events WHERE murmur_hash(device_id) MOD 10 = 0;
Expected output 
+----+-----------+---------------------+
| id | device_id | created_at          |
+----+-----------+---------------------+
|  2 | dev_abc   | 2024-01-15 10:30:00 |
|  5 | dev_xyz   | 2024-01-15 11:00:00 |
+----+-----------+---------------------+

Maintenance

Verify the component is loaded

Check that the component is registered:

SELECT * FROM mysql.component WHERE component_urn = 'file://component_percona_udf';
Expected output 
+----------------+------------------------------------------+
| component_id   | component_urn                            |
+----------------+------------------------------------------+
|              1 | file://component_percona_udf             |
+----------------+------------------------------------------+

A single row means the component is installed and will load on restart. You can also confirm the functions exist by invoking one:

SELECT fnv_64('check');
Expected output 
+----------------+
| fnv_64('check') |
+----------------+
| 123456789012345 |
+----------------+

If the component is not loaded, you get an error such as “Unknown function ‘fnv_64’”.

Uninstall the component

To remove the UDFs, uninstall the component. Any views, stored procedures, or triggers that call these functions will become invalid after uninstall.

UNINSTALL COMPONENT 'file://component_percona_udf';
Expected output 
Query OK, 0 rows affected (0.00 sec)

You need the DELETE privilege on the mysql.component system table. See UNINSTALL COMPONENT for details.

Troubleshooting

If INSTALL COMPONENT fails:

  • Check the error message for details.
  • Verify the component path 'file://component_percona_udf' is correct and that the component library exists in the directory given by SELECT @@plugin_dir;.
  • Ensure you have the required privileges (for example, INSERT on mysql.component for install, DELETE for uninstall).

For further help, see Percona Support .

Next steps

  • Post-installation — If you installed the server or the UDF component from a package, configure and secure the server next.

  • Percona Toolkit updates for 8.4 — If you use other Percona Toolkit tools (for example, pt-replica-find), see version-specific updates and terminology changes.

  • Upgrade from plugins to components — If you are migrating from the old UDF plugin to the component, see the upgrade path.

Other reading

Utility user

Percona Server for MySQL has implemented ability to have a MySQL user who has system access to do administrative tasks but limited access to user schema. This feature is especially useful to those operating MySQL As A Service.

This user has a mixed and special scope of abilities and protection:

  • Utility user does not appear in the mysql.user table and can not be modified by any other user, including root.

  • Utility user does not appear in INFORMATION_SCHEMA.USER_STATISTICS, INFORMATION_SCHEMA.CLIENT_STATISTICS or THREAD_STATISTICS tables or in any performance_schema tables .

  • Utility user’s queries may appear in the general and slow logs.

  • Utility user does not have the ability create, modify, delete or see any schemas or data not specified, except for information_schema.

  • Utility user may modify all visible, non-read-only system variables (see expanded_option_modifiers functionality).

  • Utility user may see, create, modify and delete other system users only if given access to the mysql schema.

  • Regular users may be granted proxy rights to the utility user but attempts to impersonate the utility user fail. The utility user may not be granted proxy rights on any regular user.

For example, GRANT PROXY ON utility_user TO regular_user; does not fail, but any actual attempt to impersonate as the utility user fails.

GRANT PROXY ON regular_user TO utility_user; fails when utility_user is an exact match or is more specific than than the utility user specified.

At server start, the server notes in the log output that the utility user exists and the schemas that the utility user can access.

System variables

In order to have the ability for a special type of MySQL user, which will have a very limited and special amount of control over the system and can not be see or modified by any other user including the root user, three new options have been added.

utility_user

Option Description
Command Line: Yes
Config file utility_user=<user@host>
Scope: Global
Dynamic: No
Data type String
Default NULL

Specifies a MySQL user that will be added to the internal list of users and recognized as the utility user.

Option utility_user specifies the user which the system creates and recognizes as the utility user. The host in the utility user specification follows conventions described in the MySQL manual . For example, the conventions allow wildcards and IP masks. Anonymous user names are not permitted to be used for the utility user name.

This user must not be an exact match to any other user that exists in the mysql.user table. If the server detects that the user specified with this option exactly matches any user within the mysql.user table on start up, the server reports an error and exits gracefully.

If host name wildcards are used and a more specific user specification is identified on start up, the server reports a warning and continues.

Error message 
utility_user=frank@% and [frank@localhost](mailto:frank@localhost) exists within the mysql.user table.

If a client attempts to create a MySQL user that matches this user specification exactly or if host name wildcards are used for the utility user and the user being created has the same name and a more specific host, the creation attempt fails with an error.

Error message 
utility_user=frank@% and CREATE USER [‘frank@localhost](mailto:'frank@localhost)’;

As a result of these requirements, it is strongly recommended that a very unique user name and reasonably specific host be used.

Verify the script or tools test they are running within the correct user by executing SELECT CURRENT_USER() and comparing the result against the known utility user.

utility_user_password

Option Description
Command Line: Yes
Config file utility_user_password=password
Scope: Global
Dynamic: No
Data type String
Default NULL

Specifies the password required for the utility user.

Option utility_user_password specifies the password for the utility user and must be specified or the server exits with an error.

Utility user password 
utility_user_password=Passw0rD

utility_user_schema_access

Option Description
Command Line: Yes
Config file utility_user_schema_access=schema,schema,schema
Scope: Global
Dynamic: No
Data type String
Default NULL

Specifies the schemas that the utility user has access to in a comma delimited list.

Option utility_user_schema_access specifies the name(s) of the schema(s) that the utility user will have access to read write and modify. If a particular schema named here does not exist on start up it will be ignored. If a schema by the name of any of those listed in this option is created after the server is started, the utility user will have full access to it.

Utility user schema access 
utility_user_schema_access=schema1,schema2,schema3

utility_user_privileges

Option Description
Command Line: Yes
Config file utility_user_privileges=privilege1,privilege2,privilege3
Scope: Global
Dynamic: No
Data type String
Default NULL

This variable can be used to specify a comma-separated list of extra access privileges to grant to the utility user. Supported values for the privileges list are: SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, SHUTDOWN, PROCESS, FILE, GRANT, REFERENCES, INDEX, ALTER, SHOW DATABASES, SUPER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, REPLICATION SLAVE, REPLICATION CLIENT, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT, TRIGGER, CREATE TABLESPACE

Option utility-user-privileges allows a comma-separated list of extra access privileges to grant to the utility user.

Utility user privileges 
utility-user-privileges =”CREATE,DROP,LOCK TABLES”;

utility_user_dynamic_privileges

Option Description
Command Line: Yes
Config file utility_user_dynamic_privileges=privilege1,privilege2,privilege3
Scope: Global
Dynamic: No
Data type String
Default NULL

This variable allows a comma-separated list of extra access dynamic privileges to grant to the utility user. The supported values for the dynamic privileges are:

  • APPLICATION_PASSWORD_ADMIN

  • AUDIT_ADMIN

  • BACKUP_ADMIN

  • BINLOG_ADMIN

  • BINLOG_ENCRYPTION_ADMIN

  • CLONE_ADMIN

  • CONNECTION_ADMIN

  • ENCRYPTION_KEY_ADMIN

  • FIREWALL_ADMIN

  • FIREWALL_USER

  • GROUP_REPLICATION_ADMIN

  • INNODB_REDO_LOG_ARCHIVE

  • NDB_STORED_USER

  • PERSIST_RO_VARIABLES_ADMIN

  • REPLICATION_APPLIER

  • REPLICATION_SLAVE_ADMIN

  • RESOURCE_GROUP_ADMIN

  • RESOURCE_GROUP_USER

  • ROLE_ADMIN

  • SESSION_VARIABLES_ADMIN

  • SET_USER_ID

  • SHOW_ROUTINE

  • SYSTEM_USER

  • SYSTEM_VARIABLES_ADMIN

  • TABLE_ENCRYPTION_ADMIN

  • VERSION_TOKEN_ADMIN

  • XA_RECOVER_ADMIN

Other dynamic privileges may be defined by plugins.

Option utility_user_dynamic_privileges allows a comma-separated list of extra-access dynamic privileges to grant to the utility user.

Utility user dynamic privileges 
utility_user_dynamic_privileges =”SYSTEM_USER,AUDIT_ADMIN”;

Quickstart

Overview

Percona Server for MySQL is a freely available, fully compatible, enhanced, and open source drop-in replacement for any MySQL database and provides enterprise-grade features in security, availability, data management, visibility, instrumentation, and performance.

Review Get help from Percona for ways that we can work with you.

Quickstart path

Follow this path to install Percona Server for MySQL, work with a database, and learn next steps:

  1. Choose an installation method

  2. Work with a database — create a database and run queries

  3. Clean up your installation (optional) — when you are finished

  4. Next steps — ideas for further learning

Install method

Run Percona Server for MySQL with Docker

Install Percona Server for MySQL on Ubuntu

Install Percona Server for MySQL on Oracle Linux

Additional resources

Quickstart - Run Percona Server for MySQL container images with Docker

Quickstart path: Step 1 — Install. Next: Work with a database (step 2).

Prerequisites

  • Docker Engine and Docker Compose installed and running

  • Stable internet connection

  • Basic understanding of the command-line interface (CLI)

Always adapt the commands and configurations to your specific environment and security requirements.

Start a Docker container

Choose one of the following methods:

  1. Create a directory and add a docker-compose.yml file:

    services:
  mysql:
    image: percona/percona-server:8.4.11
    container_name: psmysql
    ports:
      - "3306:3306"
    environment:
      MYSQL_ROOT_PASSWORD: secret
    volumes:
      - myvol:/var/lib/mysql
    restart: unless-stopped
volumes:
  myvol:

    To run the Docker ARM64 version of Percona Server for MySQL, use the 8.4.11-aarch64 tag instead of 8.4.11 in the image line. If needed, you can replace the secret password with a stronger password.

  2. Start the container:

    docker compose up -d
    Expected output 
    [+] Running 2/2
 ✔ Network quickstart_default  Created
 ✔ Container psmysql           Started

If you prefer to run the container manually, use the following. The command has the following options:

Option Description
-d Runs the container in detached mode.
-p 3306:3306 Maps the container’s MySQL port (3306) to the same port on your host.
--name psmysql Container name (required for the connect step below).
-e MYSQL_ROOT_PASSWORD=secret Sets the root password.
-v myvol:/var/lib/mysql Named volume for persistent storage.
percona/percona-server:8.4.11 Image and tag for the release.

You must set at least one environment variable (for example MYSQL_ROOT_PASSWORD) or the instance refuses to initialize. For more on tags and images, see percona/percona-server on the Docker Hub . For ARM64, use the 8.4.11-aarch64 tag instead of 8.4.11.

docker run -d -p 3306:3306 --name psmysql \
--platform linux/amd64 \
-e MYSQL_ROOT_PASSWORD=secret \
-v myvol:/var/lib/mysql \
percona/percona-server:8.4.11
Expected output 
Unable to find image 'percona/percona-server:8.4.11' locally
Pulling from percona/percona-server
b902d6b6048a: Pull complete
...
708ba1f9874cbc09441d18b1ca5d9c0a6f045b27e54aafe15fdd78eda8ef3ecf

Connect to the database instance

To connect to a MySQL database on a container, use the Docker exec command with the database instance connect command. You must know the name or ID of the container that runs the database server and the database credentials.

The Docker exec command runs a specified command in a running container. The database instance connect command connects to a MySQL server with the user name and password.

For this example, we have the following options:

Option Description
it Interact with the container and be a pseudo-terminal
psmysql Running container name
mysql Connects to a database instance
-u Specifies the user account used to connect
-p Use this password when connecting

You must enter the password when the server prompts you.

Run the following to connect:

docker exec -it psmysql mysql -uroot -p

You are prompted to enter the password, which is secret. If you have changed the password, use your password. You will not see any characters as you type.

Enter password:

You should see the following result.

Expected output 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 10
Server version: 8.4.11 Percona Server (GPL), Release 1, Revision 238b3c02

Copyright (c) 2009-2026 Percona LLC and/or its affiliates
Copyright (c) 2000, 2026, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

Troubleshooting

  • Connection Refusal: Ensure Docker is running and the container is active. Verify port 3306 is accessible on the container’s IP address.

  • Incorrect Credentials: Double-check the root password you set during container launch.

  • Data Loss: Always back up your data regularly outside the container volume.

Security best practices

  • Strong Passwords: Utilize complex, unique passwords for the root user and any additional accounts created within the container. The alphanumeric password should contain at least 12 characters. The password should include uppercase and lowercase letters, numbers, and symbols.

  • Network Restrictions: Limit network access to the container by restricting firewall rules to only authorized IP addresses.

  • Periodic Updates: Regularly update the Percona Server image and Docker Engine to mitigate known vulnerabilities.

  • Data Encryption: Consider encrypting the data directory within the container volume for an additional layer of security.

  • Monitor Logs: Actively monitor container logs for suspicious activity or errors.

Remember, responsible container management and robust security practices are crucial for safeguarding your MySQL deployment. By following these guidelines, you can leverage the benefits of Docker and Percona Server while prioritizing the integrity and security of your data.

Work with a database

Work with a database

Additional resources

Install Percona Server for MySQL and create a database on Ubuntu

Use the Percona repositories to install using APT.

Quickstart path: Step 1 — Install. Next: Work with a database (step 2).

The percona-release tool is a command-line utility that simplifies the management and installation of Percona software packages, providing access to the latest versions and ensuring consistency across environments. For more information, refer to the Percona Software Repositories Documentation.

Prerequisites

  • Either use sudo or run as root

  • Stable Internet access

Installation steps

Do the following steps to install the server:

  1. Update the package index:

    sudo apt update

  2. Install curl:

    sudo apt install -y curl

  3. Download and install the percona-release repository package:

    curl -O https://repo.percona.com/apt/percona-release_latest.generic_all.deb
sudo apt install -y gnupg2 lsb-release ./percona-release_latest.generic_all.deb

  4. Set up the Percona Server for MySQL 8.4 repository:

    sudo percona-release setup ps-84-lts --scheme https

  5. Enable the Percona Server for MySQL release repository:

    sudo percona-release enable ps-84-lts release --scheme https
sudo apt update

  6. Install Percona Server for MySQL:

    sudo apt install -y percona-server-server

    During installation, you may be prompted to:

    • Enter a root password (use secret for these examples, or choose your own)

    • Confirm the password

    • Choose an authentication method, if the installer asks (Strong password encryption recommended)

  7. [Optional] Secure the installation:

    Run the mysql_secure_installation script to improve security. The script helps you:

    • Set a password for the root user

    • Select a password validation policy level

    • Remove anonymous users

    • Disable root login remotely

    • Remove the test database

    • Reload the privilege table

    sudo mysql_secure_installation

  8. Check the service status and restart if needed:

    sudo systemctl status mysql
sudo systemctl restart mysql

  9. Log in to the server using the password you set during installation:

    mysql -uroot -p
Enter password:

Work with a database

The steps below walk you through creating a database and running basic queries. You can also open the Work with a database script in its own page.

Work with a database

Troubleshooting

  • Connection issues

    • Check that the MySQL service is running:

      sudo systemctl status mysql

    • If the service is not active, start it:

      sudo systemctl start mysql

    • Try connecting with the password you set during installation:

      mysql -uroot -p
Enter password:

  • Permission errors

    If MySQL reports that a user lacks permission to perform an action, grant the needed privilege. For example, to allow a user to create databases from the MySQL shell:

    GRANT CREATE ON *.* TO 'username'@'localhost';
FLUSH PRIVILEGES;

    Replace username with your MySQL user name.

  • Package installation issues

    Check the system log for errors during installation:

    sudo journalctl -u mysql -n 50

    For specific error messages, see the Percona Server for MySQL documentation or the Percona community forum.

Security best practices

  • Strong Passwords: Utilize complex and unique passwords for all users, especially the root account.

  • Minimize Permissions: Grant users only the privileges necessary for their tasks.

  • Disable Unnecessary Accounts: Remove test accounts and unused accounts.

  • Regular Backups: Implement consistent backup routines to safeguard your data.

  • Keep Software Updated: Maintain Percona Server and related packages updated with security patches.

  • Monitor Server Activity: Employ tools, like Percona Monitoring and Management , and logs to monitor server activity for suspicious behavior.

Additional resources

Install Percona Server for MySQL and create a database on Oracle Linux

Use the Percona repositories to install using YUM.

Quickstart path: Step 1 — Install. Next: Work with a database (step 2).

The percona-release tool is a command-line utility that simplifies the management and installation of Percona software packages, providing access to the latest versions and ensuring consistency across environments. For more information, refer to the Percona Software Repositories Documentation.

Prerequisites

  • Either use sudo or run as root

  • Stable Internet access

Installation steps

{.power-number}

  1. Install percona-release:

    sudo yum install -y https://repo.percona.com/yum/percona-release-latest.noarch.rpm

  2. Set up the repository for Percona Server for MySQL 8.4:

    [Optional] On Red Hat 8 systems (including Rocky Linux and AlmaLinux), disable the distribution’s MySQL module first:

    sudo dnf module disable mysql -y

    Then run:

    sudo percona-release setup ps-84-lts

  3. Enable the release repository:

    sudo percona-release enable ps-84-lts release

  4. Install Percona Server for MySQL:

    sudo yum install -y percona-server-server

  5. Start the MySQL service:

    sudo systemctl restart mysql

  6. Retrieve the temporary password:

    sudo grep 'temporary password' /var/log/mysqld.log

  7. Log in to the server using the temporary password:

    mysql -uroot -p
Enter password:

  8. Change the temporary password:

    ALTER USER 'root'@'localhost' IDENTIFIED BY '[your password]';
exit

  9. Log in again with the new password to verify:

    mysql -uroot -p
Enter password:

Secure the installation

[Optional] Run the mysql_secure_installation script to improve security. The script helps you:

  • Set a password for the root user

  • Select a password validation policy level

  • Remove anonymous users

  • Disable root login remotely

  • Remove the test database

  • Reload the privilege table

sudo mysql_secure_installation

Troubleshooting

  • Connection issues

    • Check that the MySQL service is running:

      sudo systemctl status mysql

    • If the service is not active, start it:

      sudo systemctl start mysql

    • Try connecting with the password you set during installation:

      mysql -uroot -p
Enter password:

  • Permission errors

    If MySQL reports that a user lacks permission to perform an action, grant the needed privilege. For example, to allow a user to create databases from the MySQL shell:

    GRANT CREATE ON *.* TO 'username'@'localhost';
FLUSH PRIVILEGES;

    Replace username with your MySQL user name.

  • Package installation issues

    Check the system log for errors during installation:

    sudo journalctl -u mysqld -n 50

    For specific error messages, see the Percona Server for MySQL documentation or the Percona community forum.

Security best practices

  • Keep software updated: sudo yum update regularly.

  • Strong root password: Set a complex, unique password using mysql_secure_installation.

  • Disable unused accounts and databases: Remove unnecessary elements.

  • Monitor Server Activity: Employ tools, like Percona Monitoring and Management , and logs to monitor server activity for suspicious behavior.

  • Backup data regularly: Ensure robust backups for disaster recovery.

Work with a database

Work with a database

Additional resources

First five minutes after installation

This guide walks you through the most important security and stability steps to take right after installing Percona Server for MySQL. Completing these steps helps protect your server and makes troubleshooting easier.

Quickstart path: After Install (Ubuntu), Install (Oracle Linux), or Docker. Next: Work with a database.

Step What you do
Secure the installation Run post-installation security steps so the server is not left in a default, weak state.
Create a least-privilege admin user Use a dedicated admin account with only the privileges (permissions) that account needs instead of root.
Configure an OOM-killer guardrail Reduce the chance the Linux OOM killer will abruptly terminate the MySQL process.
Enable diagnostic logging Turn on error and slow-query logging so you can diagnose issues later.
Verify the backup path Ensure the directory where backups will go exists and is writable.

When you are done, use Sanity check: First five minutes to verify that these steps were applied correctly.

Secure the installation

Before doing anything else, complete the standard post-installation security steps. These ensure the server is not left with default passwords, anonymous users (accounts with no username that can connect without proper credentials), or remote root login.

  1. Set or update the root password
    The root account is the default MySQL administrative user. If you did not set its password during install (for example, on some RPM-based systems), set one now. See Post-installation — Update the root password.

  2. Run mysql_secure_installation to:

    • Change the root password (if needed)
    • Disallow remote login for root
    • Remove anonymous users
    • Remove the test database
    • Reload privilege tables

  3. Optional: populate time zone tables

If you installed from source or generic binaries, populate the time zone tables (the system data that makes time zone–aware date and time functions work correctly).

For more information, see Post-installation — Secure the server.

Create a least-privilege admin user

Use a dedicated admin account for day-to-day administration instead of root. Give that account only the privileges the account needs (least privilege: the practice of granting only the minimum permissions necessary). Reserve root for emergencies and avoid using root for routine tasks.

  1. Connect as root (or another user that can create users and grant privileges):

    mysql -u root -p

  2. Create an admin user. Use a strong password and a specific host (the machine or network location the user can connect from; for example, localhost means the same machine only) instead of % (which means any host):

    CREATE USER 'admin'@'localhost' IDENTIFIED BY 'YourStrongPassword_12';

  3. Grant only the privileges the admin needs. Grant means to assign a permission to a user. The following is a common set for an admin who manages users, replication, and backups but does not need full SUPER (a powerful built-in privilege) or global privileges on all databases (*.*):

    GRANT CREATE USER, RELOAD, PROCESS, REPLICATION CLIENT, REPLICATION SLAVE, BACKUP_ADMIN,
      SHOW DATABASES, SELECT ON *.* TO 'admin'@'localhost';
FLUSH PRIVILEGES;

    FLUSH PRIVILEGES tells the server to reload the privilege tables so the new grants take effect immediately.

  4. Test the new user:

    mysql -u admin -p -e "SHOW DATABASES;"

  5. Use admin for daily tasks and keep root for rare, high-privilege operations.

Adjust privileges to your needs

Add or remove privileges based on what this admin will do. For example, add SHOW VIEW, CREATE ROUTINE, or EVENT if they manage views (saved queries presented as virtual tables), routines (stored procedures and functions), or scheduled events. Avoid granting SUPER, ALL, or *.* unless necessary. You can review effective privileges with SHOW EFFECTIVE GRANTS.

Configure an OOM-killer guardrail

On Linux, when the system runs low on memory, the kernel OOM killer (out-of-memory killer: a kernel process that terminates other processes to free memory) may terminate processes. MySQL can be one of them, which leads to abrupt shutdowns. You can make the MySQL service less likely to be chosen by adjusting its OOM score.

Using systemd (the Linux system and service manager that starts and manages the MySQL service), set OOMScoreAdjust so that the kernel prefers to kill other processes before MySQL. A typical value is -500 (range is -1000 to 1000; lower means less likely to be killed).

  1. Create or edit an override file for the MySQL service. On Red Hat–based systems the service is often mysqld; on Debian/Ubuntu the service may be named mysql. Adjust the service name if needed:
sudo systemctl edit mysqld

If your service is named mysql:

sudo systemctl edit mysql
  1. Add the following (use mysqld or mysql to match your service name):
[Service]
OOMScoreAdjust=-500
  1. Save and close the editor. Reload systemd and restart the MySQL service:
sudo systemctl daemon-reload
sudo systemctl restart mysqld
  1. Confirm the setting:
systemctl show mysqld -p OOMScoreAdjust

You should see OOMScoreAdjust=-500 (or the value you set).

Not a memory limit

The OOM score adjustment only influences which process the OOM killer chooses. It does not set a memory limit for MySQL. For strict limits, use cgroups or MySQL/InnoDB memory settings.

Enable diagnostic logging

Enabling the error log (a file where the server records errors, warnings, and startup messages) and the slow query log (a file that records SQL statements that run longer than a threshold you set) gives you the information you need to diagnose problems and tune performance.

  1. Locate your configuration file
    The configuration file (often my.cnf or mysqld.cnf) controls server options. Common paths: /etc/my.cnf, /etc/mysql/my.cnf, or /etc/mysql/mysql.conf.d/mysqld.cnf. Edit the file under the [mysqld] section (the section that applies to the MySQL server process).

  2. Ensure the error log is set
    The server usually writes an error log by default. Confirm or set an explicit path so you know where to look:

[mysqld]
log_error=/var/log/mysql/error.log

On some systems the directory is /var/lib/mysql (the default data directory, where the server stores database files) or similar. Create the log directory if needed and set ownership to the MySQL system user (the operating-system user that runs the MySQL process; often named mysql):

sudo mkdir -p /var/log/mysql
sudo chown mysql:mysql /var/log/mysql
  1. Enable the slow query log
    The slow query log helps you find queries that need optimization:
slow_query_log = 1
long_query_time = 2
slow_query_log_file = /var/log/mysql/slow.log

Adjust long_query_time (the time in seconds above which a query is considered “slow” and written to the log) to what you consider slow. Restart the server after changing the config.

  1. Optional: restrict log locations
    For tighter control over where logs are written, see Restrict dynamic log file locations.

Verify the backup path

Your backup strategy (for example, Percona XtraBackup for full physical backups, or mysqldump for logical backups that produce SQL) will write to a directory. Verify that path before you run a backup.

  1. Choose a backup directory
    For example, /var/backups/mysql or a dedicated volume. Do not use the data directory (where the server stores live database files) for backup output.

  2. Create the directory and set permissions
    The MySQL process (or the operating-system user that runs the backup tool) must be able to write there:

sudo mkdir -p /var/backups/mysql
sudo chown mysql:mysql /var/backups/mysql
sudo chmod 750 /var/backups/mysql
  1. Check that the backup directory is writable
    As the MySQL user (or the backup user), test write access:
sudo -u mysql touch /var/backups/mysql/.write_test && sudo -u mysql rm /var/backups/mysql/.write_test && echo "OK: backup path is writable"
  1. Document the path
    Use this path in your backup scripts, cron jobs, or documentation so that restores use the same location.

For backup strategy and tools, see Backup and restore overview. For Percona XtraBackup , see the Quickstart for 8.4 .

Next steps

Work with a database

Benefits and what to watch out for when creating databases and tables

Creating a database and table provides the following benefits:

  • Store and organize your data in a structured and consistent way.

  • Query and manipulate your data using SQL statements like SELECT, INSERT, UPDATE, and DELETE.

  • Use primary keys to uniquely identify records.

This quickstart demonstrates basic database operations. As you advance, you can add constraints, triggers, views, indexes, and other features to enforce data integrity, improve security, and optimize performance.

When you create a table, design your database schema carefully, as changing the schema later may be difficult and costly. You should also backup and restore your data regularly, as data loss or corruption may occur due to hardware failures, human errors, or malicious attacks.

You are welcome to name any items to match your organization’s standards or use your table structure and data. If you do, the results are different from the expected results.

To create a database, use the CREATE DATABASE statement. After the database is created, select the database using the USE statement.

CREATE DATABASE mydb;
Expected output 
Query OK, 1 row affected (0.01 sec)

USE mydb;
Expected output 
Database changed

Create tables

Create tables using the CREATE TABLE statement. You can specify data types, constraints, indexes, and other options for each column.

First, create the departments table:

CREATE TABLE `departments` (
    `id` INT AUTO_INCREMENT,
    `name` VARCHAR(100),
    PRIMARY KEY (`id`)
);
Expected output 
Query OK, 0 rows affected (0.01 sec)

Next, create the employees table with a foreign key to the departments table:

CREATE TABLE `employees` (
    `id` INT AUTO_INCREMENT,
    `name` VARCHAR(255),
    `department_id` INT,
    `country` VARCHAR(100),
    PRIMARY KEY (`id`),
    FOREIGN KEY (`department_id`) REFERENCES `departments`(`id`)
);
Expected output 
Query OK, 0 rows affected, 1 warning (0.03 sec)

Insert data into the tables

Insert data into the departments table first:

INSERT INTO `departments` (`name`)
VALUES
    ("Engineering"),
    ("Sales"),
    ("Marketing");
Expected output 
Query OK, 3 rows affected (0.01 sec)
Records: 3  Duplicates: 0  Warnings: 0

Insert data into the employees table using the INSERT INTO SQL statement. This statement adds multiple records into a table in one statement.

Insert data into the table using the INSERT INTO SQL statement. This statement adds multiple records into a table in one statement.

INSERT INTO `employees` (`name`,`department_id`,`country`)
VALUES
    ("Erasmus Richardson",1,"England"),
    ("Jenna French",2,"Canada"),
    ("Alfred Dejesus",1,"Austria"),
    ("Hamilton Puckett",3,"Canada"),
    ("Michal Brzezinski",1,"Poland"),
    ("Zofia Lis",2,"Poland"),
    ("Aisha Yakubu",3,"Nigeria"),
    ("Miguel Cardenas",1,"Peru"),
    ("Luke Jansen",2,"Netherlands"),
    ("Roger Pettersen",1,"Norway");
Expected output 
Query OK, 10 rows affected (0.02 sec)
Records: 10  Duplicates: 0  Warnings: 0

View all data

To view all records in a table, use SELECT * to retrieve all columns:

SELECT * FROM employees;
Expected output 
+----+---------------------+--------------+------------+
| id | name                | department_id| country    |
+----+---------------------+--------------+------------+
|  1 | Erasmus Richardson |            1 | England    |
|  2 | Jenna French        |            2 | Canada     |
|  3 | Alfred Dejesus      |            1 | Austria    |
|  4 | Hamilton Puckett    |            3 | Canada     |
|  5 | Michal Brzezinski   |            1 | Poland     |
|  6 | Zofia Lis           |            2 | Poland     |
|  7 | Aisha Yakubu        |            3 | Nigeria    |
|  8 | Miguel Cardenas     |            1 | Peru       |
|  9 | Luke Jansen         |            2 | Netherlands|
| 10 | Roger Pettersen     |            1 | Norway     |
+----+---------------------+--------------+------------+
10 rows in set (0.00 sec)

Join tables

JOIN queries combine data from multiple tables based on a related column. Use JOINs to retrieve data from related tables in a single query.

Join the employees and departments tables to display employee names with their department names:

SELECT e.id, e.name, d.name AS department, e.country 
FROM employees e 
JOIN departments d ON e.department_id = d.id;
Expected output 
+----+---------------------+------------+------------+
| id | name                | department | country    |
+----+---------------------+------------+------------+
|  1 | Erasmus Richardson | Engineering| England    |
|  2 | Jenna French        | Sales      | Canada     |
|  3 | Alfred Dejesus      | Engineering| Austria    |
|  4 | Hamilton Puckett    | Marketing  | Canada     |
|  5 | Michal Brzezinski   | Engineering| Poland     |
|  6 | Zofia Lis           | Sales      | Poland     |
|  7 | Aisha Yakubu        | Marketing  | Nigeria    |
|  8 | Miguel Cardenas     | Engineering| Peru       |
|  9 | Luke Jansen         | Sales      | Netherlands|
| 10 | Roger Pettersen     | Engineering| Norway     |
+----+---------------------+------------+------------+
10 rows in set (0.00 sec)

Run a SELECT query

SELECT queries retrieve data from one or more tables based on specified criteria. They are the most common type of query and can be used for various purposes, such as displaying, filtering, sorting, aggregating, or joining data. SELECT queries do not modify the data in the database but can affect the performance if the query involves large or complex datasets.

SELECT e.id, e.name, d.name AS department, e.country 
FROM employees e 
JOIN departments d ON e.department_id = d.id 
WHERE e.country = 'Poland';
Expected output 
+----+-------------------+------------+---------+
| id | name              | department | country |
+----+-------------------+------------+---------+
|  5 | Michal Brzezinski | Engineering| Poland  |
|  6 | Zofia Lis         | Sales      | Poland  |
+----+-------------------+------------+---------+
2 rows in set (0.00 sec)

Run an Update query

UPDATE queries modify existing data in a table. They are used to change or correct the information stored in the database. UPDATE queries can update one or more columns and rows simultaneously, depending on the specified conditions. They may also fail if they violate any constraints or rules defined on the table.

Run an UPDATE query to change a record, and then run a SELECT with a WHERE clause to verify the update.

UPDATE employees SET name = 'Zofia Niemec' WHERE id = 6;
Expected output 
Query OK, 1 row affected (0.01 sec)
Rows matched: 1  Changed: 1  Warnings: 0

SELECT name FROM employees WHERE id = 6;
Expected output 
+--------------+
| name         |
+--------------+
| Zofia Niemec |
+--------------+
1 row in set (0.00 sec)

Run an INSERT query

INSERT queries add new data to a table. They are used to populate the database with new information. INSERT queries can insert one or more rows at a time, depending on the syntax. INSERT queries may fail if they violate any constraints or rules defined on the table, such as primary keys, foreign keys, unique indexes, or triggers.

Insert a row into a table and then run a SELECT with a WHERE clause to verify the record was inserted.

INSERT INTO `employees` (`name`,`department_id`,`country`)
VALUES
("Kenzo Sasaki",1,"Japan");
Expected output 
Query OK, 1 row affected (0.01 sec)

SELECT e.id, e.name, d.name AS department, e.country 
FROM employees e 
JOIN departments d ON e.department_id = d.id 
WHERE e.id = 11;
Expected output 
+----+--------------+------------+---------+
| id | name         | department | country |
+----+--------------+------------+---------+
| 11 | Kenzo Sasaki | Engineering| Japan   |
+----+--------------+------------+---------+
1 row in set (0.00 sec)

Run a Delete query

DELETE queries remove existing data from a table. They are used to clean up the information no longer needed or relevant in the database. The DELETE queries can delete one or more rows at a time, depending on the specified conditions. They may also trigger cascading deletes on related tables if foreign key constraints are enforced.

Delete a row in the table and run a SELECT with a WHERE clause to verify the deletion.

DELETE FROM employees WHERE id >= 11;
Expected output 
Query OK, 1 row affected (0.01 sec)

SELECT e.id, e.name, d.name AS department, e.country 
FROM employees e 
JOIN departments d ON e.department_id = d.id 
WHERE e.id > 10;
Expected output 
Empty set (0.00 sec)

Count records

Use the COUNT() function to count the number of records that match a condition:

SELECT COUNT(*) FROM employees;
Expected output 
+----------+
| COUNT(*) |
+----------+
|       10 |
+----------+
1 row in set (0.00 sec)

SELECT d.name AS department, COUNT(*) AS employee_count
FROM employees e
JOIN departments d ON e.department_id = d.id
GROUP BY d.name;
Expected output 
+------------+---------------+
| department | employee_count|
+------------+---------------+
| Engineering|             5 |
| Sales      |             3 |
| Marketing  |             2 |
+------------+---------------+
3 rows in set (0.00 sec)

Next step

Additional resources

Clean up your installation

Clean up is optional. If you want to remove Percona Server for MySQL and clean up your system, follow the steps below for your installation method.

Warning

These steps will remove Percona Server for MySQL and may delete all data files (databases, tables, logs, etc.). Take a backup before proceeding if you need to preserve any data.

If you installed Percona Server for MySQL using Docker, follow these steps:

  1. Exit the MySQL command client shell if you’re still connected:

    exit

    You can also use \q or quit commands. The execution of the statement also closes the connection.

  2. Stop and remove the Docker container:

    docker container rm psmysql -f
    Expected output 
    psmysql

  3. Remove the Docker image:

    docker image rmi percona/percona-server:8.4.11

    If you are running the ARM64 version of Percona Server, use:

    docker image rmi percona/percona-server:8.4.11-aarch64
    Expected output 
    Untagged: percona/percona-server:8.4.11
Untagged: percona/percona-server@sha256:4944f9b365e0dc88f41b3b704ff2a02d1459fd07763d7d1a444b263db8498e1f
Deleted: sha256:b2588da614b1f382468fc9f44600863e324067a9cae57c204a30a2105d61d9d9
...

  4. Remove the Docker volume (if you created one):

    docker volume rm myvol
    Expected output 
    myvol

If you installed Percona Server for MySQL using APT on Ubuntu or Debian, follow these steps:

  1. Stop the Percona Server for MySQL service:

    sudo systemctl stop mysql

  2. Choose one of the following options:

    Option A: Remove packages but keep data files

    This command removes the packages but leaves data files (databases, tables, logs, configuration, etc.) behind:

    sudo apt remove percona-server*

    Option B: Remove packages and delete all data files

    Warning

    This command removes all packages and permanently deletes all data files (databases, tables, logs, etc.). Ensure you have a backup if you need this data.

    sudo apt purge percona-server*

  3. [Optional] If you used Option A and want to remove data files manually:

    Warning

    This step permanently deletes all data files. Ensure you have a backup if you need this data.

    sudo rm -rf /var/lib/mysql
sudo rm -f /etc/mysql/my.cnf

If you installed Percona Server for MySQL using YUM or DNF on Oracle Linux or RHEL, follow these steps:

  1. Stop the Percona Server for MySQL service:

    sudo systemctl stop mysql

  2. Remove the packages:

    sudo yum remove percona-server*

    Or if you’re using DNF:

    sudo dnf remove percona-server*

    These commands remove the packages but leave data files behind. If you want to remove data files as well, continue to step 3.

  3. [Optional] Remove data and configuration files:

    Warning

    This step permanently deletes all data files (databases, tables, logs, etc.). Ensure you have a backup if you need this data.

    sudo rm -rf /var/lib/mysql
sudo rm -f /etc/my.cnf

Next steps

Next steps

Additional resources

Next steps

You have the basics. To go further with Percona Server for MySQL, consider:

  • Data types — Choose the right types (integers, strings, dates, booleans) for your data.

  • Indexes — Create and use indexes to improve query performance.

  • Joins and subqueries — Combine data from multiple tables.

  • Functions, procedures, triggers, views — Encapsulate logic and automate tasks.

  • Transactions and isolation levels — Ensure data integrity with concurrent operations.

  • Backup and restore — Protect your data from loss or corruption.

  • Security — Use users, roles, privileges, and encryption.

  • Debugging and testing — Use logs, error messages, and assertions to find and fix issues.

For full details, see the Percona Server for MySQL documentation.

Other database servers you might try

Percona offers quickstart guides for MongoDB and PostgreSQL if you want to try another database server.

For backups and restores

Percona XtraBackup performs online, non-blocking full backups for Percona Server for MySQL and MySQL®.

Percona XtraBackup Quickstart overview

For monitoring and management

Percona Monitoring and Management (PMM) monitors MySQL variants and provides metrics, dashboards, and actionable performance data.

Install PMM and connect your MySQL instances to it .

For high availability

Percona XtraDB Cluster (PXC) is an open source, highly available clustering solution for MySQL based on Galera.

Percona XtraDB Cluster Quickstart overview

Advanced command-line tools

Percona Toolkit is a collection of command-line tools for MySQL and MongoDB tasks that are complex or tedious to do manually.

Percona Toolkit documentation

Operators

Percona Operator for MySQL and Percona Operator for MySQL based on Percona XtraDB Cluster simplify deploying and managing MySQL and PXC in Kubernetes, with automated failover, self-healing, and scaling.

Percona Operator for MySQL Documentation

Percona Operator for MySQL - Quickstart overview

Percona Operator for MySQL based on Percona XtraDB Cluster - Quickstart overview

Additional resources

Install

Install Percona Server for MySQL

Before installing, read the Percona Server for MySQL 8.4 Release notes.

We gather Telemetry data in the Percona packages and Docker images.

Install Percona Server for MySQL from repositories

Percona provides repositories for yum (RPM packages for Red Hat) and apt (.deb packages for Ubuntu and Debian) for software such as Percona Server for MySQL, Percona XtraBackup, and Percona Toolkit. This makes it easy to install and update your software and its dependencies through your operating system’s package manager. This is the recommended way of installing where possible.

The following guides describe the installation process for using the official Percona repositories for the .deb and .rpm packages.

Install Percona Server for MySQL on Debian and Ubuntu Install Percona Server for MySQL on Red Hat Enterprise Linux

Starting with Percona Server for MySQL 8.4.10-10, packages may be PGO or non-PGO builds depending on your platform. See Profile-Guided Optimization (PGO) and non-PGO builds for benefits, considerations, and which build you receive.

Other installation methods

Install Percona Server for MySQL from Binaries
Compile Percona Server for MySQL from Source
Run Percona Server for MySQL in a Docker container

Profile-Guided Optimization (PGO) and non-PGO builds

Starting with Percona Server for MySQL 8.4.10-10, Percona publishes a mixture of Profile-Guided Optimization (PGO) and non-PGO builds. With PGO, the compiler uses profiling data from representative workloads to guide optimization, which can improve throughput and reduce latency for typical database workloads compared with non-PGO builds.

To install Percona Server for MySQL, see Install Percona Server for MySQL from repositories.

Benefits of PGO builds

PGO builds are a good default when they are available for your platform. Advantages include the following:

  • Higher performance for typical workloads — Hot code paths are optimized using profiling data from representative database workloads, which can improve query throughput and reduce latency compared with non-PGO builds.

  • No application changes — PGO is a compile-time optimization. You install and run Percona Server for MySQL the same way as a non-PGO build; no extra configuration is required.

  • Production-oriented packaging — When Percona publishes PGO builds for a platform, they are intended for production use where performance matters.

Considerations for non-PGO builds

A non-PGO build is not inferior by design. You may receive or choose one for reasons such as the following:

  • Platform availability — Percona does not yet publish a PGO build for every operating system, architecture, or installation method. In those cases, the non-PGO package is the supported option.

  • Consistency with an existing environment — Use a non-PGO build when you need to match binaries in a test, staging, or support scenario that already runs a non-PGO package (for example, before validating an upgrade path).

  • Custom source builds — If you compile Percona Server for MySQL yourself, you control whether PGO is enabled. A non-PGO build may be appropriate when you are not using Percona’s profiling workflow or when you need a simpler reproducible build for development.

  • Workload mismatch (uncommon) — PGO optimizes for the workloads used during profiling. If your workload is highly unusual and differs sharply from typical OLTP-style database traffic, performance gains may be smaller. This is uncommon in practice; most production deployments still benefit from PGO when it is available.

When both PGO and non-PGO builds are published for the same platform, prefer the PGO build for production unless one of the considerations above applies.

Which build you receive

In most cases, you do not choose PGO or non-PGO at install time. The build type depends on how Percona packages Percona Server for MySQL for your operating system, architecture, and installation method (APT, DNF/YUM, binary tarball, or Docker). Some platforms ship PGO-enabled packages; others ship non-PGO builds while PGO support is rolled out across the release matrix.

See the Percona downloads page and the Percona Software and Platform Lifecycle page for supported platforms for this release.

Build from source

To control whether PGO is used, compile Percona Server for MySQL yourself. See Compile Percona Server for MySQL from Source.

Before you start

Percona Software Download instructions

Select the software

Do the following steps to select the software:

  1. Open Percona Software Downloads
  2. Select a product, for example, Percona Server for MySQL
  3. In Product Group, select which product, for example, Percona Server 8.4
  4. In Version, select the version, for example, PERCONA-SERVER-8.4.11-11
  5. In Platform, select the operating system, for example, DEBIAN GNU/LINUX 12.0 (“BOOKWORM”).

The Package Download Options may mix amd64 and arm64 packages. Select the correct CPU architecture for your system.

Download to a local computer

In Package Download Options, select a specific package and click Download. This action downloads the selected packages to the local computer.

Download to another computer

In Package Download Options, select a specific package and right-click on the DOWNLOAD, and in the drop-down menu, choose Copy Link Address

Paste the link in your terminal to download the selected package.

Use APT

Use an APT repository to install Percona Server for MySQL 8.4

Ready-to-use packages are available from the Percona Server for MySQL software repositories and the Percona downloads page.

Specific information on the supported platforms, products, and versions—including which Debian and Ubuntu releases and CPU architectures (DEB packages include arm64, among others)—is described in Percona Software and Platform Lifecycle .

The percona-release tool is a command-line utility that simplifies the management and installation of Percona software packages, providing access to the latest versions and ensuring consistency across environments. For more information, refer to the Percona Software Repositories Documentation.

We gather Telemetry data in the Percona packages and Docker images.

Review Get help from Percona for ways that we can work with you.

Install Percona Server for MySQL using APT

If you are new to installing Percona Server on Debian or Ubuntu, copy the first command block below and run the commands in order. Use Configure authentication when the installer asks about the default authentication plugin. After the server packages are installed, open Next Steps for securing and configuring the instance. Expand the step-by-step section when you want an explanation of each command; use Non-interactive installs and debconf and Unattended installations only for scripted automation.

Run the following commands as a root user or with sudo:

sudo apt update
sudo apt install -y curl
curl -O https://repo.percona.com/apt/percona-release_latest.generic_all.deb
sudo apt install -y gnupg2 lsb-release ./percona-release_latest.generic_all.deb
sudo percona-release setup ps-84-lts --scheme https
sudo percona-release enable ps-84-lts release --scheme https
sudo apt update
sudo apt install -y percona-server-server

The command sequence matches Install Percona Server for MySQL and create a database on Ubuntu (installation steps 1–6 there).

percona-release flag: --scheme

The command examples on the page pass --scheme https so repository URLs in APT source lists use HTTPS. The Percona Software Repositories — percona-release documentation describes the Flags section: the available flag is --scheme, with supported values HTTP and HTTPS; without the flag, the tool defaults to HTTP.

You can add --scheme https to subcommands such as setup, enable, enable-only, or disable following the same pattern as the examples. See the linked documentation for the full command reference.

Starting in 8.4, the mysql_native_password plugin is no longer loaded by default. When the package manager may prompt you during installation (depending on the package and distribution), follow the choices and steps in Configure authentication. If you choose legacy authentication but do not enable the plugin in configuration, clients may be unable to connect or the Percona Server service may fail to start.

Step-by-step: what each command does

The following sections provide detailed explanations for each step:

  1. The apt update command shown below updates the package lists for upgrades and new package installations. sudo runs the command with superuser privileges; apt update resynchronizes the package index files from the sources configured in /etc/apt/sources.list and /etc/apt/sources.list.d/.

    sudo apt update

  2. The sudo apt install -y curl command installs the curl package. curl is a command-line tool used to transfer data over networks and is required to download the Percona repository package.

    sudo apt install -y curl

  3. The curl -O command shown below downloads the percona-release_latest.generic_all.deb file from the Percona APT repository. The -O option saves the file with the same name as in the URL.

    curl -O https://repo.percona.com/apt/percona-release_latest.generic_all.deb

  4. The sudo apt install -y gnupg2 lsb-release ./percona-release_latest.generic_all.deb command installs gnupg2 (for package signature verification), lsb-release (for distribution information), and the downloaded Percona release package. Together, gnupg2, lsb-release, and the Percona release .deb configure the Percona APT repository on your system.

    sudo apt install -y gnupg2 lsb-release ./percona-release_latest.generic_all.deb

  5. The percona-release setup ps-84-lts --scheme https command disables all current Percona repository locations on the system, then enables the release repositories that match Percona Server for MySQL 8.4 for your distribution over HTTPS (see the Percona Software Repositories documentation). Omit --scheme https only if you intentionally want HTTP repository URLs (the tool default).

    sudo percona-release setup ps-84-lts --scheme https

  6. The percona-release enable ps-84-lts release --scheme https command turns on the Percona Server for MySQL release repository location with HTTPS URLs. Run apt update afterward so APT loads package indexes for that repository (including percona-server-server and related packages).

    sudo percona-release enable ps-84-lts release --scheme https
sudo apt update

  7. You can verify the repository setup by inspecting the .list files under /etc/apt/sources.list.d/ (for example percona-original-release.list, depending on your percona-release version).

  8. The sudo apt install -y percona-server-server command installs the percona-server-server package. During installation, the package manager may prompt you to select the default authentication plugin (some builds use only debconf or post-install configuration); follow the Configure authentication section when a prompt or post-install step applies.

    sudo apt install -y percona-server-server

Configure authentication

During the installation process, the package manager may prompt you to select the default authentication plugin, or you may need to adjust authentication only after install—see your installer output and Non-interactive installs and debconf when automating.

Important change in 8.4

Percona Server for MySQL 8.4 inherits the upstream change where the mysql_native_password plugin is disabled by default.

Select Option 1 if you are setting up a new server or using modern application drivers.

The recommended option uses the caching_sha2_password plugin, providing superior security and performance. No further configuration is required.

Select Option 2 only if you must support legacy applications that cannot be updated.

If you choose Option 2, you must manually enable the plugin after installation, or the server will fail to authenticate users who rely on native-password authentication.

  1. Open your configuration file (for example, /etc/mysql/mysql.conf.d/mysqld.cnf).

  2. Add the following to the [mysqld] section: 

    [mysqld]
mysql_native_password=ON

  3. Restart the service: 

    sudo systemctl restart mysql

See Configuring Percona repositories with percona-release for more information.

Percona Server for MySQL 8.4.x comes with the MyRocks storage engine. This storage engine is installed as a plugin. For information on installing and configuring MyRocks, refer to the Percona MyRocks Installation Guide.

Next Steps

After successful installation:

  • Post-installation — Configure and secure your Percona Server for MySQL installation.

  • First five minutes after installation — Security and stability steps to take right after install (secure the server, create an admin user, enable logging, verify backup path).

  • Next steps — Ideas for what to do next (backup, monitoring, data types, and related Percona products).

Non-interactive installs and debconf

Adding -y to apt install only skips APT confirmation prompts. The percona-server-server packages still run maintainer scripts that may ask debconf questions (for example the MySQL root password, whether to reuse an existing data directory, or lowercase table names, depending on your version and what is already on the disk).

To automate those prompts:

  1. Discover the questions your package version uses. On a lab host, install once interactively or inspect templates, then run:

    sudo debconf-show percona-server-server

    Template definitions are also shipped with the package (paths such as /var/lib/dpkg/info/percona-server-server.templates).

    Verify for each target: preseed names and choices can differ by distribution, point release, and percona-server-server package version. Re-run debconf-show after upgrades or when you change base image. For production automation, keep a short checklist (distro + package version → saved debconf-show output) so scripts stay aligned with what the package actually asks.

  2. Preseed answers with debconf-set-selections before apt install. For the templates shipped with many Percona Server for MySQL 8.4 packages, password prompts use the percona-server-server/root-pass and percona-server-server/re-root-pass names—for example:

    echo "percona-server-server percona-server-server/root-pass password choose-a-strong-secret" | sudo debconf-set-selections
echo "percona-server-server percona-server-server/re-root-pass password choose-a-strong-secret" | sudo debconf-set-selections

    Other prompts (for example percona-server-server/lowercase-table-names or percona-server-server/remove-data-dir) appear only in some upgrade or edge-case paths—use debconf-show output to add matching lines. Do not commit real passwords to version control or broad shell history.

  3. Optional: set DEBIAN_FRONTEND=noninteractive for the install so debconf does not try to open a UI. Noninteractive installs are usually combined with preseeding; without defaults for required questions, the configure step can still fail or leave the server in an unexpected state.

    sudo DEBIAN_FRONTEND=noninteractive apt install -y percona-server-server

Default authentication plugin: if your distribution or package adds a debconf choice for the authentication plugin, that question shows up under debconf-show for percona-server-server. If the distribution or package does not add such a question, unattended installs still follow the same rules as an interactive install: use Configure authentication after the package is installed (for example mysql_native_password=ON in my.cnf when you need legacy auth).

See also

  • Telemetry — disable collection for package installs with PERCONA_TELEMETRY_DISABLE=1 on the same apt command line.
  • Authentication methods — options for mysql_native_password, caching_sha2_password, and related settings.
  • Post-installation — secure and configure the server after packages are installed.
  • Debian Wiki — debconf — how debconf and preseeding work on Debian-derived systems.

Unattended installations

For the same apt / percona-release sequence as in Install Percona Server for MySQL using APT, add -y to each sudo apt install line where you need non-interactive APT confirmations, and use non-interactive options for percona-release as in the following snippet.

The -y flag (on apt and apt-get, --assumeyes on dnf) tells the package manager to assume affirmative answers so installs do not block on prompts. That suits scripts and unattended installs, but you do not get a last chance to review dependency changes—use it only when you accept that tradeoff.

percona-release — The setup command documents -y for non-interactive repository configuration, for example:

sudo percona-release setup -y {{pkg}} --scheme https

If you use enable or enable-only instead, see sudo percona-release --help and the Percona Software Repositories documentation for flags your version supports.

Debian and Ubuntu (apt) — Add -y to each sudo apt install (or sudo apt-get install) you run, for example sudo apt install -y curl and sudo apt install -y percona-server-server.

To disable telemetry during an unattended server install, set PERCONA_TELEMETRY_DISABLE=1 on the same line (see Telemetry for details):

sudo PERCONA_TELEMETRY_DISABLE=1 apt install -y percona-server-server

RPM-based systems (dnf / yum) — Add -y (or yes) to install commands as supported by your tool. Example with telemetry disabled:

sudo PERCONA_TELEMETRY_DISABLE=1 dnf install -y percona-server-server

Use yum instead of dnf where that is the supported tool on your OS.

Install Percona Toolkit UDFs (Optional)

Percona Server for MySQL includes user-defined functions (UDFs) from Percona Toolkit .

These UDFs provide faster checksum calculations. Install the component if you use tools that need the component (for example, pt-table-checksum) or need fast fingerprinting to compare tables or distribute rows across servers.

Use these functions for high-speed checksumming and sharding. Learn more about UDF Use Cases →

  • fnv_64: Fast hash function

  • fnv1a_64: Alternative fast hash function

  • murmur_hash: High-performance hash function

User-Defined Functions (UDFs) are custom functions you can add to MySQL to extend MySQL capabilities. The Percona Toolkit UDFs are useful for data integrity checks and performance monitoring.

To install the Percona Toolkit UDFs after installation:

INSTALL COMPONENT 'file://component_percona_udf';
Expected output 
Query OK, 0 rows affected (0.01 sec)

You can now use the UDFs in your SQL queries. For example: SELECT fnv_64('test_string');

For detailed information about the UDFs, see Percona Toolkit UDF functions.

Install the Percona testing repository using APT

Percona offers pre-release builds from the testing repository. As a superuser, run percona-release with the testing argument to enable the testing repository:

sudo percona-release enable ps-84-lts testing --scheme https

Do not run testing repository builds in production. The build may not contain all the features available in the final release and may change without notice.

DEB repository package list - Percona Server for MySQL 8.4

Package Name Description
libperconaserverclient22 The shared client library used by Percona Server and MySQL client programs. Applications that connect to MySQL servers dynamically link to this library.
libperconaserverclient22-dev Development headers and static libraries for building software that links against libperconaserverclient22. Needed when compiling client applications using the MySQL C API.
percona-mysql-router A lightweight middleware that routes connections between applications and backend MySQL or Percona Server instances. Used in high availability and cluster setups.
percona-server-client The command-line client utilities, including mysql, mysqldump, and related tools for interacting with a Percona Server instance.
percona-server-common Common configuration files, character sets, and data shared among multiple Percona Server packages. Installed automatically as a dependency.
percona-server-dbg Debug symbols for the Percona Server binaries. Useful for developers or support teams when diagnosing crashes or profiling performance.
percona-server-rocksdb The RocksDB storage engine plugin for Percona Server, providing high-performance key-value storage optimized for fast writes.
percona-server-server The main Percona Server daemon (mysqld) and supporting files. This is the actual database server that runs and manages data.
percona-server-source The source code package for Percona Server, often used for auditing, compliance, or custom builds.
percona-server-test A suite of functional and regression tests used to verify server correctness and compatibility. Useful for QA or CI environments.
percona-telemetry-agent A lightweight agent that collects anonymous usage and performance data to help Percona improve the product. Optional; can be disabled if desired.

Build APT packages

If you wish to build your own Debian/Ubuntu (dpkg) packages of Percona Server for MySQL, you first need to start with a source tarball, either from the Percona website or by generating your own by following the instructions above (Installing Percona Server for MySQL from the Git Source Tree).

Extract the source tarball:

tar xfz Percona-Server-8.4.11-11-Linux.x86_64.ssl102.tar.gz
cd Percona-Server-8.4.11-11

Copy the Debian packaging in the directory that Debian expects it to be in:

cp -ap build-ps/debian debian

Update the changelog for your distribution (here we update for the unstable distribution - sid), setting the version number appropriately. The trailing one in the version number is the revision of the Debian packaging.

dch -D unstable --force-distribution -v "8.0.13-3-1" "Update to 8.0.13-3"

Build the Debian source package:

dpkg-buildpackage -S

Use sbuild to build the binary package in a chroot:

sbuild -d sid percona-server-8.4_8.4.11-11.dsc

You can give different distribution options to dch and sbuild to build binary packages for all Debian and Ubuntu releases.

Note

PAM Authentication Plugin is not built with the server by default. In order to build the Percona Server for MySQL with PAM plugin, an additional option -DWITH_PAM=ON should be used.

Install Percona Server for MySQL 8.4 using downloaded DEB packages

Download the packages from Percona Product Downloads . If needed, Instructions for the Percona Product Download are available.

The following example downloads Percona Server for MySQL 8.4.11-11 release x86_64 packages for Ubuntu 22.04:

  1. Use wget to download the tar file:

    The download filename includes a <revision-identifier> value. This value is build-specific and must be obtained from the Percona Product Downloads page for the exact release you are installing. Select the product, version, and operating system, and find the link with the required <revision identifier> under the Download all packages button. For more details, see the Instructions for Percona Product Downloads.

    wget https://downloads.percona.com/downloads/Percona-Server-8.4/Percona-Server-8.4.11-11/binary/debian/jammy/x86_64/Percona-Server-8.4.11-11-<revision-identifier>-jammy-x86_64-bundle.tar

  2. Unpack the download to get the packages:

    tar xvf Percona-Server-8.4.11-11-<revision-identifier>-jammy-x86_64-bundle.tar
    Expected output 
    libperconaserverclient21_8.4.11-11-1.buster_amd64.deb
libperconaserverclient21-dev_8.4.11-11-1.buster_amd64.deb
percona-mysql-router_8.4.11-11-1.buster_amd64.deb
percona-server-client_8.4.11-11-1.buster_amd64.deb
percona-server-common_8.4.11-11-1.buster_amd64.deb
percona-server-dbg_8.4.11-11-1.buster_amd64.deb
percona-server-rocksdb_8.4.11-11-1.buster_amd64.deb
percona-server-server_8.4.11-11-1.buster_amd64.deb
percona-server-source_8.4.11-11-1.buster_amd64.deb
percona-server-test_8.4.11-11-1.buster_amd64.deb

  3. Install Percona Server for MySQL using dpkg. Run this command as root or use the sudo command:

    sudo dpkg -i *.deb

Warning

When installing packages manually like this, you’ll need to resolve all the dependencies and install missing packages yourself. The following packages will need to be installed before you can manually install Percona Server: mysql-common, libjemalloc1, libaio1, and libmecab2.

Apt pinning the Percona Server for MySQL 8.4 packages

Pinning allows you to stay on a release and get packages from a different version. In some cases, you can pin selected packages and avoid accidentally upgrading all the packages.

The pinning takes place in the preference file. To pin a package, set the Pin-Priority to higher numbers.

Make a new file /etc/apt/preferences.d/00percona.pref. For example, add the following to the preference file:

Package: 
Pin: release o=Percona Development Team
Pin-Priority: 1001

For more information about the pinning, you can check the official debian wiki .

Run Percona Server for MySQL 8.4 after APT repository installation

Percona Server for MySQL stores the data files in /var/lib/mysql/ by default. You can find the configuration file that is used to manage Percona Server for MySQL in /etc/mysql/my.cnf.

Note

Debian and Ubuntu installation doesn’t automatically create a special debian-sys-maint user which can be used by the control scripts to control the Percona Server for MySQL mysqld and mysqld_safe services which was the case with previous Percona Server for MySQL versions. If you still require this user you’ll need to create it manually.

Run the following commands as root or by using the sudo command

  1. Starting the service

    Percona Server for MySQL is started automatically after it gets installed unless it encounters errors during the installation process. You can also manually start it by running: service mysql start

  2. Confirming that service is running. You can check the service status by running: service mysql status

  3. Stopping the service

    You can stop the service by running: service mysql stop

  4. Restarting the service. service mysql restart

Note

Debian 9.0 (stretch) and Ubuntu 18.04 LTS (bionic) come with systemd as the default system and service manager. You can invoke all the above commands with systemctl instead of service. Currently, both are supported.

Working with AppArmor

For information on AppArmor, see Working with AppArmor.

Uninstall Percona Server for MySQL 8.4 using the APT package manager

To uninstall Percona Server for MySQL you’ll need to remove all the installed packages. Removing packages with apt remove does not remove the configuration and data files. Removing the packages with apt purge does remove the packages with configuration files and data files (all the databases). Depending on your needs you can choose which command better suits you.

  1. Stop the Percona Server for MySQL service: service mysql stop

  2. Remove the packages

    1. Remove the packages. This will leave the data files (databases, tables, logs, configuration, etc.) behind. In case you don’t need them you’ll need to remove them manually: apt remove percona-server\

    2. Purge the packages. This command removes all the packages and deletes all the data files (databases, tables, logs, and so on.): apt purge percona-server\

Use DNF

Install using DNF

Installation Method

This guide describes standard OS installation on RPM-based operating systems (RHEL, CentOS, Rocky Linux, etc.) using DNF. Ready-to-use packages are available from the Percona Server for MySQL software repositories and the [Percona downloads] page. On RHEL 8+ systems, DNF has superseded YUM, but yum commands continue to work as they are aliased to dnf.

For containerized deployments:

The percona-release tool is a command-line utility that simplifies the management and installation of Percona software packages, providing access to the latest versions and ensuring consistency across environments. For more information, refer to the Percona Software Repositories Documentation.

We gather Telemetry data in the Percona packages and Docker images.

Review Get help from Percona for ways that we can work with you.

Prerequisites

Required permissions

You need either sudo access or root access to install packages and configure system services. The installation commands in this guide use sudo, but you can run them as the root user if you prefer.

Package verification

The packages are signed with GPG keys for security. The installation process automatically installs the Percona GPG key and handles key verification, but you can manually verify packages if needed.

Security Note

Always download packages from official Percona repositories to ensure authenticity and security.

Limitations

RHEL 8+ and other EL8+ systems enable the MySQL module by default. This module hides the Percona-provided packages and the module must be disabled to make these packages visible.

Checking the MySQL Module

RHEL 8+ systems enable the MySQL module by default, which can hide or conflict with Percona’s packages. The first installation step below shows you how to check if the module is enabled. If you see an [e] marker, you must disable the module before proceeding. If you only see [d] (default), you can proceed.

Install using DNF (RHEL 8+)

Standard OS Installation

The following steps install Percona Server for MySQL directly on the host operating system using DNF. These instructions are for standard OS installations, not for Kubernetes pods or containerized environments.

All commands in this guide use sudo for privilege elevation. Follow these steps:

  1. Verify that the MySQL module is currently enabled on your system:

    sudo dnf module list mysql
    Expected output 
    Rocky Linux 9 - BaseOS                     2.2 MB/s | 2.6 MB     00:01    
Rocky Linux 9 - AppStream                  3.7 MB/s | 8.2 MB     00:02    
Rocky Linux 9 - Extras                      35 kB/s |  18 kB     00:00    
Rocky Linux 9 - AppStream
Name       Stream      Profiles                            Summary         
mysql      8.4         api, client, filter, server [d]     MySQL Module   

Hint: [d]efault, [e]nabled, [x]disabled, [i]nstalled

    The [d] next to the server profile indicates that this is the default stream. A module is only considered enabled if an [e] is present. If you see [e], it means the module is active.

  2. [Optional] If the module is listed as [e]nabled, it can cause conflicts with Percona’s packages. You must disable the module before proceeding.

    sudo dnf module disable mysql
    Expected output 
    Last metadata expiration check: 0:33:11 ago on Fri Aug 29 14:37:35 2025.
Dependencies resolved.
Nothing to do.
Complete!

    The dnf module disable command disables the MySQL module. If the module was not enabled to begin with (as shown in the dnf module list output without [e]), this command will display “Nothing to do,” which is the expected result. You can then proceed with your installation.

  3. Install the Percona repository package:

    sudo yum install https://repo.percona.com/yum/percona-release-latest.noarch.rpm
    Expected output 
    Last metadata expiration check: 1:04:21 ago on Fri Aug 29 14:37:35 2025.
percona-release-latest.noarch.rpm           69 kB/s |  28 kB     00:00    
Dependencies resolved.
===========================================================================
Package               Architecture Version       Repository          Size
===========================================================================
Installing:
percona-release       noarch       1.0-32        @commandline        28 k

Transaction Summary
===========================================================================
Install  1 Package
...
Installed:
percona-release-1.0-32.noarch                                            

Complete!

    If this step fails:

    • Check your internet connection.

    • Verify the URL is accessible: curl -I https://repo.percona.com/yum/percona-release-latest.noarch.rpm

    • Ensure you have sufficient disk space: df -h

  4. Enable the Percona Server for MySQL repository:

    sudo percona-release enable-only ps-84-lts release
    Expected output 
    * Disabling all Percona Repositories
* Enabling the Percona Server for MySQL - PS 8.4- repository
<*> All done!

    If this step fails:

    • Check if percona-release is properly installed: which percona-release

    • Verify the package name is correct for your version.

    • Check for any error messages in the output.

  5. Install the server package:

    sudo yum install percona-server-server
    Expected output 
    Percona Release release/noarch YUM reposit 6.0 kB/s | 2.5 kB     00:00    
Percona Server for MySQL - PS 8.4- release 1.5 MB/s | 2.4 MB     00:01    
Percona Telemetry release/aarch64 YUM repo 6.8 kB/s | 2.7 kB     00:00    
Dependencies resolved.
===========================================================================
Package                 Arch    Version                  Repository  Size
===========================================================================
Installing:
percona-server-server   aarch64 8.4.5-5.1.el9            ps-84-lts-release-aarch64
...
systemd-252-51.el9_6.1.aarch64                                           
systemd-pam-252-51.el9_6.1.aarch64                                       
systemd-rpm-macros-252-51.el9_6.1.noarch                                 

Complete!

    If this step fails:

    • Check available packages: yum search percona-server.

    • Ensure the repository is properly configured.

    • Check for package conflicts with existing MySQL installations.

    • Review error messages for specific issues.

See Configuring Percona repositories with percona-release for more information.

Post-installation configuration

Percona Server for MySQL 8.4.x comes with the MyRocks storage engine. This storage engine is installed as a plugin. For information on installing and configuring MyRocks, refer to the Percona MyRocks Installation Guide.

Unattended installations

The -y flag (on apt and apt-get, --assumeyes on dnf) tells the package manager to assume affirmative answers so installs do not block on prompts. That suits scripts and unattended installs, but you do not get a last chance to review dependency changes—use it only when you accept that tradeoff.

percona-release — The setup command documents -y for non-interactive repository configuration, for example:

sudo percona-release setup -y {{pkg}} --scheme https

If you use enable or enable-only instead, see sudo percona-release --help and the Percona Software Repositories documentation for flags your version supports.

Debian and Ubuntu (apt) — Add -y to each sudo apt install (or sudo apt-get install) you run, for example sudo apt install -y curl and sudo apt install -y percona-server-server.

To disable telemetry during an unattended server install, set PERCONA_TELEMETRY_DISABLE=1 on the same line (see Telemetry for details):

sudo PERCONA_TELEMETRY_DISABLE=1 apt install -y percona-server-server

RPM-based systems (dnf / yum) — Add -y (or yes) to install commands as supported by your tool. Example with telemetry disabled:

sudo PERCONA_TELEMETRY_DISABLE=1 dnf install -y percona-server-server

Use yum instead of dnf where that is the supported tool on your OS.

Next steps

After a successful installation, refer to the Post-installation documentation for detailed steps to configure and secure your Percona Server for MySQL installation.

Additional information

Red Hat certified

Percona Server for MySQL is certified for Red Hat Enterprise Linux 8. This certification is based on common and secure best practices, as well as successful interoperability with the operating system. Percona Server is listed in the Red Hat Ecosystem Catalog.

ARM support

The RPM builds contain ARM packages with the aarch64.rpm extension.

Supported platforms

Specific information on the supported platforms, products, and versions ican be found in the Percona Software and Platform Lifecycle document.

Install Percona Toolkit UDFs (optional)

Percona Server for MySQL includes user-defined functions (UDFs) from Percona Toolkit . These UDFs provide faster checksum calculations:

  • fnv_64: Fast hash function

  • fnv1a_64: Alternative fast hash function

  • murmur_hash: High-performance hash function

User-defined functions (UDFs) are custom functions you can add to MySQL to extend its capabilities. These particular UDFs are useful for data integrity checks and performance monitoring.

To install these functions after installation:

INSTALL COMPONENT 'file://component_percona_udf';
Expected output 
Query OK, 0 rows affected (0.01 sec)

UDFs installed

You can now use these functions in your SQL queries. For example: SELECT fnv_64('test_string');

For detailed information about these functions, see Percona Toolkit UDF functions.

Install the Percona testing repository (advanced users only)

Do not use testing repositories in production environments. Testing builds are pre-release versions that may contain bugs or incomplete features.

Percona offers pre-release builds from the testing repository for advanced users who want to:

  • Test new features before official release.

  • Evaluate upcoming improvements.

  • Provide feedback on development versions.

To enable the testing repository:

sudo percona-release enable ps-84-lts testing
Expected output 
* Enabling Percona Server for MySQL 8.4 LTS testing repository
* Running yum update...
Last metadata expiration check: 0:01:23 ago on Mon Jan 15 10:30:00 2024.
All packages are up to date.

Please be aware of the following limitations when using the testing repository:

  • Features may change without notice.

  • Not all features from the final release may be included.

  • May contain experimental or incomplete functionality

  • No production support for testing builds

To disable the testing repository and return to stable releases:

sudo percona-release disable testing
sudo yum update
Expected output 
* Disabling Percona testing repository
* Running yum update...
Last metadata expiration check: 0:01:23 ago on Mon Jan 15 10:30:00 2024.
All packages are up to date.

Yum repository package list - Percona Server for MySQL 8.4

Package Name Description
percona-icu-data-files Provides ICU (International Components for Unicode) data files required by Percona Server for locale, collation, and Unicode character set support.
percona-mysql-router A lightweight routing middleware that directs client connections to backend MySQL or Percona Server instances, often used in high availability setups.
percona-mysql-router-debuginfo Debug symbols for percona-mysql-router, used for troubleshooting or analyzing router crashes and performance issues.
percona-server-client Command-line client utilities for connecting to and managing MySQL/Percona Server instances, including tools like mysql and mysqldump.
percona-server-client-debuginfo Debug symbols for the client utilities, useful for diagnosing crashes or debugging client-related issues.
percona-server-debuginfo Debug symbols for the main Percona Server binaries, required when debugging or profiling the server process.
percona-server-debugsource Source code corresponding to the debug build of Percona Server, used for in-depth debugging and code-level analysis.
percona-server-devel Development files and headers for building applications that link against Percona Server client libraries.
percona-server-rocksdb The RocksDB storage engine plugin, providing high-performance key-value storage optimized for write-heavy workloads.
percona-server-rocksdb-debuginfo Debug symbols for the RocksDB plugin, useful for diagnosing issues or profiling RocksDB engine performance.
percona-server-server The main Percona Server daemon (mysqld) and associated server-side components for database management and operations.
percona-server-server-debuginfo Debug symbols for the Percona Server daemon, enabling developers to trace or debug server-level issues.
percona-server-shared Shared libraries used by both the Percona Server and client utilities, providing common functionality and APIs.
percona-server-shared-debuginfo Debug symbols for the shared libraries, useful for debugging applications that depend on these libraries.
percona-server-test A collection of tests used to verify the correctness and stability of Percona Server, typically used in QA or CI environments.
percona-server-test-debuginfo Debug symbols for the test suite, aiding developers in diagnosing issues encountered during test runs.
percona-telemetry-agent A lightweight agent that collects anonymous usage and performance data to help Percona improve its products. Optional and can be disabled.

Install Percona Server for MySQL using downloaded RPM packages

Download the packages from Percona Product Downloads . If needed, Instructions for the Percona Product Download are available.

The RPM builds for RHEL 8 and RHEL 9 contain ARM packages with the aarch64.rpm extension. This means that Percona Server for MySQL is available for users on ARM-based systems.

The following example downloads Percona Server for MySQL 8.4.11-11 release x86_64 packages for RHEL 8.

  1. Use wget to download the tar file.

    The download filename includes a <revision-identifier> value. This value is build-specific and must be obtained from the Percona Product Downloads page for the exact release you are installing. Select the product, version, and operating system, and find the link with the required <revision identifier> under the Download all packages button. For more details, see the Instructions for Percona Product Downloads.

    wget https://downloads.percona.com/downloads/Percona-Server-8.4/Percona-Server-8.4.11-11/binary/redhat/8/x86_64/Percona-Server-8.4.11-11-<revision identifier>-el8-x86_64-bundle.tar

  2. Unpack the bundle to get the packages: 

    tar xvf Percona-Server-8.4.11-11-<revision identifier>-el8-x86_64-bundle.tar

  3. To view a list of packages, run the following command:

    ls *.rpm
    The output should look like the following:

    Expected output 
    percona-icu-data-files-8.4.11-11.1.el8.x86_64.rpm
percona-mysql-router-8.4.11-11.1.el8.x86_64.rpm
percona-mysql-router-debuginfo-8.4.11-11.1.el8.x86_64.rpm
percona-server-client-8.4.11-11.1.el8.x86_64.rpm
percona-server-client-debuginfo-8.4.11-11.1.el8.x86_64.rpm
percona-server-debuginfo-8.4.11-11.1.el8.x86_64.rpm
percona-server-debugsource-8.4.11-11.1.el8.x86_64.rpm
percona-server-devel-8.4.11-11.1.el8.x86_64.rpm
percona-server-rocksdb-8.4.11-11.1.el8.x86_64.rpm
percona-server-rocksdb-debuginfo-8.4.11-11.1.el8.x86_64.rpm
percona-server-server-8.4.11-11.1.el8.x86_64.rpm
percona-server-server-debuginfo-8.4.11-11.1.el8.x86_64.rpm
percona-server-shared-8.4.11-11.1.el8.x86_64.rpm
percona-server-shared-compat-8.4.11-11.1.el8.x86_64.rpm
percona-server-shared-debuginfo-8.4.11-11.1.el8.x86_64.rpm
percona-server-test-8.4.11-11.1.el8.x86_64.rpm
percona-server-test-debuginfo-8.4.11-11.1.el8.x86_64.rpm

  4. Install jemalloc with the following command, if needed. See When to install jemalloc for guidance:

    wget https://repo.percona.com/yum/release/8/RPMS/x86_64/jemalloc-3.6.0-1.el8.x86_64.rpm

  5. An EL8-based RHEL distribution or derivatives package installation requires the mysql module to be disabled before installing the packages:

    sudo yum module disable mysql

  6. Install all the packages (for debugging, testing, etc.) with the following command:

    sudo rpm -ivh *.rpm

    Note

    When installing packages manually, you must make sure to resolve all dependencies and install any missing packages yourself.

When to install jemalloc

jemalloc is an alternative memory allocator that can improve performance and reduce memory fragmentation in certain scenarios. Consider the following when deciding whether to install jemalloc:

Install jemalloc when:

  • You have high-concurrency workloads with many threads

  • You experience memory fragmentation issues that impact performance

  • You run multi-threaded applications that perform frequent memory allocation and deallocation

  • You want to use memory profiling features to investigate memory-related issues.

  • You observe performance degradation related to memory allocation in your current setup

Do not install jemalloc when:

  • Your current memory allocator (typically glibc malloc) performs adequately for your workload

  • You have single-threaded or low-concurrency workloads where jemalloc’s benefits are minimal

  • You encounter compatibility issues with jemalloc in your environment

  • You need to debug memory issues that may be complicated by using an alternative allocator

  • Your system is already optimized and stable with the default memory allocator

Run Percona Server for MySQL

Percona Server for MySQL stores the data files in /var/lib/mysql/ by default. The configuration file used to manage Percona Server for MySQL is the /etc/my.cnf.

The following commands start, provide the server status, stop the server, and restart the server.

Note

The RHEL distributions and derivatives come with systemd as the default system and service manager so you can invoke all of the commands with sytemctl instead of service. Currently, both options are supported.

  1. Percona Server for MySQL is not started automatically on the RHEL distributions and derivatives after installation. Start the server with the following command:

    sudo service mysql start

  2. Review the service status with the following command:

    sudo service mysql status

  3. Stop the service with the following command:

    sudo service mysql stop

  4. Restart the service with the following command:

    sudo service mysql restart

SELinux and security considerations

For information on working with SELinux, see Working with SELinux.

The RHEL 8 distributions and derivatives have added system-wide cryptographic policies component. This component allows the configuration of cryptographic subsystems.

Uninstall Percona Server for MySQL

To completely uninstall Percona Server for MySQL, remove all the installed packages and data files.

  1. Stop the Percona Server for MySQL service:

    sudo service mysql stop

  2. Remove the packages:

    sudo yum remove percona-server*

  3. Remove the data and configuration files:

    Warning

    This step removes all the packages and deletes all the data files (databases, tables, logs, etc.). Take a backup before this operation in case you need the data.

    rm -rf /var/lib/mysql
rm -f /etc/my.cnf

Use binary tarballs

Install Percona Server for MySQL 8.4 from a binary tarball

A binary tarball contains a group of files, including the server binaries and support files, bundled together into one file using the tar command and compressed using gzip.

See the list of binary tarballs available based on the Percona Server for MySQL version to select the right tarball for your environment.

You can download the binary tarballs from the Linux - Generic section on the download page.

Fetch and extract the correct binary tarball. For example, for Ubuntu 22.04:

Download the tarball:

wget https://downloads.percona.com/downloads/Percona-Server-innovative-release/Percona-Server-8.4.11-11/binary/tarball/Percona-Server-8.4.11-11-Linux.x86_64.glibc2.35.tar.gz
Expected output 
--2024-01-15 10:00:00--  https://downloads.percona.com/...
Saving to: 'Percona-Server-8.4.11-11-Linux.x86_64.glibc2.35.tar.gz'
Percona-Server-8.4.11-11-Linux.x86_64.glibc2.35.tar.gz   100%[=================>]  xxx MB  xx.x MB/s    in xx s
2024-01-15 10:00:xx (xx.x MB/s) - 'Percona-Server-8.4.11-11-Linux.x86_64.glibc2.35.tar.gz' saved [xxxxxx/xxxxxx]

Extract the tarball:

tar -xzf Percona-Server-8.4.11-11-Linux.x86_64.glibc2.35.tar.gz
Expected output 
(No output on success.)

Change into the extracted directory:

cd Percona-Server-8.4.11-11-Linux.x86_64.glibc2.35
Expected output 
(No output on success.)

After extraction

The tarball does not start the server for you. To run Percona Server you must:

  1. Initialize the data directory (once) using the server executable bin/mysqld with --initialize or --initialize-insecure.

  2. Start the server (for example with bin/mysqld_safe or by configuring a systemd service).

The server executable is bin/mysqld inside the extracted directory. Do not run it for normal operation until the data directory has been initialized.

For step-by-step instructions (including creating a data directory, initializing it, starting the server, and optional boot-time setup), see Post-installation. If you did not install under /usr/local/mysql, use your extraction directory instead (for example, cd /path/to/Percona-Server-8.4.11-11-Linux.x86_64.glibc2.35 then bin/mysqld --initialize).

Binary tarball file names available based on the Percona Server for MySQL version

For later of Percona Server for MySQL, the tar files are organized by the glibc2 version. You can find this version on your operating system with the following command:

ldd --version
Expected output 
ldd (Ubuntu GLIBC 2.35-0ubuntu3.1) 2.35
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

If the glibc2 version from your operating system is not listed, then this Percona Server for MySQL version does not support that operating system.

Binary tarball file name organization

The following lists the platform and the associated full binary file name used by Percona Server for MySQL tar files 8.4.11-11.

Platform Percona Server for MySQL tarball name glibc2 version
Ubuntu 22.04 Percona-Server-8.4.11-11-Linux.x86_64.glibc2.35.tar.gz glibc2.35
Ubuntu 20.04 Percona-Server-8.4.11-11-Linux.x86_64.glibc2.31.tar.gz glibc2.31
Red Hat Enterprise 9 Percona-Server-8.4.11-11-Linux.x86_64.glibc2.34.tar.gz glibc2.34
Red Hat Enterprise 8 Percona-Server-8.4.11-11-Linux.x86_64.glibc2.28.tar.gz glibc2.28
Red Hat Enterprise 7 Percona-Server-8.4.11-11-Linux.x86_64.glibc2.17.tar.gz glibc2.17

The types of files are as follows:

Type Name Description
Full Percona-Server-<version-number>-Linux.x86_64.<glibc2-version>.tar.gz Contains all files available
Minimal Percona-Server-<version-number>-Linux.x86_64.<glibc2-version>.minimal.tar.gz Contains binaries and libraries
Debug Percona-Server-<version-number>-Linux.x86_64.<glibc2-version>.debug.tar.gz Contains the minimal build files and test files, and debug symbols

Compile from source

Install Percona Server for MySQL from a source tarball

Fetch and extract the source tarball. For example:

wget https://downloads.percona.com/downloads/Percona-Server-innovative-release/Percona-Server-8.4.11-11/binary/tarball/Percona-Server-8.4.11-11-Linux.x86_64.glibc2.35.tar.gz

Unpack the download to get the packages:

tar xfz Percona-Server-8.4.11-11-Linux.x86_64.glibc2.35.tar.gz

To complete the installation, follow the instructions in Compile Percona Server for MySQL from Source.

Compile Percona Server for MySQL from source

The following instructions install Percona Server for MySQL 8.4.

Install Percona Server for MySQL from the Git Source Tree

Percona uses the Github revision control system for development. To build the latest Percona Server for MySQL from the source tree, you will need git installed on your system.

You can now fetch the latest Percona Server for MySQL 8.4 sources.

git clone https://github.com/percona/percona-server.git
cd percona-server
git checkout 8.4
git submodule init
git submodule update

If you are going to be making changes to Percona Server for MySQL 8.4 and wanting to distribute the resulting work, you can generate a new source tarball (exactly the same way as we do for release):

cmake .
make dist

After either fetching the source repository or extracting a source tarball (from Percona or one you generated yourself), you will now need to configure and build Percona Server for MySQL.

First, run CMake to configure the build. Here you can specify all the normal build options as you do for a normal MySQL build. Depending on what options you wish to compile Percona Server for MySQL with, you may need other libraries installed on your system. Here is an example using a configure line similar to the options that Percona uses to produce binaries:

cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_CONFIG=mysql_release -DFEATURE_SET=community

Compile from source

Now, compile using make:

make

Install:

make install

Percona Server for MySQL 8.4 is installed on your system.

Docker

Running Percona Server for MySQL in a Docker Container

Percona Server for MySQL has an official Docker image hosted on Docker Hub . Download a specific version by adding the Docker tag filter for the 8.4 versions .

We gather Telemetry data in the Percona packages and Docker images.

Make sure that you are using the latest version of Docker. The APT version or the YUM version may be outdated and cause errors.

Review Get help from Percona for ways that we can work with you.

Starting a detached container

Start a container with the --detached or -d option, which runs the container in the background. In detached mode, when the root process used to run the container exits, the container exits.

The following example starts a container named ps with the latest version of Percona Server for MySQL 8.4. This action also creates the root user and uses root as the password. Please note that root is not a secure password. 

docker run -d \
  --name ps \
  -e MYSQL_ROOT_PASSWORD=root \
  percona/percona-server:8.4
Expected output 
Unable to find image 'percona/percona-server:8.4' locally
8.4: Pulling from percona/percona-server

By default, Docker pulls the image from Docker Hub if it is not available locally.

To view the container’s logs, use the following command:

docker logs ps --follow
Expected output 
Initializing database
2022-09-07T15:20:03.158128Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.4.11-11) initializing of server in progress as process 15
2022-09-07T15:20:03.167764Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2022-09-07T15:20:03.530600Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2022-09-07T15:20:04.367600Z 0 [Warning] [MY-013829] [Server] Missing data directory for ICU regular expressions: /usr/lib64/mysql/private/.
...
2022-09-07T15:20:13.706090Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060, socket: /var/lib/mysql/mysqlx.sock
2022-09-07T15:20:13.706136Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.4.11-11'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  Percona Server (GPL), Release 21, Revision c59f87d2854.

You can access the server when you see the ready for connections information in the log.

Passing Options

You can pass options with the docker run command. For example, the following command uses UTF-8 as the default setting for character set and collation for all databases:

docker run -d \
 --name ps \
 -e MYSQL_ROOT_PASSWORD=root \
 percona/percona-server:8.4 \
 --character-set-server=utf8 \
 --collation-server=utf8_general_ci

Accessing the Percona Server Container

The docker exec command lets you have a shell inside the container. This command uses it which forwards your input stream as an interactive TTY.

An example of accessing the detached container:

docker exec -it ps /bin/bash

If you need to troubleshoot, the error log is found in /var/log/ or /var/log/mysql/. The file name may be error.log or mysqld.log.

Troubleshooting

You can view the error log with the following command:

[mysql@ps] more /var/log/mysql/error.log
Expected output 
...
2017-08-29T04:20:22.190474Z 0 [Warning] 'NO_ZERO_DATE', 'NO_ZERO_IN_DATE' and 'ERROR_FOR_DIVISION_BY_ZERO' sql modes should be used with strict mode. They will be merged with strict mode in a future release.
2017-08-29T04:20:22.190520Z 0 [Warning] 'NO_AUTO_CREATE_USER' sql mode was not set.
...

Accessing the database

You can access the database either with Docker exec or using the mysql command in the container’s shell.

An example of using Docker exec to access the database:

docker exec -ti ps mysql -uroot -proot
Expected output 
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 9
...

Exiting Percona Server also exits the container.

You can also run the MySQL command-line client within the container’s shell to access the database:

[mysql@ps] mysql -uroot -proot
Expected output 
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 8
Server version: 8.4.11-11 Percona Server (GPL), Release 21, Revision c59f87d2854

Copyright (c) 2009-2022 Percona LLC and/or its affiliates
Copyright (c) 2000, 2022, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

Accessing the server from an application in another container

The image exposes the standard MySQL port 3306, so container linking makes the Percona Server instance available from other containers. To link a container running your application (in this case, from an image named app/image) with the Percona Server container, run it with the following command:

docker run -d \
  --name app \
  --link ps \
  app/image:latest

This application container will be able to access the Percona Server container via port 3306.

Storing data

There are two ways to store data used by applications that run in Docker containers:

  • Let Docker manage the storage of your data by writing the database files to disk on the host system using its internal volume management.

  • Create a data directory on the host system on high-performance storage and mount it to a directory visible from the container. This method places the database files in a known location on the host system, and makes it easy for tools and applications on the host system to access the files. The user should ensure that the directory exists, that the user accounts have required permissions, and that any other security mechanisms on the host system are set up correctly.

For example, if you create a data directory on a suitable volume on your host system named /local/datadir, you run the container with the following command:

docker run -d \
  --name ps \
  -e MYSQL_ROOT_PASSWORD=root \
  -v /local/datadir:/var/lib/mysql \
  percona/percona-server:8.4

The -v /local/datadir:/var/lib/mysql option mounts the /local/datadir directory on the host to /var/lib/mysql in the container, which is the default data directory used by Percona Server for MySQL.

Do not add MYSQL_ROOT_PASSWORD to the docker run command if the data directory contains subdirectories, files, or data.

Note

If you have SELinux enabled, assign the relevant policy type to the new data directory so that the container will be allowed to access it:

chcon -Rt svirt_sandbox_file_t /local/datadir

Port forwarding

Docker allows mapping ports on the container to ports on the host system using the -p option. If you run the container with this option, you can connect to the database by connecting your client to a port on the host machine. This ability simplifies consolidating instances to a single host.

To map the standard MySQL port 3306 to port 6603 on the host:

docker run -d \
 --name ps \
 -e MYSQL_ROOT_PASSWORD=root \
 -p 6603:3306 \
 percona/percona-server:8.4

Exiting the container

If you are in the interactive shell, use CTRL-D or exit to exit the session.

If you have a non-shell process running, interrupt the process with CTRL-C before using either CTRL-D or exit.

Stopping the container

The docker stop container command sends a TERM signal, then waits 10 seconds and sends a KILL signal. The following example stops the ps container:

docker stop ps

The default length of time before stopping a container is 10 seconds. A very large instance cannot dump the data from memory to disk within that time. With this type of instance, add the --time or the -t option to docker stop:

docker stop ps -t 600

Removing the container

To remove a stopped container, use the docker rm command.

docker rm ps

For more information

Review the Docker Docs

Docker environment variables

When running a Docker container with Percona Server, you can adjust the configuration of the instance Add one or more environment variables to the docker run command.

These variables will not affect you if you start the container with a data directory that already contains a database. Any pre-existing database remains untouched on the container startup.

The variables are optional, but you must specify at least one of the following:

  • MYSQL_DATABASE - the database schema name that is created when the container starts

  • MYSQL_USER - create a user account when the container starts

  • MYSQL_PASSWORD - used with MYSQL_USER to create a password for that user account.

  • MYSQL_ALLOW_EMPTY_PASSWORD - creates a root user with an empty password. This option is insecure and only should be used for testing or proof of concept when the database can be removed afterward. Anyone can connect as root.

  • MYSQL_ROOT_PASSWORD - this password is used for the root user account. This option is not recommended for production.

  • MYSQL_RANDOM_ROOT_PASSWORD - set this variable instead of MYSQL_ROOT_PASSWORD when you want Percona Server to generate a password for you. The generated password is available in the container’s logs only during the first start of the container. Use docker logs. You cannot retrieve the password after the first start.

To further secure your instance, use the MYSQL_ONETIME_PASSWORD variable.

These variables are visible to anyone able to run Docker inspect. 

docker inspect ps
Expected output 
...
 "Env": [
                "MYSQL_ROOT_PASSWORD=root",
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "PS_VERSION=8.4.11-11",
                "OS_VER=el8",
                "FULL_PERCONA_VERSION=8.4.11-11.el8"
               ]
...

You should use Docker secrets or volumes instead.

Percona Server for MySQL also allows adding the _FILE suffix to a variable name. This suffix lets you add the value in a path so that the value cannot be inspected from outside the container.

Use Docker Compose and named volumes

Docker Compose simplifies managing containerized services. This guide shows how to run Percona Server for MySQL 8.4 with persistent storage for data, logs, and backups using named Docker volumes.

Benefits

Creating a docker-compose.yml file offers numerous advantages for managing containerized applications effectively:

Benefit Description
Simplifies multi-container management Define and manage multiple services (containers) in a single configuration file, making it easy to run, stop, and scale your application.
Automates dependency handling Specify dependencies between services, ensuring containers start in the correct order, such as databases starting before application servers.
Enhances portability Share the docker-compose.yml file across environments (development, staging, production) to ensure consistent behavior regardless of system setup.
Supports scalability Easily scale services using the docker-compose up –scale command, allowing you to run multiple instances of specific containers.
Improves readability Centralizes configuration in a human-readable YAML format, making it easier to understand and modify compared to command-line options.
Enables reproducibility Store application settings and container configurations in version control to ensure consistent deployments.
Allows persistent data Define volumes directly in the file to persist data for services, ensuring storage remains intact even when containers are stopped.
Facilitates networking Automatically sets up networks for containers to communicate with each other without requiring manual configuration.
Simplifies environment variables management Integrate .env files to externalize sensitive information like database passwords and access tokens.
Reduces errors Avoid repetitive CLI commands by storing configurations in the file, reducing the chance of mistakes during deployment.

Directory structure

Docker automatically manages volumes, so you don’t create folders manually.

percona-compose/
├── .env
└── docker-compose.yml

Create .env

Using an .env file for MySQL Docker containers has several advantages:

Benefit Description
Keeps sensitive data secure Stores environment variables (e.g., passwords) in an .env file, keeping them out of docker-compose.yml to avoid exposure.
Simplifies configuration Centralizes environment variables, making it easier to manage and update configurations in one place.
Improves portability Enables reuse of variables across different environments (development, staging, production) without changes to configuration files.
Enhances readability Keeps docker-compose.yml or Dockerfiles cleaner by externalizing environment variables.
Facilitates collaboration Allows the use of shared templates (e.g., .env.example) for required variables, while hiding actual secrets.
Supports dynamic updates Makes it easy to update environment variables without modifying Docker configurations or scripts.

By leveraging an .env file, you streamline both security and ease of use for MySQL container deployments. This approach ensures better organization and adaptability for various environments.

MYSQL_ROOT_PASSWORD=supersecurepassword
MYSQL_DATABASE=mydb
MYSQL_USER=myuser
MYSQL_PASSWORD=myuserpassword

Create docker-compose.yml

By using a docker-compose.yml file, you streamline container orchestration, ensure consistency, and simplify collaboration across teams.

services:
  mysql:
    image: percona/percona-server:8.4
    container_name: percona-server
    ports:
      - "3306:3306"
    environment:
      MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
      MYSQL_DATABASE: ${MYSQL_DATABASE}
      MYSQL_USER: ${MYSQL_USER}
      MYSQL_PASSWORD: ${MYSQL_PASSWORD}
    volumes:
      - percona-data:/var/lib/mysql              # Database data
      - percona-logs:/var/log/mysql              # MySQL logs
      - percona-backups:/backups                 # XtraBackup output (optional, for future use)
    restart: unless-stopped
volumes:
  percona-data:
  percona-logs:
  percona-backups:

Start the Container

The command has the following options:

  • up: Starts the containers specified in the docker-compose.yml file.

  • -d: Runs the containers in detached mode, meaning they operate in the background.

docker-compose up -d
Expected output

{.text .no-copy} [+] Running 11/11 ✔ mysql Pulled 34.1s ✔ 56631da24b0d Pull complete 28.9s ✔ 5aee836c3728 Pull complete 28.9s ✔ a5fd539367b0 Pull complete 28.9s ✔ fc4a4cc146b3 Pull complete 28.9s ✔ 7a3939b8d92c Pull complete 32.1s ✔ 6fdbd2a9e883 Pull complete 32.1s ✔ 70ac4d191dd1 Pull complete 32.1s ✔ 5872370b843d Pull complete 32.1s ✔ 8310fa1d2765 Pull complete 32.1s ✔ 4437564bc659 Pull complete 32.2s [+] Running 5/5 ✔ Network percona-compose_default Created 0.0s ✔ Volume "percona-compose_percona-data" Created 0.0s ✔ Volume "percona-compose_percona-logs" Created 0.0s ✔ Volume "percona-compose_percona-backups" Created 0.0s ✔ Container percona-server Star... 0.3s

Docker automatically creates the volumes:

• percona-data: stores MySQL tables

• percona-logs: stores logs generated by the database

• percona-backups: a mount point you can use for Percona XtraBackup

Connect to the server and run a simple query

After the container is up, you can connect to the running server instance using the mysql client included in the container.

Run the following command to open a MySQL shell in the container:

docker exec -it percona-server mysql -u root -p

You must enter the root password.

Expected output 
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 9
Server version: 8.4.4-4 Percona Server (GPL), Release 32, Revision b8e378ec

Copyright (c) 2009-2025 Percona LLC and/or its affiliates
Copyright (c) 2000, 2025, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

Run a simple query:

SHOW DATABASES;
Expected output 
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mydb               |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
5 rows in set (0.02 sec)

Create a test database and table

The following query creates a test_db database and test_table:

CREATE DATABASE test_db;
USE test_db;
CREATE TABLE test_table (id INT AUTO_INCREMENT PRIMARY KEY, name VARCHAR(100));
INSERT INTO test_table (name) VALUES ('Sample Data');
SELECT * FROM test_table;
Expected output 
Query OK, 1 row affected (0.02 sec)

Database changed
Query OK, 0 rows affected (0.01 sec)

Query OK, 1 row affected (0.01 sec)

+----+-------------+
| id | name        |
+----+-------------+
|  1 | Sample Data |
+----+-------------+
1 row in set (0.00 sec)

Remember to exit when you are finished working with the server.

Use the backup volume with XtraBackup

When you run XtraBackup inside a container, either in the same network or another container, you can target /backups to store backup files.

An example of using Docker to backup the server:

docker run --rm \
  --volumes-from percona-server \
  -v percona_backups:/backup \
  percona/percona-xtrabackup:8.0 \
  xtrabackup --backup \
  --target-dir=/backup \
  --host=percona-server \
  --user=myuser \
  --password=mypassword

Best practices

  • Use named volumes to simplify backup and migration.

  • Mount logs separately for easier troubleshooting and rotation.

  • Use docker volume inspect to view volume metadata and mount points.

Shut down and clean up

You can stop the stack but retain volumes:

docker-compose down

You can also remove all resources, including volumes:

docker-compose down -v

Upgrade

Upgrade from 8.0 to 8.4 overview

Review Get help from Percona for ways that we can work with you.

Need expert guidance for your upgrade? Percona Support is ready to assist you every step of the way .

Why upgrade to Percona Server for MySQL 8.4 LTS

Long‑Term Support (LTS) releases focus on stability, predictable security patches, and a clearly defined maintenance horizon—essential qualities for production databases. Moving from 8.0 to 8.4 isn’t just a simple version bump: new defaults, deprecated options, and behavior changes can affect performance and break existing scripts. Treat the upgrade as a small project: plan the steps, pick the method that matches your downtime window, and verify the result.

Benefits of upgrading to Percona Server for MySQL 8.4

Benefit What it means for you
Security fixes Patches close known vulnerabilities, keeping your data safe from attacks.
New or enhanced features Access to functionality that improves performance, reliability, and overall capability.
Less manual effort Automation tools let you handle routine tasks without hands-on intervention.
Stay relevant The latest version helps you meet evolving customer expectations and deliver solutions more quickly.
Lower operational cost Improved efficiency, and scalability translate into cheaper day-to-day operations.

Risks of staying on an older version

Risk Potential impact
Security exposure Without the latest patches, attackers can breach, corrupt, or destroy data, harming reputation and causing financial loss.
Feature stagnation Missing new capabilities can lead to slower performance, lower productivity, and more frequent outages.
Reduced support Older versions receive less vendor assistance, resulting in longer troubleshooting times and higher support expenses.
Compatibility problems New hardware, operating system releases, or third-party applications may not work with an outdated Percona Server version, eventually leaving the server unsupported.
Unplanned upgrade pressure Unexpected hardware or OS failures can force a rushed upgrade, increasing the chance of errors.

Concerned about these risks? Percona Support can help assess and mitigate them .

Upgrade workflow

Follow this step-by-step workflow to plan and execute your upgrade from 8.0 to 8.4:

Step 1: Understand what’s changing

Review these documents to understand breaking changes, removed features, and compatibility issues:

Step 2: Complete pre-upgrade preparation

Work through the pre-upgrade checks in the upgrade checklist. This includes:

  • Verifying authentication methods and client compatibility

  • Updating replication scripts (MASTER/SLAVE → SOURCE/REPLICA syntax)

  • Identifying and addressing removed features or variables

  • Reviewing configuration defaults

  • Testing backups and restore procedures

  • Planning plugin-to-component transitions (if applicable)

Step 3: Choose your upgrade strategy

Select the upgrade method that best fits your environment:

Step 4: Execute the upgrade

Follow the step-by-step procedures for your chosen method:

Step 5: Validate the upgrade

After completing the upgrade, complete the post-upgrade validation steps in the upgrade checklist. These steps include:

  • Verifying connectivity and authentication

  • Checking replication health (if applicable)

  • Re-creating spatial indexes

  • Validating performance baselines

  • Reviewing logs and metrics

  • Testing backup and recovery

Additional reference materials

Tooling to de-risk your upgrade

  • pt-upgrade – compares query plans and execution behavior between Percona Server 8.0 and 8.4

  • Percona XtraBackup – creates hot backups and lets you test restores without downtime.

  • A full dry-run workflow - backup → restore on 8.4 → run smoke/load tests → validate → practice rollback.

These purpose‑built tools let you spot regressions early and ensure a reliable fallback plan.

Test environment is mandatory

Set up a sandbox and run the upgrade there first. This isolated environment is essential for a successful migration. If you ever need to revert to the previous version, note that there is no fully supported in-place downgrade procedure from Percona Server for MySQL 8.4 to an earlier major version. The most reliable rollback method is to restore a backup taken before the upgrade, or to use a logical dump/load or replication into the older version. Because binary compatibility may not be preserved when new features or data-format changes have been applied, the safest approach is to provision a fresh instance of the older version and reload your data, rather than expecting a simple “undo” of the upgrade.

We strongly advise upgrading to the latest LTS release (Percona Server for MySQL 8.4) to stay secure, performant, and fully supported.

Need personalized support during your upgrade? Contact Percona Support for a detailed migration plan .

Further reading

Review these upgrade-related documents:

Additional MySQL documentation

The following list summarizes a number of the changes in the 8.0 series and has useful guides that can help you perform a smooth upgrade. We strongly recommend reading this information:

Review other Percona blogs that contain upgrade information.

Understand what's changing

Breaking and incompatible changes in 8.4

Review these items before upgrading from 8.0 to 8.4. This document covers the most significant breaking behavioral changes, removed features, and removed variables that may affect your upgrade. Each entry includes the impact, replacement (if available), and recommended action.

Review the MySQL 8.4 Reference manual for more information.

Authentication and user management

Impact:

  • mysql_native_password is disabled by default in 8.4; new users default to caching_sha2_password.

  • The mysql_native_password plugin can still be loaded using --mysql-native-password=ON if needed for backward compatibility.

  • In the 9.x series, mysql_native_password will be completely removed.

  • default_authentication_plugin variable is removed.

Replacement:

  • New users default to caching_sha2_password; configure authentication via supported mechanisms without this variable.

Action:

  • Identify accounts and applications using mysql_native_password and plan migration to caching_sha2_password.
  • If necessary for temporary compatibility, you can enable mysql_native_password with --mysql-native-password=ON, but plan to migrate accounts as this plugin will be removed in future versions.
  • Validate that drivers and clients support caching_sha2_password and TLS.

Replication terminology and commands

Impact:

  • Many of the MASTER/SLAVE terms and statements are removed in 8.4 and will cause syntax errors if used.

  • Statements such as CHANGE MASTER TO, START SLAVE, STOP SLAVE, and SHOW SLAVE STATUS are no longer supported.

  • Related status variables and counters (for example, Com_show_slave_status) are also removed.

Replacement:

  • Use SOURCE/REPLICA equivalents: START REPLICA, SHOW REPLICA STATUS, CHANGE REPLICATION SOURCE TO and updated status fields.

Action:

  • Update operational scripts, automation, and runbooks to new commands before upgrading.

  • You should replace only the replication‑related statements (for example, CHANGE MASTER TO, START SLAVE, SHOW SLAVE STATUS) with their SOURCE/REPLICA equivalents, while leaving identifiers, the REPLICATION SLAVE privilege, and the slave_* system tables untouched. A scoped, manual review of each occurrence prevents breaking non‑replication code.

  • Update scripts, automation, and monitoring that reference removed statements or counters.

  • Re-test replication lifecycle: provisioning, change-source, failover.

Spatial indexes

Impact:

  • A known issue can corrupt a spatial index (R-Tree index) in MySQL 8.4.0 through 8.4.3. The corruption is triggered when an UPDATE that slightly changes a geometry’s MBR (Minimum Bounding Rectangle) is immediately followed by a DELETE of the same row. The R-Tree index can retain a dangling leaf, leading to “Incorrect key file” errors.

Action:

  • Upgrade to 8.4.4 or later, which fixes this issue.

  • If you must remain on 8.4.0-8.4.3, apply one of these workarounds:

  • Serialize UPDATE and DELETE statements (ensure they don’t execute in immediate succession)
  • Rebuild the spatial index after operations that might trigger the bug

  • Set innodb_rtree_index_update_interval=0 to disable deferred updates

  • If you’re planning an upgrade from 8.0 or an earlier 8.4 point release, schedule a pre-upgrade test that exercises an update-then-delete pattern on a table with a spatial index. Verify the index remains healthy with CHECK TABLE ... EXTENDED. This will surface the bug before you move production data.

  • As a precautionary measure, you can drop spatial indexes before upgrade and re-create them after upgrade, then verify integrity.

New reserved keywords

Impact:

  • New reserved words (for example, MANUAL, PARALLEL, QUALIFY, TABLESAMPLE) may conflict with unquoted identifiers and break queries.

Action:

Data type restrictions

Impact:

  • AUTO_INCREMENT is not permitted on FLOAT or DOUBLE.

Action:

  • Evaluate each case for the best solution.

Removed SQL function

Impact:

  • WAIT_UNTIL_SQL_THREAD_AFTER_GTIDS() function is removed (deprecated in 8.0).

Replacement:

  • WAIT_FOR_EXECUTED_GTID_SET().

Action:

  • Replace function usage in procedures, scripts, and runbooks.

Binary log retention variable

Impact:

  • expire_logs_days variable is removed.

Replacement:

  • binlog_expire_logs_seconds.

Action:

  • Adjust configuration and automation to use seconds-based retention.

Impact:

  • Built-in memcached integration variables (for example, daemon_memcached, innodb_api, and related settings) are removed.

Replacement:

  • Externalize caching at the application tier or separate cache services.

Action:

  • Remove dependencies on the built-in memcached functionality.

Further reading

Compatibility and removed items in 8.4

A successful migration requires identifying and addressing all removed parameters, variables, and functions. Using removed items in configuration files or application code will cause errors and prevent the server from starting or applications from running.

Removed server and replication system variables

Variable Name Description Replacement
avoid_temporal_upgrade Whether ALTER TABLE should upgrade pre-5.6.4 temporal columns No replacement
binlog_transaction_dependency_tracking Source of dependency information for multithreaded applier Functionality is now internal; WRITESET is the default and the only available mode
character-set-client-handshake Do not ignore client-side character set value sent during handshake No replacement
default_authentication_plugin Default authentication plugin authentication_policy
expire_logs_days Purge binary logs after a number of days binlog_expire_logs_seconds
group_replication_ip_whitelist List of hosts permitted to connect to the group No replacement
group_replication_primary_member Primary member UUID when in single-primary mode No replacement
group_replication_recovery_complete_at Recovery policies when handling cached transactions No replacement
have_openssl Whether the server supports SSL connections No replacement
have_ssl Whether the server supports SSL connections No replacement
innodb_api_... variables All innodb_api variables related to built-in memcached functionality No replacement

Removed server options, SQL statements, and status variables

Item Name Type Replacement
admin-ssl Server Option --tls-version and --admin-tls-version
authentication_fido_rp_id Server Option No replacement
--language Server Option No replacement
--old and --new Server Option No replacement
Com_change_master Status Variable Com_change_replication_source
Com_show_master_status Status Variable Com_show_binary_log_status
Com_show_slave_status Status Variable Com_show_replica_status
Com_slave_start Status Variable Com_replica_start
Com_slave_stop Status Variable Com_replica_stop
CHANGE MASTER TO SQL Statement CHANGE REPLICATION SOURCE TO
SHOW SLAVE STATUS SQL Statement SHOW REPLICA STATUS
START SLAVE SQL Statement START REPLICA
STOP SLAVE SQL Statement STOP REPLICA
SHOW MASTER STATUS SQL Statement SHOW BINARY LOG STATUS
RESET MASTER SQL Statement RESET BINARY LOGS AND GTIDS
WAIT_UNTIL_SQL_THREAD_AFTER_GTIDS() SQL Function WAIT_FOR_EXECUTED_GTID_SET()

Third-party tool compatibility

Percona XtraBackup (PXB)

  • Version-specific backups: Percona XtraBackup 8.4 creates backups of MySQL 8.4, Percona Server for MySQL 8.4, and Percona XtraDB Cluster 8.4 only.
  • Compatibility: Percona XtraBackup 8.4 does not support backing up databases from MySQL 8.0 or 9.x servers.
  • Action: Upgrade Percona XtraBackup to version 8.4 before or during the database upgrade.

Percona Operator for MySQL

  • Upgrade method: Create a new PXC 8.4 installation using the Percona Operator for PXC 8.4.
  • Data migration: Recover data from an 8.0 backup, then establish asynchronous replication between clusters.
  • In-place upgrade: Not recommended; in-place upgrades may work but are not guaranteed.

ProxySQL

  • MySQL 8.4 support: Recent versions support MySQL 8.4 and include Group Replication support for 8.4 and 9.x.
  • Authentication: ProxySQL 2.6+ supports caching_sha2_password (default in 8.4).
  • Replication terminology: Compatible with SOURCE/REPLICA syntax; can monitor replica lag and manage traffic accordingly.

Pre-upgrade validation

Use these methods to identify compatibility issues:

  • mysqlsh upgrade checker: Identifies many removed parameters and compatibility issues.
  • Manual review: Cross-reference your configuration files and application code against the removed items tables above.
  • Third-party tooling: Verify versions of backup utilities, proxies, and monitoring solutions are compatible with 8.4.

Further reading

Defaults and tuning guidance for 8.4

MySQL 8.4 updates several server defaults to align with modern CPUs, memory sizes, and SSD/NVMe storage. An in-place upgrade that blindly reuses an 8.0-era my.cnf may miss out on these improvements or cause unexpected performance behaviors. Review and re-evaluate your configuration on 8.4, or generate a new config, rather than carrying old settings forward.

Notable InnoDB default changes

InnoDB System Variable Name New Default (8.4) Previous Default (8.0)
innodb_adaptive_hash_index OFF ON
innodb_change_buffering none all
innodb_doublewrite_files 2 innodb_buffer_pool_instances * 2
innodb_doublewrite_pages 128 4
innodb_flush_method on Linux O_DIRECT if supported, otherwise fsync fsync
innodb_io_capacity 10000 200
innodb_log_buffer_size 67108864 (64 MiB) 16777216 (16 MiB)
innodb_buffer_pool_populate ON N/A (new)
innodb_numa_interleave ON OFF
temptable_max_ram 3% of total memory (1–4 GiB range) 1073741824 (1 GiB)
innodb_parallel_read_threads available logical processors / 8 (min 4) 4

Why these changes matter:

  • Higher innodb_io_capacity leverages SSD/NVMe for IO-bound workloads; legacy spinning disks may need a lower value.
  • Larger innodb_log_buffer_size reduces redo flush frequency—helpful for write-heavy workloads.
  • innodb_adaptive_hash_index default OFF favors predictability; the adaptive hash index can become a contention source under concurrency.
  • innodb_buffer_pool_populate defaults ON to enforce page faults on startup and avoid runtime stalls from page faults; disable it if you value fast startup more than runtime latency.
  • innodb_numa_interleave defaults ON to reduce memory imbalance on multi-socket systems. It only controls interleaved NUMA allocations and no longer affects whether page faults are enforced on startup.
  • innodb_change_buffering set to none reduces overhead for modern storage that handles random writes efficiently.
  • innodb_doublewrite_pages increased to 128 improves doublewrite performance on fast storage.

Configuration review checklist

Use this to adapt an 8.0 configuration to 8.4:

  • Remove overrides that merely reassert old 8.0 defaults unless they are proven necessary.
  • Re-evaluate IO settings (innodb_io_capacity, flush method) based on storage type and observed latency.
  • Confirm redo/undo settings and log buffer meet current write patterns.
  • Validate parallel read threads relative to CPU topology and workload.
  • Generate a fresh config for 8.4 when possible; only reapply carefully justified overrides.

Practical evaluation steps

  • Benchmark with your workload: establish a baseline on 8.0, then restore to 8.4 and run the same tests.
  • Compare Performance Schema metrics and wait events for regressions or new hotspots.
  • Adjust a single variable at a time; document changes and their impacts.

Further reading

Percona Toolkit updates for 8.4

Percona Toolkit has been updated to support MySQL 8.4, addressing terminology, deprecations, and authentication improvements. If your automation or runbooks use these tools, plan updates alongside the database upgrade.

Terminology alignment

  • Toolkit commands and output now use SOURCE/REPLICA terminology consistent with MySQL 8.4.

Renamed tools

  • pt-slave-find → pt-replica-find
  • pt-slave-restart → pt-replica-restart

Aliases with the old names remain for a transition period; update scripts and runbooks to the new names.

Deprecated tool

  • pt-slave-delay is deprecated and does not support MySQL 8.4. Use built-in delayed replication features instead.

Authentication and SSL

  • Enhanced SSL/TLS handling and improved support for caching_sha2_password and sha256_password authentication plugins.

What to change in your environment

  • Update automation and scripts: replace pt-slave-find with pt-replica-find, and pt-slave-restart with pt-replica-restart.
  • Remove dependencies on pt-slave-delay; use native delayed replication features instead.
  • Validate Toolkit connectivity using your TLS settings and modern authentication plugins.

Further reading

Upgrade checklist for 8.4

Thorough preparation and validation reduce risk more than any cutover tactic. Use this checklist to guide your upgrade from 8.0 to 8.4, validating each item in staging before upgrading production.

Pre-upgrade checks

Complete these checks before starting the upgrade process.

Authentication and connectivity

Impact: mysql_native_password is disabled by default in 8.4; default_authentication_plugin is removed. New accounts default to caching_sha2_password. The mysql_native_password plugin can still be loaded using --mysql-native-password=ON if needed, but it will be completely removed in the MySQL 9.x series.

Action:

  • Inventory accounts and applications that still use mysql_native_password.
  • Verify drivers/clients support caching_sha2_password and TLS as configured.
  • Plan account migration to caching_sha2_password. If temporary compatibility is needed, --mysql-native-password=ON can be used, but plan migration as this plugin will be removed in future versions.
  • See: authentication methods

Replication and operational scripts

Impact: MASTER/SLAVE syntax is removed and will cause syntax errors if used; use SOURCE/REPLICA commands.

Action:

  • Search and update scripts: START REPLICA, SHOW REPLICA STATUS, CHANGE REPLICATION SOURCE TO.
  • Validate Orchestrator/HA tooling versions for 8.4 syntax.
  • Update Percona Toolkit calls: replace pt-slave-find with pt-replica-find, and pt-slave-restart with pt-replica-restart; remove pt-slave-delay usage.
  • See: Percona Toolkit updates for 8.4

Removed features and variables

Impact: Several legacy statements, status counters, variables, and functions are removed in 8.4.

Action:

  • Replace MASTER/SLAVE statements and counters with SOURCE/REPLICA equivalents.
  • Migrate from expire_logs_days to binlog_expire_logs_seconds.
  • Replace WAIT_UNTIL_SQL_THREAD_AFTER_GTIDS() with WAIT_FOR_EXECUTED_GTID_SET().
  • Remove dependencies on built-in memcached variables/APIs.
  • See: Breaking and incompatible changes in 8.4

Reserved keywords in identifiers

Impact: New reserved words (for example, MANUAL, PARALLEL, QUALIFY, TABLESAMPLE) can break schemas and queries.

Action:

Schema constraints

Impact: AUTO_INCREMENT is not allowed on FLOAT/DOUBLE.

Action:

  • Identify and convert any FLOAT/DOUBLE AUTO_INCREMENT columns to integer types prior to upgrade.

Configuration defaults review

Impact: 8.4 changes several InnoDB defaults for modern hardware; old 8.0 configs may not be optimal and can cause behavior changes.

Action:

Spatial indexes

Impact: A known issue can corrupt a spatial index (R-Tree index) in MySQL 8.4.0 through 8.4.3. The corruption is triggered when an UPDATE that slightly changes a geometry’s MBR (Minimum Bounding Rectangle) is immediately followed by a DELETE of the same row. This issue is fixed in 8.4.4 and later.

Action:

  • If upgrading to 8.4.0-8.4.3, drop spatial indexes before upgrade (document which ones for post-upgrade re-creation) as a precautionary measure.
  • If upgrading to 8.4.4 or later, this issue is fixed; spatial indexes can remain in place.
  • Plan to re-create any spatial indexes dropped pre-upgrade after upgrade completion.
  • See: Breaking and incompatible changes in 8.4 for detailed information and workarounds.

Backup and recovery rehearsal

Action:

  • Take a hot backup with Percona XtraBackup; document restore steps and timings.
  • Restore into a clean 8.4 environment; validate startup and metadata upgrade.
  • See: Backup and restore overview

Behavior comparison and testing

Action:

  • Use pt-upgrade to compare query plans/behavior between 8.0 and 8.4.
  • Run application smoke and load tests against a restored 8.4 copy.

Plugins to components transitions

Impact: Some 8.0 plugins are removed or replaced by components in 8.4.

Action:

  • If a component exists in 8.0 (for example, data masking), transition in 8.0 before upgrading.
  • Plan configuration changes from plugin variables/--early-plugin-load to component manifests/config files.
  • See: Upgrade from plugins to components

Rollback feasibility

Action:

  • Define a rollback path (for example, keep 8.0 environment on standby or validate point-in-time recovery to 8.0-compatible readers if applicable).
  • Confirm cutover/rollback runbooks with approvers.

Post-upgrade validation

Run these checks immediately after upgrading from 8.0 to 8.4 and before widening traffic.

Connectivity and authentication

  • Verify application logins for every service account.
  • Confirm new account creations default to caching_sha2_password as expected.

Replication health (if applicable)

  • Confirm SHOW REPLICA STATUS reports healthy IO/SQL threads.
  • Exercise planned failover and change-source procedures.

Spatial indexes

  • Re-create any spatial indexes dropped pre-upgrade.
  • Run integrity checks (for example, CHECK TABLE ... EXTENDED) and representative spatial queries to verify index health.

Workload and performance baselines

  • Re-run baseline queries and workload tests; compare latency and throughput.
  • Review changes in 8.4 defaults that can affect performance (optimizer/costing, redo/undo, IO settings) and tune as needed.
  • See: Defaults and tuning guidance for 8.4

Logs and observability

  • Review error logs and warnings post-startup and during smoke tests.
  • Inspect Performance Schema metrics and application SLOs for regressions.

Backup and recovery

  • Take a fresh full backup with Percona XtraBackup.
  • Optionally perform a spot restore test to validate recovery on 8.4.

Further reading

Upgrade strategies

For critical production systems, consider engaging Percona Support to assist with your upgrade process. Our experts can help ensure a smooth transition and minimize potential risks during this sensitive operation.

Downgrade options

Review the Downgrade options to ensure that your downgrade path is supported.

In-place upgrade

Use the in-place upgrade strategy only as a last resort. This approach involves shutting down the server and replacing the server binaries or packages with new ones. The new server version then starts using the existing data directory. Configure the server to perform a slow shutdown by setting innodb_fast_shutdown=0 before shutting down.

The benefits are:

  • Lower infrastructure costs compared to creating a new environment, though nodes require testing.

  • Ability to complete an upgrade over weeks with cool-down periods between reader node upgrades.

  • Requires a failover of production traffic, and achieving minimal downtime demands robust high-availability tools.

If you use XA transactions with InnoDB, running XA RECOVER before upgrading checks for uncommitted XA transactions. If results are returned, either commit or rollback the XA transactions by issuing an XA COMMIT or XA ROLLBACK statement.

New environment with cut over

Upgrading with a new environment involves provisioning a duplicate environment with the same number of servers with the same hardware specs and same operating system as the current production nodes.

On the newly provided hardware, the target MySQL version will be installed. The new environment will be set up, and the production data will be recovered. Remember that you can use pt-config-diff to verify MySQL configurations.

Replication from the current source to the newly built environment will be established. At cutover time, all writes on the current source will be halted, and the application traffic will need to be redirected to the new source. The cutover can be done using a Virtual IP address or manually redirecting the application itself. Once writes are being received on the new environment, you are in a fail forward situation, and the old environment can be torn down.

The new environment strategy has the following pros and cons:

  • Additional infrastructure cost since a new environment must be built.

  • Ability to upgrade both the OS and the DBMS at the same time.

  • Allows upgrade of hardware easily.

  • Requires only a single cutover window.

8.0 → 8.4 migration methods

Choose the approach that matches your downtime budget, risk tolerance, and rollback needs. Always rehearse in a non-production environment first.

Note: For a complete overview of supported upgrade paths and methods, see MySQL upgrade paths and supported methods.

In-place upgrade (stop/replace/start)

Downtime: short to moderate

Risk: higher (shared data directory; fewer rollback options)

Use when: the environment is simple, downtime is acceptable, and you have strong backups and validation.

Prerequisites:

  • Complete Upgrade checklist pre-upgrade checks
  • Set innodb_fast_shutdown=0 for a clean shutdown
  • Verified backup and restore

Rollback: restore backup and revert binaries.

Logical dump and restore (clean rebuild)

Downtime: moderate to high (data size dependent)

Risk: moderate (clean metadata; slower for large datasets)

Use when: you want a pristine 8.4 instance and can accept longer downtime.

Prerequisites:

  • Sufficient capacity for parallel dump/restore
  • Application maintenance window sized to data volume

Rollback: keep 8.0 online until validation completes; redirect traffic back if needed.

Side-by-side with replication and controlled cutover

Downtime: minimal (cutover only)

Risk: lower (new environment; defined fallback until fail-forward)

Use when: you need the smallest outage and can provision a parallel environment.

Prerequisites:

  • Build a new 8.4 environment; establish replication from 8.0
  • Validate workload on the replica(s) and rehearse failover

Cutover: stop writes on 8.0, allow replica to catch up, redirect traffic (VIP/DNS), then promote 8.4.

Rollback: if issues arise before fail-forward, redirect traffic back to 8.0 and resume writes.

Further reading

MySQL upgrade paths and supported methods

MySQL supports different upgrade paths depending on the source and target versions. Choose the appropriate method based on your current version and target.

Upgrade path matrix

Upgrade Path Path Examples Supported Upgrade Methods
Within an LTS or Bugfix series 8.0.37 to 8.0.41 or 8.4.0 to 8.4.4 In-place upgrade, logical dump and load, replication, and MySQL Clone
From an LTS or Bugfix series to the next LTS series 8.0.37 to 8.4.x LTS In-place upgrade, logical dump and load, and replication
From an LTS or Bugfix release to an Innovation release before the next LTS series 8.0.34 to 8.3.0 or 8.4.0 to 9.0.0 In-place upgrade, logical dump and load, and replication
From the Innovation series to the next LTS series 8.3.0 to 8.4 LTS In-place upgrade, logical dump and load, and replication
From an Innovation series to an Innovation release after the next LTS series Not allowed, two steps are required: 8.3.0 to 8.4 LTS, and 8.4 LTS to 9.x Innovation In-place upgrade, logical dump and load, and replication

Key considerations

  • LTS to LTS: Direct upgrade from 8.0 LTS to 8.4 LTS is supported with multiple methods.
  • Innovation to Innovation: Cannot skip LTS releases; must upgrade through the LTS series first.
  • MySQL Clone: Only available for upgrades within the same major version series.
  • Replication: Available for most upgrade paths but requires careful planning for cross-version replication.

Choosing your upgrade method

  • In-place upgrade: Fastest but highest risk; requires downtime.
  • Logical dump and load: Cleanest but slowest for large datasets; requires downtime.
  • Replication: Minimal downtime but requires additional infrastructure; good for high-availability setups.
  • MySQL Clone: Fastest for same-series upgrades; requires compatible versions.

Further reading

Upgrade procedures for 8.4

Need expert guidance for your Percona Server upgrade? Percona Support is here to help .

This document provides step-by-step procedures for upgrading Percona Server for MySQL using either Percona repositories (recommended) or standalone packages.

Before beginning the upgrade process:

  1. Complete the upgrade checklist pre-upgrade checks.
  2. Create a full backup (or dump if possible) of your database.
  3. Back up your database configuration file (my.cnf) to a safe location, then modify it as needed (for example, remove deprecated variables, update settings for 8.4) before stopping the server.

  4. Stop the server using the appropriate command for your system:

    sudo systemctl stop mysql

Critical

Always test the upgrade process in a non-production environment first. For detailed upgrade procedures or if you encounter any issues during this process, our Percona Support team is available to assist you.

We recommend using the Percona repositories to upgrade your server. This method automatically handles dependencies and simplifies the upgrade process.

Find the instructions on how to enable the repositories in the following documents:

Run the following commands as root or use the sudo command.

  1. Make a full backup (or dump if possible) of your database. Copy the database configuration file, my.cnf, to another directory as a backup. If the configuration file is not backed up, it may be overwritten during the upgrade.

  2. Stop the server with the appropriate command for your system:

    sudo systemctl stop mysql

  3. Modify the database configuration file, my.cnf, as needed (for example, remove deprecated variables, update settings for 8.4).

  4. Install Percona Server for MySQL:

    sudo apt update
sudo apt install curl
curl -O https://repo.percona.com/apt/percona-release_latest.generic_all.deb 
sudo apt install gnupg2 lsb-release ./percona-release_latest.generic_all.deb
sudo apt update
sudo percona-release setup ps-84-lts
sudo apt install percona-server-server

  5. Install the storage engine packages.

    If you used the MyRocks storage engine in Percona Server for MySQL 8.4, install the percona-server-rocksdb package:

    sudo apt install percona-server-rocksdb

  6. The mysqld binary automatically runs the upgrade process if needed. To find more information, see MySQL Upgrade Process .

  7. Restart the service:

    sudo systemctl restart mysql

After the service has been successfully restarted, you can use the new Percona Server for MySQL 8.4.

Run the following commands as root or use the sudo command.

  1. Make a full backup (or dump if possible) of your database. Copy the database configuration file, for example, my.cnf, to another directory to save it.

  2. Stop the server with the appropriate command for your system:

    sudo systemctl stop mysql

  3. Check your installed packages:

    rpm -qa | grep Percona-Server

  4. Remove only the packages without dependencies and leave dependent packages. The command does not prompt for confirmation:

    rpm -qa | grep Percona-Server | xargs rpm -e --nodeps

  5. Remove the mysql-related packages:

    rpm -qa | grep '^mysql-' | xargs rpm -e --nodeps

  6. Install the percona-server-server package:

    sudo yum install https://repo.percona.com/yum/percona-release-latest.noarch.rpm
sudo percona-release setup ps-84-lts
sudo yum install percona-server-server

  7. Install the storage engine packages.

    If you used the MyRocks storage engine in the previous version, install the percona-server-rocksdb package:

    sudo yum install percona-server-rocksdb

  8. Modify your configuration file, my.cnf, as needed (for example, remove deprecated variables, update settings for 8.4). If you were using plugins that have been replaced by components in 8.4, plan the transition to components. See Upgrade from plugins to components for details.

  9. The mysqld binary automatically runs the upgrade process if needed. To find more information, see MySQL Upgrade Process .

  10. Restart the server:

    sudo systemctl restart mysql

After the service has been successfully restarted, you can use the Percona Server for MySQL 8.4.

Using standalone packages

Use this method when you cannot use repositories or need to install from manually downloaded packages. This method requires you to manually resolve dependencies.

  1. Remove the installed packages with their dependencies:

    sudo apt autoremove percona-server percona-client

  2. Do the required modifications in the database configuration file my.cnf.

  3. Download the following packages for your architecture:

    • percona-server-server
    • percona-server-client
    • percona-server-common
    • libperconaserverclient21

    The following example downloads Percona Server for MySQL 8.4.11-11 packages for Debian 12.0:

    wget https://downloads.percona.com/downloads/Percona-Server-innovative-release/Percona-Server-8.4.11-11/binary/debian/bookworm/x86_64/Percona-Server-8.4.11-11-r582ebeef-bookworm-x86_64-bundle.tar

  4. Unpack the bundle to get the packages:

    tar xvf Percona-Server-8.4.11-11-r582ebeef-bookworm-x86_64-bundle.tar

    After you unpack the bundle, you should see the following packages:

    ls *.deb
    Expected output 
    libperconaserverclient21-dev_8.4.11-11.bookworm_amd64.deb
percona-server-dbg_8.4.11-11.bookworm_amd64.deb
libperconaserverclient21_8.4.11-11.bookworm_amd64.deb      
percona-server-rocksdb_8.4.11-11.bookworm_amd64.deb
percona-mysql-router_8.4.11-11.bookworm_amd64.deb
percona-server-server_8.4.11-11.bookworm_amd64.deb
percona-server-client_8.4.11-11.bookworm_amd64.deb
percona-server-source_8.4.11-11.bookworm_amd64.deb
percona-server-common_8.4.11-11.bookworm_amd64.deb
percona-server-test_8.4.11-11.bookworm_amd64.deb

  5. Install Percona Server for MySQL:

    sudo dpkg -i *.deb

    This command installs the packages from the bundle. Another option is to download or specify only the packages you need for running Percona Server for MySQL installation (libperconaserverclient21_8.4.11-11.bookworm_amd64.deb, percona-server-client_8.4.11-11.bookworm_amd64.deb, percona-server-common_8.4.11-11.bookworm_amd64.deb, and percona-server-server_8.4.11-11.bookworm_amd64.deb).

    Warning

    When installing packages manually, you must resolve all the dependencies and install missing packages yourself. At least the following packages should be installed before installing Percona Server for MySQL 8.4.11-11: * libmecab2 * libjemalloc1 * zlib1g-dev * libaio1

  6. The mysqld binary automatically runs the upgrade process if needed. To find more information, see MySQL Upgrade Process .

  7. Restart the service:

    sudo service mysql restart

After the service has been successfully restarted, use the new Percona Server for MySQL 8.4.11-11.

  1. Check the installed packages:

    rpm -qa | grep percona-server
    Expected output 
    percona-server-shared-8.4.11-11.el9.x86_64
percona-server-shared-compat-8.4.11-11.el9.x86_64
percona-server-client-8.4.11-11.el9.x86_64
percona-server-server-8.4.11-11.el9.x86_64

    You may have the shared-compat package, which is required for compatibility.

  2. Remove the packages without dependencies:

    rpm -qa | grep percona-server | xargs rpm -e --nodeps

    It is important that you remove the packages without dependencies as many packages may depend on these (as they replace mysql) and will be removed if omitted.

    To remove the listed packages, run:

    rpm -qa | grep '^mysql-' | xargs rpm -e --nodeps

  3. Download the packages of the desired series for your architecture from the download page . The easiest way is to download the bundle which contains all the packages. The following example downloads Percona Server for MySQL 8.4.11-11 packages for RHEL 9:

    wget https://downloads.percona.com/downloads/Percona-Server-8.4/Percona-Server-8.4.11-11/binary/redhat/9/x86_64/Percona-Server-8.4.11-11-r9927a2fb-el9-x86_64-bundle.tar

  4. Unpack the bundle to get the packages:

    tar xvf Percona-Server-8.4.11-11-r9927a2fb-el9-x86_64-bundle.tar

    After you unpack the bundle, you should see the following packages:

    ls *.rpm

  5. Install Percona Server for MySQL:

    sudo rpm -ivh percona-server-server-8.4.11-11.el9.x86_64.rpm \
percona-server-client-8.4.11-11.el9.x86_64.rpm \
percona-server-shared-8.4.11-11.el9.x86_64.rpm \
percona-server-shared-compat-8.4.11-11.el9.x86_64.rpm

    This command installs only packages required to run the Percona Server for MySQL 8.4.11-11.

    You can install all the packages (for debugging, testing, etc.) with:

    sudo rpm -ivh *.rpm

    Note

    When manually installing packages, you must resolve all the dependencies and install missing ones.

  6. Modify your configuration file, my.cnf, as needed (for example, remove deprecated variables, update settings for 8.4). If you were using plugins that have been replaced by components in 8.4, plan the transition to components. See Upgrade from plugins to components for details.

    RHEL or derivatives automatically backs up the previous configuration file to /etc/my.cnf.rpmsave and installs the default my.cnf. After the upgrade/install process completes, you can restore your configuration from the backup (after removing all unsupported system variables).

  7. The mysqld binary automatically runs the upgrade process if needed. To find more information, see MySQL Upgrade Process .

  8. Restart the server:

    sudo service mysql restart

After the service has been successfully restarted, you can use the new Percona Server for MySQL 8.4.11-11.

Post-upgrade validation

After completing the upgrade, follow the post-upgrade validation steps in the upgrade checklist.

Further reading

Upgrade from plugins to components

Need help navigating plugin to component transitions? Percona Support can assist .

Percona Server for MySQL 8.4 introduces a shift from plugins to components for several key features, a change that requires a manual transition during the upgrade process. It is generally recommended to transition to the component version of a feature in the 8.0 series before performing the upgrade to 8.4 if both a plugin and a component are available for that feature.

The following plugins have changed:

Plugin 8.0 information 8.4 changes Notes
keyring_vault Only available as a plugin component_keyring_vault This plugin has been replaced by the component_keyring_vault component. The upgrade requires a manual conversion of the plugin’s configuration file from the keyring_vault_config system variable format to a JSON format.
audit_log Only available as a plugin a deprecated plugin added in 8.4.7-7 The recommended replacement is component_audit_log_filter.
audit_log_filter Only available as a plugin component_audit_log_filter This plugin has a corresponding component. The transition to the component should be performed after the upgrade to 8.4.
data_masking Available as a plugin and component component_masking_functions For this feature, it is specifically advised to transition to the component_masking_functions in the 8.0 series before upgrading to 8.4.
binlog_utils_udf and percona-udf user defined functions Previously installed via a plugin. Users must install the plugin and then run CREATE FUNCTION ... SONAME ... for each function. component_binlog_utils_udf and component_percona_udf These user-defined functions, previously installed via plugins, are now available as components. After running INSTALL COMPONENT, all functions are automatically registered, which simplifies the process.

Transition a plugin to a component

The operation to transition from a plugin to a component can be complicated. Always test the migration in a staging environment before applying the changes to the production servers. To ensure there is minimal interruption, the key preparation steps are the following:

  • Plan for downtime

  • Create a comprehensive testing strategy

  • Verify that the existing functionality transfers correctly

Before you start, review the differences between the plugin and the component. The configuration of these features changes: plugins use system variables and the --early-plugin-load option, while components rely on a separate configuration file and are loaded using a manifest.

General procedure

The general procedure for transitioning from a plugin to a component involves:

  1. Setup the component’s configuration file.

  2. Load the component using the manifest (INSTALL COMPONENT or manifest file, as applicable).

  3. Confirm that the component works. Thoroughly test the component’s functionality in your staging environment. Run queries or other operations to verify all existing functionality is correctly transferred.

  4. After confirmation, remove the original plugin.

This process should be carefully planned to minimize downtime and ensure that all existing functionality is correctly transferred.

Transition timing

The timing of the transition depends on the specific plugin:

  • Transition before upgrade: For plugins that have both plugin and component versions available in 8.0 (for example, data_masking), transition to the component in 8.0 before upgrading to 8.4.

  • Transition after upgrade: For plugins that only exist as plugins in 8.0 but have component equivalents in 8.4 (for example, audit_log_filter, keyring_vault), you must upgrade to 8.4 first, then transition to the component. The general procedure for these cases is:

  • Test the plugin functionality in 8.0 to establish a baseline.

  • Stop the service and upgrade to 8.4.

  • Review and adjust configurations as needed (for example, convert keyring_vault_config system variable to JSON format for the component).

  • Start the new 8.4 server.

  • Transition to the component following the general procedure above.

  • Verify the component works correctly in 8.4.

Further reading

Component-specific documentation

Downgrade Percona Server for MySQL

Review Get help from Percona for ways that we can work with you.

Downgrading to a 5.7 or earlier series is not supported.

Between versions within the same Long-Term Support (LTS) series, you can downgrade from 8.4.y LTS to 8.4.x LTS using the following methods:

  • Performing an in-place upgrade

  • Creating a logical dump and loading it

  • Use MySQL Clone functionality

  • Set up replication between the versions

Between one LTS or Bugfix series to the previous LTS or Bugfix series, such as moving from 8.4.x LTS to 8.0.y, you have two primary options:

  • Create a logical dump of your data and load it into the older version

  • Set up replication between the versions.

Important

This downgrade path is only supported when no new server functionality has been applied to your data.

Between an LTS or Bugfix series to an earlier Innovation series (after the previous LTS release), such as from 8.4.x LTS to 8.3.0 Innovation, you have the following options:

  • Create a logical dump of your data and load it into the older version

  • Set up replication between the versions.

Important

This downgrade path is only supported when no new server functionality has been applied to your data.

We don’t support downgrades with any 8.0.x release below 8.0.34. Releases in the range above 8.0.34 can be downgraded to any release within that range, including 8.0.34.

Downgrading risks

Downgrading has the following risks:

Risk Description
Data loss If the downgrade process has issues, you may lose your data. It is crucial that you back up your data before attempting to downgrade.
Incompatibility If you use any feature or improvement in the latest version, downgrading could result in incompatibility issues.
Performance Downgrading may result in a loss of performance
Security Newer versions have security updates that are not available in the older versions, which could lead to exposure.

Further reading

Downgrade from Percona Server for MySQL Pro

If you want to downgrade from Percona Server for MySQL Pro to the same version of Percona Server for MySQL, do the following:

  1. Set up the Percona Server for MySQL 8.4 repository

    sudo percona-release setup ps84

  2. Stop the mysql server.

    sudo systemctl stop mysql

  3. Install the server package

    sudo apt install percona-server-server

    Install other required packages. Check files in the DEB package built for Percona Server for MySQL 8.4.

  4. Start the mysql server

    sudo systemctl start mysql

Note

On Debian 12, if you want to remove the Percona Server for MySQL after the downgrade, you must stop the server manually. This behavior will be fixed in future releases.

sudo systemctl stop mysql

  1. Set up the Percona Server for MySQL 8.4 repository

    sudo percona-release setup ps84

  2. Stop the mysql server.

    sudo systemctl stop mysql

  3. Install the server package

    sudo yum --allowerasing install percona-server-server

    Install other required packages. Check files in the RPM package built for Percona Server for MySQL 8.4.

  4. Start the mysql server

    sudo systemctl start mysql

Post-Installation

First five minutes after installation

This guide walks you through the most important security and stability steps to take right after installing Percona Server for MySQL. Completing these steps helps protect your server and makes troubleshooting easier.

Quickstart path: After Install (Ubuntu), Install (Oracle Linux), or Docker. Next: Work with a database.

Step What you do
Secure the installation Run post-installation security steps so the server is not left in a default, weak state.
Create a least-privilege admin user Use a dedicated admin account with only the privileges (permissions) that account needs instead of root.
Configure an OOM-killer guardrail Reduce the chance the Linux OOM killer will abruptly terminate the MySQL process.
Enable diagnostic logging Turn on error and slow-query logging so you can diagnose issues later.
Verify the backup path Ensure the directory where backups will go exists and is writable.

When you are done, use Sanity check: First five minutes to verify that these steps were applied correctly.

Secure the installation

Before doing anything else, complete the standard post-installation security steps. These ensure the server is not left with default passwords, anonymous users (accounts with no username that can connect without proper credentials), or remote root login.

  1. Set or update the root password
    The root account is the default MySQL administrative user. If you did not set its password during install (for example, on some RPM-based systems), set one now. See Post-installation — Update the root password.

  2. Run mysql_secure_installation to:

    • Change the root password (if needed)
    • Disallow remote login for root
    • Remove anonymous users
    • Remove the test database
    • Reload privilege tables

  3. Optional: populate time zone tables

If you installed from source or generic binaries, populate the time zone tables (the system data that makes time zone–aware date and time functions work correctly).

For more information, see Post-installation — Secure the server.

Create a least-privilege admin user

Use a dedicated admin account for day-to-day administration instead of root. Give that account only the privileges the account needs (least privilege: the practice of granting only the minimum permissions necessary). Reserve root for emergencies and avoid using root for routine tasks.

  1. Connect as root (or another user that can create users and grant privileges):

    mysql -u root -p

  2. Create an admin user. Use a strong password and a specific host (the machine or network location the user can connect from; for example, localhost means the same machine only) instead of % (which means any host):

    CREATE USER 'admin'@'localhost' IDENTIFIED BY 'YourStrongPassword_12';

  3. Grant only the privileges the admin needs. Grant means to assign a permission to a user. The following is a common set for an admin who manages users, replication, and backups but does not need full SUPER (a powerful built-in privilege) or global privileges on all databases (*.*):

    GRANT CREATE USER, RELOAD, PROCESS, REPLICATION CLIENT, REPLICATION SLAVE, BACKUP_ADMIN,
      SHOW DATABASES, SELECT ON *.* TO 'admin'@'localhost';
FLUSH PRIVILEGES;

    FLUSH PRIVILEGES tells the server to reload the privilege tables so the new grants take effect immediately.

  4. Test the new user:

    mysql -u admin -p -e "SHOW DATABASES;"

  5. Use admin for daily tasks and keep root for rare, high-privilege operations.

Adjust privileges to your needs

Add or remove privileges based on what this admin will do. For example, add SHOW VIEW, CREATE ROUTINE, or EVENT if they manage views (saved queries presented as virtual tables), routines (stored procedures and functions), or scheduled events. Avoid granting SUPER, ALL, or *.* unless necessary. You can review effective privileges with SHOW EFFECTIVE GRANTS.

Configure an OOM-killer guardrail

On Linux, when the system runs low on memory, the kernel OOM killer (out-of-memory killer: a kernel process that terminates other processes to free memory) may terminate processes. MySQL can be one of them, which leads to abrupt shutdowns. You can make the MySQL service less likely to be chosen by adjusting its OOM score.

Using systemd (the Linux system and service manager that starts and manages the MySQL service), set OOMScoreAdjust so that the kernel prefers to kill other processes before MySQL. A typical value is -500 (range is -1000 to 1000; lower means less likely to be killed).

  1. Create or edit an override file for the MySQL service. On Red Hat–based systems the service is often mysqld; on Debian/Ubuntu the service may be named mysql. Adjust the service name if needed:
sudo systemctl edit mysqld

If your service is named mysql:

sudo systemctl edit mysql
  1. Add the following (use mysqld or mysql to match your service name):
[Service]
OOMScoreAdjust=-500
  1. Save and close the editor. Reload systemd and restart the MySQL service:
sudo systemctl daemon-reload
sudo systemctl restart mysqld
  1. Confirm the setting:
systemctl show mysqld -p OOMScoreAdjust

You should see OOMScoreAdjust=-500 (or the value you set).

Not a memory limit

The OOM score adjustment only influences which process the OOM killer chooses. It does not set a memory limit for MySQL. For strict limits, use cgroups or MySQL/InnoDB memory settings.

Enable diagnostic logging

Enabling the error log (a file where the server records errors, warnings, and startup messages) and the slow query log (a file that records SQL statements that run longer than a threshold you set) gives you the information you need to diagnose problems and tune performance.

  1. Locate your configuration file
    The configuration file (often my.cnf or mysqld.cnf) controls server options. Common paths: /etc/my.cnf, /etc/mysql/my.cnf, or /etc/mysql/mysql.conf.d/mysqld.cnf. Edit the file under the [mysqld] section (the section that applies to the MySQL server process).

  2. Ensure the error log is set
    The server usually writes an error log by default. Confirm or set an explicit path so you know where to look:

[mysqld]
log_error=/var/log/mysql/error.log

On some systems the directory is /var/lib/mysql (the default data directory, where the server stores database files) or similar. Create the log directory if needed and set ownership to the MySQL system user (the operating-system user that runs the MySQL process; often named mysql):

sudo mkdir -p /var/log/mysql
sudo chown mysql:mysql /var/log/mysql
  1. Enable the slow query log
    The slow query log helps you find queries that need optimization:
slow_query_log = 1
long_query_time = 2
slow_query_log_file = /var/log/mysql/slow.log

Adjust long_query_time (the time in seconds above which a query is considered “slow” and written to the log) to what you consider slow. Restart the server after changing the config.

  1. Optional: restrict log locations
    For tighter control over where logs are written, see Restrict dynamic log file locations.

Verify the backup path

Your backup strategy (for example, Percona XtraBackup for full physical backups, or mysqldump for logical backups that produce SQL) will write to a directory. Verify that path before you run a backup.

  1. Choose a backup directory
    For example, /var/backups/mysql or a dedicated volume. Do not use the data directory (where the server stores live database files) for backup output.

  2. Create the directory and set permissions
    The MySQL process (or the operating-system user that runs the backup tool) must be able to write there:

sudo mkdir -p /var/backups/mysql
sudo chown mysql:mysql /var/backups/mysql
sudo chmod 750 /var/backups/mysql
  1. Check that the backup directory is writable
    As the MySQL user (or the backup user), test write access:
sudo -u mysql touch /var/backups/mysql/.write_test && sudo -u mysql rm /var/backups/mysql/.write_test && echo "OK: backup path is writable"
  1. Document the path
    Use this path in your backup scripts, cron jobs, or documentation so that restores use the same location.

For backup strategy and tools, see Backup and restore overview. For Percona XtraBackup , see the Quickstart for 8.4 .

Next steps

Sanity check: First five minutes

This guide helps you verify that the First five minutes security and stability steps have been applied correctly. Run these checks after completing the First five minutes, or on an existing server to see if those steps are in place.

You need:

  • Access to the MySQL server as a user with enough privileges (permissions to run statements and see system data) to run the SQL checks—for example, root (the default MySQL administrative user) or the admin user you created.
  • Ability to run shell commands (and systemctl, the systemd command to inspect and manage services) on the host (the machine where MySQL runs).

Summary: what we check

Check Purpose
1. Root and security script Root has a password and basic hardening (no anonymous user, no test DB, root not remote) is done.
2. Least-privilege admin user A dedicated admin user exists and does not have full SUPER/ALL.
3. OOM-killer guardrail systemd OOM score is set so MySQL is less likely to be killed by the kernel when the system is low on memory.
4. Diagnostic logging Error log and slow query log are enabled and paths are set.
5. Backup path Backup directory exists and is writable by the MySQL (or backup) user.

1. Root and security script

Goal: Confirm the MySQL root account is not empty and that mysql_secure_installation–style hardening is in place.

1.1 Root has a password
Try connecting without a password; the connection should fail:

mysql -u root -h localhost 2>&1 | head -1

You want to see an access denied or password-related error, not a successful connection.

1.2 No anonymous user
Anonymous users (accounts with an empty username that can connect without proper credentials) should not exist:

mysql -u root -p -e "SELECT user, host FROM mysql.user WHERE user = '';"

Expected: empty result set (no rows returned by the query).

1.3 No remote root
The root user should not be allowed to connect from remote hosts—only from the same machine (root@localhost or similar):

mysql -u root -p -e "SELECT user, host FROM mysql.user WHERE user = 'root' AND host != 'localhost';"

Expected: empty result set. If you use a specific hostname for root (e.g. [email protected]), that is fine; the important thing is no root@'%' (which would allow root from any host).

1.4 Test database removed
The default test database (a sample database that is often removed for security) should not exist:

mysql -u root -p -e "SHOW DATABASES LIKE 'test';"

Expected: empty result set.

2. Least-privilege admin user

Goal: Confirm you have a dedicated admin user and that the account does not have full SUPER (a powerful built-in privilege) or global privileges on all databases (*.*) unless you intentionally grant them.

2.1 List admin-like users
Adjust the user name if you used something other than admin:

mysql -u root -p -e "SELECT user, host FROM mysql.user WHERE user IN ('admin','dba','mysqladmin') OR user NOT IN ('root','mysql.session','mysql.sys','mysql.infoschema');"

Use this to identify the account you use for day-to-day admin.

2.2 Check that admin does not have SUPER
Replace 'admin'@'localhost' with your admin user and host (the connection source; localhost means the same machine):

mysql -u root -p -e "SHOW GRANTS FOR 'admin'@'localhost';"

Expected: no line containing SUPER or ALL PRIVILEGES ON *.* unless you deliberately grant them. A limited set of grants (assigned privileges) such as CREATE USER, RELOAD, PROCESS, REPLICATION CLIENT, BACKUP_ADMIN, etc., is what we expect from the First five minutes.

2.3 (Optional) Effective privileges
To see the full effective privilege set (the combined permissions a user actually has, including from roles), use SHOW EFFECTIVE GRANTS if your server supports that statement:

mysql -u root -p -e "SHOW EFFECTIVE GRANTS FOR 'admin'@'localhost';"

3. OOM-killer guardrail

Goal: Confirm that the MySQL systemd service has a lower OOM score (a value the kernel uses when choosing which process to terminate when the system is out of memory; lower means less likely to be killed) so the kernel is less likely to kill MySQL.

3.1 Service name
On Red Hat–based systems the service is often mysqld; on Debian/Ubuntu the service may be named mysql. Check which is in use:

systemctl status mysqld 2>/dev/null || systemctl status mysql 2>/dev/null

3.2 OOMScoreAdjust
Use the same service name as above:

systemctl show mysqld -p OOMScoreAdjust
# or
systemctl show mysql -p OOMScoreAdjust

Expected: OOMScoreAdjust=-500 (or another negative value). If you see an empty value or OOMScoreAdjust=0, the First five minutes OOM guardrail is not applied.

4. Diagnostic logging

Goal: Confirm the error log and slow query log are enabled and that paths are set so you can find them.

4.1 Error log
Check that the error log (the file where the server records errors, warnings, and startup messages) is set and that the server is writing to that file:

mysql -u root -p -e "SELECT @@log_error;"

Expected: a non-empty path (e.g. /var/log/mysql/error.log or a path under the datadir—the directory where the server stores database files). Then check that the file exists and is recent:

ls -la $(mysql -u root -p -N -e "SELECT @@log_error;" 2>/dev/null)

4.2 Slow query log
Check that the slow query log (the file that records SQL statements that run longer than the long_query_time threshold) is on and where that log is written:

mysql -u root -p -e "SELECT @@slow_query_log, @@slow_query_log_file, @@long_query_time;"

Expected: slow_query_log = 1, and slow_query_log_file set to a path you can monitor. Optionally confirm the file exists and is writable after the server has been running with slow log enabled.

5. Backup path

Goal: Confirm the directory you use for backups exists and is writable by the MySQL system user (the operating-system user that runs the MySQL process or backup tool; often named mysql).

5.1 Choose your backup path
Use the same path you configured in First five minutes — Verify the backup path (e.g. /var/backups/mysql). Set that path in a variable for the next commands:

BACKUP_DIR=/var/backups/mysql

5.2 Directory exists

test -d "$BACKUP_DIR" && echo "OK: backup directory exists" || echo "FAIL: backup directory missing"

5.3 Writable by MySQL user
Replace mysql with the user that runs your backup tool if different:

sudo -u mysql touch "$BACKUP_DIR/.write_test" 2>/dev/null && sudo -u mysql rm -f "$BACKUP_DIR/.write_test" && echo "OK: backup path is writable" || echo "FAIL: backup path not writable by mysql"

Optional: run checks in one go

You can run a subset of these checks from the shell in one pass. The following assumes:

  • Service name is mysqld (change to mysql if needed).
  • Admin user is 'admin'@'localhost' (change SQL if needed).
  • Backup path is /var/backups/mysql (change if needed).
  • You will enter the root password when prompted.
echo "=== 1. No anonymous user ==="
mysql -u root -p -e "SELECT user, host FROM mysql.user WHERE user = '';"

echo "=== 2. No remote root ==="
mysql -u root -p -e "SELECT user, host FROM mysql.user WHERE user = 'root' AND host != 'localhost';"

echo "=== 3. No test database ==="
mysql -u root -p -e "SHOW DATABASES LIKE 'test';"

echo "=== 4. Admin grants (no SUPER/ALL on *.*) ==="
mysql -u root -p -e "SHOW GRANTS FOR 'admin'@'localhost';"

echo "=== 5. OOMScoreAdjust ==="
systemctl show mysqld -p OOMScoreAdjust

echo "=== 6. Error log and slow query log ==="
mysql -u root -p -e "SELECT @@log_error, @@slow_query_log, @@slow_query_log_file;"

echo "=== 7. Backup path writable ==="
BACKUP_DIR=/var/backups/mysql
test -d "$BACKUP_DIR" && sudo -u mysql touch "$BACKUP_DIR/.write_test" 2>/dev/null && sudo -u mysql rm -f "$BACKUP_DIR/.write_test" && echo "OK: writable" || echo "FAIL or missing"

Interpret the output using the expected results described in each section above.

Next steps

  • If any check fails, re-read the corresponding section in First five minutes and apply or fix the step.
  • Re-run this sanity check after changes to confirm everything passes.
  • For backup strategy and restore testing, see Backup and restore overview.

Post-installation

Depending on the type of installation, you may need to do the following tasks:

Installed using binary files or compiling from source

Task
Initialize the data dictionary
Test the server
Set service to start at boot time

Initialize the data directory

If you install the server using either the source distribution or generic binary distribution files, the data directory is not initialized, and you must run the initialization process after installation.

Run mysqld with the –initialize option or the initialize- option.

Executing mysqld with either option does the following:

  • Verifies the existence of the data directory

  • Initializes the system tablespace and related structures

  • Creates system tables including grant tables, time zone tables, and server-side help tables

  • Creates root@localhost

You should run the following steps with the mysql login.

  1. Navigate to the MySQL directory. The example uses the default location.

    cd /usr/local/mysql

  2. Create a directory for the MySQL files. The secure_file_priv uses the directory path as a value.

    mkdir mydata

    The mysql user account should have the drwxr-x--- permissions. Four sections define the permissions; file or directory, User, Group, and Others.

    The first character designates if the permissions are for a file or directory. The first character is d for a directory.

    The rest of the sections are specified in three-character sets.

    Permission User Group Other
    Read Yes Yes No
    Write Yes No No
    Execute Yes Yes No

  3. Run the command to initialize the data directory.

    bin/mysqld --initialize

Test the server

After you have initialized the data directory, and the server is started, you can run tests on the server.

This section assumes you have used the default installation settings. If you have modified the installation, navigate to the installation location. You can also add the location by Setting the Environment Variables .

You can use the mysqladmin client to access the server.

If you have issues connecting to the server, use the root user and the root account password.

sudo mysqladmin -u root -p version
Expected output 
Enter password:
mysql Ver 8.4.11-11 for debian-linux-gnu on x86_64 (Percona Server (GPL), Release '10', Revision 'f446c04')
...
Server version      8.4.11-11
Protocol version    10
Connection          Localhost via UNIX socket
UNIX socket         /var/run/mysqld/mysqld.sock
Uptime:             4 hours 58 min 10 section

Threads:    2 Questions:    16 Slow queries: 0 Opens: 139 Flush tables: 3
Open tables: 59  Queries per second avg: 0.0000

Use mysqlshow to display database and table information.

sudo mysqlshow -u root -p
Expected output 
Enter password:

+---------------------+
|      Databases      |
+=====================+
| information_schema  |
+---------------------+
| mysql               |
+---------------------+
| performance_schema  |
+---------------------+
| sys                 |
+---------------------+

Set service to run at boot time

After a generic binary installation, manually configure systemd support.

The following commands start, check the status, and stop the server:

sudo systemctl start mysqld
sudo systemctl status mysqld
sudo systemctl stop mysqld

Run the following command to start the service at boot time:

sudo systemctl enable mysqld
Run the following command to prevent a service from starting at boot time:

sudo systemctl disable mysqld

All installations

Task
Update the root password
Secure the server
Populate the time zone tables

Update the root password

During an installation on Debian/Ubuntu, you are prompted to enter a root password. On Red Hat Enterprise Linux and derivatives, you update the root password after installation.

Restart the server with the --skip-grant-tables option to allow access without a password. This option is insecure. This option also disables remote connections.

sudo systemctl stop mysqld
sudo systemctl set-environment MYSQLD_OPTS="--skip-grant-tables"
sudo systemctl start mysqld 
mysql

Reload the grant tables to be able to run the ALTER USER statement. Enter a password that satisfies the current policy.

FLUSH PRIVILEGES;
ALTER USER 'root'@'localhost' IDENTIFIED BY 'rootPassword_12';
exit

If the command fails, with ERROR 1524 (HY000): Plugin [plugin name] is not loaded., then check if the plugin is available.

SELECT PLUGIN_NAME, PLUGIN_STATUS
       FROM INFORMATION_SCHEMA.PLUGINS
       WHERE PLUGIN_NAME LIKE 'validate%';

If the result is empty or shows DISABLED, the plugin is not available. Switch the MySQL user to use the default authentication plugin, caching_sha2_password or mysql_native_password for your installation.

ALTER USER 'root'@'localhost' IDENTIFIED WITH caching_sha2_password BY 'rootPassword_12';

If, when adding the password, MySQL returns ERROR 1819 (HY000) Your password does not satisfy the current policy, run the following command to see policy requirement.

SHOW VARIABLES LIKE 'validate_password%';
Redo your password to satisfy the requirements.

Stop the server, remove the --skip-grant-tables option, start the server, and log into the server with the updated password.

sudo systemctl stop mysqld 
sudo systemctl unset-environment MYSQLD_OPTS 
sudo systemctl start mysqld 
mysql -u root -p

Secure the server

The mysql_secure_installation script improves the security of the instance.

The script does the following:

  • Changes the root password

  • Disallows remote login for root accounts

  • Removes anonymous users

  • Removes the test database

  • Reloads the privilege tables

The following statement runs the script:

mysql_secure_installation

Populate the time zone tables

The time zone system tables are the following:

  • time_zone

  • time_zone_leap_second

  • time_zone_name

  • time_zone_transition

  • time_zone_transition_type

If you install the server using either the source distribution or the generic binary distribution files, the installation creates the time zone tables, but the tables are not populated.

The mysql_tzinfo_to_sql program populates the tables from the zoneinfo directory data available in Linux.

A common method to populate the tables is to add the zoneinfo directory path to mysql_tzinfo_to_sql and then send the output into the mysql system schema .

The example assumes you are running the command with the root account. The account must have the privileges for modifying the mysql system schema.

mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root -p -D mysql

Binary logs and replication improvements

Due to continuous development, Percona Server for MySQL has incorporated a number of improvements related to replication and binary log handling. These improvements result in replication-specific behaviors that distinguish Percona Server for MySQL from standard MySQL.

Statements with a LIMIT clause

In MySQL 8.4, any UPDATE/DELETE/INSERT … SELECT statements that include a LIMIT clause are indeed considered unsafe for statement-based replication. These statements will cause MySQL to automatically switch from statement-based logging to row-based logging if binlog_format is set to MIXED.

Here’s why:

  • The LIMIT clause without an ORDER BY makes the result set non-deterministic

  • The same statement might affect different rows on the primary and replicas

Run these example statements in the MySQL client:

UPDATE table1 LIMIT 10 SET col1 = 'value';
DELETE FROM table1 LIMIT 5;
INSERT INTO table2 SELECT * FROM table1 LIMIT 3;

To make these statements safe for statement-based replication, you should do one of the following:

  • Remove the LIMIT clause

  • Add an ORDER BY clause to make the result set deterministic

Run these example statements in the MySQL client:

UPDATE table1 SET col1 = 'value' ORDER BY id LIMIT 10;
DELETE FROM table1 ORDER BY id LIMIT 5;
INSERT INTO table2 SELECT * FROM table1 ORDER BY id LIMIT 3;

The exception is when the LIMIT is used with an ORDER BY clause that uses a unique key - in this case, the statement becomes deterministic and safe for statement-based replication.

Percona Server for MySQL acknowledges statements as safe when they include either an ORDER BY PK or WHERE condition.

Relay log position fix

MySQL always updated relay log position in multi-source replications setups regardless of whether the committed transaction has already been executed or not. Percona Server omits relay log position updates for the already logged GTIDs.

Source and connection status update fix

Replica nodes configured to update source status and connection information only on log file rotation did not experience the expected reduction in load. MySQL was additionally updating this information in case of multi-source replication when replica had to skip the already executed GTID event.

Write FLUSH commands to the binary log

FLUSH commands, such as FLUSH SLOW LOGS, are not written to the binary log if the system variable binlog_skip_flush_commands is set to ON.

In the context of MySQL, the read_only and super_read_only system variables control the ability to modify data in the database. The following changes were implemented in the behavior of read_only and super_read_only modes:

  • When read_only is set to ON, any FLUSH ... command executed by a normal user (without the SUPER privilege) are not written to the binary log regardless of the value of the binlog_skip_flush_command variable.

  • When super_read_only is set to ON, any FLUSH ... command executed by any user (even by those with the SUPER privilege) are not written to the binary log regardless of the value of the binlog_skip_flush_commands variable.

An attempt to run a FLUSH command without either SUPER or RELOAD privileges results in the ER_SPECIFIC_ACCESS_DENIED_ERROR exception regardless of the value of the binlog_skip_flush_commands variable.

binlog_skip_flush_commands

Option Description
Command-line Yes
Config file Yes
Scope Global
Dynamic Yes
Default OFF

When binlog_skip_flush_commands is set to ON, FLUSH ... commands are not written to the binary log.

The binlog_skip_flush_commands setting does not impact the following commands because they are never recorded in the binary log:

  • FLUSH LOGS

  • FLUSH BINARY LOGS

  • FLUSH TABLES WITH READ LOCK

  • FLUSH TABLES ... FOR EXPORT

The FLUSH command is not recorded in the binary log and ignores the binlog_skip_flush_commands setting when executed with the NO_WRITE_TO_BINLOG keyword (or its alias LOCAL).

Keep comments with DDL commands

When you run a DDL command, such as DROP TABLE, the server does the following in the binary log.

Actions Description
Removes Comments The server deletes any comments in the original command. For example, if you use DROP TABLE my_table /* This is a comment */;, the binary log does not save the comment.
Adds Quotation Marks The server puts quotation marks around the table name. So, if you run DROP TABLE my_table;, it logs it as DROP TABLE "my_table";.

These actions simplify the logging format, but sometimes, you want the original format.

binlog_ddl_skip_rewrite

Option Description
Command-line Yes
Config file Yes
Scope Global
Dynamic Yes
Default OFF

When disabled (default setting), the server removes comments and adds quotation marks to DDL statements.

When enabled, all single-table DROP TABLE DDL statements are logged in the binary log with the following characteristics:

  • Comments are preserved, allowing any notes added to the command to remain in the binary log.

  • Quotation marks are not added.

Enable binlog_ddl_skip_rewrite

You can enable binlog_ddl_skip_rewrite at runtime:

-- Check current setting
SHOW VARIABLES LIKE 'binlog_ddl_skip_rewrite';

-- Enable feature
SET GLOBAL binlog_ddl_skip_rewrite = ON;

-- Disable feature
SET GLOBAL binlog_ddl_skip_rewrite = OFF;

to enable the variable permanently, add the following line to the my.cnf configuration file:

[mysqld]
binlog_ddl_skip_rewrite = ON

After making this change, restart the MySQL service for it to take effect.

Note

Multi-table DROP TABLE DDL statements are not supported and return an error.

Example usage

The following code demonstrates how to enable binlog_ddl_skip_rewrite and shows the feature’s effect on a DROP TABLE statement. Run these commands in the MySQL client:

SET binlog_ddl_skip_rewrite = ON;
/*comment at start*/DROP TABLE t /*comment at end*/;

Point-in-Time Recovery with binlog_utils_udf

Point-in-Time Recovery (PiTR) allows you to restore a database to any specific moment in time using binary logs. The binlog_utils_udf component provides user-defined functions (UDFs) that simplify PiTR operations by helping you:

  • Map Global Transaction Identifiers (GTIDs) to specific binary log files
  • Inspect binary log contents and timestamps
  • Locate the exact binary log files needed for recovery operations

These functions are particularly useful when you need to determine which binary log files contain specific transactions or events during recovery planning.

Prerequisites

Before using the binlog_utils_udf component, ensure the following requirements are met:

  • Percona Server for MySQL: The component is only available in Percona Server for MySQL, not in standard MySQL

  • Binary logging enabled: The server must have binary logging enabled (log_bin system variable set to ON)

  • GTID enabled: For GTID-related functions, GTID must be enabled (gtid_mode set to ON)

  • MySQL privileges: You need SYSTEM_VARIABLES_ADMIN privilege to install components. For binary log operations, BINLOG_ADMIN privilege may also be required. The SUPER privilege is deprecated in MySQL 8.0+ and should be replaced with specific dynamic privileges

Install the component

Install the component on each server where you plan to use these functions:

INSTALL COMPONENT 'file://component_binlog_utils_udf';

Verify installation

Run this command in the MySQL client:

SELECT * FROM mysql.component WHERE component_urn = 'file://component_binlog_utils_udf';

Alternatively, run this command to view all installed components:

SELECT * FROM mysql.component;

Available functions

The binlog_utils_udf component provides six functions for binary log analysis and GTID mapping:

Function Returns Description Use Case
get_binlog_by_gtid(gtid) STRING (binlog name) Returns the binary log file that contains the specified GTID. Find which binary log contains a specific transaction
get_last_gtid_from_binlog(binlog) STRING (GTID) Returns the last GTID found in the specified binary log. Identify the final transaction in a binary log file
get_gtid_set_by_binlog(binlog) STRING (GTID set) Returns all GTIDs found in the specified binary log. Get complete list of transactions in a binary log
get_binlog_by_gtid_set(gtid_set) STRING (binlog name) Returns the first binary log file that contains at least one GTID from the specified set. Find binary log containing any transaction from a GTID set
get_first_record_timestamp_by_binlog(binlog) INTEGER (timestamp) Returns the timestamp of the first event in the specified binary log. Determine when a binary log file started
get_last_record_timestamp_by_binlog(binlog) INTEGER (timestamp) Returns the timestamp of the last event in the specified binary log. Determine when a binary log file ended

Important notes

  • CAST requirement: When using these user-defined functions, you must use CAST to return a result. String functions require CAST(...AS CHAR) and timestamp functions require CAST(...AS UNSIGNED).

  • Timestamp precision: Timestamp-returning functions provide values with microsecond precision in UNIX time format. Each value represents the number of microseconds since 1970-01-01 00:00:00 UTC.

  • Binary log file names: Functions that accept a binary log name require only the short file name (for example, binlog.000001). Do not include the full path. If the input contains a path separator (/), the server returns an error.

  • Binary log directory: The server reads binary logs from the directory defined by the @@log_bin_basename system variable.

  • Return values: Functions that return binary log file names return only the short name without the path.

  • Performance considerations: These functions read binary log files directly from disk. For large binary log files, the functions may take several seconds to complete.

Simplifying UDF usage without CAST()

While CAST() is required for proper function execution, you can configure your MySQL client to handle data type conversions automatically, reducing the need to use CAST() explicitly in your queries.

Configure the MySQL client

You can set the appropriate client character set and collation to simplify UDF usage:

-- Set client character set
SET character_set_client = 'utf8mb4';

-- Set client collation
SET collation_connection = 'utf8mb4_general_ci';

Alternatively, you can configure these settings in your MySQL client configuration file (e.g., ~/.my.cnf or /etc/mysql/my.cnf):

[client]
default-character-set=utf8mb4
default-collation=utf8mb4_general_ci

By configuring these settings, the MySQL client can handle data type conversions more effectively, allowing you to use the UDF functions without explicit CAST() statements in many cases.

Note

While client configuration can simplify usage, CAST() will still work and may be necessary in some scenarios. The choice between using CAST() explicitly or relying on client configuration depends on your specific use case and preferences.

Usage examples

The following examples demonstrate how to use each function. Replace the sample arguments with values from your environment. All examples include CAST statements for proper function execution, though these may be optional if you’ve configured your MySQL client as described in the Simplifying UDF usage without CAST() section.

Find binary log by GTID

Use get_binlog_by_gtid() to locate which binary log file contains a specific transaction:

SELECT CAST(get_binlog_by_gtid('550e8400-e29b-41d4-a716-446655440000:123') AS CHAR) AS binlog;

Use case: When you know a specific GTID and need to find which binary log file contains that transaction for recovery purposes.

Get last GTID from binary log

Use get_last_gtid_from_binlog() to find the final transaction in a specific binary log file:

SELECT CAST(get_last_gtid_from_binlog('binlog.000001') AS CHAR) AS last_gtid;

Use case: Determine the last transaction processed in a binary log file before rotating to the next file.

Get all GTIDs from binary log

Use get_gtid_set_by_binlog() to retrieve all GTIDs contained in a specific binary log file:

SELECT CAST(get_gtid_set_by_binlog('binlog.000001') AS CHAR) AS gtid_set;

Use case: Get a complete list of all transactions in a binary log file for analysis or replication setup.

Find binary log by GTID set

Use get_binlog_by_gtid_set() to find the first binary log file that contains any GTID from a specified set:

SELECT CAST(get_binlog_by_gtid_set('550e8400-e29b-41d4-a716-446655440000:7,550e8400-e29b-41d4-a716-446655440000:8') AS CHAR) AS binlog;

Use case: When you have a set of GTIDs and need to find which binary log file contains at least one of those transactions.

Get binary log timestamps

Use timestamp functions to determine when events occurred in binary log files. These functions return microsecond-precision timestamps in UNIX time format.

Get first event timestamp

Find when the first event was written to a binary log file:

SELECT CAST(get_first_record_timestamp_by_binlog('binlog.000001') AS UNSIGNED) AS raw_ts;

SELECT FROM_UNIXTIME(
    CAST(get_first_record_timestamp_by_binlog('binlog.000001') AS UNSIGNED) DIV 1000000
) AS first_event_ts;

Use case: Determine when a binary log file started receiving events, useful for recovery planning.

Get last event timestamp

Find when the last event was written to a binary log file:

SELECT CAST(get_last_record_timestamp_by_binlog('binlog.000001') AS UNSIGNED) AS raw_ts;

SELECT FROM_UNIXTIME(
    CAST(get_last_record_timestamp_by_binlog('binlog.000001') AS UNSIGNED) DIV 1000000
) AS last_event_ts;

Use case: Determine when a binary log file stopped receiving events, useful for understanding binary log rotation timing.

Troubleshooting

Common issues

Function returns NULL: This usually indicates that the specified GTID or binary log file does not exist. Verify that:

  • The GTID format is correct (UUID:transaction_id)

  • The binary log file exists in the binary log directory

  • GTID is enabled on the server

Error: “Unknown function”: The component is not installed. Install the component using the INSTALL COMPONENT command.

Error: “Access denied”: You need SYSTEM_VARIABLES_ADMIN privilege to install the component and BINLOG_ADMIN privilege for binary log operations. The SUPER privilege is deprecated in MySQL 8.0+.

Performance issues: These functions read binary log files directly from disk. For large binary log files, expect execution times of several seconds.

Verify binary log files

Check which binary log files are available:

SHOW BINARY LOGS;

Check GTID status

Verify GTID is enabled:

SHOW VARIABLES LIKE 'gtid_mode';

Uninstall the component

Remove the component and all associated functions:

UNINSTALL COMPONENT 'file://component_binlog_utils_udf';

Verify removal. Run this command in the MySQL client:

SELECT * FROM mysql.component WHERE component_urn = 'file://component_binlog_utils_udf';

The query should return no rows if the component is successfully uninstalled.

Limitations

For the following variables, do not define values with one or more dot (.) characters:

A value defined with the dot (.) character is handled differently in MySQL and Percona XtraBackup and can cause unpredictable behavior.

AppArmor

Secure Percona Server for MySQL with AppArmor

The operating system has a Discretionary Access Controls (DAC) system. AppArmor supplements the DAC with a Mandatory Access Control (MAC) system. AppArmor is the default security module for Ubuntu or Debian systems and uses profiles to define how programs access resources.

AppArmor is path-based and restricts processes by using profiles. Each profile contains a set of policy rules. Some applications may install their profile along with the application. If an installation does not also install a profile, that application is not part of the AppArmor subsystem. You can also create profiles since they are simple text files stored in the /etc/apparmor.d directory.

AppArmor enhances system security by enforcing strict access controls and protecting against unauthorized access and potential threats. It achieves this by defining profiles that specify how programs interact with system resources. These profiles act as a set of rules dictating a program’s actions and the resources it can access. By confining each program to its designated profile, AppArmor limits the damage in case of a compromise and prevents unauthorized escalation of privileges. Additionally, AppArmor provides fine-grained control over program behavior, allowing administrators to tailor security policies to specific application requirements and minimize the attack surface. Overall, AppArmor is crucial in bolstering system security for MySQL developers, maintaining system integrity, and mitigating the risks associated with security breaches.

AppArmor Profiles
Manage AppArmor Profiles
Disable AppArmor
Configure AppArmor
Troubleshoot AppArmor

AppArmor profile modes

AppArmor profile modes determine how applications interact with system resources. You can mix enforce mode profiles and complain mode profiles in your server.

Mode Description
Enforce Restricts MySQL processes according to the rules defined in the profile. Any action violating these rules is denied.
Complain Allows MySQL processes to take restricted actions, but logs these actions for review.
Disabled Turns off profile restrictions entirely, allowing MySQL processes to take any action without logging.

Understanding these modes helps MySQL developers ensure that their applications can access necessary resources while maintaining system security.

Benefits

Benefit Description
Enhanced Security AppArmor profile modes, such as Enforce and Complain, help enforce security policies to prevent unauthorized access.
Easy Troubleshooting Profile modes provide flexibility in troubleshooting access issues by allowing developers to switch between modes.

Disadvantages

Disadvantage Description
Limited Flexibility Profile modes may restrict certain actions or access, potentially limiting the functionality of MySQL applications.
Complexity Understanding and managing different profile modes can be complex for beginner developers, leading to errors.
Debugging Challenges Troubleshooting issues related to profile modes, such as DENIED entries in logs, may require additional expertise.

AppArmor
Manage AppArmor Profiles
Disable AppArmor
Configure AppArmor
Troubleshoot AppArmor

Managing AppArmor profiles

Understanding AppArmor Risks in MySQL Development

While AppArmor profiles help secure your MySQL server, misconfiguring them can lead to unexpected behavior and potential security vulnerabilities. Here’s why careful review and testing are crucial when making changes:

Potential risks of misconfigured AppArmor profiles

Misconfiguration Description
Overly restrictive profiles These profiles might prevent MySQL from accessing necessary files or resources, hindering its functionality and causing errors. Imagine a profile accidentally blocking MySQL from writing to its log files, rendering them useless for troubleshooting.
Underly permissive profiles Profiles with insufficient restrictions could allow unauthorized access to MySQL’s files or functionalities. This creates a security risk, as an attacker exploiting a vulnerability might leverage a permissive profile to gain more control over the server.
Incorrect profile assignment Assigning the wrong profile to a process can lead to either of the issues mentioned above. For instance, accidentally assigning a profile meant for a different service to MySQL could have unintended consequences.

Importance of careful review and testing

By carefully reviewing and testing your AppArmor profile changes, you can minimize the risks associated with misconfigurations and ensure a secure and functional MySQL environment.

  • Review your changes thoroughly: Double-check your AppArmor profile modifications to ensure they grant MySQL the necessary permissions while maintaining security.

  • Test your changes in a safe environment: Before deploying changes to a production server, test them in a staging environment that mimics your production setup. This test allows you to identify and fix any issues caused by the AppArmor profile adjustments without impacting your live MySQL instance.

Install the utilities used to control AppArmor

Install the apparmor-utils package to work with profiles. Use these utilities to create, update, enforce, switch to complain mode, and disable profiles, as needed:

$ sudo apt install apparmor-utils
Expected output 
Reading package lists... Done
Building dependency tree
...
The following additional packages will be installed:
    python3-apparmor python3-libapparmor
...

Add the mysqld profile

Add the mysqld profile with the following procedure:

  1. Download the current version of the AppArmor:

    $ wget https://raw.githubusercontent.com/percona/percona-server/release-8.4.11-11/build-ps/debian/percona-server-server.install
    Expected output 
    ...
Saving to 'apparamor-profile`
...

  2. Move the file to /etc/apparmor.d/usr.sbin.mysqld

    $ sudo mv apparmor-profile /etc/apparmor.d/usr.sbin.mysqld

  3. Create an empty file for editing:

    $ sudo touch /etc/apparmor.d/local/usr.sbin.mysqld

  4. Load the profile:

    $ sudo apparmor_parser -r -T -W /etc/apparmor.d/usr.sbin.mysqld

  5. Restart Percona Server for MySQL:

    $ sudo systemctl restart mysql

  6. Verify the profile status:

    $ sudo aa-status
    Expected output 
    ...
processes are in enforce mode
...
/usr/sbin/mysqld (100840)
...

Check the current status

As root or using sudo, you can check the AppArmor status:

$ sudo aa-status
Expected output 
apparmor module is loaded.
34 profiles are loaded.
32 profiles in enforce mode.
...
    /usr/sbin/mysqld
...
2 profiles in complain mode.
...
3 profiles have profiles defined.
...
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.

Switch a profile to complain mode

Switch a profile to complain mode when the program is in your path with this command:

$ sudo aa-complain <program>

If needed, specify the program’s path in the command:

$ sudo aa-complain /sbin/<program>

If the profile is not stored in /etc/apparmor.d/, use the following command:

$ sudo aa-complain /path/to/profiles/<program>

Switch a profile to enforce mode

Switch a profile to the enforce mode when the program is in your path with this command:

$ sudo aa-enforce <program>

If needed, specify the program’s path in the command:

$ sudo aa-enforce /sbin/<program>

If the profile is not stored in /etc/apparmor.d/, use the following command:

$ sudo aa-enforce /path/to/profile

Disable one profile

You can disable a profile but it is recommended to Switch a Profile to Complain mode.

Use either of the following methods to disable a profile:

$ sudo ln -s /etc/apparmor.d/usr.sbin.mysqld /etc/apparmor.d/disable/
$ sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.mysqld

or

$ aa-disable /etc/apparmor.d/usr.sbin.mysqld

Reload all profiles

Run either of the following commands to reload all profiles:

$ sudo service apparmor reload

or

$ sudo systemctl reload apparmor.service

Reload one profile

To reload one profile, run the following: You may need to restart the program for some changes to take effect.

AppArmor
AppArmor Profiles
Disable AppArmor
Configure AppArmor
Troubleshoot AppArmor

Disable AppArmor

Disable AppArmor Risks

Using AppArmor might seem like an extra step, but if you disable it, your server could face security risks.

Do not disable AppArmor in production environments. This action can have the following risks:

Risk Description
Increased Attack Surface Disabling AppArmor removes security restrictions, potentially allowing unauthorized access to Percona Server for MySQL’s files and functionalities. This creates an attractive target for attackers seeking to exploit vulnerabilities or gain control of your database.
Unforeseen Security Holes AppArmor can help mitigate even unknown vulnerabilities by restricting unexpected behaviors. Disabling it leaves your system more susceptible to these hidden security holes.
Accidental Misconfigurations Even with good intentions, manual configuration of access controls can be error-prone. AppArmor provides a pre-defined security layer, reducing the risk of human error in managing permissions.

Instead, use AppArmor’s security features and configure it to fit your needs.

Disable procedure

If AppArmor must be disabled, run the following commands:

  1. Check the status.

    sudo apparmor_status

  2. Stop and disable AppArmor.

    sudo systemctl stop apparmor
sudo systemctl disable apparmor

AppArmor
AppArmor Profiles
Manage AppArmor Profiles
Configure AppArmor
Troubleshoot AppArmor

Configure AppArmor

Edit profile

Only edit /etc/apparmor.d/local/usr.sbin.mysql.

You should [switch the profile] to Complain mode before editing the file. Edit the file in any text editor. When finished, reload the profile and switch it to Enforce mode.

Configure data directory location

You can change the data directory to a non-default location, like /var/lib/mysqlcustom. You should enable audit mode to capture all actions and edit the profile to allow access to the custom location.

cat /etc/mysql/mysql.conf.d/mysqld.cnf
Expected output 
The Percona Server 8.4 configuration file.

For explanations see
https://dev.mysql.com/doc/mysql/en/server-system-variables.html

[mysqld]
pid-file    = /var/run/mysqld/mysqld.pid
socket        = /var/run/mysqld/mysqld.sock
*datadir    = /var/lib/mysqlcustom*
log-error    = /var/log/mysql/error.log

Enable audit mode for mysqld. In this mode, the security policy is enforced and all access is logged.

aa-audit mysqld

Restart Percona Server for MySQL.

sudo systemctl mysql restart

The restart fails because AppArmor has blocked access to the custom data directory location. To diagnose the issue, check the logs for the following:

  • ALLOWED - A log event when the profile is in complain mode and the action violates a policy.

  • DENIED - A log event when the profile is in enforce mode and the action is blocked.

For example, the following log entries show DENIED:

Expected output 
...
Dec 07 12:17:08 ubuntu-s-4vcpu-8gb-nyc1-01-aa-ps audit[16013]: AVC apparmor="DENIED" operation="mknod" profile="/usr/sbin/mysqld" name="/var/lib/mysqlcustom/binlog.index" pid=16013 comm="mysqld" requested_mask="c" denied_mask="c" fsuid=111 ouid=111
Dec 07 12:17:08 ubuntu-s-4vcpu-8gb-nyc1-01-aa-ps kernel: audit: type=1400 audit(1607343428.022:36): apparmor="DENIED" operation="mknod" profile="/usr/sbin/mysqld" name="/var/lib/mysqlcustom/mysqld_tmp_file_case_insensitive_test.lower-test" pid=16013 comm="mysqld" requested_mask="c" denied_mask="c" fsuid=111 ouid=111
...

Open /etc/apparmor.d/local/usr.sbin.mysqld in a text editor and edit the following entries in the Allow data dir access section.

Allow data dir access
/var/lib/mysqlcustom/ r,
/var/lib/mysqlcustom/** rwk,

In etc/apparmor.d/local/usr.sbin.mysqld, comment out, using the # symbol, the current entries in the Allow data dir access section. This step is optional. If you skip this step, mysqld continues to access the default data directory location.

Note

Edit the local version of the file instead of the main profile. Separating the changes makes maintenance easier.

Reload the profile:

apparmor_parser -r -T /etc/apparmor.d/usr.sbin.mysqld

Restart mysql:

systemctl restart mysqld

Set up a custom log location

To move your logs to a custom location, you must edit the my.cnf configuration file and then edit the local profile to allow access:

cat /etc/mysql/mysql.conf.d/mysqld.cnf
Expected output 
The Percona Server 8.4 configuration file.

For explanations see
https://dev.mysql.com/doc/mysql/en/server-system-variables.html

[mysqld]
pid-file    = /var/run/mysqld/mysqld.pid
socket        = /var/run/mysqld/mysqld.sock
datadir    = /var/lib/mysql
log-error    = /*custom-log-dir*/mysql/error.log

Verify the custom directory exists.

ls -la /custom-log-dir/
Expected output 
total 12
drwxrwxrwx  3 root root 4096 Dec  7 13:09 .
drwxr-xr-x 24 root root 4096 Dec  7 13:07 ..
drwxrwxrwx  2 root root 4096 Dec  7 13:09 mysql

Restart Percona Server.

service mysql start
Expected output 
Job for mysql.service failed because the control process exited with error code.
See "systemctl status mysql.service" and "journalctl -xe" for details.

journalctl -xe
Expected output 
...
AVC apparmor="DENIED" operation="mknod" profile="/usr/sbin/mysqld" name="/custom-log-dir/mysql/error.log"
...

The access has been denied by AppArmor. Edit the local profile in the Allow log file access section to allow access to the custom log location.

cat /etc/apparmor.d/local/usr.sbin.mysqld
Expected output 
 Site-specific additions and overrides for usr.sbin.mysqld..
 For more details, please see /etc/apparmor.d/local/README.

 Allow log file access
 /custom-log-dir/mysql/ r,
 /custom-log-dir/mysql/** rw,

Reload the profile:

apparmor_parser -r -T /etc/apparmor.d/usr.sbin.mysqld

Restart Percona Server:

systemctl restart mysqld

Set secure_file_priv directory location

By default, secure_file_priv points to the following location:

mysqlshow variables like 'secure_file_priv';
Expected output 
+------------------+-----------------------+
| Variable_name    | Value                 |
+------------------+-----------------------+
| secure_file_priv | /var/lib/mysql-files/ |
+------------------+-----------------------+

To allow access to another location, in a text editor, open the local profile. Review the settings in the Allow data dir access section:

Allow data dir access
/var/lib/mysql/ r,
/var/lib/mysql/** rwk,

Edit the local profile in a text editor to allow access to the custom location.

cat /etc/apparmor.d/local/usr.sbin.mysqld
Expected output 
Site-specific additions and overrides for usr.sbin.mysqld..
For more details, please see /etc/apparmor.d/local/README.

Allow data dir access
/var/lib/mysqlcustom/ r,
/var/lib/mysqlcustom/** rwk,

Reload the profile:

apparmor_parser -r -T /etc/apparmor.d/usr.sbin.mysqld

Restart Percona Server for MySQL:

systemctl restart mysqld

AppArmor
AppArmor Profiles
Manage AppArmor Profiles
Disable AppArmor
Troubleshoot AppArmor

Troubleshoot AppArmor profiles

Troubleshooting AppArmor profiles ensure that applications can access necessary resources without compromising system security.

Review Get help from Percona for ways that we can work with you.

Profile Modes

AppArmor profiles operate in different modes:

Mode Description
Enforce Applications are restricted by profile rules, and any violation results in denial of access.
Complain Applications are allowed to take restricted actions, but these actions are logged.
Disabled Profile restrictions are turned off, allowing applications to take any action without logging.

Check status

Use commands like aa-status to check the current status of AppArmor profiles. This check helps identify if profiles are enforcing or complaining about actions.

Switch modes

You may need to switch profiles between enforce and complain modes when troubleshooting. Use aa-enforce to switch to enforce mode and aa-complain to switch to complain mode.

Disable profiles

If necessary, profiles can be temporarily disabled. However, this is not recommended for security reasons. Use commands like ln -s or aa-disable to disable profiles.

Reload profiles

After making changes to profiles or switching modes, reloading profiles for changes to take effect is essential. Use commands like service apparmor reload or apparmor_parser -r to reload profiles.

Check Log Entries

Monitor log entries for DENIED or ALLOWED actions. DENIED entries indicate that a profile is blocking an action, while ALLOWED entries suggest that an action is permitted.

Edit Profiles

You may need to edit AppArmor profiles to troubleshoot access issues and allow specific actions. Edit the profile files in the /etc/apparmor.d/ directory to adjust access permissions.

AppArmor
AppArmor Profiles
Manage AppArmor Profiles
Disable AppArmor
Configure AppArmor

SELinux

Secure Percona Server for MySQL with SELinux

Understanding SELinux labels and their components (user, role, type, sensitivity level) Importance of SELinux context for administrators and users

MySQL SELinux Policy Explanation of SELinux policy for MySQL Compatibility of Percona Server for MySQL with CentOS 7 and CentOS 8 SELinux policies

SELinux is a mandatory access control system implemented in the Linux kernel. It’s designed to enhance system security by enforcing strict rules on how processes interact with files, directories, and other system resources. Unlike discretionary access control (DAC), where users have some control over permissions, SELinux imposes policies that must be followed regardless of user settings.

In SELinux, access policies are defined based on the context of processes and files. Each process and file is assigned a security context, which includes information about its identity and permissions. These contexts determine a process’s actions on a file or resource.

For processes, SELinux defines policies based on their security context, such as their domain and role. These policies specify which operations a process can perform and what resources it can access. For example, a web server process may be allowed to read web content files but not modify system configuration files. Similarly, files and directories are assigned security contexts that dictate how processes can access them. SELinux policies define rules governing interactions between processes and files based on their contexts. For instance, a database file may only be accessible for reading and writing by the database server process, while other processes are restricted from accessing it.

Overall, SELinux acts as a guardrail for system resources, ensuring that only authorized processes can access sensitive files and directories, thereby bolstering system security. Understanding SELinux and its access policies is crucial for maintaining a secure and robust MySQL environment.

Understanding SELinux labels and their components

Component Description
User Represents the identity of the user or process attempting an action. It helps SELinux determine which user is initiating the action.
Role Defines the role or function of a process within the system. It assists SELinux in determining the purpose or responsibility of the process.
Type Represents the type or category of an object such as files, directories, or processes. It aids SELinux in identifying the nature of the resource being accessed.
Sensitivity Level Indicates the sensitivity level or security classification of an object. It assists SELinux in enforcing security policies based on the object’s sensitivity.

Importance of SELinux context for administrators and users

Understanding SELinux context is crucial for administrators and users because it determines how processes interact with system resources. By assigning specific labels to users, roles, types, and sensitivity levels, SELinux ensures that only authorized actions are permitted. This granular control enhances system security by restricting unauthorized access and preventing malicious activities. Administrators rely on SELinux context to configure policies that align with organizational security requirements, while users benefit from a secure environment where their actions are safeguarded against potential threats. Overall, SELinux context plays a pivotal role in maintaining the integrity and confidentiality of system operations.

Explanation of SELinux Policy for MySQL

SELinux is a security feature in Linux that controls access to various resources such as files, directories, and network ports based on defined policies. For MySQL, SELinux has a specific policy that governs how the MySQL server process interacts with the system and other resources.

This policy defines rules for MySQL’s behavior, including which files it can access, which network ports it can use, and what actions it can perform. These rules help enforce security by restricting MySQL’s actions to only those that are necessary for its operation, preventing unauthorized access and potential security breaches.

The SELinux policy for MySQL ensures that the MySQL server process operates within predefined boundaries, limiting its capabilities to minimize the risk of exploitation or unauthorized access to sensitive data.

Compatibility of Percona Server for MySQL with SELinux Policies

Percona Server for MySQL is a drop-in replacement for MySQL that offers enhanced performance, scalability, and other features. When running Percona Server for MySQL on Red Hat Enterprise Linux (RHEL) 8, RHEL 9, or their derivatives, compatibility with SELinux policies is essential for ensuring secure and reliable operation.

Percona Server for MySQL is designed to be compatible with SELinux policies on these Linux distributions. This means that Percona Server for MySQL can seamlessly integrate with SELinux, allowing administrators to enforce security policies and restrictions without sacrificing the functionality or performance of the database server.

By adhering to SELinux policies, Percona Server for MySQL ensures that it operates within the confines defined by SELinux, preventing any unauthorized or potentially malicious actions that could compromise the system’s security. This compatibility with SELinux policies enhances the overall security posture of Percona Server for MySQL deployments on RHEL and its derivatives, providing peace of mind to administrators and users alike.

SELinux context example

To view the SELinux context, add the -Z switch to many of the utilities. Here is an example of the context for mysqld:

ps -eZ | grep mysqld_t
Expected output 
system_u:system_r:mysqld_t:s0    3356 ?        00:00:01 mysqld

The context has the following properties:

  • User - system_u

  • Role - system_r

  • Type or domain - mysqld_t

  • Sensitivity level - s0 3356

Most SELinux policy rules are based on the type or domain.

List SELinux types or domains associated with files

The security property that SELinux relies on is the Type security property. The type name often end with a _t. A group of objects with the same type security value belongs to the same domain.

To view the mysqldb_t types associated with the MySQL directories and files, run the following command:

ls -laZ /var/lib/ | grep mysql
Expected output 
drwxr-x--x. mysql   mysql   system_u:object_r:mysqld_db_t:s0 mysql
drwxr-x---. mysql   mysql   system_u:object_r:mysqld_db_t:s0 mysql-files
drwxr-x---. mysql   mysql   system_u:object_r:mysqld_db_t:s0 mysql-keyring

Note

If a policy type does not define the type property for an object, the default value is unconfined_t.

SELinux modes

SELinux has the following modes:

  • Disabled - No SELinux policy modules loaded, which disables policies. Nothing is reported.

  • Permissive - SELinux is active, but policy modules are not enforced. A policy violation is reported but does not stop the action.

  • Enforcing - SELinux is active, and violations are reported and denied. If there is no rule to allow access to a confined resource, SELinux denies the access.

Policy types

SELinux has several policy types:

  • Targeted - Most processes operate without restriction. Specific services are contained in security domains and defined by policies.

  • Strict - All processes are contained in security domains and defined by policies.

SELinux has confined processes that run in a domain and restricts everything unless explicitly allowed. An unconfined process in an unconfined domain is allowed almost all access.

MySQL is a confined process, and the policy module defines which files are read, which ports are opened, and so on. SELinux assumes the Percona Server for MySQL installation uses the default file locations and default ports.

If you change the default, you must also edit the policy. If you do not update the policy, SELinux, in enforcing mode, denies access to all non-default resources.

Check the SELinux mode

To check the current SELinux mode, use either of the following commands:

sestatus
Expected output 
SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             targeted
Current mode:                   enforcing
Mode from config file:          enforcing
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Memory protection checking:     actual (secure)
Max kernel policy version:      31

or

grep ^SELINUX= /etc/selinux/config
Expected output 
SELINUX=enforcing

Note

Add the -b parameter to sestatus to display the Policy booleans. The boolean values for each parameter is shown. An example of using the b parameter is the following:

sestatus -b | grep mysql
Expected output 
mysql_connect_any                           off
selinuxuser_mysql_connect_enabled

The /etc/selinux/config file controls if SELinux is disabled or enabled, and if enabled, whether SELinux operates in enforcing mode or permissive mode.

Disable SELinux

If you plan to use the enforcing mode at another time, use the permissive mode instead of disabling SELinux. During the time that SELinux is disabled, the system may contain mislabeled objects or objects with no label. If you re-enable SELinux and plan to set SELinux to enforcing, you must follow the steps to Relabel the entire file system.

On boot, to disable SELinux, set the selinux=0 kernel option. The kernel does not load the SELinux infrastructure. This option has the same effect as changing the SELINUX=disabled instruction in the configuration file and then rebooting the system.

Additional SELinux tools

Install the SELinux management tools, such as semanage or sesearch, if needed.

On RHEL 8 or compatible operating systems, use the following command as root:

yum -y install policycoreutils-python-utils

Note

You may need root privileges to run SELinux management commands.

Switch the mode in the configuration file

Switching between modes may help when troubleshooting or when modifying rules.

To permanently change the mode, edit the /etc/selinux/config file and change the SELINUX= value. You should also verify the change.

cat /etc/selinux/config | grep SELINUX= | grep -v ^#
Expected output 
SELINUX=enforcing
SELINUX=enforcing

sudo sed -i 's/^SELINUX=.*/SELINUX=permissive/g' /etc/selinux/config

cat /etc/selinux/config | grep SELINUX= | grep -v ^#
Expected output 
SELINUX=permissive
SELINUX=permissive

Reboot your system after the change.

If switching from either disabled mode or permissive mode to enforcing, see Relabel the entire file system.

Switch the mode until the next reboot

To change the mode until the next reboot, use either of the following commands as root:

setenforce Enforcing

or

setenforce 1

The following setenforce parameters are available:

setenforce parameters Also Permitted
0 Permissive
1 Enforcing

You can view the current mode by running either of the following commands:

getenforce
Expected output 
Enforcing

or

sestatus | grep -i mode
Expected output 
Current mode:                   permissive
Mode from config file:          enforcing

Switch the mode for a service

You can move one or more services into a permissive domain. The other services remain in enforcing mode.

To add a service to the permissive domain, run the following as root:

sudo semanage permissive -a mysqld_t

To list the current permissive domains, run the following command:

sudo semanage permissive -l
Expected output 
...
Customized Permissive Types

mysqld_t

Builtin Permissive Types
...

To delete a service from the permissive domain, run the following:

sudo semanage permissive -d mysqld_t

The service returns to the system’s SELinux mode. Be sure to follow the steps to Relabel the entire file system.

Relabel the entire file system

Switching from disabled or permissive to enforcing requires additional steps. The enforcing mode requires the correct contexts, or labels, to function. The permissive mode allows users and processes to label files and system objects incorrectly. The disabled mode does not load the SELinux infrastructure and does not label resources or processes.

RHEL and compatible systems, use the fixfiles application for relabeling. You can relabel the entire file system or the file contexts of an application.

For one application, run the following command:

fixfiles -R mysqld restore

To relabel the file system without rebooting the system, use the following command:

fixfiles -f -F relabel

Another option relabels the file system during a reboot. You can either add a touch file, read during the reboot operation, or configure a kernel boot parameter. The completion of the relabeling operation automatically removes the touch file.

Add the touch file as root:

touch /.autorelabel

To configure the kernel, add the autorelabel=1 kernel parameter to the boot parameter list. The parameter forces a system relabel. Reboot in permissive mode to allow the process to complete before changing to enforcing.

Note

Relabeling an entire filesystem takes time. When the relabeling is complete, the system reboots again.

Set a custom data directory

If you do not use the default settings, SELinux, in enforcing mode, prevents access to the system.

For example, during installation, you have used the following configuration:

datadir=/var/lib/mysqlcustom
socket=/var/lib/mysqlcustom/mysql.sock

Restart the service.

service mysqld restart
Expected output 
Redirecting to /bin/systemctl restart mysqld.service
Job for mysqld.service failed because the control process exited with error code.
See "systemctl status mysqld.service" and "journalctl -xe" for details.

Check the journal log to see the error code.

journalctl -xe
Expected output 
...
SELinux is preventing mysqld from getattr access to the file /var/lib/mysqlcustom/ibdata1.
...

Check the SELinux types in /var/lib/mysqlcustom.

ls -1aZ /var/lib/mysqlcustom
Expected output 
  total 164288
  drwxr-x--x.  6 mysql mysql system_u:object_r:var_lib_t:s0       4096 Dec  2 07:58  .
  drwxr-xr-x. 38 root  root  system_u:object_r:var_lib_t:s0       4096 Dec  1 14:29  ..
  ...
  -rw-r-----.  1 mysql mysql system_u:object_r:var_lib_t:s0   12582912 Dec  1 14:29  ibdata1
  ...

To solve the issue, use the following methods:

  • Set the proper labels for mysqlcustom files

  • Change the mysqld SELinux policy to allow mysqld access to var_lib_t files.

The recommended solution is to set the proper labels. The following procedure assumes you have already created and set ownership to the custom data directory location:

  1. To change the SELinux context, use semanage fcontext. In this step, you define how SELinux deals with the custom paths:

    semanage fcontext -a -e /var/lib/mysql /var/lib/mysqlcustom

    SELinux applies the same labeling schema, defined in the mysqld policy, for the /var/lib/mysql directory to the custom directory. Files created within the custom directory are labeled as if they were in /var/lib/mysql.

  2. To restorecon command applies the change.

    restorecon -R -v /var/lib/mysqlcustom

  3. Restart the mysqld service:

    service mysqld start

Set a custom log location

If you do not use the default settings, SELinux, in enforcing mode, prevents access to the location. Change the log location to a custom location in my.cnf:

log-error=/logs/mysqld.log

Verify the log location with the following command:

ls -laZ /
Expected output 
  ...
  drwxrwxrwx.   2 root root unconfined_u:object_r:default_t:s0    6 Dec  2 09:16 logs
  ...

Starting MySQL returns the following message:

service mysql start
Expected output 
Redirecting to /bin/systemctl start mysql.service
Job for mysqld.service failed because the control process exited with error code.
See "systemctl status mysqld.service" and "journalctl -xe" for details.

To view the error details, run:

journalctl -xe

The output may include a line such as:

```{.text .no-copy}
SELinux is preventing mysqld from write access to the directory logs.
```

The default SELinux policy allows mysqld to write logs into a location tagged with var_log_t, which is the /var/log location. You can solve the issue with either of the following methods:

  • Tag the /logs location properly

  • Edit the SELinux policy to allow mysqld access to all directories.

To tag the custom /logs location is the recommended method since it locks down access. Run the following commands to tag the custom location:

semanage fcontext -a -t var_log_t /logs
restorecon -v /logs

You may not be able to change the /logs directory label. For example, other applications, with their own rules, use the same directory.

To adjust the SELinux policy when a directory is shared, follow these steps:

  1. Create a local policy:

    ausearch -c 'mysqld' --raw | audit2allow -M my-mysqld

  2. This command generates the my-mysqld.te and the my-mysqld.pp files. The mysqld.te is the type enforcement policy file. The my-mysqld.pp is the policy module loaded as a binary file into the SELinux subsystem.

    An example of the my-myslqd.te file:

    module my-mysqld 1.0;

require {
    *type mysqld_t*;
    type var_lib_t;
    *type default_t*;
    class file getattr;
    *class dir write*;
}

============= mysqld_t ==============
*allow mysqld_t default_t:dir write*;
allow mysqld_t var_lib_t:file getattr;

    The policy contains rules for the custom data directory and the custom logs directory. We have set the proper labels for the data directory location, and applying this auto-generated policy would loosen our hardening by allowing mysqld to access var_lib_t tags.

  3. SELinux-generated events are converted to rules. A generated policy may contain rules for recent violations and include unrelated rules. Unrelated rules are generated from actions, such as changing the data directory location, that are not related to the logs directory. Add the --start parameter to use log events after a specific time to filter out the unwanted events. This parameter captures events when the time stamp is equal to the specified time or later. SELinux generates a policy for the current actions.

    ausearch --start 10:00:00 -c 'mysqld' --raw | audit2allow -M my-mysqld

  4. This policy allows mysqld writing into the tagged directories. Open the my_mysqld file:

    module my-mysqld 1.0;

require {
    type mysqld_t;
    type default_t;
    class dir write;
}

============= mysqld_t ==============
allow mysqld_t default_t:dir write;

  5. Install the SELinux policy module:

    semodule -i my-mysqld.pp

Restart the service. If you have a failure, check the journal log and follow the same procedure.

If SELinux prevents mysql from creating a log file inside the directory. You can view all the violations by changing the SELinux mode to permissive and then running mysqld. All violations are logged in the journal log. After this run, you can generate a local policy module, install it, and switch SELinux back to enforcing mode.

Follow this procedure:

  1. Unload the current local my-mysqld policy module:

    semodule -r my-mysqld

  2. You can put a single domain into permissive mode. Other domains on the system to remain in enforcing mode. Use semanage permissive with the -a parameter to change mysqld_t to permissive mode:

    semanage permissive -a mysqld_t

  3. Verify the mode change:

    semdule -l | grep permissive
    Expected output 
    ...
permissive_mysqld_t
...

  4. To make searching the log easier, return the time:

    date

  5. Start the service.

    service mysqld start

  6. MySQL starts, and SELinux logs the violations in the journal log. Check the journal log:

    journalctl -xe

  7. Stop the service:

    service mysqld stop

  8. Generate a local mysqld policy, using the time returned from step 4:

    ausearch --start <date-c 'mysqld' --raw | audit2allow -M my-mysqld

  9. Review the policy (the policy you generate may be different):

    cat my-mysqld.te
    Expected output 
    module my-mysqld 1.0;

require {
type default_t;
    type mysqld_t;
    class dir { add_name write };
    class file { append create open };
}

============= mysqld_t ==============
allow mysqld_t default_t:dir { add_name write };
allow mysqld_t default_t:file { append create open };

  10. Install the policy:

    semodule -i my-mysqld.pp

  11. Use semanage permissive with the -d parameter, which deletes the permissive domain for the service:

    semanage permissive -d mysqld_t

  12. Restart the service:

    service mysqld start

Note

Use this procedure to adjust the local mysqld policy module. You should review the changes which are generated to ensure the rules are not too tolerant.

Set secure_file_priv directory

Update the SELinux tags for the /var/lib/mysql-files/ directory, used for SELECT ... INTO OUTFILE or similar operations, if required. The server needs only read/write access to the destination directory.

To set secure_file_priv to use this directory, run the following commands to set the context:

semanage fcontext -a -t mysqld_db_t "/var/lib/mysql-files/(/.*)?"
restorecon -Rv /var/lib/mysql-files

Edit the path for a different location, if needed.

SELinux contexts and labels

Viewing SELinux Contexts Example of viewing SELinux context for a process using ps command Listing SELinux Types or Domains Explanation of SELinux type security property Example of listing SELinux types associated with MySQL directories and files

SELinux context is like a label that tells the system how to handle files, processes, and other resources. For example, it determines which processes can access certain files and what actions they can perform on them. Understanding SELinux context helps you know how your applications interact with the system and ensures that they have the necessary permissions to function correctly. It’s like giving each item on your computer a tag that says what it is and what it’s allowed to do. So when your application tries to access a file, SELinux checks its context to see if it’s allowed. If the context matches what’s expected, the action is allowed; if not, it’s denied. So knowing the SELinux context is essential for managing security and troubleshooting issues on your system.

Viewing SELinux context for a process using ps command

To view the SELinux context for a process using the ps command, you can add the -Z option to display the context information. Here’s how you can do it:

ps -eZ | grep <process_name>

Replace <process_name> with the process name you want to check. For example, if you want to see the SELinux context for the MySQL process, you would use:

ps -eZ | grep mysqld

The output displays the SELinux context for the specified process and typically consists of four parts: user, role, type (or domain), and sensitivity level.

Expected output 
system_u:system_r:mysqld_t:s0    3356 ?        00:00:01 mysqld
  • system_u represents the user context.
  • system_r represents the role context.
  • mysqld_t represents the type (or domain) context.
  • s0 represents the sensitivity level.

This information helps you understand how SELinux enforces security policies for the specified process.

List SELinux Types or Domains

SELinux types or domains categorize different resources on the system, such as files, directories, and processes. Each type or domain has specific permissions and restrictions associated with it, determining how resources interact with each other. To list SELinux types or domains associated with files, you can use the ls command with the -Z option. For example:

ls -laZ /var/lib/mysql
Expected output 
drwxr-x--x. mysql   mysql   system_u:object_r:mysqld_db_t:s0 mysql
drwxr-x---. mysql   mysql   system_u:object_r:mysqld_db_t:s0 mysql-files
drwxr-x---. mysql   mysql   system_u:object_r:mysqld_db_t:s0 mysql-keyring

This command lists the files and directories under /var/lib/mysql along with their SELinux context, which includes the type or domain associated with each resource. Understanding these types or domains helps manage SELinux policies and ensure proper access control for MySQL-related resources.

Manage SELinux modes

SELinux, or Security-Enhanced Linux, is a security module that provides access control policies. It enhances the system’s security by allowing administrators to define rules restricting how applications and users can access resources. SELinux operates in three different modes: Disabled, Permissive, and Enforcing.

Disabled Mode

In Disabled mode, SELinux is completely turned off. The system does not enforce any SELinux policies, and there is no SELinux security checking. Applications and processes run without any restrictions imposed by SELinux. This mode is typically used for troubleshooting or when SELinux is not needed.

To set SELinux to Disabled mode, you need to edit the SELinux configuration file. Open the file /etc/selinux/config with a text editor and set the SELINUX parameter to disabled:

$ SELINUX=disabled

Save the file and reboot the system for the change to take effect.

Permissive Mode

In Permissive mode, SELinux policies are not enforced, but violations are logged. This mode is useful for troubleshooting and for understanding what SELinux would block without actually blocking anything. Applications and processes run as if SELinux is not enforcing policies, but administrators can see which actions would have been denied if SELinux were enforcing.

To set SELinux to Permissive mode, you can edit the SELinux configuration file /etc/selinux/config and set the SELINUX parameter to permissive:

$ SELINUX=permissive

Save the file and reboot the system. Alternatively, you can change to Permissive mode temporarily without rebooting by running the following command as root:

$ setenforce 0

Enforcing Mode

In Enforcing mode, SELinux enforces all policies and denies access based on the rules defined in the policy. This mode is the default and most secure mode. SELinux actively restricts actions of applications and processes based on the policies in place. Any violation of the rules results in access being denied and logged.

To set SELinux to Enforcing mode, edit the SELinux configuration file /etc/selinux/config and set the SELINUX parameter to enforcing:

$ SELINUX=enforcing

Save the file and reboot the system. To change to Enforcing mode temporarily without rebooting, you can use the following command as root:

$ setenforce 1

How to check the SELinux mode

You can check which mode SELinux is currently running in by using a few terminal commands.

Use the sestatus command

To check the current SELinux mode, you can use the sestatus command. This command shows the status of SELinux, including the mode it is operating in. Type the following command and press Enter:

$ sestatus
Expected output 
SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             targeted
Current mode:                   enforcing
Mode from config file:          enforcing
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Max kernel policy version:      31
Result Description
Current mode This line shows the mode SELinux is currently operating in. It can be “enforcing”, “permissive”, or “disabled”.
Enforcing SELinux is actively enforcing its policies and blocking any actions that are not allowed.
Permissive SELinux is not blocking actions, but it logs any actions that would be blocked in enforcing mode.
Disabled SELinux is completely turned off, and no policies are enforced or logged.
Mode from config file This line shows the mode that SELinux is configured to use at boot time, which might be different from the current mode if changes were made without rebooting.

Use the getenforce command

Another command to check the current SELinux mode is getenforce. Type the following command and press Enter:

$ getenforce
Expected output 
Enforcing

Check the configuration file

You can also check the SELinux configuration file to see what mode SELinux is set to use when the system boots. Open the configuration file located at /etc/selinux/config using a text editor. For example, you can use cat to view the file contents:

$ cat /etc/selinux/config
Expected output 
SELINUX=enforcing

How to switch the SELinux mode

Switching the SELinux mode changes how the Security-Enhanced Linux (SELinux) system controls access and enforces policies on your system.

Switch SELinux mode temporarily

To switch SELinux mode temporarily, use the setenforce command. This change will last until the system is rebooted.

$ sudo setenforce 1

To check if the mode has changed, run sestatus.

Switch SELinux Mode permanently

To make the change permanent, you need to edit the SELinux configuration file. This file is usually located at /etc/selinux/config.

  1. Open the configuration file with a text editor. For example, using nano:

    $ sudo nano /etc/selinux/config

  2. Look for the line that starts with SELINUX=. It will be followed by the current mode: enforcing, permissive, or disabled.

  3. Change the value to the desired mode.

  4. Save the file and exit the text editor.

To apply the permanent change, reboot the system. After the system restarts, check the SELinux mode with sestatus to ensure the change took effect.

Changing SELinux mode for a service

SELinux (Security-Enhanced Linux) controls access and permissions for processes and users on a Linux system. SELinux has different modes: Enforcing, Permissive, and Disabled. When you change the SELinux mode for a specific service, you can control how strictly SELinux policies apply to that service. This can be useful when you need to test or troubleshoot services without disabling SELinux entirely.

Step 1: Identify the Service

First, identify the service for which you want to change the SELinux mode. For example, let’s say you want to change the SELinux mode for the Apache web server (httpd).

Step 2: Check current SELinux context

Check the current SELinux context of the service to understand its current mode and permissions. You can use the ps command with -Z option to view the SELinux context of a running process.

$ ps -eZ | grep httpd

This command displays the SELinux context for all httpd processes.

Step 3: Create a Custom SELinux Policy Module

To change the SELinux mode for a specific service, you create a custom SELinux policy module. This module will move the service to a permissive domain while keeping the rest of the system in enforcing mode.

Create a policy file, for example, httpd_permissive.te:

$ nano httpd_permissive.te

Add the following content to the file:

policy_module(httpd_permissive, 1.0)

gen_permissive(httpd_t)

This policy module tells SELinux to make the httpd_t domain permissive.

Step 4: Compile and Install the Policy Module

Compile the policy module using the checkmodule and semodule_package commands:

$ checkmodule -M -m -o httpd_permissive.mod httpd_permissive.te
$ semodule_package -o httpd_permissive.pp -m httpd_permissive.mod

Install the compiled policy module using the semodule command:

$ semodule -i httpd_permissive.pp

This installs the custom SELinux policy module, making the httpd service run in permissive mode.

Step 5: Verify the Changes

Restart the service to apply the changes:

$ systemctl restart httpd

Check the SELinux context again to ensure the httpd service is running in the permissive domain:

$ ps -eZ | grep httpd

You should see the httpd processes with a permissive context.

Step 6: Monitor Logs and Adjust Policies

While the service is in a permissive domain, SELinux logs any policy violations without enforcing them. Monitor the logs to identify and resolve issues. Use audit2allow to generate new policies if needed:

$ ausearch -m avc -c httpd | audit2allow -M httpd_custom
$ semodule -i httpd_custom.pp

This command sequence helps you create and install new SELinux policies based on logged violations, refining your SELinux configuration.

Additional SELinux tools and management

Installing SELinux management tools

To install SELinux management tools on Red Hat Enterprise Linux 8 or later, run the following command as root:

yum -y install policycoreutils-python-utils

Ensure you have root privileges to execute these commands.

Switching SELinux mode

SELinux can operate in three modes: Disabled, Permissive, and Enforcing.

To switch SELinux mode until the next reboot, use either of the following commands as root:

setenforce Enforcing
or 
setenforce 1

To view the current SELinux mode, use either of the following commands:

getenforce
or 
sestatus | grep -i mode

Managing SELinux policies

Using the semanage command

To add a service to the permissive domain, execute the following as root:

semanage permissive -a <service_name>

To delete a service from the permissive domain, run:

semanage permissive -d <service_name>

List the current Permissive domains

To list the current permissive domains, use the following command:

semanage permissive -l

Troubleshoot SELinux issues

Review Get help from Percona for ways that we can work with you.

Relabel the Entire File System

Relabeling the entire file system is updating SELinux contexts for all files and directories. This operation ensures that SELinux can enforce its policies correctly.

When relabeling the entire file system for SELinux, you should use fixfiles when you want to initiate the relabeling process manually. This command is useful when you need to perform the relabeling operation immediately or if you want to specify additional options, such as forcing the operation with the -f flag.

On the other hand, .autorelabel is used when you want the relabeling process to occur automatically during system boot. This method is convenient when scheduling the relabeling task without manual intervention. The .autorelabel in the root directory triggers the relabeling process during the boot sequence, ensuring that all files and directories are relabeled according to the SELinux policy.

Manually relabeling

This command relabels the entire file system without requiring a system reboot.

fixfiles -f relabel
The command fixfiles -f relabel is a directive used within the context of SELinux, a security feature in Linux systems. This specific command instructs the system to forcefully reapply SELinux labels, also known as contexts, to files and directories.

Here’s a breakdown of what each part of the command does:

Option Description
fixfiles This is the name of the command being executed. It’s a tool provided by SELinux specifically designed to fix file contexts.
-f This is an option passed to the fixfiles command. In this context, the -f option stands for “force”. It tells the fixfiles command to perform the relabeling operation forcefully, regardless of the current state or any potential errors.
relabel This is an argument passed to the fixfiles command. It specifies the action that fixfiles should take: relabel the files and directories on the system.

When you run fixfiles -f relabel, SELinux goes through all files and directories on the system and applies the appropriate SELinux labels to each one. These labels are crucial for SELinux to enforce its security policies effectively. They determine how processes and users can interact with the files and directories, ensuring that only authorized actions are allowed.

This command is typically used in scenarios where there may have been changes to the file system that require SELinux labels to be updated. For example, if files or directories have been moved or copied from one location to another or if SELinux policies have been modified, running fixfiles -f relabel ensures that the SELinux labels remain consistent with the system’s current configuration.

It’s important to note that running fixfiles -f relabel can be a resource-intensive operation and may take some time to complete, especially on systems with many files and directories. Additionally, since it forcefully relabels all files and directories, use it cautiously and preferably during maintenance windows to minimize potential disruptions to system operations.

Automatic relabeling

Creating the .autorelabel file initiates a relabeling process that often requires a reboot to apply the changes effectively. During this reboot, SELinux relabels all files based on their defined policies.

touch /.autorelabel

This command creates a file named .autorelabel in the root directory of the Linux filesystem. The “touch” command creates a new file.

The purpose of the .autorelabel file is to trigger an automatic relabeling of the entire filesystem when the system boots up. Relabeling involves assigning security labels to files and directories based on SELinux policies. This process ensures that all files and directories have the correct security context, which is essential for SELinux to enforce its security policies effectively.

Creating this file tells the system to perform a relabeling operation during the next boot. This operation can be useful in situations where SELinux policies or file contexts have been modified, and we want to ensure that all files are correctly labeled according to the updated policies.

It’s important to note that the .autorelabel file contains no data or configuration. The file acts as a trigger for the relabeling process. Once the relabeling is complete, the system automatically removes the .autorelabel file.

Set Custom Data Directory

Setting a custom data directory for the server involves configuring SELinux contexts to allow the server to access the new directory properly.

It would be best to use semanage when you defining or modifying SELinux policy rules related to a custom data directory. This command allows you to manage SELinux policy modules, including adding, deleting, and modifying SELinux policy rules for specific file contexts or directories.

It would be best to use restorecon when you restore the default SELinux context for files and directories, including those in a custom data directory. restorecon resets the SELinux context of specified files or directories to match the default context defined in the SELinux policy. It’s typically used after file or directory modifications to ensure they have the correct SELinux context.

Use semanage

The following command configures the SELinux context for a custom data directory in the server.

semanage fcontext -a -t mysqld_db_t "/path/to/custom/data(/.*)?"

Each part of the command is as follows:

Option Description
semanage Command-line tool used to manage SELinux policy settings.
fcontext Sub-command of semanage specifically used to manage file contexts, which define how SELinux labels files and directories.
-a Stands for “add” and indicates the intention to add a new file context configuration.
-t mysqld_db_t Specifies the type of context to assign to the specified path. In this case, mysqld_db_t is the SELinux type context for The server database files.
"/path/to/custom/data(/.*)?" Path to the custom data directory in the Server setup. The (/.*)? part is a regular expression pattern matching any files or subdirectories within the specified directory.

This command tells SELinux to label all files and subdirectories within the /path/to/custom/data directory with the SELinux type context mysqld_db_t. This operation ensures that SELinux treats these files and directories as part of the server’s database, allowing the server to access them according to its SELinux policy.

Use restorecon

restorecon -Rv /path/to/custom/data

The restorecon -Rv /path/to/custom/data command restores the SELinux context for a specific directory and subdirectory. Here’s what each part of the command does:

  • restorecon: This is the main command used to restore the SELinux context of files and directories.
  • -R: This option stands for “recursive” and indicates that the command should operate recursively on all files and subdirectories within the specified directory.
  • -v: This option stands for “verbose” and instructs the command to display detailed information about the actions it performs, providing feedback on which files and directories had their SELinux context restored.

The /path/to/custom/data part of the command should be replaced with the actual path to the directory for which you want to restore the SELinux context.

Typically, restorecon does not require a system reboot. It simply restores the SELinux context for the specified directory and its contents. However, if you’re experiencing issues with SELinux after running the command, a system reboot may be necessary to ensure all changes take effect.

Setting Custom Log Location

When setting a custom log location for the server, SELinux permissions may need adjustment to allow the server to write to the new directory.

This command associates the var_log_t type with the custom log directory and contents.

semanage fcontext -a -t var_log_t "/path/to/custom/logs(/.*)?"

This command restores SELinux contexts recursively for the custom log directory, ensuring proper permissions for the server to write logs.

restorecon -Rv /path/to/custom/logs

Setting secure_file_priv Directory

When configuring the server’s secure_file_priv directory, you must update the SELinux tags to allow the server to access this directory.

This command associates the mysqld_db_t type with the secure_file_priv directory and its contents.

semanage fcontext -a -t mysqld_db_t "/path/to/secure_file_priv(/.*)?"

This command restores SELinux contexts recursively for the secure_file_priv directory, ensuring proper permissions for the server file operations.

restorecon -Rv /path/to/secure_file_priv

Develop

Fundamental SQL operations

SQL basics

SQL stands for Structured Query Language. It’s a powerful tool used to communicate with databases. Think of a database as a digital filing cabinet where you store and organize information. SQL is like the language you use to talk to that filing cabinet and ask questions or tell it what you want to do with the data inside.

With SQL, you can do a variety of tasks:

  • Retrieve Data: You can ask the database to give you specific information, like all the names of customers who bought a certain product.

  • Insert Data: You can add new information into the database, such as adding a new customer’s details.

  • Update Data: If information changes, like a customer’s address, you can update it in the database.

  • Delete Data: If information is no longer needed, you can remove it from the database.

SQL provides a standardized way to interact with a database. It uses simple commands and statements to perform these tasks, making it easy to learn and use for managing data effectively.

Fundamental SQL links:

Common SQL commands

SQL commands used by MySQL can be categorized into different types based on their purposes: Data Definition Language (DDL), Data Manipulation Language (DML), Data Control Language (DCL), and Transaction Control Language (TCL).

Data Manipulation Language (DML)

DML commands manage data within database tables.

Common DML commands include:

  • [SELECT]: This command retrieves data from a table or from several tables when you use a join. In this example, it selects all columns with a wildcard (*) from the customers table.

Be careful when using wildcards. They return every column, which can slow down queries, use more memory and network resources, and put extra load on both the server and the client.

```sql
SELECT * FROM customers;
```

  • [INSERT]: This command adds new records to a table.

    INSERT INTO customers (name, city) VALUES ('John Doe', 'Berlin');

  • [UPDATE]: This command modifies existing records in a table.

    UPDATE customers SET city = 'Madrid' WHERE id = 1;

  • [DELETE]: This command removes records from a table.

    DELETE FROM customers WHERE id = 1;

Data Definition Language (DDL)

DDL commands define, modify, and remove database objects such as tables, indexes, and views.

Common DDL commands include:

  • CREATE: This command creates new database objects like tables, indexes, and views.

    CREATE TABLE employees (id INT, name VARCHAR(50));

  • ALTER: This command modifies the structure of existing database objects.

    ALTER TABLE employees ADD COLUMN city VARCHAR(50);

  • DROP: This command removes database objects from the database.

    DROP TABLE employees;

Data Control Language (DCL)

DCL commands control access to database objects and define privileges.

Common DCL commands include:

  • GRANT: This command grants specific privileges to database users.

    GRANT SELECT, INSERT ON employees TO 'user1'@'localhost';

  • REVOKE: This command revokes privileges from database users.

    REVOKE INSERT ON employees FROM 'user2'@'localhost';

Transaction Control Language (TCL)

TCL commands manage transactions within a database.

Common TCL commands include:

  • COMMIT: This command saves changes made during the current transaction to the database.

    COMMIT;

  • ROLLBACK: This command undoes changes made during the current transaction and restores the database to its previous state.

    ROLLBACK;

Fundamental SQL links:

SELECT statement

The syntax of a SELECT statement in MySQL is straightforward. You start with the keyword SELECT, followed by the columns from which you want to retrieve data. You can specify the table from which to retrieve data using the FROM keyword. Optionally, you can include conditions to filter the results using the WHERE clause.

The following table is a breakdown of the syntax:

Syntax Description
SELECT This keyword indicates that you want to retrieve data from the database.
Columns Specify the columns you want to retrieve data from. You can use the asterisk (*) to select all columns or specify individual column names separated by commas.
FROM Use the FROM keyword to specify the table from which you want to retrieve data.
WHERE (optional) If you want to filter the results based on specific conditions, you can use the WHERE clause. This clause allows you to specify conditions using comparison operators like =, >, <, etc., and logical operators like AND, OR, NOT. 
SELECT column1, column2
FROM table_name
WHERE condition;
  • SELECT column1, column2 specifies that you want to retrieve data from column1 and column2.
  • FROM table_name specifies the table from which you want to retrieve data.
  • WHERE condition is an optional clause that filters the results based on the specified condition.

Fundamental SQL links:

INSERT statement

In MySQL, the INSERT statement adds new rows of data to a table. It follows a simple syntax pattern that beginners can easily understand.

Trade-Offs Description
Advantages - Allows for efficient addition of new data into the database.
- Provides flexibility to insert data into specific columns or all columns of a table.
- Supports inserting multiple rows with a single INSERT statement.
- Can be used in conjunction with SELECT statements to insert data from one table into another.
Disadvantages - May result in performance overhead, especially when inserting large volumes of data or when indexes need to be updated.
- Requires proper error handling to deal with constraints, such as primary key or unique constraints, to prevent duplicate entries.
- Limited functionality for bulk inserts compared to specialized tools or techniques like bulk loading utilities.

Syntax of the INSERT Statement:

Option Description
INSERT INTO This keyword indicates that you are performing an insertion operation into a table.
table_name This is the name of the table where you want to insert the data.
column1, column2, … These are optional and specify the columns into which you want to insert data. If omitted, values must be provided for all columns in the table, in the same order as they are defined in the table.
VALUES This keyword introduces the list of values to be inserted into the specified columns. Alternatively, you can use the SELECT statement to retrieve data from another table and insert it into the specified columns.
value1, value2, … These are the values to be inserted into the corresponding columns. The number and order of values must match the number and order of columns specified in the INSERT INTO clause.

The number of values in the VALUES clause must always match the number of columns specified or the total number of columns in the table.

To insert data into a table, you use the INSERT INTO statement followed by the table name and a list of column names (if specified) or the VALUES keyword, followed by the values you want to insert into the table.

INSERT INTO table_name (column1, column2, ...)
VALUES (value1, value2, ...);

In this example, we are doing the following:

  • Inserting a new row into the “employees” table.

  • The values 1, ‘John Doe’, and 50000 are being inserted into the “id”, “name”, and “salary” columns, respectively.

INSERT INTO employees (id, name, salary)
        VALUES (1, 'John Doe', 50000);

Fundamental SQL links:

UPDATE statement

Purpose of the UPDATE Statement

The UPDATE statement modifies existing records in a table. It allows developers to change the values of one or more columns in a specific row or set of rows based on certain conditions.

Advantages and Disadvantages of Using the UPDATE Statement:

Trade-offs Description
Advantages - Allows for updating existing data without the need to delete and re-insert records.
- Provides flexibility in modifying specific columns or rows based on specified conditions.
- Can be used in conjunction with WHERE clause to update only selected rows, reducing unnecessary updates and improving performance.
- Supports bulk updates, allowing multiple rows to be modified in a single statement.
Disadvantages - Incorrectly formulated UPDATE statements can lead to unintended data changes or data loss.
- Lack of proper WHERE clause can result in updating all rows in a table, potentially causing data corruption or performance issues.
- May cause locking and contention issues in high-concurrency environments, impacting the performance of other queries accessing the same table.

Syntax of an UPDATE Statement:

Option Description
UPDATE table_name This clause specifies the name of the table you want to modify.
SET column_name1 = value1, column_name2 = value2, ... This clause defines which columns you want to update and their corresponding new values. You can update multiple columns by separating them with commas.
WHERE condition (optional) This clause specifies a condition that filters which rows in the table will be affected by the update. If omitted, all rows in the table will be updated. 
UPDATE table_name
SET column1 = value1, column2 = value2, ...
[WHERE condition];

In this example, the statement does the following:

  • Modifies the salary column for employees in the ‘Sales’ department.

  • Increases the salary of each employee by 10% (salary * 1.1).

UPDATE employees
SET salary = salary * 1.1
WHERE department = 'Sales';

Fundamental SQL links:

DELETE statement

The DELETE statement removes one or more rows from a table based on specified conditions. It allows developers to selectively delete data from a table, providing a way to manage and maintain the database by removing unnecessary or outdated records.

Advantages and Disadvantages of Using DELETE Statement

Trade-offs Description
Advantages Allows selective removal of specific rows from a table, helping to maintain data integrity and manage database resources efficiently.
Can be combined with WHERE clause to delete rows that meet certain conditions, providing flexibility in data manipulation.
Provides a straightforward way to remove unwanted data without affecting the structure of the table or other related tables.
Disadvantages Deleting large amounts of data can impact performance and may require careful consideration to avoid unintended consequences.
Deletes are permanent and irreversible, so it’s crucial to double-check conditions and backup data before executing DELETE queries.

Syntax of DELETE Statement

The statement has the following options:

Option Description
DELETE FROM table_name This clause specifies the table from which you want to delete rows.
WHERE condition (Optional) This clause filters the rows to be deleted based on a specific condition. If omitted, all rows in the table will be deleted.

The syntax of the DELETE statement is as follows:

DELETE FROM table_name
[WHERE condition];

Example of DELETE Statement

This example deletes all rows from the orders table where the order_date is before January 1, 2023.

DELETE FROM orders
WHERE order_date < '2023-01-01';

Fundamental SQL links:

SQL operators

Purpose of SQL Operators

SQL operators are symbols or keywords used to perform operations on data in SQL queries. They allow developers to manipulate and compare data, perform calculations, and filter results based on specified conditions.

Advantages and Disadvantages of Using SQL Operators:

Trade-Offs Description
Advantages - Enables developers to perform various operations on data, such as arithmetic calculations, comparisons, logical operations, and string concatenation.
- Provides flexibility in crafting complex queries to extract, transform, and manipulate data according to specific requirements.
- Enhances query efficiency by allowing filtering and sorting of data directly within SQL queries, reducing the need for post-processing in application code.
Disadvantages - May introduce complexity to queries, especially when multiple operators are combined or when dealing with complex logical conditions.
- Requires careful consideration of operator precedence and evaluation order to ensure the desired results are obtained.
- Can sometimes result in less readable or maintainable queries, particularly for developers unfamiliar with the SQL syntax or operators being used.

Syntax of Using SQL Operators:

Option Description
Arithmetic Arithmetic operators such as +, -, *, /, and % are used to perform mathematical calculations on numeric data.
Comparison Comparison operators like =, <>, <, >, <=, and >= are used to compare values and determine their relationship.
Logical Logical operators such as AND, OR, and NOT are used to perform logical operations on boolean values or expressions.
Concatenation The CONCAT() function or || operator is used to concatenate strings together.
Bitwise Bitwise operators like &, |, ^, ~, <<, and >> are used to perform bitwise operations on binary data.
Assignment The = and := operators are used to assign values to variables or columns.
In The IN operator is used to check whether a value matches any value in a list or subquery.
Like The LIKE operator is used to compare a value to a pattern using wildcard characters % and _.

Example of Using SQL Operators:

  • Arithmetic Operator Example:
SELECT 10 * 5; -- Multiplication
  • Comparison Operator Example:
SELECT * FROM products WHERE price > 100; -- Select products with price greater than 100
  • Logical Operator Example:
SELECT * FROM customers WHERE age >= 18 AND age <= 30; -- Select customers aged between 18 and 30
  • Concatenation Operator Example:
SELECT CONCAT(first_name, ' ', last_name) AS full_name FROM employees; -- Concatenate first name and last name
  • Bitwise Operator Example:
SELECT id, name FROM permissions WHERE permission_flags & 4 = 4; -- Select permissions with specific flag
  • Assignment Operator Example:
SET @total_sales := 500; -- Assigning a value to a variable
  • In Operator Example:
SELECT * FROM products WHERE category_id IN (1, 2, 3); -- Select products in specified categories
  • Like Operator Example:
SELECT * FROM customers WHERE city LIKE 'Ber%'; -- Select customers in cities starting with Ber

These examples illustrate how SQL operators are used in Percona Server for MySQL queries to perform various data operations.

Fundamental SQL links:

Advanced SQL features

Common data types

Choosing the correct data type for each column ensures data accuracy, efficiency, and reliability within the database. The following describes the purpose of a data type in Percona Server for MySQL:

  • Purpose:

  • Data types define the kind of data that can be stored in a column of a table.

  • They enforce constraints on the values that can be inserted into the column, ensuring data integrity.

  • Data types determine how the data is stored in memory and on disk, optimizing storage space and performance.

  • They provide a way to specify the format and range of acceptable values for numeric, string, date, and other types of data.

  • Data types facilitate efficient sorting, indexing, and searching of data within the database.

  • Importance:

  • Choosing the appropriate data type for each column is crucial for efficient database design and performance.

  • Data types help prevent data corruption and inconsistency by enforcing strict rules for data storage and manipulation.

  • They enable database administrators and developers to define the structure of the database accurately and ensure compatibility with application requirements.

  • Understanding data types allows for effective data modeling and schema design, leading to well-organized and scalable databases.

The following is a description of common data types:

Integer Types

Integers are whole numbers without any fractional part. Percona Server for MySQL offers different sizes of integer types to accommodate various ranges of values.

Data Type name Description
TINYINT A very small integer that can hold values from -128 to 127 (signed) or 0 to 255 (unsigned).
SMALLINT A small integer that can hold values from -32768 to 32767 (signed) or 0 to 65535 (unsigned).
MEDIUMINT A medium-sized integer that can hold values from -8388608 to 8388607 (signed) or 0 to 16777215 (unsigned).
INT or INTEGER A standard-sized integer that can hold values from -2147483648 to 2147483647 (signed) or 0 to 4294967295 (unsigned).
BIGINT A large integer that can hold values from -9223372036854775808 to 9223372036854775807 (signed) or 0 to 18446744073709551615 (unsigned).

Floating-Point Types

Floating-point types are used to represent numbers with a fractional part.

Data Type name Description
FLOAT A single-precision floating-point number that can hold up to 7 decimal digits of precision.
DOUBLE or REAL A double-precision floating-point number that can hold up to 15 decimal digits of precision.

Fixed-Point Types

Fixed-point types are used to represent exact numeric values.

  • DECIMAL or NUMERIC: A fixed-point number with user-defined precision and scale.

String Types

String types are used to store text data.

Data Type name Description
CHAR A fixed-length string that can hold up to 255 characters.
VARCHAR A variable-length string that can hold up to 65535 characters.
TEXT A string with a maximum length of 65535 characters.
BLOB A binary large object that can hold up to 65535 bytes.

Date and Time Types

Date and time types are used to store date and time information.

Data Type name Description
DATE A date value in the format YYYY-MM-DD.
TIME A time value in the format HH:MM:SS.
DATETIME A combination of date and time values in the format YYYY-MM-DD HH:MM:SS.
TIMESTAMP A timestamp value representing the number of seconds since the Unix epoch (January 1, 1970).

Advanced SQL features

Functions

A function in MySQL is a reusable block of code that performs a specific task and returns a value. It allows users to encapsulate logic, modularize code, and perform complex calculations or data manipulations.

Advantages of Using Functions:

Benefits Description
Reusability Functions can be reused multiple times in different parts of a SQL statement or query, reducing code duplication and promoting code modularity and maintainability.
Encapsulation Functions encapsulate logic and calculations, making it easier to understand and manage complex operations within the database.
Performance Functions can improve query performance by reducing the amount of data transferred between the database server and the client application.
Customization Functions allow users to create custom data transformations and calculations tailored to specific business requirements, enhancing the flexibility of the database.

Disadvantages of Using Functions:

Disadvantages Description
Performance Functions may introduce performance overhead, particularly if they involve complex computations or require access to large datasets.
Maintenance Functions require maintenance to keep them synchronized with changes to the underlying data model or business logic. Changes may impact the behavior of dependent queries.
Portability Functions written in MySQL may not be compatible with other database systems, limiting the portability of applications and databases.
Security Improperly designed or implemented functions may pose security risks, such as SQL injection vulnerabilities or unauthorized access to sensitive data.

Create function

CREATE FUNCTION calculate_discount (total_amount DECIMAL(10, 2)) RETURNS DECIMAL(10, 2)
    -> BEGIN
    ->     DECLARE discount DECIMAL(10, 2);
    ->     IF total_amount > 100 THEN
    ->         SET discount = total_amount * 0.1;
    ->     ELSE
    ->         SET discount = 0;
    ->     END IF;
    ->     RETURN discount;
    -> END;

Call function

SELECT calculate_discount(120);

Drop function

DROP FUNCTION IF EXISTS calculate_discount;

Advanced SQL features

SQL conventions

Sure, here’s a description of common SQL style conventions with examples using common MySQL commands:

Naming Conventions

Naming conventions refer to the rules and guidelines for naming database objects such as tables, columns, indexes, and stored procedures.

  • Use descriptive names: Choose names that clearly describe the purpose or content of the database object.
CREATE TABLE users (
    user_id INT AUTO_INCREMENT PRIMARY KEY,
    username VARCHAR(50),
    email VARCHAR(100)
);
  • Avoid abbreviations: Prefer full and meaningful words over abbreviations to enhance readability and understanding.
ALTER TABLE customers
ADD COLUMN date_of_birth DATE;

Indentation and Formatting

Indentation and formatting conventions improve the readability and maintainability of SQL code.

  • Indent SQL statements: Indent SQL statements consistently to show the logical structure of queries and commands.
SELECT
    user_id,
    username,
    email
FROM
    users
WHERE
    user_id = 1;
  • Use consistent casing: Use consistent casing for keywords, identifiers, and SQL functions to improve code consistency.
SELECT
    first_name,
    last_name,
    CONCAT_WS(' ', first_name, last_name) AS full_name
FROM
    customers;

Comments

Comments are annotations added to SQL code to explain its purpose, logic, or any other relevant information.

  • Document intent: Use comments to document the intent or purpose of SQL statements and code blocks.
-- Retrieve all active users
SELECT * FROM users WHERE status = 'active';
  • Avoid redundant comments: Avoid adding comments that merely repeat the code without adding meaningful information.
-- This query retrieves all users
SELECT * FROM users;

These SQL style conventions help maintain consistency, readability, and clarity in SQL code, making it easier to understand, debug, and maintain.

Advanced SQL features

Common SQL errors

Error handling in SQL commands involves managing and responding to errors that may occur during database operations. It ensures that the database remains consistent and provides feedback to users when errors occur.

SELECT statement

When executing a SELECT statement, errors may occur due to invalid syntax, missing tables, or insufficient permissions.

SELECT * FROM non_existent_table;
ERROR 1146 (42S02): Table 'database_name.non_existent_table' doesn't exist

INSERT Statement

Errors can occur during INSERT operations if data violates constraints or exceeds column limits.

INSERT INTO table_name (id, name) VALUES (1, 'John');
ERROR 1136 (21S01): Column count doesn't match value count at row 1

UPDATE Statement

UPDATE statements may encounter errors when attempting to modify non-existent rows or violating constraints.

UPDATE table_name SET non_existent_column = 'value';
ERROR 1054 (42S22): Unknown column 'non_existent_column' in 'field list'

DELETE Statement

Errors in DELETE statements can occur if the WHERE clause condition is invalid or violates constraints.

DELETE FROM table_name WHERE id = 'non_numeric_value';
ERROR 1054 (42S22): Unknown column 'non_numeric_value' in 'where clause'

DDL Statements (CREATE, ALTER, DROP)

DDL statements may fail due to syntax errors, existing object conflicts, or insufficient privileges.

CREATE TABLE existing_table (id INT PRIMARY KEY);
ERROR 1050 (42S01): Table 'existing_table' already exists

Advanced SQL features

SQL syntax

SQL (Structured Query Language) is a standardized language used to communicate with databases. Percona Server for MySQL follows SQL syntax, which consists of commands and statements for performing various operations on databases and their objects.

The SQL syntax includes commands for data manipulation (e.g., SELECT, INSERT, UPDATE, DELETE), data definition (e.g., CREATE, ALTER, DROP), data control (e.g., GRANT, REVOKE), and transaction control (e.g., COMMIT, ROLLBACK).

Syntax type Description
Data Manipulation MySQL supports powerful data manipulation features, allowing you to retrieve, insert, update, and delete data
Data Definition With MySQL, you can define the structure of your database objects such as tables, indexes, views, and stored procedures
Data Control MySQL provides commands for controlling access to database objects and defining user privileges
Transaction Management MySQL supports transactions, which allow you to group multiple SQL statements into a single unit of work
Stored Procedures MySQL allows you to define stored procedures and functions using SQL syntax
Triggers MySQL supports triggers, which are special types of stored procedures that automatically execute in response to specific events
Indexes MySQL provides features for optimizing query performance, including the ability to create indexes on columns
Views MySQL allows you to create views, which are virtual tables generated from SQL queries
Data Types MySQL supports a wide range of data types for storing different types of data

These features make MySQL a powerful and versatile database management system, capable of handling a wide range of database tasks efficiently and effectively using SQL syntax.

While MySQL SQL syntax may deviate from the standard SQL syntax in some aspects, it generally aims to be compatible with standard SQL to ensure interoperability with other database systems and tools. However, developers should be aware of these differences and consult the MySQL documentation for guidance when writing SQL queries and statements.

MySQL SQL syntax largely adheres to the standard SQL syntax, but there are some differences and extensions that set it apart:

Syntax Description
Data Types MySQL supports additional data types beyond the standard SQL specification, such as ENUM, SET, and BOOLEAN. These data types provide additional flexibility but may not be compatible with other database systems.
String Quoting MySQL allows both single quotes (') and double quotes (") for string literals, while standard SQL typically only uses single quotes. Additionally, MySQL supports backticks (`) for quoting identifiers, which is not standard SQL syntax.
Case Sensitivity By default, MySQL treats table and column names as case-insensitive, while standard SQL treats them as case-sensitive. However, this behavior can be changed by adjusting the server configuration.
LIMIT Clause MySQL uses the LIMIT clause to restrict the number of rows returned by a query, while standard SQL uses the FETCH FIRST or OFFSET clauses for similar functionality.
AUTO_INCREMENT MySQL uses the AUTO_INCREMENT attribute to automatically generate unique values for a column, while standard SQL uses IDENTITY or sequences for this purpose.
SQL Functions MySQL provides additional built-in functions and extensions beyond the standard SQL functions. For example, MySQL has functions like GROUP_CONCAT() and IFNULL(), which may not be available in other database systems.
Storage Engines MySQL supports multiple storage engines, each with its own set of features and capabilities. This option allows users to choose the most suitable storage engine for their specific requirements, but it introduces differences in behavior and syntax.

Advanced SQL features

Stored Procedures

A stored procedure is a set of pre-defined SQL statements stored in the database and executed as a single unit. It allows users to execute complex operations without rewriting the same code multiple times.

Benefit Description
Code Reusability Stored procedures can be reused multiple times in different parts of an application, reducing code duplication.
Improved Performance By executing multiple SQL statements in a single call, stored procedures can reduce network traffic and improve performance.
Enhanced Security Users can execute stored procedures without needing direct access to underlying tables, improving security and data integrity.
Centralized Logic Business logic is encapsulated within stored procedures, making it easier to manage and maintain.
Disadvantage Description
Difficulty in Debugging Stored procedures can be challenging to debug, as they are executed on the database server rather than within the application code.
Vendor Lock-in Stored procedures are specific to a particular database system, making it difficult to migrate to another database platform.
Limited Portability Stored procedures written in one database system may not be compatible with other systems, limiting portability and interoperability.

Stored Procedure examples

Create a Stored Procedure

DELIMITER //
CREATE PROCEDURE GetCustomerDetails (IN customerId INT)
    BEGIN
        SELECT * FROM customers WHERE id = customerId;
    END //
DELIMITER ;

Call a Stored Procedure

CALL GetCustomerDetails(123);

Modify a Stored Procedure

DELIMITER //
ALTER PROCEDURE GetCustomerDetails (IN customerId INT)
    BEGIN
        SELECT name, city FROM customers WHERE id = customerId;
    END //
DELIMITER ;

Drop a Stored Procedure

DROP PROCEDURE IF EXISTS GetCustomerDetails;

Advanced SQL features

Error handling in stored procedures

Error handling in stored procedures allows developers to gracefully handle exceptions and errors that may occur during the execution of the procedure. It enables better control over error messages and the ability to perform custom actions in response to errors.

Advantages of Using Error Handling:

Benefits Description
Graceful Error handling provides a way to handle exceptions gracefully, preventing unexpected termination of the procedure and providing users with meaningful error messages.
Customized Developers can customize error handling to perform specific actions based on the type of error encountered, such as logging errors, rolling back transactions, or retrying operations.
Control Error handling gives developers greater control over error propagation and recovery, allowing them to handle errors at different levels of granularity and complexity.
Robustness By implementing error handling, developers can make stored procedures more robust and resilient to unexpected conditions, enhancing the overall stability and reliability of the system.

Disadvantages of Using Error Handling:

Disadvantages Description
Complexity Error handling can introduce additional complexity to stored procedures, making them harder to understand, debug, and maintain, especially when dealing with nested error handling.
Overhead Implementing error handling may add overhead in terms of code complexity and execution time, particularly for procedures with extensive error-checking logic or frequent error conditions.
Performance Error handling may impact performance, especially in scenarios where error-checking logic needs to be executed repeatedly or in tight loops, leading to increased CPU and resource utilization.
Dependency Error handling can create dependencies between stored procedures and error-handling routines, making it challenging to modify or refactor procedures without affecting error handling.

To add error handling to a stored procedure, developers can use constructs like DECLARE, SIGNAL, RESIGNAL, and HANDLER to declare variables, raise errors, and handle exceptions. Here’s an example of error handling in a stored procedure:

DELIMITER //
CREATE PROCEDURE my_procedure()
    BEGIN
        DECLARE exit handler for sqlexception
        BEGIN
            -- Handle SQL exceptions
            ROLLBACK;
            SELECT 'An error occurred: ' || SQLSTATE();
        END;

        -- Procedure logic here
    END //
DELIMITER ;

In this example, the DECLARE statement declares an exit handler for SQL exceptions. Inside the handler block, the procedure rolls back any changes made and returns a custom error message with the SQL state.

CALL my_procedure();

This command executes the stored procedure and triggers the error handling logic if an exception occurs during execution.

Advanced SQL features

Variables in stored procedures, functions, and triggers

To add a variable in MySQL, you use the DECLARE keyword within the context of a stored program, such as a stored procedure, function, or trigger. The DECLARE keyword is used to define a new variable along with its data type and optionally, its initial value.

Value Description
variable_name This is the name of the variable you want to declare. Variable names must follow the rules for identifiers in MySQL.
data_type This specifies the data type of the variable, such as INT, VARCHAR, DECIMAL, DATE, etc.
default_value This is an optional parameter that specifies the default value for the variable. If not provided, the variable will be initialized to NULL by default. 
DECLARE variable_name data_type [DEFAULT default_value];

  • When you declare a variable using the DECLARE keyword, you are essentially telling MySQL to reserve space in memory to store a value of the specified data type.

  • Variables in MySQL are scoped to the block in which they are declared. This means they can only be used within the block of code (for example, stored procedure, function) in which they are declared.

  • Variables can be used to store and manipulate values within the context of the stored program. They are commonly used for temporary storage of intermediate results, loop counters, or parameters passed to the program.

DECLARE total_sales DECIMAL(10, 2) DEFAULT 0.0;

This statement has the following settings:

Description Value
total_sales is the name of the variable. total_sales
DECIMAL(10, 2) specifies that total_sales will hold decimal numbers with a precision of 10 digits and a scale of 2 decimal places. DECIMAL(10, 2)
DEFAULT 0.0 sets the initial value of total_sales to 0.0. If not provided, the default value would be NULL. DEFAULT 0.0

Advanced SQL features

Triggers

Using triggers

A trigger is a database object that automatically performs a specified action in response to certain events on a table or view. It allows users to enforce business rules, maintain data integrity, and automate tasks within the database.

Advantages of Using Triggers

Benefits Description
Data Integrity Triggers can enforce data integrity constraints by automatically validating or modifying data before it is inserted, updated, or deleted in a table.
Audit Trails Triggers can be used to create audit trails by recording changes made to the database, including who made the changes and when they occurred.
Simplified Triggers simplify application logic by moving complex business rules and validation checks into the database, reducing the amount of code needed in the application layer.
Automated Triggers automate repetitive tasks, such as updating denormalized data or sending notifications, by executing predefined actions in response to specified events.

Disadvantages of Using Triggers

Disadvantages Description
Complexity Triggers can add complexity to the database schema and make it harder to understand and maintain, especially when dealing with multiple triggers and complex logic.
Performance Triggers may impact database performance, particularly if they involve complex operations or are triggered frequently, leading to increased overhead and slower response times.
Debugging Triggers can be difficult to debug and troubleshoot, as they are executed automatically in response to events and may not provide detailed error messages or logging information.
Dependency Triggers create dependencies between database objects, making it challenging to modify or refactor the database schema without considering the impact on existing triggers.

Create a before_insert trigger

CREATE TRIGGER before_insert_customer
    BEFORE INSERT ON customers
    FOR EACH ROW
    BEGIN
        SET NEW.created_at = NOW();
    END;

Create an after_update trigger

CREATE TRIGGER after_update_inventory
    AFTER UPDATE ON inventory
    FOR EACH ROW
    BEGIN
        INSERT INTO inventory_changes (product_id, old_quantity, new_quantity, change_date)
        VALUES (OLD.product_id, OLD.quantity, NEW.quantity, NOW());
    END;

Drop a before_insert trigger

DROP TRIGGER IF EXISTS before_insert_customer;

Drop an after_update trigger

DROP TRIGGER IF EXISTS after_update_inventory;

Advanced SQL features

Troubleshoot SQL code

Review Get help from Percona for ways that we can work with you.

Troubleshooting SQL Code

To troubleshoot SQL code, follow these steps:

Action Description
Review Error Messages Carefully read any error messages returned by the MySQL server. They often provide valuable clues about what went wrong.
Check Syntax Verify that the SQL syntax is correct. A single typo or missing keyword can cause errors.
Verify Table and Column Names Ensure that table and column names are spelled correctly and match the actual names in the database.
Test in Isolation Test each part of the SQL statement separately to identify which part is causing the issue.
Use Logging Tools Enable query logging or use debugging tools to track the execution of SQL queries and identify any issues.
Review Documentation Consult the MySQL documentation to understand the correct usage of SQL statements and functions.
Seek Help Don’t hesitate to ask for help from more experienced developers or consult online forums and communities for assistance.

Troubleshooting SQL Code example:

Suppose you have the following SQL query that is not returning the expected results:

SELECT * FORM users WHERE age = 30;

After reviewing the error message returned by MySQL, you notice a typo in the query. The keyword “FORM” should be “FROM”. After correcting the typo, the query becomes:

SELECT * FROM users WHERE age = 30;

Now, the query should execute successfully and return the desired results.

Advanced SQL features

JSON

JSON in Percona Server for MySQL

JSON stands for JavaScript Object Notation. It is a lightweight data-interchange format that is easy for humans to read and write. It is also easy for machines to parse and generate. Percona Server for MySQL supports JSON data type, allowing you to store JSON documents in your database.

The JSON data type in Percona Server for MySQL is a handy way to store and work with flexible, semi-structured data right in your database. Think of it as a way to save JSON objects directly into your tables, so you don’t have to convert them into a rigid format.

When you use the JSON data type, the database stores your data in a special binary format that’s optimized for speed and space which is faster and more efficient than just saving JSON as plain text.

The JSON data type is great when your data doesn’t fit into a fixed structure or if it’s likely to change over time. The following are examples of when you would use the JSON data type:

  • Storing user preferences or settings.

  • Capturing logs or other dynamic data.

  • Handling complex objects without adding a ton of columns to your table.

JSON has the following features:

Feature Details
Validation Built-In Percona Server checks your JSON data when you insert or update it to make sure it’s valid. If something’s wrong, you’ll know right away.
Powerful Querying You can dig into specific parts of your JSON data using built-in functions like the following:
- JSON_EXTRACT() to pull out specific keys or values.
- JSON_CONTAINS() to check if a key or value exists.
- JSON_SET() to update parts of your JSON object without replacing the whole thing.
Indexing for Speed If you often query a particular key inside your JSON, you can create a generated column based on that key and index it, making queries much faster.

Use JSON in your database

The following is an example using JSON in your database.

CREATE TABLE user_data (
    id INT PRIMARY KEY AUTO_INCREMENT,
    name VARCHAR(50),
    settings JSON
);

INSERT INTO user_data (name, settings)
VALUES ('John', '{"theme": "dark", "notifications": {"push": true, "sms": false}}');

SELECT JSON_EXTRACT(settings, '$.theme') AS theme
FROM user_data
WHERE name = 'John';

  • The settings column stores JSON data.

  • You can use JSON_EXTRACT() to get the value of a specific key, like theme.

JSON in Percona Server for MySQL gives you have the flexibility of NoSQL with the reliability and querying power of a relational database.

Create a table with JSON Data Type

Create a table that includes a column with the JSON data type.

CREATE TABLE users (
    id INT AUTO_INCREMENT PRIMARY KEY,
    name VARCHAR(255) NOT NULL,
    info JSON
);

The columns are the following:

  • id is an auto-incremented primary key.

  • name is a column for storing the user’s name.

  • info is a column for storing JSON data.

Insert JSON Data

Insert the JSON data into the table using the INSERT statement. The name column stores the user’s name. The info column stores JSON data using the JSON_OBJECT function. This function creates a JSON object with key-value pairs.

INSERT INTO users (name, info) VALUES (
    'John Doe',
    JSON_OBJECT('age', 30, 'city', 'Berlin', 'phone', '+44 20 7946 0958')
);

Query JSON Data

You can query JSON data using the SELECT statement. The name column retrieves the user’s name. The info->>'$.age' expression retrieves the value of the age key from the JSON object stored in the info column.

SELECT name, info->>'$.age' AS age FROM users;

Update JSON Data

You can update JSON data using the UPDATE statement. The JSON_SET function updates the value of the age key in the JSON object stored in the info column. The WHERE clause specifies that only the row with the name ‘John Doe’ should be updated.

UPDATE users
SET info = JSON_SET(info, '$.age', 31)
WHERE name = 'John Doe';

Delete JSON Data

You can delete JSON data using the DELETE statement. This statement removes rows from the users table where the city key in the JSON object stored in the info column has the value ‘New York’.

DELETE FROM users WHERE info->>'$.city' = 'Berlin';

Add New Key-Value Pairs to JSON Data

You can add new key-value pairs to existing JSON data using the JSON_SET function. The JSON_SET function adds a new key phone with the value ‘+44 20 7946 0958’ to the JSON object stored in the info column.

UPDATE users
SET info = JSON_SET(info, '$.phone', '+44 20 7946 0958')
WHERE name = 'John Doe';

Remove Key-Value Pairs from JSON Data

You can remove key-value pairs from existing JSON data using the JSON_REMOVE function. This function removes the phone key from the JSON object stored in the info column.

UPDATE users
SET info = JSON_REMOVE(info, '$.phone')
WHERE name = 'John Doe';

Use JSON Functions

Percona Server for MySQL provides several functions to work with JSON data.

JSON_EXTRACT

You can extract data from a JSON document using the JSON_EXTRACT function. This function extracts the value of the city key from the JSON object stored in the info column.

SELECT JSON_EXTRACT(info, '$.city') AS city FROM users WHERE name = 'John Doe';

JSON_ARRAY

You can create a JSON array using the JSON_ARRAY function. This function creates a JSON array with the values ‘apple’, ‘banana’, and ‘cherry’.

INSERT INTO users (name, info) VALUES (
    'Jane Smith',
    JSON_ARRAY('apple', 'banana', 'cherry')
);

JSON_CONTAINS

You can check if a JSON document contains a specific value using the JSON_CONTAINS function. This function checks if the info column contains the value ‘New York’ for the city key.

SELECT name FROM users WHERE JSON_CONTAINS(info, '"New York"', '$.city');

Manage

Database management

Introduction to Databases and SQL

Introduction to databases

A database in the server is a structured collection of data. It helps store, organize, and manage various types of information like customer details, product inventories, financial records, and more. Using a database allows you to store data in an organized manner, making it easy to retrieve, update, and manipulate as needed.

Advantages

Using a database in servers has several benefits. The table below lists these advantages:

Advantages Description
Efficient Storage Databases store data in an organized way, making it easy to manage large volumes of information.
Quick Retrieval You can quickly find and retrieve specific data using SQL queries.
Data Integrity Databases ensure data accuracy and consistency through constraints and relationships.
Scalability Databases can handle growing amounts of data and users efficiently.
Security Databases provide robust security features to protect sensitive data.

Disadvantages

While databases offer many advantages, there are also some drawbacks to consider. The table below outlines these disadvantages:

Disadvantages Description
Complex Setup Setting up and configuring a database can be complex and time-consuming.
Maintenance Databases require regular maintenance and updates to function optimally.
Resource Intensive Databases can consume significant server resources, impacting performance.
Backup and Recovery Proper backup and recovery processes are necessary to prevent data loss.
Cost Licensing and operational costs for databases can be high, especially for large-scale deployments.

Permissions required

To create a database on a server, a user must have the CREATE privilege. This privilege allows the user to create new databases and tables within those databases.

Using SQL Commands with a database

Create a database

You use the CREATE DATABASE command to create a new database in the server. This command tells the server to create a new database with the specified name. For example, to create a database named my_database, you execute the following command:

CREATE DATABASE my_database;

This command creates a new, empty database called my_database. You can then start adding tables and data to this database.

Select a database

After creating a database, you need to select it to start working with it. Use the USE command to specify which database you want to use for your SQL statements. For example, to select the my_database database, you execute the following command:

USE my_database;

This command tells the server to use my_database for all subsequent SQL commands. Now, any SQL operations you perform will apply to my_database.

Database management

Introduction to database tables

A database table is a collection of data organized into rows and columns. Each table consists of records (rows) and fields (columns). Tables help organize and manage data efficiently.

Advantages

Advantages Description
Organized Data Tables allow you to organize data into rows and columns, making it easy to understand and manage.
Efficient Queries You can use SQL queries to quickly search, filter, and retrieve data from tables.
Data Integrity Tables support constraints like primary keys and foreign keys, ensuring data integrity and consistency.
Scalability You can add or modify tables as your data grows, making it easy to scale your database.
Relational Management Tables allow you to create relationships between different sets of data, making it easier to manage complex datasets.

Disadvantages

Disadvantages Description
Complexity Designing and maintaining tables, especially with relationships, can become complex and time-consuming.
Performance Issues Large tables with many rows can lead to performance issues, requiring optimization and indexing.
Storage Overhead Tables with many columns or large data types can consume significant storage space.
Maintenance Regular maintenance tasks, such as backups and indexing, are necessary to ensure optimal performance and data integrity.
Learning Curve Beginners may find it challenging to learn SQL and understand how to design and manage tables effectively.

Permissions required

To create a table in a database, you need appropriate permissions granted to your database user account. These permissions are typically managed by the database administrator (DBA) or system administrator. Database permissions control what actions a user can perform on a database. In the context of creating a table, the user needs specific permissions related to database management.

Permission Description
CREATE TABLE The most fundamental permission required to create a table is the CREATE TABLE permission. This permission allows the user to create new tables within the database.
CREATE In addition to CREATE TABLE, the user might also need the more general CREATE permission. This permission grants the ability to create other database objects besides tables, such as indexes, views, or stored procedures.
ALTER Depending on the database configuration, the user might also need the ALTER permission. This permission allows the user to modify the structure of existing tables, such as adding or removing columns.

Create a table

To create a table, use the CREATE TABLE command. Follow it with the table name and define the columns and their data types. For example, to create a table named customers with columns for id, name, and city, use this command:

CREATE TABLE customers (
    id INT AUTO_INCREMENT PRIMARY KEY,
    name VARCHAR(100),
    city VARCHAR(100)
);

Database management

Create a table

Creating a table is essential to organizing and storing your data effectively when working with a database. Here’s a step-by-step guide on how to create a table in such a database:

Permissions Required

To create a table in a database, you need appropriate permissions. Typically, you’ll need the CREATE TABLE privilege, which allows you to create new tables within a database. This privilege is usually granted to database users by database administrators or through predefined user roles. If you do not have the necessary permissions, you’ll need to contact your database administrator to grant them.

Define the table structure

Now, define the structure of your table by specifying its columns along with their data types and any additional properties. Each column represents a different attribute of your data.

Here’s the syntax for creating a table:

CREATE TABLE table_name (
    column1_name data_type constraints,
    column2_name data_type constraints,
    ...
);

Replace table_name with the desired name for your table. For each column, provide a name, data type, and constraints such as NOT NULL, PRIMARY KEY, AUTO_INCREMENT.

Create the table

Execute the CREATE TABLE command to create the table in the database. For example, to create a table named employees with columns for id, name, and salary, you would run the following SQL command:

CREATE TABLE employees (
    id INT PRIMARY KEY AUTO_INCREMENT,
    name VARCHAR(50) NOT NULL,
    salary DECIMAL(10, 2)
);

This command creates a table named employees with three columns: id, name, and salary. The id column is an integer type and serves as the primary key with auto-increment functionality. The name column is a variable-length string, and the salary column is a decimal number with a precision of 10 digits and a scale of 2.

Verify Table Creation

After executing the CREATE TABLE command, verify that the table has been successfully created. You can use various SQL commands such as SHOW TABLES or DESCRIBE table_name to check the existence and structure of the newly created table.

SHOW TABLES;
DESCRIBE employees;

Database management

Modify a table

The ALTER TABLE command acts like a toolkit that allows you to change the structure of existing tables. You can add new sections (columns), remove old ones, or change how information is stored (data types). This command helps you adapt your database to new needs or improve efficiency.

Things to Watch Out For

  • Data loss: Be careful when modifying tables! Deleting a section (column) or changing its format might erase existing data in that section.

  • Slowdowns: Altering large tables or making complex changes can slow down the database, especially during busy times. It might take longer for things to work while the changes are applied.

  • Locks: MySQL might temporarily lock the tables you’re working on when making changes. This operation means other users can’t access or modify that data until the changes are complete, which can cause delays for others.

Modify table example

After a table has been created, you may need to modify its structure or properties. Percona Server for MySQL provides the ALTER TABLE command for making such modifications. You can add, modify, or drop columns, change data types, add constraints, and more using this command.

The following is an example using an ALTER TABLE command:

ALTER TABLE users
ADD COLUMN age INT,
MODIFY COLUMN city VARCHAR(100),
DROP COLUMN username;

Database management

Isolation levels

In databases, isolation levels define how transactions interact with each other and the data they access. They determine the level of concurrency and consistency in a multi-user database environment.

In MySQL, there are four isolation levels available, each offering different trade-offs between concurrency and consistency:

Each isolation level offers a different balance between concurrency and consistency, and the choice depends on the application’s specific requirements. By selecting the appropriate isolation level, developers can ensure their MySQL database applications’ desired data integrity and performance level.

Read Uncommitted

In the Read Uncommitted isolation level, transactions can read data that has been modified by other transactions but not yet committed. This level allows for the highest concurrency but can lead to dirty reads.

SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;

-- Perform a SELECT query to read uncommitted data
SELECT * FROM table_name;

Read Committed

In Read Committed isolation level, transactions can only read data that has been committed by other transactions. This level prevents dirty reads but allows for non-repeatable reads and phantom reads.

SET TRANSACTION ISOLATION LEVEL READ COMMITTED;

-- Perform a SELECT query to read committed data
SELECT * FROM table_name;

Repeatable Read

In Repeatable Read isolation level, transactions can only read data that has been committed by other transactions at the start of the transaction. This level prevents dirty reads and non-repeatable reads but allows for phantom reads.

SET TRANSACTION ISOLATION LEVEL REPEATABLE READ;

-- Perform a SELECT query to read data consistently within the transaction
SELECT * FROM table_name;

Serializable

In Serializable isolation level, transactions are executed serially, preventing any concurrent access to the data. This level provides the highest level of isolation but can lead to reduced concurrency and potential deadlock situations.

SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;

-- Perform a SELECT query within a serializable transaction
SELECT * FROM table_name;

These examples demonstrate how to set and use different isolation levels in SQL transactions, each providing consistency and concurrency control.

Database management

Transaction management

A database transaction is a unit of work performed within a database management system (DBMS) that must be executed atomically and consistently. A transaction represents a series of operations (such as queries, inserts, updates, or deletes) that are treated as a single, indivisible unit. Transactions ensure data integrity by guaranteeing that all of the transaction’s operations are completed successfully and permanently saved to the database (committed) or none of them are applied (rolled back).

Percona Server for MySQL provides features for managing transactions to ensure the consistency and reliability of data.Transactions in Percona Server for MySQL are typically managed using the following commands and techniques:

  • START TRANSACTION: This command begins a new transaction. Once started, all subsequent SQL statements will be part of the transaction until it is either committed or rolled back.

  • COMMIT: The COMMIT command is used to save the changes made during the transaction to the database permanently. Once committed, the changes become visible to other transactions.

  • ROLLBACK: The ROLLBACK command is used to undo the changes made during the transaction and restore the database to its state before the transaction begins. It cancels any modifications made within the transaction.

  • SAVEPOINT: SAVEPOINTs are markers within a transaction that allow you to set points to which you can later roll back. They provide a way to partially undo changes within a transaction without rolling back the entire transaction.

Transactions in Percona Server for MySQL are ACID-compliant, meaning they adhere to the principles of Atomicity, Consistency, Isolation, and Durability:

Type Description
Atomicity Transactions are atomic, meaning that all the operations within a transaction are treated as a single unit of work. Either all operations are completed successfully, or none of them are applied.
Consistency Transactions ensure that the database remains in a consistent state before and after the transaction. Constraints, triggers, and other rules are enforced to maintain data integrity.
Isolation Transactions are isolated from each other, meaning that the changes made within one transaction are not visible to other transactions until the transaction is committed.
Durability Once a transaction is committed, the changes made to the database are permanent and cannot be lost, even in the event of system failure.

Percona Server for MySQL supports different transaction isolation levels, such as READ UNCOMMITTED, READ COMMITTED, REPEATABLE READ, and SERIALIZABLE, which control how transactions interact with each other and with the data in the database.

Database management

Views

A view is a virtual table generated from a SQL query. It allows users to simplify complex queries, hide sensitive data, and provide a customized view of the database without altering the underlying schema.

Advantages of Using Views

Benefits Description
Simplification Views simplify complex queries by encapsulating them into a single, reusable object. They provide a convenient way to abstract and hide the complexity of underlying tables.
Security Views can enhance security by restricting access to sensitive data. Users can be granted access to views containing only the necessary columns, without direct access to the tables.
Customization Views enable users to create customized perspectives of the data, presenting only the relevant information needed for specific tasks or reports.
Performance Views can improve query performance by pre-computing and caching results, reducing the need to repeatedly execute complex queries.

Disadvantages of Using Views

Disadvantages Description
Complexity Views can introduce complexity to the database schema and query execution plan, making it harder to optimize and troubleshoot performance issues.
Overhead Views may incur overhead in terms of storage and processing resources, particularly for materialized views or views involving joins and aggregation functions.
Maintenance Views require maintenance to keep them synchronized with the underlying tables. Changes to the base tables may impact the results returned by the view.
Limited Use Views have limitations in terms of updateability and support for certain SQL operations, such as ordering or grouping by columns not present in the underlying tables.

Create view

CREATE VIEW customer_orders AS
    SELECT customers.name, orders.order_id, orders.total_amount
    FROM customers
    JOIN orders ON customers.customer_id = orders.customer_id;

CREATE VIEW recent_orders AS
   SELECT *
    FROM orders
    WHERE order_date >= CURDATE() - INTERVAL 30 DAY;

Drop view

DROP VIEW IF EXISTS customer_orders;

DROP VIEW IF EXISTS recent_orders;

Database management

Authentication methods

An authentication method is a way to verify the identity of a user trying to access the database. It defines how the server checks if the credentials provided are correct and whether the user can connect.

Version changes

MySQL 8.4 disables the deprecated mysql_native_password authentication plugin by default.

Enable the mysql_native_password plugin explicitly. Use one of the following configuration methods:

  • Add the --mysql-native-password=ON option when starting the MySQL server.

  • Edit your MySQL configuration file. In the [mysqld] section, add the line mysql_native_password=ON.

Either option re-enables the plugin for backward compatibility. Adopt a stronger authentication method whenever possible.

Common Authentication Methods

Method Description
Caching SHA-2 Pluggable Authentication Uses SHA-256 for password hashing. MySQL hashes the user’s password and compares it to the stored hash. It caches authentication data for better performance. Suitable for modern setups with strong security and performance. However, it may not work with older MySQL clients.
MySQL Native Authentication An older method that uses SHA-1 for password hashing. It offers wide compatibility, making it useful for legacy systems or applications that don’t support newer methods. However, it has reduced security and is best avoided unless necessary for compatibility.
PAM Pluggable Authentication Integrates MySQL with Linux’s Pluggable Authentication Modules (PAM). MySQL relies on the operating system for authentication, allowing for various authentication mechanisms. Useful in environments needing centralized authentication management, but setup can be complex.
LDAP Authentication MySQL connects to an LDAP server to authenticate users. Ideal for managing large, distributed systems, enabling centralized user management, and integrating with existing directory services. The main drawback is the added complexity of maintaining an LDAP server.
Kerberos Authentication Uses the Kerberos protocol for authentication. Provides strong security and single sign-on across multiple services. Common in enterprise environments but requires a complex Kerberos infrastructure.
FIDO Pluggable Authentication Supports FIDO (Fast IDentity Online) authentication devices. Used in high-security environments for robust two-factor authentication. Requires special hardware like security keys and may face user resistance.
OpenID Connect Authentication Authenticates users with signed JSON Web Tokens (JWTs) from an external Identity Provider. Supported providers include Keycloak, Okta, and Microsoft Entra ID. Suitable for organizations that already operate a central identity service and want password-less, single-sign-on access to MySQL. Requires a secure connection (TLS, socket, or shared memory). Supports group-to-role mapping and group-based proxy users.
Auth Socket Authentication Uses the operating system’s socket-based authentication, matching the connecting user with the system user that owns the MySQL process. Ideal for local administrative access but limited to local machine use and not suitable for remote or multi-user environments.

Review effective privileges with SHOW EFFECTIVE GRANTS

In MySQL, SHOW GRANTS has the following limitations:

  • Shows only explicitly granted privileges

  • Does not show inherited anonymous user privileges

  • Does not show privileges inherited through roles unless the USING clause is specified

Other privileges might be available to the account but are not displayed. For example:

-- Create named and anonymous users
CREATE USER 'user1'@'localhost';
CREATE USER ''@'localhost';

-- Grant privilege to anonymous user
GRANT SELECT ON db.* TO ''@'localhost';

-- Check user1's grants
SHOW GRANTS FOR 'user1'@'localhost';
Expected output 
GRANT USAGE ON *.* TO 'user1'@'localhost'

Even though ‘user1’@’localhost’ can use SELECT on db.*, this privilege does not appear in SHOW GRANTS.

Percona Server for MySQL’s SHOW EFFECTIVE GRANTS command provides a comprehensive view of a user’s permissions. It reveals not only the privileges directly granted to the user but also those inherited from other accounts, such as anonymous users or roles. This includes system-level, database-level, and table-level privileges, giving you a complete picture of the user’s access rights within the database.

The benefits are:

  • Shows complete privilege picture

  • Helps identify privilege sources

  • Simplifies security audits

  • Makes troubleshooting easier

  • Reveals inherited privileges

Example

If we create the following users:

CREATE USER grantee@localhost IDENTIFIED BY 'grantee1';
Expected output 
Query OK, 0 rows affected (0.50 sec)

CREATE USER grantee IDENTIFIED BY 'grantee2';
Expected output 
Query OK, 0 rows affected (0.09 sec)

CREATE DATABASE db2;
Expected output 
Query OK, 1 row affected (0.20 sec)

GRANT ALL PRIVILEGES ON db2.* TO grantee WITH GRANT OPTION;
Expected output 
Query OK, 0 rows affected (0.12 sec)
  • SHOW EFFECTIVE GRANTS output before the change:
SHOW EFFECTIVE GRANTS;
Expected output 
+----------------------------------------------------------------------------------------------------------------+
| Grants for grantee@localhost                                                                                   |
+----------------------------------------------------------------------------------------------------------------+
| GRANT USAGE ON *.* TO 'grantee'@'localhost' IDENTIFIED BY PASSWORD '*9823FF338D44DAF02422CF24DD1F879FB4F6B232' |
+----------------------------------------------------------------------------------------------------------------+
1 row in set (0.04 sec)

Although the grant for the db2 database isn’t shown, grantee user has enough privileges to create the table in that database:

user@trusty:~mysql -ugrantee -pgrantee1 -h localhost

CREATE TABLE db2.t1(a int);
Expected output 
Query OK, 0 rows affected (1.21 sec)
  • The output of SHOW EFFECTIVE GRANTS after the change shows all the privileges for the grantee user:
SHOW EFFECTIVE GRANTS;
Expected output 
+-------------------------------------------------------------------+
| Grants for grantee@localhost                                      |
+-------------------------------------------------------------------+
| GRANT USAGE ON *.* TO 'grantee'@'localhost' IDENTIFIED BY PASSWORD|
| '*9823FF338D44DAF02422CF24DD1F879FB4F6B232'                       |
| GRANT ALL PRIVILEGES ON `db2`.* TO 'grantee'@'%' WITH GRANT OPTION|
+-------------------------------------------------------------------+
2 rows in set (0.00 sec)

Restrict dynamic log file locations

The secure_log_path system variable plays a crucial role in enhancing the security and organization of log files within a MySQL database environment by restricting where dynamic log files can be stored.

In a MySQL environment, restricting dynamic log locations offers several benefits:

Benefit Details
Enhanced security It prevents unauthorized modification of log files, protecting sensitive information and audit trails.
Improved compliance It helps meet regulatory requirements for data security and auditability.
Simplified administration It centralizes log files, making them easier to manage and monitor.
Increased reliability It reduces the risk of accidental log file deletion or corruption.

The disadvantages could be:

  • Reduced flexibility: Cannot change the log file locations easily

  • Increased complexity: Adds an extra layer of configuration and management

  • Performance impact: Writing to log files on slower storage media may increase overhead and potentially affect the overall performance of the MySQL server.

The benefits of restricting dynamic log locations in MySQL outweigh the disadvantages, especially in security-conscious environments.

secure_log_path

Variable Name Description
Command-line –secure-log-path
Dynamic No
Scope Global
Data type String
Default empty string

The secure_log_path variable controls where specific log files are stored. This variable expects a directory name as a string value. By default, the value is an empty string, allowing older applications to continue functioning without requiring a secure log path.

This variable affects the following options:

Option Description
slow_query_log=ON Enables the storage of the slow query log file.
slow_query_log_file Sets the name and location of the slow query log file.
general_log=ON Enables the storage of the general log file.
general_log_file Sets the name and location of the general log file.
buffered_error_log_filename Sets the name and location of the buffered error log file.
buffered-error-log-size Specifies the size of the buffer for error logging in bytes.

The secure_log_path variable is read-only and must be set up in a configuration file or the command line.

Value Description
Empty string The variable only adds a warning to the error log and does nothing. The log files are located in the default directory, /var/lib/mysql.
Directory name If the value contains a directory name, then the slow query log and the general log must be located in that directory. An attempt to move either of these files outside of the specified directory results in an error.

By establishing a controlled logging environment through the secure_log_path variable, MySQL administrators can significantly enhance both the security and manageability of their logs, reducing risks associated with unauthorized access and data integrity.

The example of the secure_log_path variable usage

Run the following commands as root:

  1. Create the direcory to store the log files.

    mkdir /var/lib/mysqld-logs

  2. Enable the following options and set them up with the created directory in /etc/my.cnf configuration file.

    [mysqld]
secure_log_path=/var/lib/mysqld-logs
general-log=ON
general-log-file=/var/lib/mysqld-logs/general_log
slow-query-log=ON
slow-query-log-file=/var/lib/mysqld-logs/slow_log
buffered-error-log-size=1000
buffered-error-log-filename=/var/lib/mysqld-logs/buffered_log

  3. Change the owner and group of the /var/lib/mysqld-logs directory and all its subdirectories and files to mysql.

    chown -R mysql:mysql /var/lib/mysqld-logs

  4. Restart the MySQL server.

    systemctl restart mysql

  5. Check that the slow query log and the general log are enabled for the MySQL server.

    mysql -e "select @@slow_query_log, @@general_log, @@secure_log_path"
    Expected output 
    +------------------+---------------+-----------------------+
| @@slow_query_log | @@general_log | @@secure_log_path     |
+------------------+---------------+-----------------------+
|                1 |             1 | /var/lib/mysqld-logs/ |
+------------------+---------------+-----------------------+

  6. Check that the slow query log and the general log are stored in the /var/lib/mysqld-logs directory. As root (or a user with read access to the directory), run:

    cd /var/lib/mysqld-logs/
ls -lrth
    Expected output 
    -rw-r-----. 1 mysql mysqld-logs 240 Aug 18 11:56 localhost-slow.log
-rw-r-----. 1 mysql mysqld-logs 565 Aug 18 11:56 localhost.log

MySQL Clone plugin

The MySQL Clone plugin lets you clone data from either a local server or from a remote server. The plugin creates a physical snapshot of the data stored in InnoDB, which includes schemas, tables, tablespaces, and data dictionary metadata. The cloned data is a functional data directory and can be used for provisioning a server .

The following table lists the cloning operation types:

Cloning operation type Description
Local Clones the data from the server where the operation is initiated to either a directory on the same server or a server node.
Remote Clones the data from the donor to the joiner over the network.

When replicating a large number of transactions, the Clone plugin may be a more efficient solution.

Install the Clone plugin

The Clone plugin must be installed on both the donor and the joiner servers at either server startup or at runtime. To install the plugin at runtime, run the following command:

INSTALL PLUGIN clone SONAME 'mysql_clone.so';

Review the INFORMATION_SCHEMA.PLUGINS table or run the SHOW PLUGINS command to verify the installation. The following is an example of querying the PLUGINS table.

SELECT PLUGIN_NAME, PLUGIN_STATUS FROM INFORMATION_SCHEMA.PLUGINS WHERE PLUGIN_NAME='clone';

The result lists the Clone plugin and the status.

Clone data

The SQL statement used to clone data depends on if the operation is local or remote. The following code is an example of cloning data from a remote server:

CLONE INSTANCE FROM `root@remote.server:13336` IDENTIFIED BY `user`;

Replace the user name, host name, and port number with the settings from the donor server.

Limitations

The MySQL 8.4 clone plugin enforces several functional and scope limitations that define which instances administrators clone and which data or metadata a clone operation includes.

Version and instance limits

  • The clone plugin only supports cloning within the same MySQL server series, so administrators clone between patch releases such as 8.4.1 and 8.4.13, but not between different major series such as 8.0 and 8.4.

  • Each clone operation targets only one MySQL instance, and a single operation never clones multiple instances at once.

Network and protocol restrictions

  • The clone plugin uses the classic MySQL protocol, so the X Protocol port defined by mysqlx_port does not serve as the donor port in remote cloning operations with CLONE INSTANCE .

  • The donor MySQL server instance does not accept clone connections that pass through MySQL Router, so administrators connect directly to the donor.

Configuration and logs excluded

  • The clone plugin preserves the configuration of the recipient server instance, so a clone operation does not copy the donor server configuration or persisted system variable settings.

  • The plugin does not copy binary logs, so the recipient does not receive the donor’s binary log files as part of the clone .

Storage engine behavior

  • The clone plugin copies only data stored in the InnoDB storage engine and excludes data that other storage engines store.

  • The plugin creates MyISAM and CSV tables on the recipient as empty tables, even when those tables reside in special schemas such as the sys schema .

Local cloning and tablespaces

  • During local cloning, the clone plugin does not support general tablespaces that use an absolute path, because cloning those tablespaces would create a conflicting file that uses the same absolute path on the destination host.

Manage components

INSTALL COMPONENT

The INSTALL COMPONENT does the following:

  • Installs the component
  • Activates the component

If an error, such as a misspelled component name, occurs, the statement fails and nothing happens.

You can install multiple components at the same time.

Example

The following is an example of the INSTALL COMPONENT statement.

INSTALL COMPONENT 'file://componentA';

UNINSTALL COMPONENT

The UNINSTALL COMPONENT does the following:

  • Deactivates the component
  • Uninstalls the component

If the statement does not undo any persisted variables.

If an error, such as a misspelled component name, occurs, the statement fails and nothing happens.

You can uninstall multiple components at the same time.

Required privilege

The statement requires the DELETE privilege for the mysql.component system table. Executing the statement removes the registration row from this table.

Example

The following is an example of the UNINSTALL COMPONENT statement.

mysql > UNINSTALL COMPONENT 'file://componentA' ;

Find more information in the UNINSTALL COMPONENT document.

Back up and restore

Backup and restore overview

Backups are data snapshots that are taken at a specific time and are stored in a common location in a common format. A backup is only useful for a defined time.

The following scenarios require a backup to recover:

Reason Description
Hardware or host failure Issues with disks, such as stalls or broken disks. With cloud services, the instance can be unaccessible or broken.
Corrupted data This issue can be caused by power outages, the database failed to write correctly and close the file.
User mistake Deleting data or an update overwriting good data with bad data
Natural disaster or data center failure Power outage, flooding, or internet issues
Compliance Required to comply with regulations and standards

Strategies

Define a backup and restore strategy for each of your databases. The strategies should have the following practices:

Practice Description
Retention How long should you keep the backups. This decision should be based on the organization’s data governance policies and the expense of storing the backups. The schedule for backups should match the retention schedule.
Document Document the strategy and any related policies. The documents should include information about the process and any tools used during backup or restore.
Encrypt Encrypt the backup and secure the storage locations
Test Test the backups on a timely basis.

The backup strategy defines type and the backup frequency, the hardware required, how the backups are verified, and storing the backups, which also includes the backup security. The strategy uses the following metrics:

Metric Description
Recovery Time Objective (RTO) How long can the system be down?
Recovery Point Objective (RPO) How much data can the organization lose?

The restore strategy defines which user account has the restore responsibility and how and frequency of testing the restore process.

These strategies require planning, implementation, and rigorous testing. You must test your restore process with each type of backup used to validate the backup and measure the recovery time. Automate this testing as much as possible. You should also document the process. In case of disaster, you can follow the procedures in the document without wasting time.

If you are using replication, consider using a dedicated replica for backups because the operation can cause a high CPU load.

Physical backup or logical backup

A backup can be either a physical backup or a logical backup.

Physical backups

A physical backup copies the files needed to store and recover the database. They can be data files, configuration files, logs, and other types of files. The physical database can be stored in the cloud, in offline storage, on disc, or tape.

Percona XtraBackup takes a physical backup. You can also use RDS/LVM Snapshots or the MySQL Enterprise Backup.

If the server is stopped or down, you can copy the datadir with the cp command or the rsync command.

Logical backups

A logical backup contains the structural details. This type of backup contains tables, views, procedures, and functions.

Tools like mysqldump , mydumper mysqlpump , and mysql shell take a logical backup.

Comparison

Comparison Physical backup Logical backup
Content The physical database files The tables, users, procedures, and functions
Restore speed Restore can be quick Restore can be slower and does not include file information.
Storage Can take more space Based on what is selected, the backup can be smaller

Backup locks

Percona Server for MySQL offers the LOCK TABLES FOR BACKUP statement as a lightweight alternative to FLUSH TABLES WITH READ LOCK for both physical and logical backups.

FLUSH TABLES WITH READ LOCK

The FLUSH TABLES WITH READ LOCK statement performs two main actions:

  • Flushes any changes to the tables currently in memory but have not yet been written to disk. This operation ensures that the data on the disk is up-to-date with the most recent changes.

  • Acquires a read lock on all tables. This means that other sessions can continue to read from the tables but cannot execute any write operations (like INSERT, UPDATE, or DELETE) until the lock is released. This action is beneficial when creating a consistent data snapshot for backup purposes.

Please note that FLUSH TABLES WITH READ LOCK applies to all databases unless you specify tables individually. Also, this statement causes an implicit commit.

The following command connects to the server and executes the FLUSH TABLES WITH READ LOCK statement. After running this command, no write operations can be performed until the lock is released.

mysql --user="user" --password="password" --host="host" --execute="FLUSH TABLES WITH READ LOCK"

LOCK TABLES FOR BACKUP

LOCK TABLES FOR BACKUP uses a new MDL lock type to block updates to non-transactional tables and DDL statements for all tables. If there is an active LOCK TABLES FOR BACKUP lock then all DDL statements and all updates to MyISAM, CSV, MEMORY, ARCHIVE, and MyRocks tables will be blocked in the Waiting for backup lock status, visible in PERFORMANCE_SCHEMA or PROCESSLIST.

LOCK TABLES FOR BACKUP does not affect SELECT queries for all mentioned storage engines. Against InnoDB, MyRocks, Blackhole and Federated tables, the LOCK TABLES FOR BACKUP does not apply to the INSERT, REPLACE, UPDATE, DELETE statements: Blackhole tables have no relevance to backups and Federated tables are ignored by both logical and physical backup tools.

Unlike FLUSH TABLES WITH READ LOCK, LOCK TABLES FOR BACKUP does not flush tables, i.e. storage engines are not forced to close tables, and tables are not expelled from the table cache. As a result, LOCK TABLES FOR BACKUP only waits for conflicting statements to complete (i.e. DDL and updates to non-transactional tables). It never waits for SELECTs, or UPDATEs to InnoDB or MyRocks tables to complete, for example.

If an “unsafe” statement is executed in the same connection that is holding a LOCK TABLES FOR BACKUP lock, the statement fails with the following error:

Expected output 
ERROR 1880 (HY000): Can't execute the query because you have a conflicting backup lock

UNLOCK TABLES releases the lock acquired by LOCK TABLES FOR BACKUP.

The intended use case for Percona XtraBackup is:

LOCK TABLES FOR BACKUP
... copy .frm, MyISAM, CSV, etc. ...
UNLOCK TABLES
... get binlog coordinates ...
... wait for redo log copying to finish ...

Privileges

The LOCK TABLES FOR BACKUP requires the BACKUP_ADMIN privilege.

Interaction with other global locks

The LOCK TABLES FOR BACKUP has no effect if the current connection already owns a FLUSH TABLES WITH READ LOCK lock, as that lock is more restrictive. If FLUSH TABLES WITH READ LOCK is executed in a connection that has acquired LOCK TABLES FOR BACKUP, FLUSH TABLES WITH READ LOCK fails with an error.

If the server is operating in the read-only mode (for example, read_only set to 1), statements that are unsafe for backups will be either blocked or fail with an error, depending on whether they are executed in the same connection that owns a LOCK TABLES FOR BACKUP lock or other connections.

MyISAM index and data buffering

MyISAM key buffering is normally write-through, i.e. by the time each update to a MyISAM table is completed, all index updates are written to disk. The only exception is the delayed key writing feature which is disabled by default.

When the global system variable delay_key_write is set to ALL, key buffers for all MyISAM tables are not flushed between updates, so a physical backup of those tables may result in broken MyISAM indexes. To prevent this, LOCK TABLES FOR BACKUP will fail with an error if delay_key_write is set to ALL. An attempt to set delay_key_write to ALL when there’s an active backup lock will also fail with an error.

Another option to involve delayed key writing is to create MyISAM tables with the DELAY_KEY_WRITE option and set the delay_key_write variable to ON (which is the default). In this case, LOCK TABLES FOR BACKUP will not be able to prevent stale index files from appearing in the backup. Users are encouraged to set delay_key_writes to OFF in the configuration file, my.cnf, or repair MyISAM indexes after restoring from a physical backup created with backup locks.

MyISAM may also cache data for bulk inserts, e.g. when executing multi-row INSERTs or LOAD DATA statements. Those caches, however, are flushed between statements, so do not affect physical backups as long as all statements updating MyISAM tables are blocked.

The mysqldump Command

The mysqldump tool has also been extended with a new option, lock-for-backup (disabled by default). When used together with the --single-transaction option, the option makes mysqldump issue a LOCK TABLES FOR BACKUP before starting the dump operation to prevent unsafe statements that would normally result in an inconsistent backup.

When used without the --single-transaction option, lock-for-backup is automatically converted to lock-all-tables.

The option lock-for-backup is mutually exclusive with lock-all-tables, i.e. specifying both on the command line will lead to an error.

If the backup locks feature is not supported by the target server, but lock-for-backup is specified on the command line, mysqldump aborts with an error.

System Variables

have_backup_locks

Option Description
Command Line: Yes
Config file No
Scope: Global
Dynamic: No
Data type Boolean
Default value YES

This is a server variable implemented to help other utilities decide what locking strategy can be implemented for a server. When available, the backup locks feature is supported by the server and the variable value is always YES.

Status variables

Com_lock_tables_for_backup

Option Description
Scope: Global/Session
Data type Numeric

This status variable indicates the number of times the corresponding statements have been executed.

Client command line parameter

lock-for-backup

Option Description
Command Line: Yes
Scope: Global
Dynamic: No
Data type String
Default value Off

When used together with the –-single-transaction option, the option makes mysqldump issue LOCK TABLES FOR BACKUP before starting the dump operation to prevent unsafe statements that would normally result in an inconsistent backup.

Extended mysqldump

Backup locks support

When used together with the –single-transaction option, the lock-for-backup option makes mysqldump issue LOCK TABLES FOR BACKUP before starting the dump operation to prevent unsafe statements that would normally result in an inconsistent backup.

More information can be found in Backup Locks.

Compressed columns support

mysqldump supports the Compressed columns with dictionaries feature.

More information can be found in Compressed columns with dictionaries.

Taking backup by descending primary key order

–order-by-primary-desc tells mysqldump to take the backup by descending primary key order (PRIMARY KEY DESC) which can be useful if the storage engine is using the reverse order column for a primary key.

RocksDB support

mysqldump detects when MyRocks is installed and available. If there is a session variable named rocksdb_skip_fill_cache, mysqldump sets the variable to 1.

mysqldump automatically enables rocksdb_bulk_load if the the target server supports the variable.

Start transaction with consistent snapshot

Percona Server for MySQL has ported MariaDB enhancement for START TRANSACTION WITH CONSISTENT SNAPSHOTS feature to the group commit implementation. This enhancement makes binary log positions consistent with InnoDB transaction snapshots.

This feature obtains logical backups with correct positions without running a FLUSH TABLES WITH READ LOCK. Binary log position can be obtained by two newly implemented status variables: Binlog_snapshot_file and Binlog_snapshot_position. After starting a transaction using the START TRANSACTION WITH CONSISTENT SNAPSHOT, these two variables provide you with the binlog position that corresponds to the state of the database when the consistent snapshot is created and ignores which other transactions have been committed since the snapshot was created.

Snapshot cloning

The Percona Server for MySQL implementation extends the START TRANSACTION WITH CONSISTENT SNAPSHOT syntax with the optional FROM SESSION clause:

START TRANSACTION WITH CONSISTENT SNAPSHOT FROM SESSION <session_id>;

When specified, all participating storage engines and binary log instead of creating a new snapshot of data (or binary log coordinates), create a copy of the snapshot which has been created by an active transaction in the specified session. session_id is the session identifier reported in the Id column of SHOW PROCESSLIST.

Currently snapshot cloning is only supported by XtraDB and the binary log. As with the regular START TRANSACTION WITH CONSISTENT SNAPSHOT, snapshot clones can only be created with the REPEATABLE READ isolation level.

For XtraDB, a transaction with a cloned snapshot will only see data visible or changed by the donor transaction. That is, the cloned transaction will see no changes committed by transactions that started after the donor transaction, not even changes made by itself. Note that in case of chained cloning the donor transaction is the first one in the chain. For example, if transaction A is cloned into transaction B, which is in turn cloned into transaction C, the latter will have read view from transaction A (i.e., the donor transaction). Therefore, it will see changes made by transaction A, but not by transaction B.

mysqldump

mysqldump has been updated to use new status variables automatically when they are supported by the server and both –single-transaction and –source-data are specified on the command line. Along with the mysqldump improvements introduced in Backup Locks there is now a way to generate mysqldump backups that are guaranteed to be consistent without using FLUSH TABLES WITH READ LOCK even if --source-data is requested.

System variables

have_snapshot_cloning

Option Description
Command Line: Yes
Config file No
Scope: Global
Dynamic: No
Data type Boolean

This server variable is implemented to help other utilities detect if the server supports the FROM SESSION extension. When available, the snapshot cloning feature and the syntax extension to START TRANSACTION WITH CONSISTENT SNAPSHOT are supported by the server, and the variable value is always YES.

Status variables

Binlog_snapshot_file

Option Description
Scope: Global
Data type String

Binlog_snapshot_position

Option Description
Scope: Global
Data type Numeric

These status variables are only available when the binary log is enabled globally.

Secure

LDAP features

Using LDAP authentication plugins

Lightweight Directory Access Protocol (LDAP) provides an alternative method to access existing directory servers. These servers maintain information about individuals, groups, and organizations.

Percona Server for MySQL supports two LDAP authentication plugins:

Authentication method Description
Simple LDAP authentication The Percona plugin is a free and Open Source implementation of the MySQL Enterprise Simple LDAP authentication plugin.
Simple Authentication and Security Layer (SASL)-based LDAP authentication Supports the SCRAM-SHA-1 SASL mechanism only.

For all LDAP plugin system variables, see LDAP authentication plugin system variables.

Choose an authentication method

Both plugins authenticate users against an LDAP server. Both plugins support LDAP group to MySQL role mapping. The main difference is how the client sends the password to the database server.

Factor Simple LDAP authentication SASL-based LDAP authentication
Password on the wire to the database server Cleartext SASL-protected
Client plugin mysql_clear_password authentication_ldap_sasl_client
Client requirement Enable --enable-cleartext-plugin Install the SASL client plugin library on the client host
Connection security Require TLS between the client and the database server TLS recommended; password stays off the wire to the database server
LDAP server requirement Standard LDAP bind LDAP server must support SASL with the SCRAM-SHA-1 mechanism
Additional dependencies OpenLDAP client library on the database server SASL server, SASL client library on the client host, SCRAM-SHA-1 configuration

Choose simple LDAP authentication when you can require TLS on the client connection and you want fewer dependencies.

Choose SASL-based LDAP authentication when the password must stay off the wire to the database server and your LDAP environment supports SCRAM-SHA-1.

Plugin names and file names

The plugins require manual installation. The database server does not load them by default.

Each method uses these plugin and file names:

Plugin or file Plugin name or file name
Server-side plugin authentication_ldap_simple
Client-side plugin mysql_clear_password
Library file authentication_ldap_simple.so
Plugin or file Plugin name or file name
Server-side plugin authentication_ldap_sasl
Client-side plugin authentication_ldap_sasl_client
Library files authentication_ldap_sasl.so
authentication_ldap_sasl_client.so

Install authentication_ldap_sasl_client.so on each client host that uses SASL-based LDAP authentication. Place the library in the MySQL client plugin directory. The mysql client loads the plugin when you connect to an account that uses authentication_ldap_sasl.

Prerequisites for authentication

The LDAP authentication plugins require the following:

  • Run an LDAP server.

  • Add user accounts to the LDAP server for authentication.

  • Install the OpenLDAP client library on the same system as the plugin.

The SASL-based LDAP authentication additionally requires the following:

  • Configure the LDAP server to communicate with a SASL server.

  • Install the SASL client library on the same system as the client plugin.

  • Configure services to use the SCRAM-SHA-1 SASL mechanism.

Install the plugins

Install the plugins at server start or at runtime.

Load the plugins at server start

Add these statements to my.cnf to load simple LDAP authentication:

[mysqld]
plugin-load-add=authentication_ldap_simple.so
authentication_ldap_simple_server_host=127.0.0.1
authentication_ldap_simple_bind_base_dn='dc=percona, dc=com'

Restart the database server for the changes to take effect.

Add these statements to my.cnf to load SASL-based LDAP authentication:

[mysqld]
plugin-load-add=authentication_ldap_sasl.so
authentication_ldap_sasl_server_host=127.0.0.1
authentication_ldap_sasl_bind_base_dn='dc=percona, dc=com'

Load the plugins at runtime

INSTALL PLUGIN authentication_ldap_simple SONAME 'authentication_ldap_simple.so';

Set and persist values at runtime:

SET PERSIST authentication_ldap_simple_server_host='127.0.0.1';
SET PERSIST authentication_ldap_simple_bind_base_dn='dc=percona, dc=com';

INSTALL PLUGIN authentication_ldap_sasl SONAME 'authentication_ldap_sasl.so';

Set and persist values at runtime:

SET PERSIST authentication_ldap_sasl_server_host='127.0.0.1';
SET PERSIST authentication_ldap_sasl_bind_base_dn='dc=percona, dc=com';

Verify the plugin

Confirm the plugin is loaded

After you install a plugin, confirm that the database server loaded it and set the status to ACTIVE.

Run this statement:

SHOW PLUGINS;

Or query the INFORMATION_SCHEMA.PLUGINS table:

SELECT PLUGIN_NAME, PLUGIN_STATUS
FROM INFORMATION_SCHEMA.PLUGINS
WHERE PLUGIN_NAME = 'authentication_ldap_simple';
Expected output 
+----------------------------+----------------+
| PLUGIN_NAME                | PLUGIN_STATUS  |
+----------------------------+----------------+
| authentication_ldap_simple | ACTIVE         |
+----------------------------+----------------+

SELECT PLUGIN_NAME, PLUGIN_STATUS
FROM INFORMATION_SCHEMA.PLUGINS
WHERE PLUGIN_NAME = 'authentication_ldap_sasl';
Expected output 
+--------------------------+----------------+
| PLUGIN_NAME              | PLUGIN_STATUS  |
+--------------------------+----------------+
| authentication_ldap_sasl | ACTIVE         |
+--------------------------+----------------+

Check the server error log if the plugin is missing or the status is not ACTIVE.

Confirm authentication works

Create an LDAP user account. Connect with the mysql client as shown in Examples. A successful connection confirms LDAP authentication.

Configure the LDAP connection

Configure the database server to LDAP server connection separately from the client to database server connection. TLS on the client connection does not encrypt traffic between the database server and the LDAP server.

This configuration example uses simple LDAP authentication. For SASL-based LDAP authentication, use the matching authentication_ldap_sasl_* variables.

[mysqld]
authentication_ldap_simple_server_host=ldap.example.com
authentication_ldap_simple_server_port=636
authentication_ldap_simple_bind_base_dn='dc=example,dc=com'
authentication_ldap_simple_user_search_attr=uid
authentication_ldap_simple_bind_root_dn='cn=ldapadmin,ou=services,dc=example,dc=com'
authentication_ldap_simple_bind_root_pwd='LDAP_BIND_PASSWORD'
authentication_ldap_simple_ssl=ON
authentication_ldap_simple_ca_path=/etc/mysql/ldap-ca.pem

The database server uses these settings as follows:

Variable Purpose
authentication_ldap_simple_server_host, authentication_ldap_simple_server_port LDAP server address. Port 636 is typical for LDAPS.
authentication_ldap_simple_bind_base_dn Base distinguished name (DN) for LDAP searches.
authentication_ldap_simple_user_search_attr LDAP attribute that holds the user name. Use uid for OpenLDAP. Use sAMAccountName for Active Directory.
authentication_ldap_simple_bind_root_dn, authentication_ldap_simple_bind_root_pwd Credentials the plugin uses to search LDAP for users and groups.
authentication_ldap_simple_ssl Enable LDAPS (ldaps://). Use authentication_ldap_simple_tls for STARTTLS on port 389.
authentication_ldap_simple_ca_path Certificate authority file for LDAPS or STARTTLS.

For group membership and role mapping, configure authentication_ldap_simple_group_search_attr and authentication_ldap_simple_group_search_filter when the default cn attribute or filter does not match your directory layout.

To configure a fallback LDAP server, set authentication_ldap_simple_fallback_server_host and authentication_ldap_simple_fallback_server_port. The plugin connects to the fallback server when the primary server is unavailable.

Active Directory

For Active Directory, use a configuration like this:

[mysqld]
authentication_ldap_simple_server_host=ad.example.com
authentication_ldap_simple_server_port=636
authentication_ldap_simple_bind_base_dn='dc=example,dc=com'
authentication_ldap_simple_user_search_attr=sAMAccountName
authentication_ldap_simple_bind_root_dn='cn=ldapbind,ou=service accounts,dc=example,dc=com'
authentication_ldap_simple_bind_root_pwd='LDAP_BIND_PASSWORD'
authentication_ldap_simple_ssl=ON

Match the LDAP group name in authentication_ldap_simple_group_role_mapping to the group name returned from Active Directory. Adjust authentication_ldap_simple_group_search_attr when your directory stores group names in an attribute other than cn.

How authentication works

Each server-side LDAP plugin requires a specific client-side plugin.

The database server uses authentication_ldap_simple. The client uses mysql_clear_password. The client sends the password to the database server as cleartext. Require a secure connection between the client and the database server.

The database server uses authentication_ldap_sasl. The client uses authentication_ldap_sasl_client. The client sends the password through SASL messages within the LDAP protocol. The password stays off the wire to the database server.

The database server rejects the connection when the client user name and host name do not match a server account.

The authentication flow is as follows:

  1. The client connects with a user name and password.

  2. The database server finds the matching MySQL account.

  3. The LDAP server searches for the user entry.

  4. The LDAP server validates the password.

  5. The authentication plugin determines the authenticated user name from the LDAP entry.

  6. The database server checks privileges for the client user name or the authenticated user name.

The LDAP server locates the user entry in one of two ways:

  • The MySQL account includes an authentication string with the user DN. Add the DN with IDENTIFIED WITH <plugin-name> BY '<auth-string>' or AS '<auth-string>'. Authentication fails when the DN or password is wrong.

  • The MySQL account has no authentication string. The database server uses the client user name and the configured search attribute to find the LDAP entry.

Authentication fails when the LDAP server finds no match or multiple matches.

When the LDAP entry has no group attribute, the plugin returns the client user name as the authenticated name. When the LDAP entry has a group attribute, the plugin returns the group value as the authenticated name.

When the client user name and authenticated user name match, the database server checks privileges for the client user name. When the names differ, the database server looks for an account that matches the authenticated name.

External roles

When an LDAP user logs in, the database server checks LDAP group membership.

When the user belongs to a configured group, the database server grants the mapped database roles.

Configure the LDAP group to MySQL role mapping with authentication_ldap_simple_group_role_mapping or authentication_ldap_sasl_group_role_mapping.

The mapping value is a plain comma-separated string. Use <ldap_group>=<mysql_role> pairs:

<ldap_group>=<mysql_role>,<ldap_group2>=<mysql_role2>,

The LDAP group name must match the group name from LDAP. By default, the database server reads the group name from the cn attribute. The MySQL role must exist before login.

When the user belongs to multiple mapped groups, the database server grants each matching role.

Group membership lookup requires authentication_ldap_simple_bind_root_dn and authentication_ldap_simple_bind_root_pwd, or the SASL equivalents. See Map LDAP groups to MySQL roles.

The database server applies external roles as follows:

  • Grant or revoke external role privileges on each user connection.

  • Read group membership from the LDAP group attribute.

  • Check LDAP group membership only at user login.

  • Revoke privileges on the next connection after group removal.

External roles and LDAP proxy mapping

Percona Server for MySQL supports two LDAP authorization models:

Model Configuration Result
External roles Global authentication_ldap_*_group_role_mapping variable The database server grants MySQL roles to the connecting user
LDAP proxy mapping Authentication string with #group=proxied_user syntax on a MySQL account The connecting user acts as a different MySQL account

External roles grant privileges through MySQL roles. LDAP proxy mapping switches the session to another MySQL user account.

Choose external roles when LDAP groups should map directly to MySQL roles. Choose LDAP proxy mapping when LDAP groups should map to existing MySQL user accounts.

Create a user using simple LDAP authentication

Use one of these methods to add or modify a user.

Specify the authentication_ldap_simple plugin in the IDENTIFIED WITH clause of a CREATE USER or ALTER USER statement:

CREATE USER ... IDENTIFIED WITH authentication_ldap_simple;

The database server assigns the specified plugin to the account.

An optional authentication string stores the LDAP user DN. The string uses the format cn,ou,dc,dc:

CREATE USER ... IDENTIFIED WITH authentication_ldap_simple BY 'cn=[user name],ou=[organization unit],dc=[domain component],dc=com'

When the account includes an authentication string, the database server skips these system variables unless authentication_ldap_simple_group_role_mapping is set:

When the account has no authentication string, the database server uses these system variables.

When authentication_ldap_simple_group_role_mapping is set, the database server also requires authentication_ldap_simple_bind_root_dn and authentication_ldap_simple_bind_root_pwd.

Create a user using SASL-based LDAP authentication

Use one of these methods to add or modify a user.

Specify the authentication_ldap_sasl plugin in the IDENTIFIED WITH clause of a CREATE USER or ALTER USER statement:

CREATE USER ... IDENTIFIED WITH authentication_ldap_sasl;

An optional authentication string stores the LDAP user DN. The string uses the format cn,ou,dc,dc. You can use BY or AS for the authentication string:

CREATE USER ... IDENTIFIED WITH authentication_ldap_sasl BY 'cn=[user name],ou=[organization unit],dc=[domain component],dc=com'

When the account includes an authentication string, the database server skips these system variables unless authentication_ldap_sasl_group_role_mapping is set:

When the account has no authentication string, the database server uses these system variables.

When authentication_ldap_sasl_group_role_mapping is set, the database server also requires authentication_ldap_sasl_bind_root_dn and authentication_ldap_sasl_bind_root_pwd.

Examples

These examples configure an LDAP user and verify authentication with a client connection.

The sample LDAP user DN is:

uid=ldapuser,ou=testusers,dc=percona,dc=com

Create a database server account for ldapuser:

CREATE USER 'ldapuser'@'localhost' IDENTIFIED WITH authentication_ldap_simple BY 'uid=ldapuser,ou=testusers,dc=percona,dc=com';

The authentication string does not include the LDAP password. The client user must provide the LDAP password at connect time.

mysql --user=ldapuser --password --enable-cleartext-plugin

Enter the LDAP password for ldapuser when prompted. A successful connection confirms that LDAP authentication works.

The client sends the password as cleartext. This behavior is required for a server-side LDAP library without SASL. To reduce risk:

  • Require clients to enable the mysql_clear_password plugin with --enable-cleartext-plugin.

  • Require clients to connect over an encrypted connection to the database server.

Create a database server account for ldapuser:

CREATE USER 'ldapuser'@'localhost' IDENTIFIED WITH authentication_ldap_sasl AS 'uid=ldapuser,ou=testusers,dc=percona,dc=com';

The authentication string does not include the LDAP password. The client user must provide the LDAP password at connect time.

mysql --user=ldapuser --password

Enter the LDAP password for ldapuser when prompted. A successful connection confirms that LDAP authentication works.

The client and database server SASL LDAP plugins exchange SASL messages within the LDAP protocol.

Map LDAP groups to MySQL roles

This example maps two LDAP groups to MySQL roles. The LDAP groups have a cn value of mysql-admins and mysql-readers. The example uses simple LDAP authentication. For SASL-based LDAP authentication, use the authentication_ldap_sasl_* variables with the same mapping string format.

Create the MySQL roles and grant privileges:

CREATE ROLE 'mysql_admins', 'mysql_readers';
GRANT SELECT, INSERT, UPDATE, DELETE ON app_db.* TO 'mysql_admins';
GRANT SELECT ON app_db.* TO 'mysql_readers';

Set the mapping and LDAP bind credentials at server start in my.cnf:

[mysqld]
authentication_ldap_simple_group_role_mapping='mysql-admins=mysql_admins,mysql-readers=mysql_readers,'
authentication_ldap_simple_bind_root_dn='cn=ldapadmin,ou=services,dc=percona,dc=com'
authentication_ldap_simple_bind_root_pwd='LDAP_BIND_PASSWORD'

Set the values at runtime instead:

SET PERSIST authentication_ldap_simple_group_role_mapping = 'mysql-admins=mysql_admins,mysql-readers=mysql_readers,';
SET PERSIST authentication_ldap_simple_bind_root_dn = 'cn=ldapadmin,ou=services,dc=percona,dc=com';
SET PERSIST authentication_ldap_simple_bind_root_pwd = 'LDAP_BIND_PASSWORD';

Create an LDAP user account:

CREATE USER 'ldapuser'@'%' IDENTIFIED WITH authentication_ldap_simple;

Connect as ldapuser with the LDAP password. When ldapuser belongs to the mysql-admins LDAP group, the database server grants the mysql_admins role on connection. Verify the role assignment:

SHOW GRANTS;
SELECT CURRENT_ROLE();

When the role appears in the output but privileges are missing, activate the role:

SET ROLE 'mysql_admins';
SHOW GRANTS;

To activate all granted roles at login, enable activate_all_roles_on_login .

Troubleshoot authentication

When LDAP authentication fails, check these items:

  • Confirm the plugin status is ACTIVE. See Confirm the plugin is loaded.

  • Review the server error log.

  • Increase the logging level with authentication_ldap_simple_log_status or authentication_ldap_sasl_log_status. Valid values range from 1 to 6. Higher values write more detail to the error log.

  • Confirm the database server can reach the LDAP server on the configured host and port.

  • Confirm authentication_ldap_*_bind_root_dn and authentication_ldap_*_bind_root_pwd are correct when the plugin searches LDAP for users or groups.

  • Confirm authentication_ldap_*_user_search_attr matches your directory. OpenLDAP directories often use uid. Active Directory directories often use sAMAccountName.

  • Confirm the LDAP group name in authentication_ldap_*_group_role_mapping matches the group name returned from LDAP.

  • Confirm each mapped MySQL role exists and has the expected privileges before login.

  • For simple LDAP authentication, confirm the client uses --enable-cleartext-plugin and connects over TLS.

  • For SASL-based LDAP authentication, confirm authentication_ldap_sasl_client.so is installed on the client host.

Uninstall the plugins

When you installed either plugin at server startup, remove those options from the my.cnf file. Remove any startup options that set LDAP system variables. Restart the database server.

When you installed the plugin at runtime, run these statements:

UNINSTALL PLUGIN authentication_ldap_simple;

When you used SET PERSIST, run RESET PERSIST to remove the settings.

When you installed the plugin at runtime, run these statements:

UNINSTALL PLUGIN authentication_ldap_sasl;

When you used SET PERSIST, run RESET PERSIST to remove the settings.

LDAP authentication plugin system variables

Authentication system variables

The installation adds the following variables:

Variable name Description
authentication_ldap_sasl_bind_base_dn Base distinguished name
authentication_ldap_sasl_bind_root_dn Root distinguished name
authentication_ldap_sasl_bind_root_dn_pwd Password for the root distinguished name
authentication_ldap_sasl_ca_path Absolute path of the certificate authority
authentication_ldap_sasl_fallback_server_host If the primary server is unavailable, the authentication plugin attempts to connect to the fallback server
authentication_ldap_sasl_fallback_server_port The port number for the fallback server
authentication_ldap_sasl_group_role_mapping A list of LDAP group names - MySQL role pairs
authentication_ldap_sasl_group_search_attr Name of the attribute that specifies the group names in the LDAP directory entries
authentication_ldap_sasl_group_search_filter Custom group search filter
authentication_ldap_sasl_init_pool_size Initial size of the connection pool to the LDAP server
authentication_ldap_sasl_log_status logging level
authentication_ldap_sasl_max_pool_size Maximum size of the pool of connections to the LDAP server
authentication_ldap_sasl_server_host LDAP server host
authentication_ldap_sasl_server_port LDAP server TCP/IP port number
authentication_ldap_sasl_ssl If plugin connections to the LDAP server use the SSL protocol (ldaps://)
authentication_ldap_sasl_tls If plugin connections to the LDAP server are secured with STARTTLS (ldap://)
authentication_ldap_sasl_user_search_attr Name of the attribute that specifies user names in the LDAP directory entries
authentication_ldap_simple_bind_base_dn Base distinguished name
authentication_ldap_simple_bind_root_dn Root distinguished name
authentication_ldap_simple_bind_root_dn_pwd Password for the root distinguished name
authentication_ldap_simple_ca_path Absolute path of the certificate authority
authentication_ldap_simple_fallback_server_host If the primary server is unavailable, the authentication plugin attempts to connect to the fallback server
authentication_ldap_simple_fallback_server_port The port number for the fallback server
authentication_ldap_simple_group_role_mapping A list of LDAP group names - MySQL role pairs
authentication_ldap_simple_group_search_attr Name of the attribute that specifies the group names in the LDAP directory entries
authentication_ldap_simple_group_search_filter Custom group search filter
authentication_ldap_simple_init_pool_size Initial size of the connection pool to the LDAP server
authentication_ldap_simple_log_status logging level
authentication_ldap_simple_max_pool_size Maximum size of the pool of connections to the LDAP server
authentication_ldap_simple_server_host LDAP server host
authentication_ldap_simple_server_port LDAP server TCP/IP port number
authentication_ldap_simple_ssl If plugin connections to the LDAP server use the SSL protocol (ldaps://)
authentication_ldap_simple_tls If plugin connections to the LDAP server are secured with STARTTLS (ldap://)
authentication_ldap_simple_user_search_attr Name of the attribute that specifies user names in the LDAP directory entries

The following variables are described in detail:

authentication_ldap_sasl_bind_base_dn

Option Description
Command-line –authentication-ldap-sasl-bind-base-dn=value
Scope Global
Dynamic Yes
Data type String
Default NULL

The base distinguished name (DN) for SASL-based LDAP authentication. You can limit the search scope by using the variable as the base of the search.

authentication_ldap_sasl_bind_root_dn

Option Description
Command-line –authentication-ldap-sasl-bind-root-dn=value
Scope Global
Dynamic Yes
Data type String
Default NULL

The root distiguished name (DN) used to authenticate SASL-based LDAP. When performing a search, this variable is used with authentication_ldap_sasl_bind_root_pwd as the authenticating credentials to the LDAP server.

authentication_ldap_sasl_bind_root_pwd

Option Description
Command-line –authentication-ldap-sasl-bind-root-pwd=value
Scope Global
Dynamic Yes
Data type String
Default NULL

The root password used to authenticate against SASL-based LDAP server. This variable is used with authentication_ldap_sasl_bind_root_dn.

authentication_ldap_sasl_ca_path

Option Description
Command-line –authentication-ldap-sasl-ca_path=value
Scope Global
Dynamic Yes
Data type String
Default NULL

The certificate authority’s absolute path used to verify the LDAP certificate.

authentication_ldap_sasl_fallback_server_host

Option Description
Command-line –authentication-ldap-sasl-fallback-server-host
Scope Global
Dynamic Yes
Type Sting
Default NULL

Use with authentication_ldap_sasl_fallback_server_port.

If the primary server is unavailable, the authentication plugin attempts to connect to the fallback server and authenticate using that server.

authentication_ldap_sasl_fallback_server_port

Option Description
Command-line –authentication-ldap-sasl-fallback-server-port
Scope Global
Dynamic Yes
Type Integer
Default NULL

Use with authentication_ldap_sasl_fallback_server_host.

If the primary server is unavailable, the authentication plugin attempts to connect to the fallback server and authenticate using that server.

If the fallback server host has a value, and the fallback port is 0, users can specify multiple fallback servers.

Use this format to specify multiple fallback servers: authentication_ldap_sasl_fallback_server_host="ldap(s)://host:port,ldap(s)://host2:port2, for example.

authentication_ldap_sasl_group_role_mapping

Option Description
Command-line –authentication-ldap-sasl-group-role-mapping=value
Scope Global
Dynamic Yes
Data type String
Default Null

When an LDAP user logs in, the server checks if the LDAP user is a member of the specified group. If the user is, then the server automatically grants the database server roles to the user.

The variable has this format: <ldap_group>=<mysql_role>,<ldap_group2>=<mysql_role2>,.

authentication_ldap_sasl_group_search_attr

Option Description
Command-line –authentication-ldap-sasl-group-search-attr=value
Scope Global
Dynamic Yes
Data type String
Default cn

The attribute name that specifies group names in the LDAP directory entries for SASL-based LDAP authentication.

authentication_ldap_sasl_group_search_filter

Option Description
Command-line –authentication-ldap-sasl-group-search-filter=value
Scope Global
Dynamic Yes
Data type String
Default (|(&(objectClass=posixGroup)(memberUid=%s))(&(objectClass=group)(member=%s)))

The custom group search filter for SASL-based LDAP authentication.

authentication_ldap_sasl_init_pool_size

Option Description
Command-line –authentication-ldap-sasl-init-pool-size=value
Scope Global
Dynamic Yes
Data type Integer
Default 10
Minimum value 0
Maximum value 32767
Unit connections

The initial size of the connection pool to the LDAP server for SASL-based LDAP authentication.

authentication_ldap_sasl_log_status

Option Description
Command-line –authentication-ldap-sasl-log-status=value
Scope Global
Dynamic Yes
Data type Integer
Default 1
Minimum value 1
Maximum value 6

The logging level for messages written to the error log for SASL-based LDAP authentication.

authentication_ldap_sasl_max_pool_size

Option Description
Command-line –authentication-ldap-sasl-max-pool-size=value
Scope Global
Dynamic Yes
Data type Integer
Default 1000
Minimum value 0
Maximum value 32767
Unit connections

The maximum connection pool size to the LDAP server in SASL-based LDAP authentication. The variable is used with authentication_ldap_sasl_init_pool_size.

authentication_ldap_sasl_server_host

Option Description
Command-line –authentication-ldap-sasl-server-host=value
Scope Global
Dynamic Yes
Data type String
Default NULL

The LDAP server host used for SASL-based LDAP authentication. The LDAP server host can be an IP address or a host name.

authentication_ldap_sasl_server_port

Option Description
Command-line –authentication-ldap-sasl-server-port=value
Scope Global
Dynamic Yes
Data type Integer
Default 389
Minimum value 1
Maximum value 32376

The LDAP server TCP/IP port number used for SASL-based LDAP authentication.

authentication_ldap_sasl_ssl

Option Description
Command-line –authentication-ldap-sasl-ssl=value
Scope Global
Dynamic Yes
Data type Boolean
Default OFF

If this variable is enabled, the plugin connects to the server with SSL.

authentication_ldap_sasl_tls

Option Description
Command-line –authentication-ldap-sasl-tls=value
Scope Global
Dynamic Yes
Data type Boolean
Default OFF

If this variable is enabled, the plugin connects to the server with TLS.

authentication_ldap_sasl_user_search_attr

Option Description
Command-line –authentication-ldap-sasl-user-search-attr=value
Scope Global
Dynamic Yes
Data type String
Default uid

The attribute name that specifies the user names in LDAP directory entries in SASL-based LDAP authentication.

authentication_ldap_simple_bind_base_dn

Option Description
Command-line –authentication-ldap-simple-bind-base-dn=value
Scope Global
Dynamic Yes
Data type String
Default NULL

The base distinguished name (DN) for simple LDAP authentication. You can limit the search scope by using the variable as the base of the search.

authentication_ldap_simple_bind_root_dn

Option Description
Command-line –authentication-ldap-simple-bind-root-dn=value
Scope Global
Dynamic Yes
Data type String
Default NULL

The root distinguished name (DN) used to authenticate simple LDAP. When performing a search, this variable is used with authentication_ldap_simple_bind_root_pwd as the authenticating credentials to the LDAP server.

authentication_ldap_simple_bind_root_pwd

Option Description
Command-line –authentication-ldap-simple-bind-root-pwd=value
Scope Global
Dynamic Yes
Data type String
Default NULL

The root password used to authenticate against simple LDAP server. This variable is used with authentication_ldap_simple_bind_root_dn.

authentication_ldap_simple_ca_path

Option Description
Command-line –authentication-ldap-simple-ca_path=value
Scope Global
Dynamic Yes
Data type String
Default NULL

The certificate authority’s absolute path used to verify the LDAP certificate.

authentication_ldap_simple_fallback_server_host

Option Description
Command-line –authentication-ldap-simple-fallback-server-host
Scope Global
Dynamic Yes
Type Sting
Default NULL

Use with authentication_ldap_simple_fallback_server_port.

If the primary server is unavailable, the authentication plugin attempts to connect to the fallback server and authenticate using that server.

authentication_ldap_simple_fallback_server_port

Option Description
Command-line –authentication-ldap-simple-fallback-server-port
Scope Global
Dynamic Yes
Type Integer
Default NULL

Use with authentication_ldap_simple_fallback_server_host.

If the primary server is unavailable, the authentication plugin attempts to connect to the fallback server and authenticate using that server.

If the fallback server host has a value, and the fallback port is 0, users can specify multiple fallback servers.

Use this format to specify multiple fallback servers: authentication_ldap_simple_fallback_server_host="ldap(s)://host:port,ldap(s)://host2:port2, for example.

authentication_ldap_simple_group_role_mapping

Option Description
Command-line –authentication-ldap-simple-group-role-mapping=value
Scope Global
Dynamic Yes
Data type String
Default Null

When an LDAP user logs in, the server checks if the LDAP user is a member of the specified group. If the user is, then the server automatically grants the database server roles to the user.

The variable has this format: <ldap_group>=<mysql_role>,<ldap_group2>=<mysql_role2>,.

authentication_ldap_simple_group_search_attr

Option Description
Command-line –authentication-ldap-simple-group-search-attr=value
Scope Global
Dynamic Yes
Data type String
Default cn

The attribute name that specifies group names in the LDAP directory entries for simple LDAP authentication.

authentication_ldap_simple_group_search_filter

Option Description
Command-line –authentication-ldap-simple-group-search-filter=value
Scope Global
Dynamic Yes
Data type String
Default (|(&(objectClass=posixGroup)(memberUid=%s))(&(objectClass=group)(member=%s)))

The custom group search filter for simple LDAP authentication.

authentication_ldap_simple_init_pool_size

Option Description
Command-line –authentication-ldap-simple-init-pool-size=value
Scope Global
Dynamic Yes
Data type Integer
Default 10
Minimum value 0
Maximum value 32767
Unit connections

The initial size of the connection pool to the LDAP server for simple LDAP authentication.

authentication_ldap_simple_log_status

Option Description
Command-line –authentication-ldap-simple-log-status=value
Scope Global
Dynamic Yes
Data type Integer
Default 1
Minimum value 1
Maximum value 6

The logging level for messages written to the error log for simple LDAP authentication.

authentication_ldap_simple_max_pool_size

Option Description
Command-line –authentication-ldap-simple-max-pool-size=value
Scope Global
Dynamic Yes
Data type Integer
Default 1000
Minimum value 0
Maximum value 32767
Unit connections

The maximum connection pool size to the LDAP server in simple LDAP authentication. The variable is used with authentication_ldap_simple_init_pool_size.

authentication_ldap_simple_server_host

Option Description
Command-line –authentication-ldap-simple-server-host=value
Scope Global
Dynamic Yes
Data type String
Default NULL

The LDAP server host used for simple LDAP authentication. The LDAP server host can be an IP address or a host name.

authentication_ldap_simple_server_port

Option Description
Command-line –authentication-ldap-simple-server-port=value
Scope Global
Dynamic Yes
Data type Integer
Default 389
Minimum value 1
Maximum value 32376

The LDAP server TCP/IP port number used for simple LDAP authentication.

authentication_ldap_simple_ssl

Option Description
Command-line –authentication-ldap-simple-ssl=value
Scope Global
Dynamic Yes
Data type Boolean
Default OFF

If this variable is enabled, the plugin connects to the server with SSL.

authentication_ldap_simple_tls

Option Description
Command-line –authentication-ldap-simple-tls=value
Scope Global
Dynamic Yes
Data type Boolean
Default OFF

If this variable is enabled, the plugin connects to the server with TLS.

authentication_ldap_simple_user_search_attr

Option Description
Command-line –authentication-ldap-simple-user-search-attr=value
Scope Global
Dynamic Yes
Data type String
Default uid

The attribute name that specifies the user names in LDAP directory entries in simple LDAP authentication.

Data masking

Data masking overview

Data masking protects sensitive information by changing or replacing values so that readers see altered data instead of the originals. Use masking when you need to present, demonstrate, or test software without revealing real data. Masking keeps a similar data format, which reduces the risk of recovering originals.

Masking is a data transformation, not a substitute for access control. It works alongside MySQL privileges (RBAC—role-based access control). RBAC is MySQL’s privilege system: it determines which users and roles can read, write, or administer tables. Users who have permission to read the table see the unmasked result; the component does not by itself restrict access or prevent privilege escalation. You still need to grant only the minimum required privileges so that only authorized users can access the data at all. For small value domains (for example, gender or state codes), inference or frequency analysis may still reveal or narrow down originals. Treat masking as one layer of protection, not a guarantee of irreversibility.

Typical use cases:

Use case Description
Development and testing Supply non-production environments with masked copies of sensitive fields (for example, payment card numbers, Social Security numbers, addresses) so applications can be validated without exposing real data.
Compliance (GDPR, HIPAA, CCPA, and similar) Anonymize or pseudonymize personal data so that analysis and reporting can use the data while meeting regulatory requirements for protection of personal information.
Sharing data with third parties Provide vendors or partners with datasets where sensitive columns are masked so that accurate personal details are not exposed.
Customer service and training Give support or training staff access to data that looks realistic but does not contain real customer identifiers or PII.
Analysis and reporting on masked data Run queries and reports on privacy-safe data. Generated values do not guarantee the same statistical distribution as originals; use for privacy-safe environments or design generation to preserve distributions where needed.

Use masking as part of a broader data-protection and access-control strategy, not as the only safeguard.

The component offers two kinds of functions: those that mask existing values (for example, replace digits with a character) and those that generate replacement values (for example, random SSNs or emails). See Data masking component functions for the full catalog and to find a function by task.

Next steps: Install the data masking component, then follow the Data masking quickstart to create a test database and try masking. The quickstart covers required privileges and working examples.

Version updates

Percona Server for MySQL 8.4.4-4 adds an internal term cache that speeds up dictionary lookups. The cache affects gen_blocklist() and gen_dictionary(). Lookups are faster than querying the dictionary table each time, especially when processing many rows.

Memory use grows with the size of your dictionaries. For very large dictionaries, consider server memory and monitor resource use.

Cache and table can get out of sync if you change the dictionary table directly instead of using the management functions. Use the management functions to add or remove terms:

If the cache and table do get out of sync, call masking_dictionaries_flush() to resync them. The function returns 1 when successful.

Replication: Dictionary changes on the source are written to the binary log and applied to the replica’s table, but the replica’s term cache does not update immediately. During that lag, queries on the replica can see stale or inconsistent dictionary data. In sensitive or high-traffic setups, unmasked or incorrectly masked data may be visible until the cache is refreshed.

To reduce that risk, set the component_masking_functions.dictionaries_flush_interval_seconds variable to a positive value (for example, 60). A background process then refreshes the cache at that interval so replicas stay in sync. The default is 0 (no automatic refresh).

Limitations and security considerations

This component is a data-presentation tool, not a data-redaction tool. The component changes how data is shown in query results; the component does not redact stored data or enforce a full secure data lifecycle.

Referential integrity: The component does not enforce consistency across tables. The same logical identifier (for example, a customer ID) can be masked to different values in different tables. If you need consistent masked identities for joins or testing, use deterministic masking so the same input always maps to the same output (for example, gen_blocklist() with the same dictionaries, or a single mapping in application logic).

Access control: Masking in a SELECT only protects that query. Users with SELECT on the base table can bypass masking by querying the table directly. Restrict access to views or stored procedures that apply the masking functions.

Views are not a complete boundary. Users with SHOW CREATE VIEW can see underlying table and column names; users with FILE or other privileges may read data outside the SQL layer. Secure the rest of the stack separately. Do not grant UPDATE or INSERT to users who should only see masked data; writing masked values back into the database corrupts real data. Treat masking as read-only presentation and control write access separately.

Partial masking: Functions that leave part of a value visible (for example, last four digits) can allow re-identification when combined with other data. Consider k-anonymity and your threat model.

Logs (slow query log, general query log, binary logs) may record query text or parameters. Restrict log access or redact as required. The component transforms values at query time; stored data is not altered. Behavior for empty strings or malformed identifiers is not fully specified; poor data quality can undermine masking.

Granting privileges to mysql.session on the dictionary table has security implications; the documentation does not prove the absence of privilege escalation. Regional functions (for example, mask_canada_sin, mask_uk_nin, mask_iban) are not documented as compliant with current regulatory or format requirements in those jurisdictions; verify for your use case.

Masking is applied per query, so the same column can appear differently in different contexts. Use views or standardized patterns for a consistent policy. For very large dictionaries, performance and memory use depend on size and workload; see the function list for cache behavior and operational notes.

Additional resources

Install the data masking component

Prerequisites

  • Percona Server for MySQL with the data masking component available for your version.
  • The component_masking_functions library must be present in the server plugin directory. The plugin_dir system variable defines that directory; the server resolves file://component_masking_functions in INSTALL COMPONENT relative to plugin_dir. To verify the library is available, check that the file exists in the directory reported by: 
    SHOW VARIABLES LIKE 'plugin_dir';
    If the library is missing, INSTALL COMPONENT fails at load time; check the server error log and your installation package or deployment to ensure the component library is installed in the plugin directory.

The component has the following parts:

  • A system table, mysql.masking_dictionaries, used to store terms and dictionaries (you must create this table; see below).
  • The loadable component component_masking_functions, which provides the masking functions.

The MASKING_DICTIONARIES_ADMIN privilege is required for dictionary management functions; the privilege is registered when the component is loaded.

Install the component

Follow the steps in order. The table must exist and match the required schema before you run INSTALL COMPONENT; the component does not create the table. If the table is missing or the schema is wrong (for example, a typo in column names or types), the component can load in a broken state or fail.

  1. Create the masking_dictionaries table in the mysql schema.

    Ensure the statement completes and is committed before you run step 2. The schema below is required; do not alter column names or types unless a future Percona Server release documents a different schema. You are responsible for creating and maintaining this table; if the component’s expected schema changes in an upgrade, release or upgrade documentation will describe any required ALTER TABLE or migration that you must apply.

    CREATE TABLE IF NOT EXISTS
mysql.masking_dictionaries(
    Dictionary VARCHAR(256) NOT NULL,
    Term VARCHAR(256) NOT NULL,
    UNIQUE INDEX dictionary_term_idx (Dictionary, Term)
) ENGINE = InnoDB DEFAULT CHARSET=utf8mb4;

  2. Install the component and loadable functions.

    INSTALL COMPONENT 'file://component_masking_functions';

    If the command fails, check the server error log and confirm the library is present in plugin_dir (see Prerequisites). The component is registered in mysql.component and is loaded again on server restart. On replicas or in high-availability setups, the instance may be in read_only or super_read_only mode; component loading in those states follows server behavior. If the component does not load on a replica (for example, because the library is missing or configuration differs), masking is not available on that instance until the cause is resolved.

    On Percona Server for MySQL 8.4.4-1 and later, dictionary-based functions (gen_dictionary, gen_blocklist) use the built-in mysql.session user for internal queries. Grant mysql.session the required privileges on the masking_dictionaries table as described in Permissions in the data masking function list. Granting these privileges allows the server to read and modify the dictionary table for masking; the table may contain lookup data. Rely on your normal access controls and hardening for the mysql schema and dictionary contents.

  3. Grant MASKING_DICTIONARIES_ADMIN to users who will manage dictionaries.

    The privilege is registered when the component loads. Run this step only after INSTALL COMPONENT has succeeded. If GRANT fails with an unknown-privilege or similar error, the component may not be loaded or your Percona Server version may not support this privilege; verify the component is loaded (for example, check mysql.component) and consult the documentation for your version.

    The following functions require MASKING_DICTIONARIES_ADMIN:

    • masking_dictionary_term_add
    • masking_dictionary_term_remove
    • masking_dictionary_remove
    GRANT MASKING_DICTIONARIES_ADMIN ON *.* TO <user>;

Uninstall the data masking component

Data masking component functions

Data masking quickstart — create a test database and try masking functions.

Get started with Data Masking

This quickstart gets the data masking component installed and lets you try the component with a test database. You need access to a Percona Server for MySQL 8.4 server. Follow the steps to install, set permissions, create sample data, and run masking examples. For production use and limitations, see the Data masking overview.

Prerequisites

  • Percona Server for MySQL (latest recommended).
  • The component_masking_functions library in the server plugin directory. Run SHOW VARIABLES LIKE 'plugin_dir'; and confirm the library exists in that directory. If the library is missing, see Install the data masking component.

Install the component and set permissions

Do these once, in order. Full details: Install the data masking component.

  1. Create the dictionary table (required before step 2):

    CREATE TABLE IF NOT EXISTS
mysql.masking_dictionaries(
    Dictionary VARCHAR(256) NOT NULL,
    Term VARCHAR(256) NOT NULL,
    UNIQUE INDEX dictionary_term_idx (Dictionary, Term)
) ENGINE = InnoDB DEFAULT CHARSET=utf8mb4;
    Expected output 
    Query OK, 0 rows affected (0.01 sec)

  2. Install the component:

    INSTALL COMPONENT 'file://component_masking_functions';
    Expected output 
    Query OK, 0 rows affected (0.00 sec)

  3. Grant mysql.session access to the dictionary table (required for gen_dictionary and gen_blocklist on 8.4.4-1 and later):

    GRANT SELECT, INSERT, UPDATE, DELETE ON mysql.masking_dictionaries TO 'mysql.session'@'localhost';
    Expected output 
    Query OK, 0 rows affected (0.00 sec)

  4. Grant dictionary management to your user (required for the dictionary-based masking section below; skip if you will not try that section). Replace <user> and <host> with the MySQL user you connect as (for example, root and localhost):

    GRANT MASKING_DICTIONARIES_ADMIN ON *.* TO '<user>'@'<host>';
    Expected output 
    Query OK, 0 rows affected (0.00 sec)

    No FLUSH PRIVILEGES is needed; the grant takes effect immediately. If the privilege does not appear, reconnect to the server.

Create the test database and tables

Run the following script to create the database and both tables with sample rows. The script drops the database if the database already exists so you can run the script again without creating duplicate rows. The column types and values are chosen so you can try masking functions in the next sections.

DROP DATABASE IF EXISTS masking_demo;
CREATE DATABASE masking_demo;
USE masking_demo;

CREATE TABLE contacts (
    id          INT AUTO_INCREMENT PRIMARY KEY,
    name        VARCHAR(100),
    ssn         VARCHAR(11),
    card_no     VARCHAR(19),
    email       VARCHAR(255),
    phone       VARCHAR(20),
    notes       VARCHAR(255)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

INSERT INTO contacts (name, ssn, card_no, email, phone, notes)
VALUES
    ('Priya Sharma',  '123-45-6789', '4111111111111111', '[email protected]',  '1-555-010-1234', 'VIP'),
    ('Yuki Tanaka',   '987-65-4321', '5500000000000004', '[email protected]',    '1-555-010-5678', 'Newsletter'),
    ('Fatima Hassan', '111-22-3333', '340000000000009',  '[email protected]',  '1-555-010-9012', NULL);

CREATE TABLE contacts_intl (
    id          INT AUTO_INCREMENT PRIMARY KEY,
    name        VARCHAR(100),
    country     VARCHAR(50),
    canada_sin  VARCHAR(11),
    uk_nin      VARCHAR(20),
    iban        VARCHAR(40)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

INSERT INTO contacts_intl (name, country, canada_sin, uk_nin, iban)
VALUES
    ('Jean Lefebvre',   'Canada',  '506-948-819', NULL, NULL),
    ('Siobhan O''Brien', 'UK',     NULL, 'CT264683D', NULL),
    ('Hans Mueller',    'Germany', NULL, NULL, 'DE27 1002 02003 77495 4156');
Expected output 
Query OK, 0 rows affected (0.00 sec)
Query OK, 1 row affected (0.00 sec)
Database changed
Query OK, 0 rows affected (0.01 sec)
Query OK, 3 rows affected (0.00 sec)
Query OK, 0 rows affected (0.00 sec)
Query OK, 3 rows affected (0.00 sec)

View raw data

SELECT id, name, ssn, card_no, email, phone FROM contacts;
Expected output 
+----+--------------+-------------+------------------+-------------------------+----------------+
| id | name         | ssn         | card_no          | email                   | phone          |
+----+--------------+-------------+------------------+-------------------------+----------------+
|  1 | Priya Sharma | 123-45-6789 | 4111111111111111 | [email protected]| 1-555-010-1234 |
|  2 | Yuki Tanaka  | 987-65-4321 | 5500000000000004 | [email protected] | 1-555-010-5678 |
|  3 | Fatima Hassan| 111-22-3333 | 340000000000009  | [email protected]| 1-555-010-9012 |
+----+--------------+-------------+------------------+-------------------------+----------------+
3 rows in set (0.00 sec)

Mask sensitive columns in queries

The examples below use masking functions in SELECT; the table data is unchanged. Expected outputs in this guide are illustrative; results from random or dictionary-based functions vary on each run and may not match the examples. To enforce masking for other users, use a view (see Expose masked data via a view) and restrict access to the base table. For limitations and security, see the Data masking overview.

Payment card and SSN

mask_pan leaves the last four digits visible; mask_ssn does the same for SSN.

SELECT
    name,
    mask_pan(card_no)   AS card_masked,
    mask_ssn(ssn)      AS ssn_masked
FROM contacts;
Expected output 
+--------------+------------------+-------------+
| name         | card_masked      | ssn_masked  |
+--------------+------------------+-------------+
| Priya Sharma | XXXXXXXXXXXX1111 | ***-**-6789 |
| Yuki Tanaka  | XXXXXXXXXXXX0004 | ***-**-4321 |
| Fatima Hassan| XXXXXXXXXX0009   | ***-**-3333 |
+--------------+------------------+-------------+
3 rows in set (0.00 sec)

Inner and outer parts of a string

mask_inner keeps the first and last character(s); mask_outer masks the left and right ends. The third and fourth arguments are the number of characters to keep unmasked on the left and right.

SELECT
    name,
    mask_inner(name, 1, 1)     AS name_inner_masked,
    mask_outer(phone, 2, 4)    AS phone_outer_masked
FROM contacts;
Expected output 
+--------------+------------------+------------------+
| name         | name_inner_masked| phone_outer_masked|
+--------------+------------------+------------------+
| Priya Sharma | PXXXXXXXXXXa     | XX5-010-1234      |
| Yuki Tanaka  | YXXXXXXXXa      | XX5-010-5678      |
| Fatima Hassan| FXXXXXXXXXXn     | XX5-010-9012      |
+--------------+------------------+------------------+
3 rows in set (0.00 sec)

Generated values (for testing)

gen_rnd_email() and gen_rnd_ssn() return new values each time. They are useful for generating test data, not for masking existing values in place.

SELECT
    id,
    gen_rnd_email(5, 8, 'test.example') AS generated_email,
    gen_rnd_ssn()                         AS generated_ssn
FROM contacts;
Expected output 
+----+---------------------------+-------------+
| id | generated_email           | generated_ssn|
+----+---------------------------+-------------+
|  1 | [email protected]| 912-34-5678 |
|  2 | [email protected]| 987-65-4321 |
|  3 | [email protected]| 923-45-6789 |
+----+---------------------------+-------------+
3 rows in set (0.00 sec)

Output will vary on each run and may not match the expected results above. For constraints and caveats, see Data masking component functions and variables.

International data masking functions

The component includes functions for region-specific identifiers: Canadian Social Insurance Number (SIN) (mask_canada_sin), United Kingdom National Insurance Number (NIN) (mask_uk_nin), and International Bank Account Number (IBAN) (mask_iban). The script above already created the contacts_intl table. View the raw data, then apply the region-specific masking functions:

SELECT name, country, canada_sin, uk_nin, iban FROM contacts_intl;
Expected output 
+----------------+---------+-------------+----------------+------------------------------+
| name           | country | canada_sin  | uk_nin     | iban                         |
+----------------+---------+-------------+----------------+------------------------------+
| Jean Lefebvre  | Canada  | 506-948-819 | NULL       | NULL                         |
| Siobhan O'Brien| UK      | NULL        | CT264683D | NULL                         |
| Hans Mueller   | Germany | NULL        | NULL       | DE27 1002 02003 77495 4156   |
+----------------+---------+-------------+----------------+------------------------------+
3 rows in set (0.00 sec)

SELECT
    name,
    country,
    mask_canada_sin(canada_sin)  AS canada_sin_masked,
    mask_uk_nin(uk_nin)          AS uk_nin_masked,
    mask_iban(iban)              AS iban_masked
FROM contacts_intl;
Expected output 
+----------------+---------+------------------+---------------+----------------------------+
| name           | country | canada_sin_masked| uk_nin_masked | iban_masked                |
+----------------+---------+------------------+---------------+----------------------------+
| Jean Lefebvre  | Canada  | XXX-XXX-XXX      | NULL          | NULL                       |
| Siobhan O'Brien| UK      | NULL             | CT*******     | NULL                       |
| Hans Mueller   | Germany | NULL             | NULL          | DE** **** **** **** ****   |
+----------------+---------+------------------+---------------+----------------------------+
3 rows in set (0.00 sec)

Each function returns NULL when the input is NULL. For more options (for example, custom mask characters) and the corresponding generators (gen_rnd_canada_sin, gen_rnd_uk_nin, gen_rnd_iban), see Data masking component functions and variables.

Expose masked data via a view

To ensure that only masked data is visible, define a view that applies the masking functions and grant SELECT on the view (not on the base table) to users who should see masked data. Example:

DROP VIEW IF EXISTS contacts_masked;
CREATE VIEW contacts_masked AS
SELECT
    id,
    name,
    mask_pan(card_no)       AS card_no,
    mask_ssn(ssn)          AS ssn,
    mask_outer(email, 2, 4) AS email,
    mask_outer(phone, 2, 4) AS phone,
    notes
FROM contacts;

SELECT id, name, card_no, ssn, email, phone FROM contacts_masked;
Expected output 
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.00 sec)

+----+--------------+------------------+-------------+--------------------------+------------------+
| id | name         | card_no          | ssn         | email                     | phone            |
+----+--------------+------------------+-------------+--------------------------+------------------+
|  1 | Priya Sharma | XXXXXXXXXXXX1111 | ***-**-6789 | XXriya.sharma@exampXXXX  | XX5-010-1234     |
|  2 | Yuki Tanaka  | XXXXXXXXXXXX0004 | ***-**-4321 | XXuki.tanaka@exampXXXX   | XX5-010-5678     |
|  3 | Fatima Hassan| XXXXXXXXXX0009   | ***-**-3333 | XXtima.hassan@examXXXX  | XX5-010-9012     |
+----+--------------+------------------+-------------+--------------------------+------------------+
3 rows in set (0.00 sec)

The view uses mask_outer(..., 2, 4) for email and phone. Grant SELECT on the view (not the base table) to users who should see masked data only. For access control and limitations, see the Data masking overview.

Optional: dictionary-based masking

Functions such as gen_dictionary and gen_blocklist use the mysql.masking_dictionaries table.

You need the step 4 grant (MASKING_DICTIONARIES_ADMIN) and, on 8.4.4-1 and later, the step 3 grant to mysql.session described in the install guide.

Add a small dictionary with masking_dictionary_term_add, then try gen_dictionary.

The role column is chosen at random from the dictionary; your output will vary and may not match the expected results.

SELECT masking_dictionary_term_add('roles', 'Engineer');
SELECT masking_dictionary_term_add('roles', 'Analyst');
SELECT masking_dictionary_term_add('roles', 'Manager');

SELECT id, name, gen_dictionary('roles') AS role FROM contacts;
Expected output (role column varies; your output may not match) 
+-----------------------------------------------+
| masking_dictionary_term_add('roles', 'Engineer')|
+-----------------------------------------------+
|                                             1 |
+-----------------------------------------------+
1 row in set (0.00 sec)

+----------------------------------------------+
| masking_dictionary_term_add('roles', 'Analyst')|
+----------------------------------------------+
|                                            1 |
+----------------------------------------------+
1 row in set (0.00 sec)

+----------------------------------------------+
| masking_dictionary_term_add('roles', 'Manager') |
+----------------------------------------------+
|                                            1 |
+----------------------------------------------+
1 row in set (0.00 sec)

+----+---------------+----------+
| id | name          | role     |
+----+---------------+----------+
|  1 | Priya Sharma  | Manager  |
|  2 | Yuki Tanaka   | Engineer |
|  3 | Fatima Hassan | Manager  |
+----+--------------+----------+
3 rows in set (0.00 sec)

To remove the dictionary when finished: SELECT masking_dictionary_remove('roles'); (masking_dictionary_remove). For large dictionaries or production use, see the Data masking overview and function list.

Clean up (optional)

To remove the test database and all objects in the database (tables, the contacts_masked view, and data):

DROP DATABASE IF EXISTS masking_demo;
Expected output 
Query OK, 0 rows affected (0.01 sec)

Additional resources

Data masking component functions and variables

The feature is in tech preview. This page is a catalog of individual functions and variables. For the security model, use cases, and when to apply masking (including limitations and failure modes), see Data masking overview and the Limitations and security considerations section. Performance characteristics (latency, complexity, scalability) are not documented; for large result sets or SLA-critical workloads, benchmark and evaluate per-row overhead. The component does not provide a function to test whether a value is already masked; auditing that masking has been applied to required rows must be done by application logic, known patterns, or other tooling.

To find a function by what the function does, use the “Find by task” table or the full list below. Each function name links to that function’s definition later on this page.

Find by task

If you want to… Function(s)
Mask or generate a payment card number (PAN) mask_pan, mask_pan_relaxed, gen_rnd_pan
Mask or generate a US Social Security number (SSN) mask_ssn, gen_rnd_ssn
Mask or generate a Canadian SIN mask_canada_sin, gen_rnd_canada_sin
Mask or generate a UK National Insurance number mask_uk_nin, gen_rnd_uk_nin
Mask or generate an IBAN mask_iban, gen_rnd_iban
Mask or generate a UUID mask_uuid, gen_rnd_uuid
Generate a random email or US phone number gen_rnd_email, gen_rnd_us_phone
Mask the middle or the ends of any string mask_inner, mask_outer
Replace a value with a random term from a dictionary gen_dictionary, gen_blocklist
Generate a number in a range gen_range
Manage dictionaries (add/remove terms, flush cache) masking_dictionary_term_add, masking_dictionary_term_remove, masking_dictionary_remove, masking_dictionaries_flush

Full list (by name)

Name Details
gen_blocklist(str, from_dictionary_name, to_dictionary_name) If the input is in the source dictionary, returns a random term from the target dictionary; otherwise returns the input
gen_dictionary(dictionary_name) Returns a random term from a dictionary
gen_range(lower, upper) Returns a number from a range
gen_rnd_canada_sin() Generates a Canadian Social Insurance number
gen_rnd_email([name_size, surname_size, domain]) Generates a random string in name.surname@domain format
gen_rnd_iban([country, size]) Generates an International Bank Account number
gen_rnd_pan() Generates a Primary account number for a payment card
gen_rnd_ssn() Generates a US Social Security number
gen_rnd_uk_nin() Generates a United Kingdom National Insurance number
gen_rnd_us_phone() Generates a US phone number
gen_rnd_uuid() Generates a Universally Unique Identifier
mask_canada_sin(str [,mask_char]) Masks the Canadian Social Insurance number
mask_iban(str [,mask_char]) Masks the International Bank Account number
mask_inner(str, margin1, margin2 [,mask_char]) Masks the inner part of a string
mask_outer(str, margin1, margin2 [,mask_char]) Masks the outer part of the string
mask_pan(str [,mask_char]) Masks the Primary Account number for a payment card
mask_pan_relaxed(str [,mask_char]) Partially masks the Primary Account number for a payment card
mask_ssn(str [,mask_char]) Masks the US Social Security number
mask_uk_nin(str [,mask_char]) Masks the United Kingdom National Insurance number
mask_uuid(str [,mask_char]) Masks the Universally Unique Identifier
masking_dictionaries_flush() Resyncs the internal dictionary term cache
masking_dictionary_remove(dictionary_name) Removes the dictionary
masking_dictionary_term_add(dictionary_name, term_name) Adds a term to the masking dictionary
masking_dictionary_term_remove(dictionary_name, term_name) Removes a term from the masking dictionary

Permissions

In Percona Server for MySQL 8.4.4-1, dictionary-related functions no longer run internal queries as the root user without a password. Following MySQL best practices, many admins disable the root user, which previously caused these functions to stop working. The server now uses the built-in mysql.session user to execute dictionary queries.

However, for dictionary operations to work, you need to grant the mysql.session user SELECT, INSERT, UPDATE, and DELETE privileges on the masking_dictionaries table. Granting UPDATE and DELETE allows the server to modify or remove dictionary data. Compromise of the server or abuse of that user could allow dictionary tampering or deletion, causing dictionary-based functions to fail or return unexpected values; protect the server and the mysql schema accordingly.

GRANT SELECT, INSERT, UPDATE, DELETE ON mysql.masking_dictionaries TO 'mysql.session'@'localhost';

If you change the value of the masking_functions.masking_database system variable to something other than mysql, make sure to update the GRANT query to match the new value.

GRANT SELECT, INSERT, UPDATE, DELETE ON <masking_functions.masking_database>.masking_dictionaries TO 'mysql.session'@'localhost';

Limitations and operational notes

  • Character set and collation: Several functions accept a mask_char that can be in a different character set than the input string. When the character sets differ, the server must transcode the mask character; in some collations or storage contexts this can produce illegal byte sequences or errors. Prefer using a mask_char in the same character set as the input (or the target column) to avoid corruption or storage failures.

  • Schema and format: Masking functions return strings (for example, containing X or *). If the target column or application enforces strict validation (numeric-only, regex, fixed width, or checksums such as Luhn for payment cards), the masked value may cause INSERT or UPDATE failures or break application logic. The component does not guarantee format-preserving or type-preserving output; choose mask character and usage so that results remain valid for the target schema and application.

  • Randomness: Functions that return “random” values (for example, gen_dictionary, gen_blocklist, gen_rnd_*) do not document that the randomness is cryptographically secure or unpredictable. The selection or generation logic is not necessarily resistant to prediction; do not rely on that logic to resist inference attacks (for example, an attacker with partial data might infer mappings).

  • Collision and uniqueness: The gen_rnd_* functions do not document entropy source or collision probability. At scale (for example, millions of rows), duplicate generated values are possible and can violate unique constraints or make the dataset unsuitable for integration testing. Evaluate uniqueness requirements before relying on these generators for large datasets.

  • Default mask character: The default masking character (for example, X or *) may not suit all use cases. For fixed-width fields, padding, or systems that expect specific formats or checksums (for example, Luhn for payment cards), the default can break validation. Choose mask character and format to match the target schema and application requirements.

  • Performance at scale: Applying masking functions to very large result sets (for example, tens or hundreds of millions of rows) can add significant CPU and memory cost. No performance guarantees or benchmarks are documented. Test and monitor when using these functions in heavy reporting or large SELECTs to avoid resource exhaustion or denial-of-service risk.

  • Aggregation and inference: Row-level masking affects how values are displayed in results; such masking does not necessarily protect against inference via aggregates. Depending on implementation, SUM(), AVG(), or other aggregates may be computed over unmasked data in the execution engine. Do not assume that masking in the result set prevents leakage through aggregate queries; consider differential privacy or aggregate-level protection if that is a threat.

  • Boundary behavior: For mask_outer, when the sum of margin1 and margin2 is greater than or equal to the string length, the entire string is replaced with the mask character (the whole string is masked). For mask_inner, when that sum is greater than or equal to the string length, no masking occurs (the string is returned unchanged).

  • Validation: The component does not provide a function to check whether a value is already masked. To audit that required rows or columns have been masked, use application logic, schema conventions, or external tooling.

gen_blocklist(str, from_dictionary_name, to_dictionary_name)

If str is present in the dictionary named from_dictionary_name, returns a randomly selected term from the dictionary named to_dictionary_name. Otherwise returns str unchanged. The selection is not documented as cryptographically secure; do not rely on the selection to resist prediction or inference (see Limitations and operational notes).

Version update

Percona Server for MySQL 8.4.4-4 introduces an internal term cache. The server now uses in-memory data structures for lookups instead of querying the <masking_functions.masking_database>.masking_dictionaries table every time. This improvement boosts performance, especially when handling multiple rows.

Parameters

Parameter Optional Description Type
str No The input value to look up and optionally replace String
from_dictionary_name No The dictionary in which str must be found for replacement to occur String
to_dictionary_name No The dictionary from which a replacement term is chosen at random String

Returns

If str is in from_dictionary_name, a randomly selected term from to_dictionary_name. If str is not in from_dictionary_name or either dictionary is missing, str unchanged. If to_dictionary_name does not exist, NULL. Returns NULL if str is NULL. The character set of the returned string is the same as str.

Example

SELECT gen_blocklist('apple', 'fruit', 'nut');
Expected output 
+-----------------------------------------+
| gen_blocklist('apple', 'fruit', 'nut')  |
+-----------------------------------------+
| walnut                                  |
+-----------------------------------------+

gen_dictionary(dictionary_name)

Returns a term from a dictionary selected at random. The selection is not documented as cryptographically secure; do not rely on the selection to resist prediction or inference (see Limitations and operational notes).

Version update

Percona Server for MySQL 8.4.4-4 introduces an internal term cache. The server now uses in-memory data structures for lookups instead of querying the <masking_functions.masking_database>.masking_dictionaries table every time. This improvement boosts performance, especially when handling multiple rows.

Parameters

Parameter Optional Description Type
dictionary_name No Select the random term from this dictionary String

Returns

A random term from the dictionary listed in dictionary_name in the utf8mb4 character set. Returns NULL if the dictionary_name does not exist.

Example

SELECT gen_dictionary('trees');
Expected output 
+--------------------------------------------------+
| gen_dictionary('trees')                          |
+--------------------------------------------------+
| Norway spruce                                    |
+--------------------------------------------------+

gen_range(lower, upper)

Returns a number from a defined range.

Parameters

Parameter Optional Description Type
lower No The lower boundary of the range Integer
upper No The upper boundary of the range Integer

The upper parameter value must be an integer either greater than or equal to the lower parameter value.

Returns

An integer, selected at random, from an inclusive range defined by the lower parameter value and the upper parameter value, or NULL if the upper boundary is less than the lower boundary.

Example

SELECT gen_range(10, 100);
Expected output 
+--------------------------------------+
| gen_range(10,100)                    |
+--------------------------------------+
| 56                                   |
+--------------------------------------+

gen_rnd_canada_sin()

Generates a Canada Social Insurance Number (SIN).

Important

Only use this function for testing because the result could be a legitimate SIN. Use mask_canada_sin to disguise the result if you must publish the result.

Parameters

None.

Returns

Returns a Canada SIN formatted in three groups of three digits (for example, 123-456-789) in the utf8mb4 character set. To ensure the number is consistent, the number is verified with the Luhn algorithm .

Example

SELECT gen_rnd_canada_sin();
Expected output 
+-------------------------+
| gen_rnd_canada_sin()    |
+-------------------------+
| 506-948-819             |
+-------------------------+

gen_rnd_email([name_size, surname_size, domain])

Generates a random string in the name.surname@domain format.

Parameters

Parameter Optional Description Type
name_size Yes Specifies the number of characters in the name part. The default number is five. The minimum number is one. The maximum number is 1024. Integer
surname_size Yes Specifies the number of characters in the surname part. The default number is seven. The minimum number is one. The maximum number is 1024. Integer
domain Yes Specifies the domain name used. The default value is example.com. String

Returns

A generated string in the same character set as domain. If the domain value is not specified, then the string is in the utf8mb4 character set. The name and surname are random lower-case letters (a - z). For columns with a unique constraint, duplicates are possible at scale; see Collision and uniqueness in Limitations and operational notes.

Example

SELECT gen_rnd_email(name_size=4, surname_size=5, domain='example.test');
Expected output 
+----------------------------------------+
| gen_rnd_email(4, 5, 'example.test')   |
+----------------------------------------+
| [email protected]               |
+----------------------------------------+

gen_rnd_iban([country, size])

Generates an International Bank Account Number (IBAN).

Important

Generating an IBAN with a valid country code should only be used for testing. The function does not check if the generated value is a legitimate bank account. If you must publish the result, consider using mask_iban to disguise the result. The function does not perform a checksum on the bank account number.

Parameters

Parameter Optional Description Type
country Yes A two-character country code String
size Yes Number of characters Integer

If the country is not specified, the default value is ZZ. The value must be two upper-case characters (A-Z) or an error is returned.

The default value for size is 16. The minimum value is 15. The maximum value is 34.

Returns

The function returns a string that is the length of the size value. The string consists of country (two characters) followed by the (size - 2) random digits.

The character set is the same as the country parameter or if that parameter is not specified, the character set is utf8mb4.

Example

SELECT gen_rnd_iban();
Expected output 
+-------------------+
| gen_rnd_iban()    |
+-------------------+
|ZZ78959120078536   |
+-------------------+

gen_rnd_pan()

Generates a Primary Account Number (PAN) for a payment card that passes basic checksum validation.

The generated PAN can be one of the following:

  • American Express

  • Visa

  • Mastercard

  • Discover

Important

Generating the PAN should only be used for testing. The function does not check if the generated value is a legitimate primary account number. If you must publish the result, consider using mask_pan or mask_pan_relaxed() to disguise the result.

Parameters

None

Returns

A random PAN string in utf8mb4 character set.

Example

SELECT gen_rnd_pan();
Expected output 
+-------------------+
| gen_rnd_pan()     |
+-------------------+
| 1234567898765432  |
+-------------------+

gen_rnd_ssn()

Generates a United States Social Security Account Number (SSN).

Parameters

None

Returns

An SSN string in nine-digit format “AAA-GG-SSSS” in the utf8mb4 character set. The number has three parts, the first three digits are the area number, the group number, and the serial number. The generated SSN uses ‘900’ or greater numbers for the area number. These numbers are not legitimate because they are outside the approved range.

Example

SELECT gen_rnd_ssn();
Expected output 
+----------------+
| gen_rnd_ssn()  |
+----------------+
| 970-03-0370    |
+----------------+

gen_rnd_uk_nin()

Generates a United Kingdom National Insurance Number (NIN).

Important

This function should only be used for testing. The function does not check if the generated value is a legitimate United Kingdom National Insurance number. If you must publish the result, consider masking the result with mask_uk_nin.

Parameters

None.

Returns

A NIN string in the utf8mb4 character set. The string is nine (9) characters in length, always starts with ‘AA’ and ends with ‘C’.

Example

SELECT gen_rnd_uk_nin();
Expected output 
+----------------------+
| gen_rnd_uk_nin()     |
+----------------------+
| AA123456C            |
+----------------------+

gen_rnd_us_phone()

Generates a United States phone number with the 555 area code. The ‘555’ area code represents fictional numbers.

Parameters

None

Returns

Returns a United States phone number in the utf8mb4 character set.

Example

SELECT gen_rnd_us_phone();
Expected output 
+--------------------+
| gen_rnd_us_phone() |
+--------------------+
| 1-555-249-2029     |
+--------------------+

gen_rnd_uuid()

Generates a version 4 Universally Unique Identifier (UUID).

Parameters

None.

Returns

Returns a UUID as a string in the utf8mb4 character set.

Example

SELECT gen_rnd_uuid();
Expected output 
+------------------------------------+
| gen_rnd_uuid()                     |
+------------------------------------+
|9a3b642c-06c6-11ee-be56-0242ac120002|
+------------------------------------+

mask_canada_sin(str [,mask_char])

Masks a Canada Social Insurance Number (SIN).

Parameters

Parameter Optional Description Type
str No The string to be masked String
mask_char Yes The masking character String

The str accepts an alphanumeric string.

If you do not specify a mask_char, the default character is X. The mask_char value can be a multibyte character in any character set and may not be the same character set as str. See Limitations and operational notes for character set and collation caveats when mask_char differs from the input.

Returns

A string with the selected characters masked by a specified mask_char or the default value for that parameter. The function supports multibyte characters in any character set. The character set of the return value is the same as str.

An error is reported if str length is an incorrect length.

Returns a NULL if you invoke this function with NULL as the primary argument.

Example

SELECT mask_canada_sin('555-555-555');
Expected output 
+--------------------------------+
| mask_canada_sin('555-555-555') |
+--------------------------------+
| XXX-XXX-XXX                    |
+--------------------------------+

mask_iban(str [,mask_char])

Masks an International Bank Account Number (IBAN).

Parameters

Parameter Optional Description Type
str No The string to be masked String
mask_char Yes Character used for masking String

The str accepts either of the following:

  • No separator symbol

  • Groups of four characters. These groups can be separated by a space or any separator character.

The default value for mask_char is *. The value can be a multibyte character in any character set and may not be the same character set as str. See Limitations and operational notes for character set and collation caveats when mask_char differs from the input.

Returns

Returns the masked string. The character set of the result is the same as the character set of str.

An error is reported if the str length is incorrect.

Returns NULL if you invoke this function with NULL as the primary argument.

Example

SELECT mask_iban('DE27 1002 02003 77495 4156');
Expected output 
+---------------------------------------------+
| mask_iban('DE27 1002 02003 77495 4156')     |
+---------------------------------------------+
| DE** **** **** **** ****                    |
+---------------------------------------------+

mask_inner(str, margin1, margin2 [,mask_char])

Returns the string where a selected inner portion is masked with a substitute character.

Parameters

Parameter Optional Description Type
str No The string to be masked String
margin1 No The number of characters on the left end of the string to remain unmasked Integer
margin2 No The number of characters on the right end of the string to remain unmasked Integer
mask_char Yes The masking character String

The margin1 value cannot be a negative number. A value of 0 (zero) masks all characters.

The margin2 value cannot be a negative number. A value of 0 (zero) masks all characters.

If the sum of margin1 and margin2 is greater than or equal to the string length, no masking occurs. See Limitations and operational notes for character set and collation caveats when mask_char differs from the input.

If the mask_char is not specified, the default is ‘X’. The mask_char value can be a multibyte character in any character set and may not be the same character set as str.

Returns

A string with the selected characters masked by a specified mask_char or that parameter’s default value in the character set of str.

Returns NULL if you invoke this function with NULL as the primary argument.

Example

SELECT mask_inner('123456789', 1, 2);
Expected output 
+-----------------------------------+
| mask_inner('123456789', 1, 2)     |
+-----------------------------------+
| 1XXXXXX89                          |
+-----------------------------------+

mask_outer(str, margin1, margin2 [,mask_char])

Returns the string where a selected outer portion is masked with a substitute character.

Parameters

Parameter Optional Description Type
str No The string to be masked String
margin1 No On the left end of the string, mask this designated number of characters Integer
margin2 No On the right end of the string, mask this designated number of characters Integer
mask_char Yes The masking character String

The margin1 cannot be a negative number. A value of 0 (zero) does not mask any characters.

The margin2 cannot be a negative number. A value of 0 (zero) does not mask any characters.

If the sum of margin1 and margin2 is greater than or equal to the string length, the entire string is replaced with the mask character (the whole string is masked). See Limitations and operational notes for character set and collation caveats when mask_char differs from the input.

If the mask_char is not specified, the default is ‘X’. The mask_char value can be a multibyte character in any character set and may not be the same character set as str.

Returns

A string with the selected characters masked by a specified mask_char or that parameter’s default value in the same character set as str.

Returns NULL if you invoke this function with NULL as the primary argument.

Example

SELECT mask_outer('123456789', 2, 2);
Expected output 
+------------------------------------+
| mask_outer('123456789', 2, 2)     |
+------------------------------------+
| XX34567XX                          |
+------------------------------------+

mask_pan(str [,mask_char])

Returns a masked payment card Primary Account Number (PAN). The mask replaces the PAN number with the specified character except for the last four digits.

Parameters

Parameter Optional Description Type
str No The string to be masked String
mask_char Yes The masking character String

The str contains a minimum of 14 or a maximum of 19 alphanumeric characters.

If the mask_char is not specified, the default value is ‘X’. The mask_char value can be a multibyte character in any character set and may not be the same character set as str. See Limitations and operational notes for character set and collation caveats when mask_char differs from the input. For numeric-only columns or systems that validate Luhn checksums, the default character may be invalid; see Schema and format and Default mask character in that section.

Returns

A string with the selected characters masked by a specified mask_char or that parameter’s default value. The character set of the result is the same character set as str.

An error occurs if the str parameter is not the correct length.

Returns NULL if you invoke this function with NULL as the primary argument.

Example

SELECT mask_pan (gen_rnd_pan());
Expected output 
+------------------------------------+
| mask_pan(gen_rnd_pan())            |
+------------------------------------+
| XXXXXXXXXXX2345                    |
+------------------------------------+

mask_pan_relaxed(str [,mask_char])

Returns a masked payment card Primary Account Number (PAN). Leaves the first six and last four digits unmasked; the rest is masked by the specified character or X.

Parameters

Parameter Optional Description Type
str No The string to be masked String
mask_char Yes The specified character for masking String

The str must contain a minimum of 14 or a maximum of 19 alphanumeric characters.

If the mask_char is not specified, the default value is ‘X’.

Returns

A string with the first six and last four digits unmasked and the rest masked by a specified mask_char or that parameter’s default value (X). The character set of the result is the same character set as str.

The mask_char value can be a multibyte character in any character set and may not be the same character set as str. See Limitations and operational notes for character set and collation caveats when mask_char differs from the input.

Reports an error if the str parameter is not the correct length.

Returns NULL if you invoke this function with NULL as the primary argument.

Example

SELECT mask_pan_relaxed(gen_rnd_pan());
Expected output 
+------------------------------------------+
| mask_pan_relaxed(gen_rnd_pan())          |
+------------------------------------------+
| 520754XXXXXX4848                         |
+------------------------------------------+

mask_ssn(str [,mask_char])

Returns a masked United States Social Security Number(SSN). The mask replaces the SSN number with the specified character except for the last four digits.

Parameters

Parameter Optional Description Type
str No The string to be masked String
mask_char Yes The masking character String

The str accepts either of the following:

  • Nine integers, no separator symbol
  • Nine integers in the AAA-GG-SSSS pattern. The - (dash symbol) is the separator character.

If the mask_char is not specified, the default value is *. The mask_char value can be a multibyte character in any character set and may not be the same character set as str. See Limitations and operational notes for character set and collation caveats when mask_char differs from the input.

Returns

A string with the selected characters masked by a specified mask_char or that parameter’s default value in the same character set as str.

Reports an error if the value of the str is an incorrect length.

Returns a NULL value if you invoke this function with NULL as the primary argument.

Example

SELECT mask_ssn('555-55-5555', 'X');
Expected output 
+-----------------------------+
| mask_ssn('555-55-5555','X') |
+-----------------------------+
| XXX-XX-5555                 |
+-----------------------------+

mask_uk_nin(str [,mask_char])

Returns a masked United Kingdom National Insurance Number (NIN). The mask replaces the NIN with the specified character except for the first two characters.

Parameters

Parameter Optional Description Type
str No The string to be masked String
mask_char Yes The masking character String

The str accepts an alpha-numeric string and does not check format and the str can use any separator character.

If the mask_char is not specified, the default value is *. The mask_char value can be a multibyte character in any character set and may not be the same character set as str. See Limitations and operational notes for character set and collation caveats when mask_char differs from the input.

Returns

Returns a string with the selected characters masked by a specified mask_char or that parameter’s default value in the same character set as str.

An error occurs if the str parameter is not the correct length.

Returns a NULL value if you invoke this function with NULL as the primary argument.

Example

SELECT mask_uk_nin ('CT 26 46 83 D');
Expected output 
+------------------------------------+
| mask_uk_nin('CT 26 46 83 D')       |
+------------------------------------+
| CT ** ** ** *                      |
+------------------------------------+

mask_uuid(str [,mask_char])

Masks a Universally Unique Identifier (UUID).

Parameters

Parameter Optional Description Type
str No The string to be masked String
mask_char Yes The masking character String

The str format is ********-****-****-****-************.

If the mask_char is not specified, the default value is ‘*’. The mask_char value can be a multibyte character in any character set and may not be the same character set as str. See Limitations and operational notes for character set and collation caveats when mask_char differs from the input.

Returns

A string with the characters masked by a specified mask_char or that parameter’s default value in the same character set as str.

An error occurs if the length of str is incorrect.

Returns NULL if you invoke this function with NULL as the primary argument.

Example

SELECT mask_uuid('9a3b642c-06c6-11ee-be56-0242ac120002');
Expected output 
+-------------------------------------------------------+
| mask_uuid('9a3b642c-06c6-11ee-be56-0242ac120002')     |
+-------------------------------------------------------+
|********-****-****-****-************                   |
+-------------------------------------------------------+

masking_dictionaries_flush()

Resyncs the internal dictionary term cache.

Parameters

None

Returns

Returns an integer value of 1 (one) when successful.

Example

SELECT masking_dictionaries_flush();
Expected output 
+------------------------------+
| masking_dictionaries_flush() |
+------------------------------+
|                          1   |
+------------------------------+

masking_dictionary_remove(dictionary_name)

Removes all of the terms and then removes the dictionary.

Requires the MASKING_DICTIONARIES_ADMIN privilege.

Parameters

Parameter Optional Description Type
dictionary_name No The dictionary to be removed String

Returns

Returns an integer value of 1 (one) if the operation is successful. Returns the integer value of 0 (zero) for a failure.

Example

SELECT masking_dictionary_remove('trees');
Expected output 
+------------------------------------------+
| masking_dictionary_remove('trees')       |
+------------------------------------------+
|                                        1 |
+------------------------------------------+

masking_dictionary_term_add(dictionary_name, term_name)

Adds a term to the dictionary and requires the MASKING_DICTIONARIES_ADMIN privilege.

Parameters

Parameter Optional Description Type
dictionary_name No The dictionary where the term is added String
term_name No The term added to the selected dictionary String

Returns

Returns an integer value of 1 (one) if the operation is successful. Returns an integer value of 0 (zero) for a failure. If the dictionary_name does not exist, the operation creates the dictionary.

The operation uses INSERT IGNORE and can have the following outcomes:

  • The term_name is truncated if the term_name length is greater than maximum length of the Term field in the mysql.masking_dictionaries table.

  • The character of the dictionary_name is not supported by the Dictionary field in mysql.masking_dictionaries table, the character is implicitly converted to ‘?’.

  • If the character of the term_name is not supported by the Term field in the mysql.masking_dictionaries table, the character is implicitly converted to ‘?’.

The following command returns the table information:

DESCRIBE mysql.masking_dictionaries;

The result returns the table structure.

Expected output 
+------------+--------------+------+-----+---------+-------+
| Field      | Type         | Null | Key | Default | Extra |
+------------+--------------+------+-----+---------+-------+
| Dictionary | varchar(256) | NO   | PRI | NULL    |       |
| Term       | varchar(256) | NO   | PRI | NULL    |       |
+------------+--------------+------+-----+---------+-------+
2 rows in set (0.02 sec)

Modify the table with an ALTER TABLE statement, if needed.

Example

SELECT masking_dictionary_term_add('trees','pine');
Expected output 
+-----------------------------------------------+
| masking_dictionary_term_add('trees', 'pine')  |
+-----------------------------------------------+
|                                             1 |
+-----------------------------------------------+

masking_dictionary_term_remove(dictionary_name, term_name)

Removes the selected term from the dictionary.

Requires the MASKING_DICTIONARIES_ADMIN privilege.

Parameters

Parameter Optional Description Type
dictionary_name No The dictionary that contains the term_name String
term_name No The term to be removed String

Returns

Returns an integer value of 1 (one) if the operation is successful. Returns the integer value of 0 (zero) for a failure.

Returns NULL if the operation fails. An operation can fail if the following occurs:

  • The term_name is not available in the dictionary specified by dictionary_name
  • The dictionary_name could not be found

Example

SELECT masking_dictionary_term_remove('trees','pine');
Expected output 
+-------------------------------------------------------+
| masking_dictionary_term_remove('trees', 'pine')       |
+-------------------------------------------------------+
|                                                     1 |
+-------------------------------------------------------+

System variables

Name Details
dictionaries_flush_interval_seconds (integer, unsigned) The number of seconds between updates to the internal dictionary cache to match changes in the dictionaries table.
masking_database(string) Set a different database name to use for the dictionaries table.

dictionaries_flush_interval_seconds(integer, unsigned)

Option Description
command-line Yes
scope Global
data type unsigned integer
default 0

Variable name: component_masking_functions.dictionaries_flush_interval_seconds. You can set the variable at runtime (for example, with SET GLOBAL) or on the command line.

Percona Server for MySQL 8.4.4-4 adds this variable. The number of seconds between synchronizations of the dictionaries table and the internal dictionary cache. The default value is 0 seconds (disabled). The minimum value is 1 second. The maximum value is 31,536,000 seconds (1 year).

Replication: On replicas that use row-based replication, the dictionary term cache is not updated immediately when dictionary changes are applied from the binary log. Set this variable to a positive value (for example, 60) so that a background process periodically refreshes the cache and keeps replicas in sync with the source.

masking_database(string)

Option Description
Scope: Global
Read, Write, or Read-Only: Read-Only
Data type String
Default value “mysql”

Specify the name of the database that holds the masking_dictionaries table. By default, the setting uses the mysql database.

Uninstall the data masking component

Uninstalling removes the masking functions and (if you choose) the dictionary table. The steps below prioritize a clean removal. Before you run them, address dependencies, privileges, and the risk of exposing previously masked data.

Before you uninstall

  • Views, stored procedures, and triggers: Any view, routine, or trigger that calls masking functions (for example, in a SELECT or default expression) depends on the component. After UNINSTALL COMPONENT, those objects become invalid. Queries that use them can fail with an error such as a missing function or component. Before uninstalling, identify and either drop or replace those objects, or ensure no application uses them. If you leave them in place and uninstall, applications that query the view may fail or, if they fall back to base tables, may see unmasked data. Plan to replace masking-based views with restricted views, dummy tables, or access denial so that uninstallation does not inadvertently expose raw data.

  • Privileges: The install procedure grants MASKING_DICTIONARIES_ADMIN to users and may grant privileges to mysql.session on mysql.masking_dictionaries. Uninstalling does not revoke these. For a clean state, revoke them before or after uninstall. To revoke from mysql.session, the table must still exist, so do that before dropping the table. Example (adjust for your database if you changed masking_database):

REVOKE MASKING_DICTIONARIES_ADMIN ON *.* FROM <user>;
REVOKE SELECT, INSERT, UPDATE, DELETE ON mysql.masking_dictionaries FROM 'mysql.session'@'localhost';

  • System variables: If you set masking-related variables (for example, component_masking_functions.dictionaries_flush_interval_seconds or the database used for the dictionaries table), remove or reset them in your configuration (for example, my.cnf or the command line). After the component is uninstalled, references to component variables in the config can produce “unknown variable” warnings or errors on server restart and can confuse configuration management tools.

  • In-flight use: Ensure no views, routines, or applications are actively using masking functions when you uninstall. Uninstalling while queries or background processes depend on the component can cause those operations to fail.

Uninstall steps

The following steps uninstall the component; the second step optionally drops the masking_dictionaries table.

  1. Uninstall the component and loadable functions.

    UNINSTALL COMPONENT 'file://component_masking_functions';

    After this step, any view, stored procedure, or trigger that references masking functions is invalid and will cause errors when used.

  2. Optionally drop the masking_dictionaries table.

    Dropping this table permanently removes all dictionary data and cannot be undone. Skip this step if you plan to reinstall or upgrade and want to keep the table and data. If you might need the dictionary data later, back up the table or export the data before dropping.

    DROP TABLE mysql.masking_dictionaries;

    If you granted privileges to mysql.session on this table and did not revoke them in “Before you uninstall,” the grant entries may remain in the system until you revoke them; revoking requires the table to exist, so revoke before dropping the table if you want a full cleanup.

Install the data masking component

Data masking component functions

PAM authentication plugin

Percona PAM Authentication Plugin is a free and Open Source implementation of the MySQL’s authentication plugin. This plugin acts as a mediator between the MySQL server, the MySQL client, and the PAM stack. The server plugin requests authentication from the PAM stack, forwards any requests and messages from the PAM stack over the wire to the client (in cleartext) and reads back any replies for the PAM stack.

PAM plugin uses dialog as its client side plugin. Dialog plugin can be loaded to any client application that uses libperconaserverclient/libmysqlclient library.

Here are some of the benefits that Percona dialog plugin offers over the default one:

  • It correctly recognizes whether PAM wants input to be echoed or not, while the default one always echoes the input on the user’s console.

  • It can use the password which is passed to MySQL client via “-p” parameter.

  • Dialog client installation bug has been fixed.

Percona offers two versions of this plugin:

  • Full PAM plugin called auth_pam. This plugin uses dialog.so. It fully supports the PAM protocol with arbitrary communication between client and server.

  • Oracle-compatible PAM called auth_pam_compat. This plugin uses mysql_clear_password which is a part of Oracle MySQL client. It also has some limitations, such as, it supports only one password input. You must use -p option in order to pass the password to auth_pam_compat.

These two versions of plugins are physically different. To choose which one you want used, you must use IDENTIFIED WITH ‘auth_pam’ for auth_pam, and IDENTIFIED WITH ‘auth_pam_compat’ for auth_pam_compat.

Version specific information

A plugin may not be supported in later releases of MySQL or Percona Server for MySQL since version changes may introduce incompatible changes.

Installation

This plugin requires manual installation because it isn’t installed by default.

INSTALL PLUGIN auth_pam SONAME 'auth_pam.so';

After the plugin has been installed it should be present in the plugins list. To check if the plugin has been correctly installed and active

SHOW PLUGINS;
Expected output 
...
| auth_pam                       | ACTIVE   | AUTHENTICATION     | auth_pam.so | GPL     |

Configuration

In order to use the plugin, authentication method should be configured. Simple setup can be to use the standard UNIX authentication method (pam_unix).

Note

To use pam_unix, mysql will need to be added to the shadow group in order to have enough privileges to read the /etc/shadow.

A sample /etc/pam.d/mysqld file:

auth       required     pam_unix.so
account    required     pam_unix.so

For added information in the system log, you can expand it to be:

auth       required     pam_warn.so
auth       required     pam_unix.so audit
account    required     pam_unix.so audit

Creating a user

After the PAM plugin has been configured, users can be created with the PAM plugin as authentication method

CREATE USER 'newuser'@'localhost' IDENTIFIED WITH auth_pam;

This will create a user newuser that can connect from localhost who will be authenticated using the PAM plugin. If the pam_unix method is being used user will need to exist on the system.

Supplementary groups support

Percona Server for MySQL has implemented PAM plugin support for supplementary groups. Supplementary or secondary groups are extra groups a specific user is member of. For example user joe might be a member of groups: joe (his primary group) and secondary groups developers and dba. A complete list of groups and users belonging to them can be checked with cat /etc/group command.

This feature enables using secondary groups in the mapping part of the authentication string, like “mysql, developers=joe, dba=mark”. Previously only primary groups could have been specified there. If user is a member of both developers and dba, PAM plugin will map it to the joe because developers matches first.

Known issues

Default mysql stack size is not enough to handle pam_encryptfs module. The workaround is to increase the MySQL stack size by setting the thread-stack variable to at least 512KB or by increasing the old value by 256KB.

PAM authentication can fail with mysqld: pam_unix(mysqld:account): Fork failed: Cannot allocate memory error in the /var/log/secure even when there is enough memory available. Current workaround is to set vm.overcommit_memory to 1:

echo 1 /proc/sys/vm/overcommit_memory

and by adding the vm.overcommit_memory = 1 to /etc/sysctl.conf to make the change permanent after reboot. Authentication of internal (i.e. non PAM) accounts continues to work fine when mysqld reaches this memory utilization level. NOTE: Setting the vm.overcommit_memory to 1 will cause kernel to perform no memory overcommit handling which could increase the potential for memory overload and invoking of OOM killer.

SSL improvements

Percona Server for MySQL passes Elliptic Curve Cryptography (ECC) ciphers to OpenSSL by default.

Note

Although documented as supported, elliptic-curve crypto-based ciphers do not work with MySQL.

FIDO authentication plugin

Percona Server for MySQL supports the Fast Identify Online (FIDO) authentication method that uses a plugin. The FIDO authentication provides a set of standards that reduces the reliance on passwords.

The server-side fido authentication plugin enables authentication using external devices. If this plugin is the only authentication plugin used by the account, this plugin allows authentication without a password. Multi-factor authentication can use non-FIDO MySQL authentication methods, the FIDO authentication method, or a combination of both.

All distributions include the client-side authentication_fido_client plugin. This plugin allows clients to connect to accounts that use authentication_fido and authenticate on a server that has that plugin loaded.

Plugin and library file names

The plugin and library file names are listed in the following table.

Plugin or file name Plugin or library file name
Server-side plugin authentication_fido
Client-side plugin authentication_fido_client
Library file authentication_fido.so

Install the FIDO authentication plugin

The library file must be stored in the directory named by the plugin_dir variable.

At server startup, use the --plugin_load_add option with the library name. The option must be added each time the server starts.

[mysqld]
...
plugin-load-add=authentication_fido.so
...

INSTALL PLUGIN authentication_fido SONAME `authentication_fido.so`;

Verify installation

Use the SHOW PLUGINS statement or query the INFORMATION_SCHEMA.PLUGINS table to verify that the plugin was loaded successfully and is active.

Check the server error log if the plugin is not loaded.

FIDO authentication strategies

FIDO can be used with non-FIDO authentication. FIDO can be used to create 1FA accounts that do not require passwords.

Use FIDO authentication with non-FIDO authentication

A FIDO device is associated with the account using FIDO authentication. The FIDO device must be registered before the account can be used in a one-time process. This device must be available and the user must perform whatever FIDO device action required, such as adding a thumbprint, or the registration fails.

The registration can only be performed by the user named by the account. An error occurs if a user attempts the registration for another user.

The device registration can be performed on the mysql client or MySQL Shell. Use the --fido-register-factor option with the factor or factors for the device. For example, if you are using FIDO as a second authentication method, which is a common practice, the statement is --fido-register-factor=2.

Any authentication factors that proceed the FIDO registration must succeed before the registration continues.

The server checks the user account information to determine if the FIDO device requires registration. If the device must be registered, the server switches the client session to sandbox mode. The registration must be completed before any other activity. In this mode, only ALTER USER statements are permitted. If the session is started with --fido-register-factor, the client generates the statements required to register. After the registration is complete, the session is switched out of sandbox mode and the client can proceed as normal.

After the device is registered, the server updates the mysql.user system table for that account with the device registration status and stores the public key and credential ID.

The user must use the same FIDO device during registration and authentication. If the device is reset or the user attempts to use a different device, the authentication fails. To use a different device, the registered device must be unregistered and you must complete the registration process again.

Use FIDO authentication as the only method

If FIDO is used as the only method of authentication, the method does not use a password. The authentication uses a method such as a biometric scan or a security key.

The user creates an account with the PASSWORDLESS_USER_ADMIN privilege and the CREATE USER privilege.

The first element of the authentication_policy value must be an asterisk(*). Do not start with the plugin name. Configuring the authentication policy value has more information.

You must include the INITIAL AUTHENTICATION IDENTIFIED BY clause in the CREATE USER statement. The server does accept the statement without the clause but the account is unusable because the user cannot connect to the server to register the device.

The CREATE USER syntax is the following:

CREATE USER <username>@<hostname> IDENTIFIED WITH authentication_fido INITIAL AUTHENTICATION IDENTIFIED BY '<password>';

During registration, the user must authenticate with the password. After the device is registered, the server deletes the password and modifies the account to make FIDO the only authentication method.

Unregister a FIDO device

If the FIDO device is replaced or lost, the following actions occur:

Action required Who can perform the action
Unregister the previous device The account owner or any user with the CREATE USER privilege can unregister the device
Register the new device The user planning to use the device must register the new device

Unregister a device with the following statement:

ALTER USER `username`@`hostname` {2|3} FACTOR UNREGISTER;

OpenID Connect authentication

OpenID Connect authentication

Tip

Want to try OIDC end-to-end on a single server? Follow the Get started with OpenID Connect authentication walkthrough. The sections that follow are the full reference.

What is OpenID Connect authentication?

OpenID Connect (OIDC) is an identity layer on top of the OAuth 2.0 framework. The user authenticates against an external Identity Provider (IDP) before connecting to MySQL. Supported providers include Keycloak, Okta, and Microsoft Entra ID.

The IDP issues a signed JSON Web Token (JWT), called the Identity token. The client transmits the token to Percona Server for MySQL during the authentication handshake. The server verifies the token and grants the connection without exchanging a password.

For an overview of the supported authentication methods, see Authentication methods. To compare OIDC with related plugins, see LDAP authentication, PAM authentication, and FIDO authentication. To configure the encrypted transport that OIDC requires, see SSL improvements.

Plugin capabilities

The plugin provides the following capabilities:

  • Verify signed Identity tokens issued by one or more configured Identity Providers.

  • Refresh signing keys from a JSON Web Key Set (JWKS) endpoint at runtime.

  • Enforce an optional aud audience check per IDP.

  • Map IDP group claims to MySQL roles for the duration of the session.

  • Proxy multiple IDP identities to a single MySQL account through GRANT PROXY.

  • Support the signature algorithms listed in Supported signature algorithms.

Proxy support is a Percona-specific addition. The upstream MySQL OIDC plugin does not include this capability.

The server-side plugin pairs with the authentication_openid_connect_client client-side plugin distributed with Percona Server for MySQL.

Supported signature algorithms

The plugin supports the following signature algorithms:

Algorithm Description
RS256 RSASSA-PKCS1-v1_5 with SHA-256
RS384 RSASSA-PKCS1-v1_5 with SHA-384
RS512 RSASSA-PKCS1-v1_5 with SHA-512
ES256 ECDSA with SHA-256
ES384 ECDSA with SHA-384
ES512 ECDSA with SHA-512
PS256 RSASSA-PSS with SHA-256
PS384 RSASSA-PSS with SHA-384
PS512 RSASSA-PSS with SHA-512

Plugin and library file names

The library file must reside in the directory named by the plugin_dir system variable. The file name suffix may differ on your platform.

The following table lists the plugin components:

Plugin or file Name
Server-side plugin auth_openid_connect
Client-side plugin authentication_openid_connect_client
Server library file auth_openid_connect.so
Client library file authentication_openid_connect_client.so

How does OpenID Connect authentication work?

The plugin processes a connection in the following sequence:

  1. The user authenticates against the IDP through an out-of-band flow. The IDP returns a JWT Identity token.

  2. The user writes the token to a file readable by the operating system account that runs the client. The token must not exceed 10 KB.

  3. The client reads the token file referenced by the --authentication-openid-connect-client-id-token-file option. See Connect with a client for the option syntax. The client validates that the file contains a well-formed JWT.

  4. The client confirms the connection uses TLS, a Unix domain socket, or shared memory. The client refuses to send the token over plaintext TCP.

  5. The client transmits the token to the server during the authentication handshake.

  6. The server confirms the connection is secure. The server then receives and decodes the token.

  7. The server validates the token against the configuration of the referenced IDP.

The plugin accepts the token only when every check in the following table passes:

Check Requirement
JWT structure Parses as a valid JWT
Signature algorithm One of the supported signature algorithms (see Supported signature algorithms)
Signature Verified by a configured public key, selected by the kid header
Expiration (exp claim) Timestamp in the future
Issuer (iss claim) Equals the configured issuer-name for the IDP
Subject (sub claim) Equals the user value when IDENTIFIED ... AS includes a user field (direct authentication only)
Audience (aud claim) Matches an allowed audience when audiences is configured

The plugin selects the authentication mode from the fields present in the account’s IDENTIFIED ... AS JSON:

Fields in the AS JSON Mode The plugin authenticates as
identity_provider, user Direct authentication The handshake account (no proxying)
identity_provider, group Named-group proxying The literal value of group
identity_provider only Anonymous proxying The first entry in the token’s groups claim

The sub claim is verified against user only in direct authentication. Proxy modes verify group membership instead.

Both proxy modes require group-claim, so the plugin can read group membership from the token. In named-group proxying, the plugin verifies that the configured group appears in the claim. The session then authenticates as the proxied account named after that group. In anonymous proxying, the plugin uses the first entry in the claim as the proxied account.

In both proxy modes, the proxied account name equals the group value from the token. MySQL limits account names to 32 characters, so group names longer than 32 characters cannot be used by the proxying feature. Prefer group names over group IDs in the claim. See Proxying.

The proxying account must hold PROXY privilege on the proxied account. For end-to-end examples and the safety considerations of proxy modes, see Proxying.

When group-claim and group-role are both configured, the plugin reads the group claim from the token. The plugin grants the connection any matching MySQL roles for the session.

The plugin denies the connection on any failed check. The plugin writes a diagnostic message to the server error log.

The plugin validates the token only at connection time. A connection remains active when the token expires later in the session.

Prerequisites

Before you configure OpenID Connect authentication, gather the resources in the following table:

Resource Description
Identity Provider OIDC-compliant IDP that issues signed Identity tokens
IDP issuer URL URL plus either a JWKS endpoint or a static set of public keys exported as JWKs
Secure transport TLS or a Unix domain socket between the client and the server
Token delivery method Wrapper script or other tool that calls the IDP token endpoint and writes tokens to a file

The plugin has been tested with Keycloak. Any IDP that exposes a standard JWKS endpoint is compatible.

Install the plugin

INSTALL PLUGIN requires the INSERT privilege on mysql.plugin. Editing my.cnf requires file-system access on the server host.

Choose one of the following installation methods:

Run this statement to load the plugin during a session:

INSTALL PLUGIN auth_openid_connect SONAME 'auth_openid_connect.so';
Expected output 
Query OK, 0 rows affected (0.01 sec)

Add the following lines to the [mysqld] section of my.cnf:

[mysqld]
plugin-load-add=auth_openid_connect.so

Restart the server for the change to take effect.

Verify the installation

Run the SHOW PLUGINS statement, or query INFORMATION_SCHEMA.PLUGINS. Confirm that the plugin loaded successfully:

SELECT PLUGIN_NAME, PLUGIN_STATUS
FROM INFORMATION_SCHEMA.PLUGINS
WHERE PLUGIN_NAME = 'auth_openid_connect';
Expected output 
+---------------------+---------------+
| PLUGIN_NAME         | PLUGIN_STATUS |
+---------------------+---------------+
| auth_openid_connect | ACTIVE        |
+---------------------+---------------+
1 row in set (0.00 sec)

A PLUGIN_STATUS value other than ACTIVE, or an empty result set, indicates that the plugin failed to load. Check the server error log for the cause.

Register the update_jwks() UDF

The plugin library also provides the update_jwks() user-defined function (UDF). Register the function once after the plugin loads. The UDF refreshes cached JWKS public keys at runtime. For details, see Refresh JWKS keys.

CREATE FUNCTION update_jwks RETURNS INTEGER
SONAME 'auth_openid_connect.so';
Expected output 
Query OK, 0 rows affected (0.01 sec)

Confirm the function is registered:

SELECT * FROM mysql.func WHERE name = 'update_jwks';
Expected output 
+-------------+-----+------------------------+----------+
| name        | ret | dl                     | type     |
+-------------+-----+------------------------+----------+
| update_jwks |   2 | auth_openid_connect.so | function |
+-------------+-----+------------------------+----------+
1 row in set (0.00 sec)

An empty result set indicates that the function did not register. Check the server error log for the cause.

Configure the plugin

The plugin reads the list of trusted Identity Providers from a single JSON document. The administrator supplies the document through the auth_openid_connect_configuration system variable. The variable accepts the document inline or as a path to a file.

SET GLOBAL requires the SYSTEM_VARIABLES_ADMIN privilege. SET PERSIST_ONLY additionally requires PERSIST_RO_VARIABLES_ADMIN.

Configuration schema

Each top-level key in the JSON document is the IDP name. User accounts reference this name in the IDENTIFIED ... AS clause. The value is an object with the following members:

Key Required Description
audiences No An array of allowed aud claim values. The plugin rejects the token when the aud claim does not match. The plugin omits the audience check when this key is absent.
group-claim No The name of the JWT claim that lists the user group memberships, such as groups. The claim value must be a string or an array of strings. Group names longer than 32 characters cannot be used by the proxying feature.
group-role No An array of single-key objects that map IDP group names to MySQL role names. The mapping pairs with group-claim.
issuer-name Yes The exact value of the iss claim that the IDP issues. The plugin matches this value against the token’s iss claim.
jwks-url When keys is absent The HTTPS URL of the IDP JWKS endpoint. The plugin fetches and caches keys from the URL. The plugin can refresh keys at runtime through the update_jwks() UDF. HTTP URLs are accepted for testing only and emit a warning.
keys When jwks-url is absent An array of JSON Web Key (JWK) objects that verify token signatures. Each entry must include kty (RSA or EC), kid, and the algorithm-specific parameters. RSA keys require n and e. EC keys require crv, x, and y.

Tip

Use jwks-url in production. IDPs rotate signing keys on a schedule, and jwks-url lets the plugin fetch the current keys without a configuration change.

Use keys only when jwks-url is impractical, for example:

  • The IDP runs without a JWKS endpoint, such as a test scenario where tokens are signed by a script.

  • The server cannot reach the JWKS endpoint, such as a deployment behind a firewall that blocks outbound HTTPS. For deployments that route egress through a corporate forward proxy, see Route JWKS traffic through an HTTP proxy.

Set the configuration variable

The system variable auth_openid_connect_configuration accepts a string with one of the following prefixes:

  • JSON:// followed by the configuration JSON inline.

  • FILE:// followed by an absolute path to a file that contains the configuration JSON.

The prefix check is case-insensitive. Setting the variable to a value without a recognized prefix fails with ERROR 1231 (42000). The same error applies when the file cannot be read or parsed. Detailed messages appear in the server error log.

Example configuration

The following JSON document configures two trusted IDPs. The first resolves keys at runtime through a jwks-url. The second uses static keys to verify token signatures.

The n value is truncated for brevity. Replace ptR4...QEASRw with the full base64url-encoded RSA modulus from the IDP signing key.

{
  "my-keycloak": {
    "issuer-name": "https://keycloak.example.com/realms/master",
    "jwks-url": "https://keycloak.example.com/realms/master/protocol/openid-connect/certs",
    "audiences": [ "mysql-oidc" ],
    "group-claim": "groups",
    "group-role": [
      { "/accounting": "accounting" },
      { "/marketing": "marketing" }
    ]
  },
  "oidc-idp": {
    "issuer-name": "https://idp.example.com/realms/dummy",
    "keys": [
      {
        "kid": "rsa-key-1",
        "kty": "RSA",
        "n": "ptR4YxjdrF2RrYiY9XYH3KcXKzlS6b2foGAeHN9dViAs5y...QEASRw",
        "e": "AQAB",
        "use": "sig",
        "alg": "RS256"
      }
    ],
    "audiences": [
      "ee2811b9-10b8",
      "https://api.example.com"
    ],
    "group-claim": "groups",
    "group-role": [
      { "acc": "accounting" },
      { "eng": "engineering" }
    ]
  }
}

The subsections that follow show three ways to load the JSON configuration into the auth_openid_connect_configuration system variable.

Configure in my.cnf

Most deployments set the variable in my.cnf so the configuration applies on every server start. Save the example as /etc/mysql/oidc/idps.json, then add the variable to the [mysqld] section alongside the plugin load directive:

[mysqld]
plugin-load-add=auth_openid_connect.so
auth_openid_connect_configuration='FILE:///etc/mysql/oidc/idps.json'

The plugin-load-add line matches the option shown in Install the plugin. The same prefixes apply to the variable: FILE:// for an external file or JSON:// for an inline document. Inline JSON in my.cnf requires careful quote escaping and is harder to maintain. Prefer the FILE:// form for my.cnf deployments.

The variable also accepts a --auth_openid_connect_configuration command-line argument when starting mysqld directly. This form suits scripted automation.

Restart the server for the change to take effect.

Configure from a file

To set the variable in a running server, use SET GLOBAL with the FILE:// prefix:

SET GLOBAL auth_openid_connect_configuration =
  'FILE:///etc/mysql/oidc/idps.json';
Expected output 
Query OK, 0 rows affected (0.00 sec)

Configure inline

To set the variable in a running server without an external file, use SET GLOBAL with the JSON:// prefix:

SET GLOBAL auth_openid_connect_configuration =
  'JSON://{"my-keycloak":{"issuer-name":"https://keycloak.example.com/realms/master","jwks-url":"https://keycloak.example.com/realms/master/protocol/openid-connect/certs","audiences":["mysql-oidc"]}}';
Expected output 
Query OK, 0 rows affected (0.00 sec)

Persist the configuration

Warning

A SET GLOBAL assignment does not survive a server restart. Persist the configuration through SET PERSIST_ONLY or my.cnf to avoid an authentication outage on the next restart.

Skip this step if you already set the variable in my.cnf. The my.cnf value loads on every server start and does not require SET PERSIST_ONLY.

Otherwise, persist the configuration with the following statement:

SET PERSIST_ONLY auth_openid_connect_configuration =
  'FILE:///etc/mysql/oidc/idps.json';
Expected output 
Query OK, 0 rows affected (0.00 sec)

Create a user (individual account)

In direct authentication, one MySQL account maps to one user identity in the IDP. The mapping is encoded as a JSON object in the IDENTIFIED ... AS clause:

CREATE USER 'mysql_oidc_user'@'%'
  IDENTIFIED WITH 'auth_openid_connect'
  AS '{"identity_provider": "my-keycloak", "user": "4c28d537-a635-4b6d-957f-58e3c8860bcc"}';
Expected output 
Query OK, 0 rows affected (0.01 sec)

The clause requires two fields:

  • identity_provider must match a top-level key in auth_openid_connect_configuration.

  • user must match the sub claim in the Identity tokens that the IDP issues for this user.

For Keycloak, the sub claim contains the user UUID. For other providers, the claim may contain an email address or another stable identifier.

The server validates the JSON at connection time, not at user creation. The connection fails when either field is missing. The connection also fails when the configuration does not contain the referenced IDP.

Grant privileges to the account with GRANT, the same as any other MySQL account.

For deployments where many IDP users share a single MySQL account, see Proxying. One MySQL proxied account maps to one IDP group. The IDP users in that group inherit the same privileges.

Obtain an Identity token

The plugin requires the ID token issued by the IDP. The ID token is distinct from the access token. Production environments use the OAuth 2.0 flow that the IDP recommends. Common choices include authorization code with Proof Key for Code Exchange (PKCE) and device authorization.

The following example uses Keycloak and the password grant. Replace placeholders with values from your IDP:

Password grant is for testing only

The password grant (grant_type=password) is deprecated in OAuth 2.1 and omitted from modern best-practice guidance. Use this flow only for local automation, scripts, and internal testing.

For production CLI clients, use the Device Authorization Grant or authorization code with PKCE. These flows do not require the client to handle user credentials directly.

curl -s -X POST \
  https://keycloak.example.com/realms/master/protocol/openid-connect/token \
  -d 'grant_type=password' \
  -d 'client_id=mysql-oidc' \
  -d 'scope=openid' \
  -d 'username=alice' \
  -d 'password=<password>' \
  | jq -r .id_token > /run/user/1000/id_token.jwt

chmod 600 /run/user/1000/id_token.jwt

The scope=openid parameter is required. Without scope=openid, the IDP returns an access token but no ID token. The client_id must reference a client registered in the IDP. The client must permit the relevant grant type.

Keycloak client configuration

For Keycloak, the ID token aud claim equals the client_id. Set audiences in the plugin configuration to match. The preceding example uses client_id=mysql-oidc, so the configuration uses "audiences": ["mysql-oidc"].

Public clients (Access Type: public) send only the client_id. Confidential clients (Access Type: confidential) require a client_secret parameter:

-d 'client_secret=<CLIENT_SECRET>'

Retrieve the secret from the Keycloak admin console under Clients > mysql-oidc > Credentials.

The file must contain only the raw compact-serialized JWT. The JWT is three base64url segments joined by dots, with no surrounding whitespace, JSON wrapper, or Bearer prefix.

For other IDPs, see the vendor documentation. Examples: Okta token endpoint and Microsoft Entra ID OAuth 2.0 token endpoint .

Connect with a client

A client locates the Identity token file through the --authentication-openid-connect-client-id-token-file option. The plugin reads no other source. The plugin does not consult an environment variable.

Pass the option on the command line:

mysql --host=mysql.example.com \
      --ssl-mode=REQUIRED \
      --user=mysql_oidc_user \
      --authentication-openid-connect-client-id-token-file=/run/user/1000/id_token.jwt

For persistent client configuration, set the option in the [client] section of my.cnf:

[client]
authentication-openid-connect-client-id-token-file=/run/user/1000/id_token.jwt

The MySQL client reads [client] at startup. A command-line value overrides the value from my.cnf.

The client refuses to authenticate when any of the following conditions occur:

  • The --authentication-openid-connect-client-id-token-file option is missing from both the command line and my.cnf.

  • The token file is missing, empty, or larger than 10 KB.

  • The token file does not contain a syntactically valid JWT.

  • The connection between the client and server is not secured by TLS, a Unix socket, or shared memory.

Map groups to MySQL roles

When group-claim and group-role are both configured for an IDP, the plugin inspects the named claim at connection time. The plugin associates each matching MySQL role with the connection. The user activates the role with SET ROLE .

Consider the following configuration fragment:

"group-claim": "groups",
"group-role": [
  { "acc": "accounting" },
  { "eng": "engineering" }
]

Suppose a token has the groups claim with the value ["acc", "hr"]. The connection then exhibits the following behavior:

  • SET ROLE accounting succeeds: acc maps to the accounting role.

  • SET ROLE engineering fails: eng is not present in the token.

  • SET ROLE hr fails: hr has no role mapping.

Grant and revoke external roles at connection time

Percona Server maintains a container of external roles for each user account. External roles are MySQL roles that an authentication plugin grants when a user connects.

At each connection, the server compares the roles returned by the plugin against the roles already recorded in the container:

  • The server grants any returned role that is not yet in the container. Roles already in the container remain granted.

  • The server revokes any external role that exists in the container but was not returned by the plugin.

  • The server updates the container to match the roles actually granted to the session.

The OIDC plugin determines group membership from the group claim in the Identity token. The claim name is configured with group-claim. The plugin maps matching groups to MySQL roles through group-role.

External role privileges are applied at connection time, not continuously. Percona Server does not synchronize with the Identity Provider between connections. A user removed from a group at the IDP retains external role privileges until the next connection. The token on that connection must no longer contain the group.

The roles must already exist on the server. The plugin does not create roles automatically.

The group claim must be a JSON array of strings or a single string. Any other type causes authentication to fail with the message cannot parse groups claim in the token.

Keycloak claim shape

The plugin reads group-claim as a top-level field in the token. The plugin does not traverse nested objects. A claim path such as realm_access.roles is not supported.

Keycloak places realm roles inside realm_access.roles and client roles inside resource_access.<client>.roles by default. Configure a Keycloak client scope mapper that emits a flat top-level claim:

  • For Keycloak groups, add a Group Membership mapper with Full group path enabled. Prefer group names over group IDs. Group names longer than 32 characters cannot be used by the proxying feature.

  • For realm roles, add a User Realm Role mapper with Multivalued enabled.

Set Token Claim Name to match the value of group-claim in the plugin configuration. See the Keycloak Protocol Mappers documentation for the exact UI paths.

Proxying

The plugin supports MySQL proxy users so multiple IDP identities can share a single MySQL account. The proxy target is selected by IDP group membership. For the field combinations that select each mode, see How does OpenID Connect authentication work?.

Proxying reduces administrative overhead in deployments with many IDP users. One MySQL proxied account serves every IDP user in a group. DBAs do not maintain a MySQL account per user, and they do not synchronize IDP and MySQL user lists. The pattern matches how MySQL LDAP authentication is typically deployed.

Warning

Proxy modes do not verify the sub claim. Any token signed by a configured IDP that contains the required group is accepted. Use proxy modes only when group membership is your trust boundary.

Group name length limit

In named-group and anonymous proxying, the plugin maps each group value from the ID token to a MySQL account of the same name. MySQL limits account names to 32 characters.

Group names in the token’s group claim must not exceed 32 characters. A longer value prevents creation of the proxied MySQL account and causes authentication to fail. Keep group names at 32 characters or fewer. If your IDP issues tokens with long group IDs in the groups claim, check whether the IDP can be configured to use group names instead.

Configure the plugin and IDP before you use the proxy examples. The IDP must issue tokens with a groups claim. For the configuration baseline, see Configure the plugin and Map groups to MySQL roles.

The two subsections that follow show end-to-end examples for both proxy modes. Both use the mysql_no_login plugin so proxy target accounts cannot authenticate directly.

Named-group proxying

In named-group proxying, the connecting MySQL user names the IDP group for the session. Named-group proxying suits users who belong to multiple groups and need to choose between them per session.

Create one MySQL account per group. The group field in the AS JSON pins the account to one IDP group:

CREATE USER 'accounting'@'%'
  IDENTIFIED WITH 'auth_openid_connect'
  AS '{"identity_provider": "my-keycloak", "group": "/accounting"}';

CREATE USER 'marketing'@'%'
  IDENTIFIED WITH 'auth_openid_connect'
  AS '{"identity_provider": "my-keycloak", "group": "/marketing"}';
Expected output 
Query OK, 0 rows affected (0.01 sec)
Query OK, 0 rows affected (0.01 sec)

Create a mysql_no_login proxy target for each group, then grant PROXY:

CREATE USER '/accounting'@'%' IDENTIFIED WITH 'mysql_no_login';
GRANT PROXY ON '/accounting'@'%' TO 'accounting'@'%';

CREATE USER '/marketing'@'%' IDENTIFIED WITH 'mysql_no_login';
GRANT PROXY ON '/marketing'@'%' TO 'marketing'@'%';
Expected output 
Query OK, 0 rows affected (0.01 sec)
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.01 sec)
Query OK, 0 rows affected (0.00 sec)

The proxy target accounts hold the privileges that the proxied session inherits. Grant privileges to the targets with GRANT, the same as any other MySQL account.

The token holder selects a group by choosing the matching MySQL user at connect time. The token must include the chosen group in the groups claim:

mysql -u marketing \
      --authentication-openid-connect-client-id-token-file=id_token.jwt

After connecting, the session reflects the proxied target:

SELECT USER(), CURRENT_USER();
Expected output 
+---------------------+----------------+
| USER()              | CURRENT_USER() |
+---------------------+----------------+
| marketing@localhost | /marketing@%   |
+---------------------+----------------+
1 row in set (0.00 sec)

USER() shows the handshake account (marketing). CURRENT_USER() shows the proxied target (/marketing@%), which holds the privileges the session inherits.

Anonymous proxying

In anonymous proxying, the connecting username is irrelevant. The plugin proxies to a target named after the first group in the token’s groups claim.

Group ordering hazard

The plugin selects the proxy target from index 0 of the groups claim. JSON arrays preserve insertion order, but the IDP controls that order. An alphabetical sort, an added group, or an IDP policy change can shift index 0 silently. The session then inherits a different proxy target without warning.

Anonymous proxying is safe only when one of these conditions holds:

  • Each IDP user belongs to exactly one privilege-bearing group.

  • The IDP guarantees a stable position for the privileged group at index 0.

Otherwise, use Named-group proxying. The target is selected by the IDENTIFIED ... AS clause and is not affected by token ordering.

Create a single anonymous MySQL account that any handshake username matches:

CREATE USER ''@''
  IDENTIFIED WITH 'auth_openid_connect'
  AS '{"identity_provider": "my-keycloak"}';
Expected output 
Query OK, 0 rows affected (0.01 sec)

Create a mysql_no_login proxy target for each group, then grant PROXY to the anonymous account:

CREATE USER '/accounting'@'%' IDENTIFIED WITH 'mysql_no_login';
GRANT PROXY ON '/accounting'@'%' TO ''@'';

CREATE USER '/marketing'@'%' IDENTIFIED WITH 'mysql_no_login';
GRANT PROXY ON '/marketing'@'%' TO ''@'';
Expected output 
Query OK, 0 rows affected (0.01 sec)
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.01 sec)
Query OK, 0 rows affected (0.00 sec)

Connect with any handshake username. The first group in the token selects the proxied target:

mysql -u anyname \
      --authentication-openid-connect-client-id-token-file=id_token.jwt

After connecting, the session reflects the proxied target:

SELECT USER(), CURRENT_USER();
Expected output 
+-------------------+----------------+
| USER()            | CURRENT_USER() |
+-------------------+----------------+
| anyname@localhost | /marketing@%   |
+-------------------+----------------+
1 row in set (0.00 sec)

If a token has multiple groups, only the first group selects the target. To make the choice explicit, use named-group proxying instead.

Refresh JWKS keys

When you configure an IDP with jwks-url, the plugin loads keys at startup. The plugin also reloads keys on every assignment to auth_openid_connect_configuration. Keys may rotate at the IDP between configuration changes. The update_jwks() UDF refreshes cached keys without changing the configuration. Register the UDF as part of plugin installation. See Register the update_jwks() UDF.

The plugin does not refresh keys automatically. The plugin ignores HTTP cache headers such as Cache-Control, Expires, and ETag. Cached keys remain valid until a refresh trigger fires.

The plugin defines three refresh triggers:

  • Server start with a configured jwks-url

  • Assignment to auth_openid_connect_configuration

  • A successful call to update_jwks()

Schedule the UDF when your IDP rotates keys on a fixed cadence. See Schedule periodic refresh for an event-based example.

Call the UDF without arguments to refresh keys for every configured IDP that has a jwks-url:

SELECT update_jwks();
Expected output 
+---------------+
| update_jwks() |
+---------------+
|             1 |
+---------------+
1 row in set (0.05 sec)

Call the UDF with one string argument to refresh a single IDP:

SELECT update_jwks('my-keycloak');
Expected output 
+-----------------------------+
| update_jwks('my-keycloak')  |
+-----------------------------+
|                           1 |
+-----------------------------+
1 row in set (0.04 sec)

The following table lists the return values:

Return value Meaning
>= 0 The number of IDPs whose keys were successfully refreshed.
-1 The named IDP is not in the configuration.
-2 An unexpected error occurred during the refresh. Details are written to the server error log.

Schedule periodic refresh

Schedule the UDF to align the cache with the key rotation policy of the IDP. A MySQL event is the simplest option. The event runs inside the server. External tooling such as cron or a Kubernetes CronJob is a valid alternative.

The following event refreshes keys for every configured IDP once per hour:

CREATE EVENT update_oidc_keys
  ON SCHEDULE EVERY 1 HOUR
  DO SELECT update_jwks();
Expected output 
Query OK, 0 rows affected (0.00 sec)

The event scheduler must be running for the event to fire. Confirm and enable the scheduler at runtime:

SHOW VARIABLES LIKE 'event_scheduler';
SET GLOBAL event_scheduler = ON;

To enable the scheduler on every server start, set the variable in my.cnf:

[mysqld]
event_scheduler=ON

The account that runs CREATE EVENT requires the EVENT privilege on the schema that holds the event.

Choose an interval that matches the key rotation cadence of the IDP. Hourly suits most deployments. Consult the IDP documentation for the exact cadence.

Route JWKS traffic through an HTTP proxy

The plugin fetches jwks-url content with libcurl. The plugin sets no proxy options of its own. The IDP configuration JSON exposes no proxy field.

Libcurl reads standard proxy environment variables from the mysqld process environment:

  • https_proxy for HTTPS JWKS endpoints

  • http_proxy for HTTP JWKS endpoints

  • no_proxy for hostnames that bypass the proxy

The variables must reach mysqld, not the interactive shell that the DBA uses. Set them in the service unit on systems that use systemd:

[Service]
Environment="https_proxy=http://proxy.example.com:8080"
Environment="no_proxy=keycloak.internal,.example.lan,127.0.0.1"

After editing the unit, reload systemd and restart mysqld:

systemctl daemon-reload
systemctl restart mysqld

Restart required

The plugin reads the environment at process start. The plugin applies updated proxy values only after mysqld restarts.

For environments that block all outbound traffic, configure each IDP with a static keys array instead of jwks-url. See Configuration schema for the required JWK fields. Distribute the JWK set to the database host out of band.

Uninstall the plugin

Warning

Drop every UDF that depends on the plugin before you uninstall the plugin. A stale function definition remains when UNINSTALL PLUGIN runs while a UDF still references the library. Drop scheduled events that call the UDF as well.

If you scheduled a refresh event, drop the event first:

DROP EVENT IF EXISTS update_oidc_keys;
Expected output 
Query OK, 0 rows affected (0.00 sec)

Drop the UDF:

DROP FUNCTION IF EXISTS update_jwks;
Expected output 
Query OK, 0 rows affected (0.00 sec)

Uninstall the plugin:

UNINSTALL PLUGIN auth_openid_connect;
Expected output 
Query OK, 0 rows affected (0.00 sec)

System variable reference

auth_openid_connect_configuration

The following table summarizes the system variable. For the full list of system variables, see Percona Server system variables.

Variable name Default value Scope Dynamic Valid values
auth_openid_connect_configuration {} Global Yes A string prefixed with JSON:// or FILE://

The variable holds the configuration of trusted OpenID Connect Identity Providers. The value must start with JSON:// for an inline JSON document. The value must start with FILE:// for an absolute path to a JSON file. The configuration schema appears in Configure the plugin.

The variable accepts assignments from SET GLOBAL, SET PERSIST_ONLY, the my.cnf option file, and the mysqld command line. For examples, see Set the configuration variable.

The plugin validates the variable at assignment. The server rejects an unknown prefix, malformed JSON, or an unreadable file with ERROR 1231 (42000).

A configuration that references an unreachable jwks-url is accepted at assignment. The plugin writes a warning to the server error log. Authentication against that IDP fails until the keys load successfully. Run update_jwks() after the IDP becomes reachable.

User-defined function reference

update_jwks()

The following table summarizes the function:

Property Value
Return type INTEGER
Library auth_openid_connect.so
Arguments Zero or one string

The function refreshes cached JWKS public keys for one IDP or for all configured IDPs. For details on return values, see Refresh JWKS keys.

Troubleshoot connection failures

The plugin writes a single diagnostic line to the server error log on each rejected connection. The client receives a generic authentication failure to avoid leaking configuration details.

The following table follows a Symptoms, Diagnosis, and Solution model for the most frequent failures:

Symptom Diagnosis Solution
unsecure connection, use TLS, socket or memory in the server log The client connected over plaintext TCP. Reconnect with TLS, a Unix socket, or shared memory.
IDP not found: <name> in the server log The identity_provider value in the user IDENTIFIED ... AS clause does not match a key in the configuration. Update the user definition or the configuration so the IDP names align.
audience not authorized in the server log The token aud claim does not match any value in the audiences configuration. For Keycloak, the ID token aud equals the client_id. Set audiences to match the aud claim in the ID token. For Keycloak, use the client_id value (for example, ["mysql-oidc"]).
invalid sysvar prefix, expected FILE:// or JSON:// in the server log The auth_openid_connect_configuration value lacks a valid prefix. Reset the variable with the correct JSON:// or FILE:// prefix.
JWKS configuration is insecure, use HTTPS warning in the server log The jwks-url uses http://. Replace the URL with an https:// endpoint before production use.
JWKS: HTTP GET from <url> failed in the server log The IDP is unreachable or returned a non-2xx status. The host may also require an outbound HTTP proxy. Verify network reachability. For corporate egress, see Route JWKS traffic through an HTTP proxy. Run update_jwks() after the IDP recovers.
incorrect number of keys in the server log The token has no kid header but the plugin loaded multiple keys for the IDP. Set keys to a single entry that matches the IDP signing key.
user is not a member of the required group in the server log The account uses named-group proxying but the token’s groups claim does not contain the configured group. Verify the user’s group membership at the IDP, or have the user select a different MySQL account that maps to a group they belong to.
CREATE USER fails for a proxy target, or proxy authentication fails The group value in the token exceeds the 32-character MySQL account name limit, or the claim contains a group ID instead of a group name. Group names longer than 32 characters cannot be used by the proxying feature. Prefer group names over group IDs. See Proxying.

Get started with OpenID Connect authentication

Configure OpenID Connect (OIDC) authentication against a Keycloak realm in the following walkthrough. You install the server-side plugin, configure trust for Keycloak, create a MySQL user, obtain an ID token, and connect. Allow about 20 minutes. For production guidance, the configuration schema, and troubleshooting, see OpenID Connect authentication.

Before you begin

Gather the resources in the following list:

  • Percona Server for MySQL 8.4 with the auth_openid_connect.so library in plugin_dir.

  • A reachable Keycloak server with a configured realm, an OIDC client, and at least one user assigned to a group.

  • TLS, a Unix domain socket, or shared memory between the MySQL client and the server.

  • curl and jq on the workstation that obtains the ID token.

  • A MySQL administrator account with the INSERT privilege on mysql.plugin, the SYSTEM_VARIABLES_ADMIN privilege, and the CREATE USER privilege.

Quickstart values

The walkthrough uses the following example values. Replace each one with the equivalent value from your environment.

Item Example value Description
Keycloak host keycloak.example.com Hostname of the Keycloak server
Realm master Keycloak realm name
OIDC client ID mysql-oidc Client registered in the realm
IDP name in MySQL my-keycloak Top-level key in the JSON configuration
Keycloak username alice User in the realm
sub claim value 4c28d537-a635-4b6d-957f-58e3c8860bcc User UUID issued by Keycloak
Keycloak group /accounting Group assigned to the user
MySQL role accounting Role mapped to the group

Install the OIDC plugin

Load the plugin during the current server session:

INSTALL PLUGIN auth_openid_connect SONAME 'auth_openid_connect.so';
Expected output 
Query OK, 0 rows affected (0.01 sec)

Confirm the plugin loaded:

SELECT PLUGIN_NAME, PLUGIN_STATUS
FROM INFORMATION_SCHEMA.PLUGINS
WHERE PLUGIN_NAME = 'auth_openid_connect';
Expected output 
+---------------------+---------------+
| PLUGIN_NAME         | PLUGIN_STATUS |
+---------------------+---------------+
| auth_openid_connect | ACTIVE        |
+---------------------+---------------+
1 row in set (0.00 sec)

To load the plugin at server start instead, see Install the plugin.

Configure trust for Keycloak

Set the auth_openid_connect_configuration system variable to a JSON document that describes the Keycloak realm. The plugin fetches signing keys from the JWKS endpoint of the realm:

SET GLOBAL auth_openid_connect_configuration = 'JSON://{
  "my-keycloak": {
    "issuer-name": "https://keycloak.example.com/realms/master",
    "jwks-url": "https://keycloak.example.com/realms/master/protocol/openid-connect/certs",
    "audiences": ["mysql-oidc"],
    "group-claim": "groups",
    "group-role": [
      { "/accounting": "accounting" }
    ]
  }
}';
Expected output 
Query OK, 0 rows affected (0.01 sec)

SET GLOBAL lasts only until the next restart. To persist the configuration, run the same statement with SET PERSIST_ONLY. See Persist the configuration.

Create a role, a database, and an OIDC user

Create a small database that the OIDC user will read through the role:

CREATE DATABASE oidc_demo;
CREATE TABLE oidc_demo.invoices (id INT, amount DECIMAL(10, 2));
INSERT INTO oidc_demo.invoices VALUES (1, 99.95), (2, 145.00);
Expected output 
Query OK, 1 row affected (0.00 sec)
Query OK, 0 rows affected (0.01 sec)
Query OK, 2 rows affected (0.00 sec)

Create the MySQL role and grant access to the table:

CREATE ROLE accounting;
GRANT SELECT ON oidc_demo.* TO accounting;
Expected output 
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.00 sec)

Create the OIDC-authenticated user. Replace the user value with the sub claim that Keycloak issues for the account:

CREATE USER 'alice'@'%'
  IDENTIFIED WITH 'auth_openid_connect'
  AS '{"identity_provider": "my-keycloak", "user": "4c28d537-a635-4b6d-957f-58e3c8860bcc"}';
Expected output 
Query OK, 0 rows affected (0.01 sec)

The OIDC user does not need an explicit GRANT for oidc_demo. The user inherits access from the accounting role when the token includes the /accounting group.

Obtain an ID token from Keycloak

Run the following on the workstation. The script writes the ID token to /run/user/1000/id_token.jwt. Replace the placeholder password with the password for alice:

curl -s -X POST \
  https://keycloak.example.com/realms/master/protocol/openid-connect/token \
  -d 'grant_type=password' \
  -d 'client_id=mysql-oidc' \
  -d 'scope=openid' \
  -d 'username=alice' \
  -d 'password=<password>' \
  | jq -r .id_token > /run/user/1000/id_token.jwt

chmod 600 /run/user/1000/id_token.jwt

The password grant suits scripted demos. Production deployments use authorization code with Proof Key for Code Exchange (PKCE) or device authorization. For details on each step, see Obtain an Identity token.

Note

The plugin requires the ID token, not the access token. The preceding example selects the id_token field from the Keycloak response.

The example assumes a public Keycloak client. Confidential clients require an additional -d 'client_secret=<SECRET>' parameter. See Obtain an Identity token for details.

Connect with the OIDC token

Connect as alice and pass the token file:

mysql --host=mysql.example.com \
      --ssl-mode=REQUIRED \
      --user=alice \
      --authentication-openid-connect-client-id-token-file=/run/user/1000/id_token.jwt

A successful authentication opens the MySQL prompt. The plugin denies the connection on a missing token, an expired token, or an unsecured transport.

Verify the role mapping

Confirm the connected identity:

SELECT CURRENT_USER();
Expected output 
+----------------+
| CURRENT_USER() |
+----------------+
| alice@%        |
+----------------+
1 row in set (0.00 sec)

Activate the role and read from the table:

SET ROLE accounting;
SELECT * FROM oidc_demo.invoices;
Expected output 
Query OK, 0 rows affected (0.00 sec)

+------+--------+
| id   | amount |
+------+--------+
|    1 |  99.95 |
|    2 | 145.00 |
+------+--------+
2 rows in set (0.00 sec)

SET ROLE accounting succeeds when the token contains /accounting in the groups claim. A user without the group sees ERROR 3530 (HY000).

Clean up

Remove the demo objects when you finish:

DROP USER 'alice'@'%';
DROP ROLE accounting;
DROP DATABASE oidc_demo;
Expected output 
Query OK, 0 rows affected (0.01 sec)
Query OK, 0 rows affected (0.00 sec)
Query OK, 1 row affected (0.01 sec)

To remove the plugin and any UDFs, see Uninstall the plugin.

Additional resources

Encryption functions

Percona Server for MySQL adds encryption functions and variables to manage the encryption range. The functions may take an algorithm argument. Encryption converts plaintext into ciphertext using a key and an encryption algorithm.

You can also use the user-defined functions with the PEM format keys generated externally by the OpenSSL utility.

A digest uses plaintext and generates a hash value. This hash value can verify if the plaintext is unmodified. You can also sign or verify on digests to ensure that the original plaintext was not modified. You cannot decrypt the original text from the hash value.

When choosing key lengths, consider the following:

  • Encryption strength increases with the key size and, also, the key generation time.

  • If performance is important and the functions are frequently used, use symmetric encryption. Symmetric encryption functions are faster than asymmetric encryption functions. Moreover, asymmetric encryption has restrictions on the maximum length of a message being encrypted. For example, for RSA the algorithm maximum message size is the key length in bytes (key length in bits / 8) minus 11.

Version updates

Percona Server for MySQL 8.4.4 adds the following:

  • Support for pkcs1, oaep, or no padding for RSA encrypt and decrypt operations

    pkcs1 padding explanation RSAES-PKCS1-v1_5 RSA encryption padding scheme prevents patterns that attackers could exploit by including a random sequence of bytes, which ensures that the ciphertext is different no matter how many times it is encrypted.
    oaep padding explanation
    The RSAES-OAEP - Optimal Asymmetric Encryption Padding RSA encryption padding scheme adds a randomized mask generation function. This function makes it more difficult for attackers to exploit the encryption algorithm’s weaknesses or recover the original message.
    no padding explanation Using no padding means the plaintext message is encrypted without adding an extra layer before performing the RSA encryption operation.

  • Support for pkcs1 or pkcs1_pss padding for RSA sign and verify operations

    pkcs1 padding explanation The RSASSA-PKCS1-v1_5 is a deterministic RSA signature padding scheme that hashes a message, pads the hash with a specific structure, and encrypts it with the signer’s private key for signature generation.
    pkcs1_pss padding explanation The RSASSA-PSS - `Probabilistic Signature Scheme’ is an RSA signature padding scheme used to add randomness to a message before signing it with a private key. This randomness helps to increase the security of the signature and makes it more resistant to various attacks.

  • encryption_udf.legacy_paddding system variable

  • Character set awareness

Charset Awareness

All component_encryption_udf functions now handle character sets intelligently:

• Algorithms, digest names, padding schemes, keys, and parameters in PEM format: Automatically converted to the ASCII charset at the MySQL level before passing to the functions.

• Messages, data blocks, and signatures used for digest calculation, encryption, decryption, signing, or verification: Automatically converted to the binary charset at the MySQL level before passing to the functions.

• Function return values in PEM format: Assigned the ASCII charset.

• Function return values for operations like digest calculation, encryption, decryption, and signing: Assigned the binary charset.

Use user-defined functions

You can also use the user-defined functions with the PEM format keys generated externally by the OpenSSL utility.

A digest uses plaintext and generates a hash value. This hash value can verify if the plaintext is unmodified. You can also sign or verify on digests to ensure that the original plaintext was not modified. You cannot decrypt the original text from the hash value.

When choosing key lengths, consider the following:

  • Encryption strength increases with the key size and generation time.

  • If performance is essential and the functions are frequently used, use symmetric encryption. Symmetric encryption functions are faster than asymmetric encryption functions. Moreover, asymmetric encryption restricts the maximum length of a message being encrypted. For example, the algorithm’s maximum message size for RSA is the key length in bytes (key length in bits / 8) minus 11.

Install component_encryption_udf

Use the Install Component Statement to add the component_encryption_udf component. The functions and variables are available. The user-defined functions and the Encryption threshold variables are auto-registered. There is no requirement to invoke CREATE FUNCTION ... SONAME ....

The INSERT privilege on the mysql.component system table is required to run the INSTALL COMPONENT statement. The operation adds a row to this table to register the component.

The following is an example of the installation command:

INSTALL COMPONENT 'file://component_encryption_udf';

Note

When you build Percona Server for MySQL from source code, the Encryption UDF component is included by default. To exclude it, use the -DWITH_ENCRYPTION_UDF=OFF option with cmake.

Functions

The following table and sections describe the functions. For examples, see function examples.

Function Name
asymmetric_decrypt()
asymmetric_derive()
asymmetric_encrypt()
asymmetric_sign()
asymmetric_verify()
create_asymmetric_priv_key()
create_asymmetric_pub_key()
create_dh_parameters()
create_digest()

The following table describes the encryption threshold variables which can be used to set the maximum value for a key length based on the type of encryption used.

Variable Name
encryption_udf.dh_bits_threshold
encryption_udf.dsa_bits_threshold
encryption_udf.legacy_padding
encryption_udf.rsa_bits_threshold

User-defined functions described

asymmetric_decrypt(algorithm, crypt_str, key_str)

Decrypts an encrypted string using the algorithm and a key string.

Returns

A plaintext as a string.

Parameters

The following are the function’s parameters:

  • algorithm - the encryption algorithm supports RSA in decrypting the string.

  • crypt_str - an encrypted string produced by certain encryption functions like AES_ENCRYPT(). This string is typically stored as a binary or blog data type.

  • key_str - a string in the PEM format. The key string must have the following attributes:

  • Valid

  • Public or private key string corresponding with the private or public key string used with the asymmetric_encrypt function.

  • padding - An optional parameter introduced in Percona Server for MySQL 8.4.4. It is used with the RSA algorithm and supports RSA encryption padding schemes like pkcs1, or oaep. If you skip this parameter, the system determines its value based on the encryption_udf.legacy_padding variable.

asymmetric_derive(pub_key_str, priv_key_str)

Derives a symmetric key using a public key generated on one side and a private key generated on another.

asymmetric_derive output

A key as a binary string.

asymmetric_derive parameters

The pub_key_str must be a public key in the PEM format and generated using the Diffie-Hellman (DH) algorithm.

The priv_key_str must be a private key in the PEM format and generated using the Diffie-Hellman (DH) algorithm.

asymmetric_encrypt(algorithm, str, key_str)

Encrypts a string using the algorithm and a key string.

asymmetric_encrypt output

A ciphertext as a binary string.

asymmetric_encrypt parameters

The parameters are the following:

  • algorithm - the encryption algorithm supports RSA to encrypt the string.

  • str - measured in bytes. The length of the string must not be greater than the key_str modulus length in bytes - 11 (additional bytes used for PKCS1 padding)

  • key_str - a key (either private or public) in the PEM format

  • padding - An optional parameter introduced in Percona Server for MySQL 8.4.4. It is used with the RSA algorithm and supports RSA encryption padding schemes like pkcs1, or oaep. If you skip this parameter, the system determines its value based on the encryption_udf.legacy_padding variable.

asymmetric_sign(algorithm, digest_str, priv_key_str, digest_type)

Signs a digest string using a private key string.

asymmetric_sign output

A signature is a binary string.

asymmetric_sign parameters

The parameters are the following:

  • algorithm - the encryption algorithm supports either RSA or DSA in encrypting the string.

  • digest_str - the digest binary string that is signed. Invoking create_digest generates the digest.

  • priv_key_str - the private key used to sign the digest string. The key must be in the PEM format.

  • digest_type - the OpenSSL version installed on your system determines the available hash functions. The following table lists these functions:

    OpenSSL 1.0.2 OpenSSL 1.1.0 OpenSSL 1.1.1 OpenSSL 3.0.x
    md5 md5 md5 md5
    sha1 sha1 sha1 sha1
    sha224 sha224 sha224 sha224
    sha384 sha384 sha384 sha384
    sha512 sha512 sha512 sha512
    md4 md4 md4 md4
    sha md5-sha1 md5-sha1 md5-sha1
    ripemd160 ripemd160 ripemd160 sha512-224
    whirlpool whirlpool sha512-224 sha512-256
    blake2b512 sha512-256 sha3-224
    blake2s256 whirlpool sha3-256
    sm3 sha3-384
    blake2b512 sha3-512
    blake2s256
    sha3-224
    sha3-384
    sha3-512
    shake128
    shake256

  • padding - An optional parameter introduced in Percona Server for MySQL 8.4.4. It is used with the RSA algorithm and supports RSA signature padding schemes like pkcs1, or pkcs1_pss. If you skip this parameter, the system determines its value based on the encryption_udf.legacy_padding variable.

asymmetric_verify(algorithm, digest_str, sig_str, pub_key_str, digest_type)

Verifies whether the signature string matches the digest string.

asymmetric_verify output

A 1 (success) or a 0 (failure).

asymmetric_verify parameters

The parameters are the following:

  • algorithm - supports either ‘RSA’ or ‘DSA’.

  • digest_str - invoking create_digest generates this digest binary string.

  • sig_str - the signature binary string. Invoking asymmetric_sign generates this string.

  • pub_key_str - the signer’s public key string. This string must correspond to the private key passed to asymmetric_sign to generate the signature string. The string must be in the PEM format.

  • digest_type - the supported values are listed in the digest type table of create_digest

  • padding - An optional parameter introduced in Percona Server for MySQL 8.4.4. It is used with the RSA algorithm and supports RSA signature padding schemes like pkcs1, or pkcs1_pss. If you skip this parameter, the system determines its value based on the encryption_udf.legacy_padding variable.

create_asymmetric_priv_key(algorithm,(key_len | dh_parameters))

Generates a private key using the given algorithm and key length for RSA or DSA or Diffie-Hellman parameters for DH. For RSA or DSA, if needed, execute KILL [QUERY|CONNECTION] <id> to terminate a long-lasting key generation. The DH key generation from existing parameters is a quick operation. Therefore, it does not make sense to terminate that operation with KILL.

create_asymmetric_priv_key output

The key as a string in the PEM format.

create_asymmetric_priv_key parameters

The parameters are the following:

  • algorithm - the supported values are ‘RSA’, ‘DSA’, or ‘DH’.

  • key_len - the supported key length values are the following:

    • RSA - the minimum length is 1,024. The maximum length is 16,384.

    • DSA - the minimum length is 1,024. The maximum length is 9,984.

    Note

    The key length limits are defined by OpenSSL. To change the maximum key length, use either encryption_udf.rsa_bits_threshold or encryption_udf.dsa_bits_threshold.

  • dh_parameters - Diffie-Hellman (DH) parameters. Invoking create_dh_parameter creates the DH parameters.

create_asymmetric_pub_key(algorithm, priv_key_str)

Derives a public key from the given private key using the given algorithm.

create_asymmetric_pub_key output

The key as a string in the PEM format.

create_asymmetric_pub_key parameters

The parameters are the following:

  • algorithm - the supported values are ‘RSA’, ‘DSA’, or ‘DH’.

  • priv_key_str - must be a valid key string in the PEM format.

create_dh_parameters(key_len)

Creates parameters for generating a Diffie-Hellman (DH) private/public key pair. If needed, execute KILL [QUERY|CONNECTION] <id> to terminate the generation of long-lasting parameters.

Generating the DH parameters can take more time than generating the RSA keys or the DSA keys. OpenSSL defines the parameter length limits. To change the maximum parameter length, use encryption_udf.dh_bits_threshold.

create_dh_parameters output

A string in the PEM format and can be passed to create_asymmetric_priv_key.

create_dh_parameters parameters

The parameters are the following:

  • key_len - the range for the key length is from 1024 to 10,000. The default value is 10,000.

create_digest(digest_type, str)

Creates a digest from the given string using the given digest type. The digest string can be used with asymmetric_sign() and asymmetric_verify().

create_digest output

The digest of the given string as a binary string

create_digest parameters

The parameters are the following:

  • digest_type - the OpenSSL version installed on your system determines the available hash functions. The following table lists these functions:

    OpenSSL 1.0.2 OpenSSL 1.1.0 OpenSSL 1.1.1 OpenSSL 3.0.x
    md5 md5 md5 md5
    sha1 sha1 sha1 sha1
    sha224 sha224 sha224 sha224
    sha384 sha384 sha384 sha384
    sha512 sha512 sha512 sha512
    md4 md4 md4 md4
    sha md5-sha1 md5-sha1 md5-sha1
    ripemd160 ripemd160 ripemd160 sha512-224
    whirlpool whirlpool sha512-224 sha512-256
    blake2b512 sha512-256 sha3-224
    blake2s256 whirlpool sha3-256
    sm3 sha3-384
    blake2b512 sha3-512
    blake2s256 sm3
    sha3-224 blake2b512
    sha3-384 blake2s256
    sha3-512 blake2b512
    shake128 blake2s256
    shake256

  • str - String used to generate the digest string.

Encryption threshold variables

The maximum key length limits are defined by OpenSSL. Server administrators can limit the maximum key length using the encryption threshold variables.

The variables are automatically registered when component_encryption_udf is installed.

Variable Name
encryption_udf.dh_bits_threshold

encryption_udf.dh_bits_threshold

The variable sets the maximum limit for the create_dh_parameters() user-defined function and takes precedence over the OpenSSL maximum length value.

Option Description
command-line Yes
scope Global
data type unsigned integer
default 10000

The range for this variable is from 1024 to 10,000. The default value is 10,000.

encryption_udf.dsa_bits_threshold

The variable sets the threshold limits for create_asymmetric_priv_key() user-defined function when the function is invoked with the DSA parameter and takes precedence over the OpenSSL maximum length value.

Option Description
command-line Yes
scope Global
data type unsigned integer
default 9984

The range for this variable is from 1,024 to 9,984. The default value is 9,984.

encryption_udf.legacy_padding

The variable enables or disables the legacy padding scheme for certain encryption operations.

Option Description
command-line Yes
scope Global
data type Boolean
default OFF

This system variable is a BOOLEAN type and set to OFF by default.

This variable controls how the functions asymmetric_encrypt(), asymmetric_decrypt(), asymmetric_sign(), and asymmetric_verify() behave when you don’t explicitly set the padding parameter.

The asymmetric_encrypt() and asymmetric_decrypt() functions, when the encryption is RSA, can accept an optional parameter, padding. You can set this parameter to no, pkcs1, or oaep. If you don’t specify this parameter, it defaults based on the encryption_udf.legacy_padding value.

The padding schemes have the following limitations:

Padding Scheme Details
oaep The message you encrypt can be as long as your RSA key size in bytes - 42 bytes.
no The message length must exactly match your RSA key size in bytes. For example, if your key is 1024 bits (128 bytes), the message must also be 128 bytes. If it doesn’t match, it will cause an error.
pkcs1 Your message can be equal to or smaller than the RSA key size - 11 bytes. For instance, with a 1024-bit RSA key, your message can’t be longer than 117 bytes.

Similarly, asymmetric_sign() and asymmetric_verify() also have an optional padding parameter, either pkcs1 or pkcs1_pss. If not explicitly set, it follows the default based on encryption_udf.legacy_padding. You can only use the padding parameter with RSA algorithms.

Additional resources

For more information, read Digital Signatures: Another layer of Data Protection in Percona Server for MySQL

encryption_udf.rsa_bits_threshold

The variable sets the threshold limits for the create_asymmetric_priv_key user-defined function when the function is invoked with the RSA parameter and takes precedence over the OpenSSL maximum length value.

Option Description
command-line Yes
scope Global
data type unsigned integer
default 16384

The range for this variable is from 1,024 to 16,384. The default value is 16,384.

Examples

Code examples for the following operations:

  • Set the threshold variables

  • Create a private key

  • Create a public key

  • Encrypt data

  • Decrypt data

-- Set Global variable
SET GLOBAL encryption_udf.dh_bits_threshold = 4096;

-- Set Global variable
SET GLOBAL encryption_udf.rsa_bits_threshold = 4096;

-- Create private key
SET @private_key = create_asymmetric_priv_key('RSA', 3072);

-- Create public key
SET @public_key = create_asymmetric_pub_key('RSA', @private_key);

-- Encrypt data using the private key (you can also use the public key)
SET @ciphertext = asymmetric_encrypt('RSA', 'This text is secret', @private_key);

-- Decrypt data using the public key (you can also use the private key)
-- The decrypted value @plaintext should be identical to the original 'This text is secret'
SET @plaintext = asymmetric_decrypt('RSA', @ciphertext, @public_key);

Code examples for the following operations:

  • Generate a digest string

  • Generate a digest signature

  • Verify the signature against the digest

-- Generate a digest string
SET @digest = create_digest('SHA256', 'This is the text for digest');

-- Generate a digest signature
SET @signature = asymmetric_sign('RSA', @digest, @private_key, 'SHA256');

-- Verify the signature against the digest
-- The @verify_signature must be equal to 1
SET @verify_signature = asymmetric_verify('RSA', @digest, @signature, @public_key, 'SHA256');

Code examples for the following operations:

  • Generate a DH parameter

  • Generates two DH key pairs

  • Generate a symmetric key using the public_1 and the private_2

  • Generate a symmetric key using the public_2 and the private_1

 -- Generate a DH parameter
SET @dh_parameter = create_dh_parameters(3072);

 -- Generate DH key pairs
SET @private_1 = create_asymmetric_priv_key('DH', @dh_parameter);
SET @public_1 = create_asymmetric_pub_key('DH', @private_1);
SET @private_2 = create_asymmetric_priv_key('DH', @dh_parameter);
SET @public_2 = create_asymmetric_pub_key('DH', @private_2);

-- Generate a symmetric key using the public_1 and private_2
-- The @symmetric_1 must be identical to @symmetric_2
SET symmetric_1 = asymmetric_derive(@public_1, @private_2);

-- Generate a symmetric key using the public_2 and private_1
-- The @symmetric_2 must be identical to @symmetric_1
SET symmetric_2 = asymmetric_derive(@public_2, @private_1);

Code examples for the following operations:

  • Create a private key using a SET statement

  • Create a private key using a SELECT statement

  • Create a private key using an INSERT statement

SET @private_key1 = create_asymmetric_priv_key('RSA', 3072);
SELECT create_asymmetric_priv_key('RSA', 3072) INTO @private_key2;
INSERT INTO key_table VALUES(create_asymmetric_priv_key('RSA', 3072));

Uninstall component_encryption_udf

You can deactivate and uninstall the component using the Uninstall Component statement.

UNINSTALL COMPONENT 'file://component_encryption_udf';

UUID_VX component

UUID_VX component

A Universally Unique Identifier (UUID) is a 128-bit number used to identify information uniquely in computer systems. It is often represented as a 32-character hexadecimal string divided into five groups separated by hyphens.

Benefit Description
Global Uniqueness UUIDs ensure that each identifier is unique across different databases and systems without needing a central authority to manage the IDs. This prevents ID conflicts when merging data from multiple sources.
Decentralized Generation Since UUIDs can be generated independently by different systems, there is no need for coordination. This is particularly useful in distributed environments where systems might not have constant communication with each other.
Scalability UUIDs support scalability in distributed databases. New records can be added without worrying about generating duplicate IDs, even when data is inserted concurrently across multiple nodes.
Improved Data Merging When data from various sources is combined, UUIDs prevent conflicts, making the merging process simpler and more reliable.
Security UUIDs, especially those generated randomly (like UUIDv4), are hard to predict, adding a layer of security when used as identifiers.

The following table describes the UUID versions:

UUID Version Description
Version 1 (Time-based) - Generated using the current time and a node identifier (usually the MAC address).
- Ensures uniqueness over time and across nodes.
Version 2 (DCE Security) - Similar to version 1 but includes additional information such as POSIX UID/GID.
- Often used in environments requiring enhanced security.
Version 3 (Name-based, MD5 hash) - Generated from a namespace identifier and a name (string).
- Uses the MD5 hashing algorithm to ensure the UUID is derived from the namespace and name.
Version 4 (Random) - Generated using random numbers.
- Offers high uniqueness and is easy to generate without requiring specific inputs.
Version 5 (Name-based, SHA-1 hash) - Similar to version 3 but uses the SHA-1 hashing algorithm.
- Provides a stronger hash function than MD5.
Version 6 (Time-ordered) - A reordered version of UUIDv1 for better indexing and storage efficiency.
- Combines timestamp and random or unique data.
Version 7 (Unix Epoch Time) - Combines a high-precision timestamp with random data.
- Provides unique, time-ordered UUIDs that are ideal for database indexing.
Version 8 (Custom) - Reserved for user-defined purposes and experimental uses.
- Allows custom formats and structures according to specific requirements.

UUID version 4 (UUIDv4) generates a unique identifier using random numbers. This randomness ensures a high level of uniqueness without needing a central authority to manage IDs. However, using UUIDv4 as a primary key in a distributed database is not recommended. The random nature of UUIDv4 leads to several issues:

Issue Description
Inefficient Indexing UUIDv4 does not follow any order, causing inefficient indexing. Databases struggle to keep records organized, leading to slower query performance.
Fragmentation The random distribution of UUIDv4 can cause data fragmentation, making database storage less efficient.
Storage Space UUIDs are larger (128 bits) than traditional integer keys, consuming more storage space and memory.

For better performance and efficiency in a distributed database, consider using UUIDv7, which incorporates timestamps for some order levels.

UUID version 7 (UUIDv7) creates time-ordered identifiers by encoding a Unix timestamp with millisecond precision in the first 48 bits. It uses 6 bits to specify the UUID version and variant, while the remaining 74 bits are random. This time-ordering results in nearly sequential values, which helps improve index performance and locality in distributed systems.

Install the UUID_VX component

INSTALL COMPONENT 'file://component_uuid_vx_udf';
Expected output 
Query OK, 0 rows affected (0.03 sec)

Character sets available

The following character sets are used in the component:

Character set Description
ascii Used everywhere UUID strings are returned by functions or accepted as function arguments.
utf8mb4 Used for string arguments in hash-based UUID generators, like UUID_V3() and UUID_V5() functions.
binary Used for arguments in the BIN_TO_UUID_VX() function and for results from the UUID_VX_TO_BIN() function.

Functions available in UUID_VX

The following functions are compatible with all UUID versions:

Function name Argument Description
BIN_TO_UUID_VX() One string argument that must be a hexadecimal of exactly 32 characters (16 bytes) The function returns a UUID with binary data from the argument. It returns an error for all other inputs.
IS_MAX_UUID_VX() One string argument that represents a UUID in standard or hexadecimal form. The function returns true if the argument is a valid UUID and is a MAX UUID. It returns false for all other inputs. If the argument is NULL, it returns NULL. If the argument cannot be parsed as a UUID, the function throws an error.
IS_NIL_UUID_VX() One string argument representing a UUID in standard or hexadecimal form. The function returns true if the string is a NIL UUID. If the argument is NULL, it returns NULL. If the argument is not a valid UUID, it throws an error.
IS_UUID_VX() One string argument that represents a UUID in either standard or hexadecimal form. The function returns true if the argument is a valid UUID. If the argument is NULL, it returns NULL. For any other input, it returns false.
MAX_UUID_VX() No argument This function generates a MAX UUID, which has all 128 bits set to one (FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF). This function result is the opposite of the NIL UUID.
NIL_UUID_VX() No argument. This function generates a NIL UUID, which has all 128 bits set to zero (00000000-0000-0000-0000-000000000000).
UUID_VX_TO_BIN() One string argument, formatted as a UUID or in hexadecimal form The function converts the string arugment to its binary representation.
UUID_VX_VARIANT() One string argument that represents a UUID in either standard or hexadecimal format. The function returns the UUID version (1-8) or an error if the argument is not a valid UUID or returns NULL if the input is NULL.
UUID_VX_VERSION() One string representing a UUID in standard or hexadecimal form. The function returns version of UUID(1-8). The function throws an error if the argument is not a valid UUID in formatted or hexadecimal form or returns a NULL if the argument is NULL. If the argument is a valid UUID string but has an unknown value (outside of the 1-8 range) the function returns -1.

Examples of functions for all UUID versions

SELECT is_uuid_vx('01900bf6-0eb0-715a-80f4-636367e07777');
Expected output 
+----------------------------------------------------+
| is_uuid_vx('01900bf6-0eb0-715a-80f4-636367e07777') |
+----------------------------------------------------+
|                                                  1 |
+----------------------------------------------------+

SELECT uuid_vx_version('01900bf6-0eb0-715a-80f4-636367e07777');
Expected output 
+---------------------------------------------------------+
| uuid_vx_version('01900bf6-0eb0-715a-80f4-636367e07777') |
+---------------------------------------------------------+
|                                                       7 |
+---------------------------------------------------------+

 SELECT uuid_vx_variant('01900bf6-0eb0-715a-80f4-636367e07777');
Expected output 
+---------------------------------------------------------+
| uuid_vx_variant('01900bf6-0eb0-715a-80f4-636367e07777') |
+---------------------------------------------------------+
|                                                       1 |
+---------------------------------------------------------+

UUID generator functions

The following functions generate specific UUID versions:

UUID Version Arguement Description
UUID_V1() No argument Generates a version 1 UUID based on a timestamp. If possible, use UUID_V7() instead.
UUID_V3() One or two arguments: the first argument is a string that is hashed with MD5 and used in the UUID; the second argument is optional and specifies a namespace (integer values: DNS: 0, URL: 1, OID: 2, X.500: 3; default is 1 or URL). Generates a version 3 UUID based on a name. Note: MD5 is outdated and not secure. Use with caution and avoid exposing sensitive data.
UUID_V4() No argument The function generates a version 4 UUID using random numbers and is similar to the built-in UUID() function.
UUID_V5() One or two arguments: the first argument is a string that is hashed with SHA1 and used in the UUID; the second argument is optional and specifies a namespace (integer values: DNS: 0, URL: 1, OID: 2, X.500: 3; default is 1 or URL). Generates a version 5 UUID based on a name. Note: SHA1 is better than MD5 but still not secure. Use with caution and avoid exposing sensitive data.
UUID_V6() No argument Generates a version 6 UUID based on a timestamp. If possible, use UUID_V7() instead.
UUID_V7() Can have either no argument or a one integer argument: the argument is the number of milliseconds to adjust the timestamp forward or backward (negative values). Generates a version 7 UUID based on a timestamp. If there is no argument, no timestamp shift occurs. Timestamp shift can hide the actual creation time of the record.

The UUID_v3() function and UUID_v5() function do not validate the string argument, such as whether the URL is formatted correctly or the DNS name is correct. These functions generate a string hash and then add that hash to a UUID with the defined namespace. The user specifies the string.

UUID generator examples

UUID version 1:

SELECT uuid_v1();
Expected output 
+--------------------------------------+
| uuid_v1()                            |
+--------------------------------------+
| 14c22f93-2962-11ef-9078-c3abf1c446bb |
+--------------------------------------+

UUID version 3 takes one argument and uses the default UUID namespace as “URL”.

SELECT uuid_v3('http://example.com');
Expected output 
+--------------------------------------+
| uuid_v3('http://example.com')        |
+--------------------------------------+
| d632b50c-7913-3137-ae9a-2d93f56e70d5 |
+--------------------------------------+

UUID version 3 takes one argument and the explicit UUID namespace is “URL”.

SELECT uuid_v3('http://example.com', 1);
Expected output 
+--------------------------------------+
| uuid_v3('http://example.com')        |
+--------------------------------------+
| d632b50c-7913-3137-ae9a-2d93f56e70d5 |
+--------------------------------------+

UUID version 3 takes one argument, with the explicit UUID namespace set to “DNS”.

SELECT uuid_v3('example.com',0);
Expected output 
+--------------------------------------+
| uuid_v3('example.com',0)             |
+--------------------------------------+
| 9073926b-929f-31c2-abc9-fad77ae3e8eb |
+--------------------------------------+

UUID version 4:

SELECT uuid_v4();
Expected output 
+--------------------------------------+
| uuid_v4()                            |
+--------------------------------------+
| a408e4ad-9b98-4edb-a105-40f22648a928 |
+--------------------------------------+

UUID version 5:

SELECT uuid_v5("http://example.com");
Expected output 
+--------------------------------------+
| uuid_v5("http://example.com")        |
+--------------------------------------+
| 8c9ddcb0-8084-5a7f-a988-1095ab18b5df |
+--------------------------------------+

UUID version 6:

SELECT uuid_v6();
Expected output 
+--------------------------------------+
| uuid_v6()                            |
+--------------------------------------+
| 1ef29686-2168-64a7-b9a2-adb13f80f118 |
+--------------------------------------+

UUID version 7 generation:

SELECT uuid_v7();
Expected output 
+--------------------------------------+
| uuid_v7()                            |
+--------------------------------------+
| 019010f6-0426-70f0-80b0-b63decd3d7d1 |
+--------------------------------------+
1 row in set (0.00 sec)

UUID version 7 with timestamp offset in 84000 seconds in the future

SELECT uuid_v7(84000000);
Expected output 
+--------------------------------------+
| uuid_v7(84000000)                    |
+--------------------------------------+
| 019015f8-c7c4-70b4-8043-fe241c2be36c |
+--------------------------------------+

Time-based functions

The following functions are used only with time-based UUIDs, specifically versions 1, 6, and 7.

Function name Argument Description
UUID_VX_TO_TIMESTAMP() One string argument Returns a timestamp string like “2024-05-29 18:04:14.201”. If the argument is not parsable as UUID v.1,6,7, the function throws an error. The function always uses UTC time, regardless of system settings or time zone settings in MySQL.
UUID_VX_TO_TIMESTAMP_TZ() One string argument Returns a timestamp string with the time zone like “Wed May 29 18:05:07 2024 GMT”. If the argument is not parsable as UUID v.1,6,7, the function throws an error. The function always uses UTC time (GMT time zone), regardless of system settings or time zone settings in MySQL.
UUID_VX_TO_UNIXTIME() One string argument Returns a number of milliseconds since the Epoch. If the argument is not parsable as UUID v.1,6,7, the function throws an error.

Timestamp-based function examples

SELECT uuid_vx_to_timestamp('01900bf6-0eb0-715a-80f4-636367e07777');
Expected output 
+--------------------------------------------------------------+
| uuid_vx_to_timestamp('01900bf6-0eb0-715a-80f4-636367e07777') |
+--------------------------------------------------------------+
| 2024-06-12 10:19:53.392                                      |
+--------------------------------------------------------------+
1 row in set (0.00 sec)

SELECT uuid_vx_to_timestamp_tz('01900bf6-0eb0-715a-80f4-636367e07777');
Expected output 
+-----------------------------------------------------------------+
| uuid_vx_to_timestamp_tz('01900bf6-0eb0-715a-80f4-636367e07777') |
+-----------------------------------------------------------------+
| Wed Jun 12 10:19:53 2024 GMT                                    |
+-----------------------------------------------------------------+

SELECT uuid_vx_to_unixtime('01900bf6-0eb0-715a-80f4-636367e07777');
Expected output 
+-------------------------------------------------------------+
| uuid_vx_to_unixtime('01900bf6-0eb0-715a-80f4-636367e07777') |
+-------------------------------------------------------------+
|                                               1718187593392 |
+-------------------------------------------------------------+

Uninstall the UUID_VX component

UNINSTALL COMPONENT 'file://component_uuid_vx_udf';
Expected output 
Query OK, 0 rows affected (0.03 sec)

Data-at-rest-encryption

Data at Rest Encryption

Data security is a concern for institutions and organizations. Transparent Data Encryption (TDE) or Data at Rest Encryption encrypts data files. Data at rest is any data that is not accessed or changed frequently, stored on different types of storage devices. Encryption ensures that if an unauthorized user accesses the data files from the file system, the user cannot read the contents.

If the user uses master key encryption, the MySQL keyring plugin stores the InnoDB master key, used for the master key encryption implemented by MySQL. The master key is also used to encrypt redo logs, and undo logs, along with the tablespaces.

The InnoDB tablespace encryption has the following components:

  • The database instance has a master key for tablespaces and a master key for binary log encryption.

  • Each tablespace has a tablespace key. The key is used to encrypt the Tablespace data pages. Encrypted tablespace keys are written on the tablespace header. In the master key implementation, the tablespace key cannot be changed unless you rebuild the table.

Two separate keys allow the master key to be rotated in a minimal operation. When the master key is rotated, each tablespace key is decrypted and re-encrypted with the new master key. The key rotation only reads and writes to the first page of each tablespace file (.ibd).

An InnoDB tablespace file is comprised of multiple logical and physical pages. Page 0 is the tablespace header page and keeps the metadata for the tablespace. The encryption information is stored on page 0 and the tablespace key is encrypted.

An encrypted page is decrypted at the I/O layer, added to the buffer pool, and used to access the data. A buffer pool page is not encrypted. The page is encrypted by the I/O layer before the page is flushed to disk.

Review Get help from Percona for ways that we can work with you.

Percona XtraBackup support

Percona XtraBackup version 8.4 supports the backup of encrypted general tablespaces.

Percona XtraBackup only supports features that are Generally Available (GA) in Percona Server for MySQL. Due to time constraints, a GA feature may be supported in a later Percona XtraBackup release. Review the Percona XtraBackup release notes for more information.

Keyring components

Keyring components overview

Percona Server supports a keyring that enables internal server components to store sensitive information securely for later retrieval.

Warning

Enable only one keyring component at a time for each server instance. Enabling multiple keyring components is not supported and may result in data loss.

Percona Server supports the following keyring components:

Get Started with component keyring

Use the keyring file component

Use the keyring vault component

Use the Key Management Interoperability Protocol (KMIP)

Use the Amazon Key Management Service (AWS KMS)

Get Started with component keyring

Enable the component keyring (component_keyring_file) in Percona Server 8.4 for encryption at rest. In MySQL 8.4, the supported path is the component keyring, not legacy keyring plugins.

Important

Enable only one keyring at a time. Do not use legacy keyring plugins (such as keyring_file or keyring_vault) together with the component keyring.

If you are upgrading from 8.0 or another release and already have data encrypted with a legacy keyring plugin, do not enable the component keyring without a migration plan. Data encrypted with the old plugin will not be readable by the new component; existing encrypted tables can become unreadable.

See Upgrade components and your upgrade documentation before switching. For migrating keys from a legacy keyring to the component keyring, check MySQL and Percona documentation for your version (for example, the mysql_migrate_keyring utility where applicable).

This guide is based on Configuring the Component Keyring in Percona Server and PXC 8.4 (Percona Community).

Scope: This quickstart assumes a typical package install and standard paths. Minimal images, custom builds, tarball installs, or locked-down environments may need different paths and steps. Confirm your binary location, plugin directory, and manifest path before following the steps below.

Prerequisites

  • Percona Server for MySQL 8.4 installed
  • sudo or root access to the server filesystem (to create the keyring directory, manifest, and component config)
  • Service name may be mysql or mysqld depending on your system
  • For the encryption examples (tables, system tablespace, redo/undo), the MySQL user you connect as must have the required privileges (for example, CREATE TABLESPACE for system tablespace). OS-level sudo does not grant MySQL privileges.

This guide assumes you have OS-level access to install and configure the keyring. In managed or hosted environments where you have only SQL privileges and no filesystem access, your provider must configure the keyring or offer an alternative. This quickstart does not cover managed or hosted environments with only SQL access.

Step 1: Register the component (manifest)

The manifest file (mysqld.my) tells MySQL which component to load. The component’s own configuration file (Step 3) configures how it behaves: where to store keys and whether the keyring is read-only.

MySQL loads components from a JSON manifest. A single typo, trailing comma, or missing quote can prevent the server from starting.

Before restarting, you can validate JSON syntax locally with jq . /usr/sbin/mysqld.my (if jq is installed). When the JSON is valid, jq . prints the file contents back; invalid JSON produces an error.

Note: jq only checks syntax. It does not verify that the component library exists in the plugin directory or that the server will load it. A successful jq run does not guarantee the server will start with the keyring. Do not paste configuration that contains paths or other sensitive details into third-party websites.

  1. Confirm where your server expects the manifest. The server reads the manifest from a location tied to the mysqld binary (often the same directory as the binary). Check the path of the running binary (for example, inspect your systemd unit file or run readlink -f /proc/$(pgrep -f mysqld)/exe on Linux).

    If you installed from a tarball or a non-standard repo, the path may not be /usr/sbin. Create the manifest in the location you confirmed. If the manifest is in the wrong place, the server ignores it and starts without the keyring; encryption (for example, CREATE TABLE ... ENCRYPTION='Y') will then fail with no clear error.

    A common path is /usr/sbin/mysqld.my. Custom builds or container images may use /usr/bin, /usr/local/sbin, or another directory. Some packages or security modules (for example, AppArmor or SELinux) may block writing under /usr/sbin. Create the manifest file at the path you confirmed:

    sudo vi /usr/sbin/mysqld.my

  2. Add the following JSON (check spelling and quotes):

    {
  "components": "file://component_keyring_file"
}

  3. Set ownership and permissions:

    sudo chown root:root /usr/sbin/mysqld.my
sudo chmod 644 /usr/sbin/mysqld.my

Step 2: Create the keyring directory

The keyring directory holds encryption keys. Restrict access.

sudo mkdir -p /var/lib/mysql-keyring
sudo chown mysql:mysql /var/lib/mysql-keyring
sudo chmod 750 /var/lib/mysql-keyring

Rule of thumb: mysql owns the directory; only MySQL (and root for administration) should access it.

If you use a custom Linux user, a Docker volume, or a non-standard install, ensure the keyring directory is owned by the user that runs MySQL and that the MySQL process has read access to the manifest and component config files. “Permission denied” on startup usually means ownership or permissions on the manifest and component config paths.

In hardened environments (for example, security-hardened AMIs), extended attributes (ACLs) or immutable flags (for example, chattr +i) may also block access. If chmod/chown succeed but MySQL still cannot access the file or directory, check with getfacl or lsattr and adjust or remove restrictions as your security policy allows.

In orchestrated or container environments (for example, Kubernetes, OpenShift), file ownership and permissions are often managed by the platform (for example, Security Context Constraints or admission controllers). Manual chown/chmod may fail or be overwritten when the pod restarts. Check your platform documentation for how to set permissions or run as the correct user.

Back up the keyring file: data recovery depends on it

Do not delete the keyring file (for example, /var/lib/mysql-keyring/component_keyring_file) to “clean up” or for any other reason. If that file is lost or deleted, all data encrypted with it is unrecoverable. There is no way to decrypt tablespaces, redo logs, or undo logs without the keyring.

Life cycle: Back up the keyring file and its directory as part of your normal backup strategy. Include the keyring in your restore procedures so that after a restore you can start MySQL with the same keys and access your encrypted data. If you move or clone the server, copy the keyring file to the new location before starting the server.

Step 3: Configure the keyring component

Verify the component library exists before pointing the manifest at it. In a MySQL session (if the server is already running), run SELECT @@plugin_dir;. Then list that directory and confirm the component_keyring_file library (or the file name your distribution uses) is present. On Linux you can run ls -la "$(mysql -N -e 'SELECT @@plugin_dir;')" and look for a file related to component_keyring_file.

Minimal or slim container images may omit the library. Install the package that provides the keyring component or use a full image.

Create the component configuration file in the MySQL plugin directory. Use the path returned by @@plugin_dir (typical paths: /usr/lib64/mysql/plugin on RHEL-style, /usr/lib/mysql/plugin on Debian/Ubuntu).

  1. Go to the plugin directory (use the path from @@plugin_dir or adjust for your distribution):

    cd /usr/lib64/mysql/plugin

  2. Create the config file:

    sudo vi component_keyring_file.cnf

  3. Add (adjust path if you used a different keyring directory):

    {
  "path": "/var/lib/mysql-keyring/component_keyring_file",
  "read_only": false
}

    Use "read_only": true if you want to prevent runtime changes to the keyring. You can validate the component config file with jq . component_keyring_file.cnf before restarting.

  4. Set ownership and permissions:

    sudo chown root:root component_keyring_file.cnf
sudo chmod 640 component_keyring_file.cnf

Step 4: Restart MySQL

Loading the component from the manifest requires a server restart. On a typical system:

sudo systemctl restart mysql

Or sudo systemctl restart mysqld, depending on your system.

In orchestrated or managed environments (for example, Kubernetes or some cloud DB services), restart may be controlled by a scheduler or not allowed mid-workflow. Plan the keyring setup during a maintenance window or coordinate with your provider. There is no supported way to load the keyring component at runtime without a restart when using the file-based manifest.

If the server does not start, check the error log. To see where it is written, run SELECT @@log_error; in a previous MySQL session, or look in common locations such as /var/log/mysql/error.log or the server datadir. JSON syntax errors in the manifest or component config (trailing commas, missing quotes, wrong brackets) are reported there. Fix the JSON and restart again.

Step 5: Verify the keyring is loaded

In a MySQL session:

SELECT * FROM performance_schema.keyring_component_status;

You should see component_keyring_file with Component_status = Active and the correct Data_file path.

Example:

+---------------------+-----------------------------------------------+
| STATUS_KEY          | STATUS_VALUE                                  |
+---------------------+-----------------------------------------------+
| Component_name      | component_keyring_file                        |
| Component_status    | Active                                        |
| Data_file           | /var/lib/mysql-keyring/component_keyring_file |
| Read_only           | No                                            |
+---------------------+-----------------------------------------------+

Seeing “Active” here means only that the keyring component is loaded and ready. Your data is not encrypted until you enable encryption for tables, tablespaces, or logs in the next section.

Treat Step 5 as “keyring ready”. Actual protection comes from applying encryption in the Data at rest encryption section below.

Data at rest encryption

With the keyring loaded, the server can encrypt data on disk (transparent data encryption, or TDE). Keys are stored in the keyring; you enable encryption per tablespace or for redo/undo logs.

Performance: Enabling system-wide encryption (redo logs, undo logs, and the system tablespace) consumes extra CPU cycles and I/O. Encryption has a measurable cost: throughput can drop and latency can increase. The impact depends on workload. Consider enabling encryption where it matters for compliance or risk, and measure performance under your load before and after.

Tables and schemas

Create a new table with encryption:

CREATE TABLE myapp.sensitive_data (
  id INT PRIMARY KEY,
  payload VARCHAR(255)
) ENCRYPTION='Y';

Encrypt an existing table:

ALTER TABLE myapp.existing_table ENCRYPTION='Y';

Set default encryption for a schema so new tables are encrypted by default:

ALTER SCHEMA myapp DEFAULT ENCRYPTION='Y';

System tablespace

Encrypt the system tablespace (data dictionary, mysql system tablespace):

ALTER TABLESPACE mysql ENCRYPTION='Y';

Requires the CREATE TABLESPACE privilege on the MySQL user you use (having OS sudo does not grant MySQL privileges).

Encrypting the system tablespace can be a long-running, high-I/O operation. On a large instance it may take minutes or hours and can increase load or cause blocking. Do not assume it is safe to run on a heavily loaded production system without a maintenance window. Plan for a maintenance window and expect significant I/O. See Encrypt system tablespace.

Redo and undo logs

Enable encryption for redo and undo log files so recovery and rollback data are encrypted on disk.

All SET GLOBAL settings are temporary and are lost at restart. For redo/undo log encryption, use the config file as the primary method. Do not enable it only with SET GLOBAL or your protection will vanish after the next reboot (for example, after a kernel update or crash).

Add these options to your MySQL config file (for example, my.cnf) so they persist:

innodb_redo_log_encrypt = ON
innodb_undo_log_encrypt = ON

Restart the server (or add the options and plan a restart). If these settings are only set with SET GLOBAL and not in the config file, the server will stop encrypting new redo/undo pages after a restart with no clear indication.

New log pages are encrypted when written; existing pages are unchanged until they are rewritten. See Log encryption.

Verify encryption

The following checks report metadata and schema settings only. They do not prove that data on disk is actually encrypted or that the keyring has served a key for that tablespace.

InnoDB encrypts data when the option is set. These queries only confirm that the option is set (the “label”), not that the “lock” is engaged on disk.

Tables: List tables that have the encryption option set. Such tables show ENCRYPTION="Y" in CREATE_OPTIONS:

SELECT TABLE_SCHEMA, TABLE_NAME, CREATE_OPTIONS
FROM INFORMATION_SCHEMA.TABLES
WHERE CREATE_OPTIONS LIKE '%ENCRYPTION%';

Tablespaces: Check whether a tablespace has the encryption flag set in INNODB_TABLESPACES. Bit 13 (value 8192) is set when the tablespace is marked encrypted:

SELECT name, (flag & 8192) != 0 AS encrypted
FROM INFORMATION_SCHEMA.INNODB_TABLESPACES;

Schemas: List schemas that have default encryption (new tables in schemas with default encryption are encrypted by default):

SELECT SCHEMA_NAME, DEFAULT_ENCRYPTION
FROM INFORMATION_SCHEMA.SCHEMATA
WHERE DEFAULT_ENCRYPTION = 'YES';

Redo and undo logs: Confirm whether redo and undo log encryption is enabled:

SHOW GLOBAL VARIABLES LIKE 'innodb_redo_log_encrypt';
SHOW GLOBAL VARIABLES LIKE 'innodb_undo_log_encrypt';

For more (binary logs, doublewrite, temporary files, and INNODB_TABLESPACES_ENCRYPTION), see Verify encryption.

Clean up

If you ran the examples above and want to return the instance to its previous state, run the following. Skip any step that does not apply (for example, if you did not create myapp or encrypt the system tablespace).

Disable redo and undo log encryption. If you added them to my.cnf, remove or comment out those lines and restart; otherwise they will turn back on at next restart.

To turn off only for the current run:

SET GLOBAL innodb_redo_log_encrypt = OFF;
SET GLOBAL innodb_undo_log_encrypt = OFF;

Revert system tablespace encryption (only if you ran ALTER TABLESPACE mysql ENCRYPTION='Y'):

ALTER TABLESPACE mysql ENCRYPTION='N';

Remove the example schema default and tables. If you created the myapp schema or the example tables used in this guide:

ALTER SCHEMA myapp DEFAULT ENCRYPTION='N';
DROP TABLE IF EXISTS myapp.sensitive_data;

If you encrypted a different existing table, revert it with ALTER TABLE schema_name.table_name ENCRYPTION='N';.

The keyring remains loaded and the keyring files on disk are unchanged. To remove the keyring entirely, delete the component from mysqld.my, remove or rename component_keyring_file.cnf, and restart MySQL.

Next steps

  • Use data at rest encryption (for example, encrypt tablespaces, redo/undo logs) with the keyring in place.
  • Verify encryption for your tablespaces and logs.
  • For Percona XtraDB Cluster (PXC): keyring file is not replicated; copy the keyring file from the bootstrap node to other nodes before starting them. See the Percona Community blog for PXC-specific steps and the 8.4.4/8.4.5 plugin-path workaround.

Operational notes

  • Treat the keyring file as a secret: restrict access and include it in your secure backup strategy.
  • Back up the keyring file and its directory. If the keyring is lost or damaged (for example, after a migration or permission change), you cannot decrypt data that was encrypted with the keyring. Recovery is not possible. Duplicate or back up the keyring before major changes to the server or filesystem.
  • If the keyring is lost and you have encrypted data, recovery is not possible.
  • To change the master key (for example, for rotation or compliance), use ALTER INSTANCE ROTATE INNODB MASTER KEY; see Rotate the master encryption key. Do not delete or replace the keyring file manually to “reset” or rotate. You will lose access to all data encrypted with it.
  • For MySQL 8.4 and later, components are the supported keyring model; avoid mixing with legacy keyring plugins.

Use the keyring file component

The keyring_file component is part of the component-based MySQL infrastructure which extends the server capabilities.

Important

Percona Server for MySQL 8.4 does not support the keyring_file plugin.

See the MySQL documentation on the component installation and on the keyring_file component usage for more information.

Install a keyring component through a manifest file. During startup, the server reads the manifest. Each component reads a corresponding configuration file during initialization.

Do not load keyring components with either of the following methods:

Method Why it fails
--early-plugin-load option Loads plugins only, not components
INSTALL COMPONENT statement Registers components in the mysql.component table, which the server loads after InnoDB initialization

Components that InnoDB requires at startup must load earlier.

Create a global manifest file named mysqld.my in the installation directory. Optionally, create a local manifest file with the same name in a data directory.

To install a keyring component, complete the following steps:

  1. Write a manifest in valid JSON format

Each component uses the same manifest mechanism. Each component reads its own configuration file. The name of the configuration file matches the component name with a .cnf extension. For example, component_keyring_vault reads component_keyring_vault.cnf.

Manifest files

Manifest fields

The manifest is a JavaScript Object Notation (JSON) object with the following fields:

Field Type Required Description
components string Conditional URL-style locator for the component to load. The format is file://<component_name>. Required when the manifest declares the component directly.
read_local_manifest boolean No When true, the global manifest delegates to the local manifest in the data directory. When false or absent, the global manifest declares the component directly.

Manifest examples

Replace <component_name> in the following examples with one of the available components.

The following example shows a global manifest that loads a component directly:

{
  "components": "file://<component_name>"
}

The following example shows a global manifest that delegates to the local manifest:

{
  "read_local_manifest": true
}

The following example shows a local manifest:

{
  "components": "file://<component_name>"
}

Manifest locations

Percona Server reads two manifest files at startup:

Manifest Location Purpose
Global manifest The directory that contains the mysqld binary Default manifest for the server installation
Local manifest Data directory Per-instance override for hosts that run multiple instances with different keyring components

Both files use the name mysqld.my. The global manifest declares the component directly or delegates to the local manifest.

Why the manifest is the only supported load path

The keyring must load before InnoDB opens an encrypted page. Any mechanism that depends on the Structured Query Language (SQL) layer therefore loads the keyring too late.

A mysqld startup proceeds in the following order:

  1. mysqld parses startup configuration and reads the manifest file from the server installation directory.

  2. The server loads each component named in the manifest.

  3. InnoDB initializes, replays the redo log, and opens tablespaces.

  4. The SQL layer accepts connections.

The keyring must load between steps 1 and 3. The following table explains why each alternative mechanism fails to load the keyring within this window:

Mechanism Reason for failure
INSTALL COMPONENT The statement runs as SQL and cannot execute until step 4. The registration record lives in mysql.component, an InnoDB table that the server reads only after InnoDB initializes. The system tablespace cannot decrypt without the keyring, which creates a circular dependency. Crash recovery also runs before the SQL layer, so the encrypted redo log must remain readable without SQL.
--early-plugin-load The flag applies to legacy keyring plugins, not components. Plugins and components load through separate subsystems. The flag cannot locate component entry points.

A component registered through INSTALL COMPONENT on a running server does not persist across restarts. On the next restart, InnoDB cannot unwrap tablespace keys because no manifest file exists on disk. A missing or malformed mysqld.my therefore prevents startup for any instance with encrypted tablespaces.

Install a keyring component

To install a keyring component:

  1. Stop the mysqld process.

  2. Create the manifest file at the chosen location. Use one of the examples in Manifest examples as a template.

  3. Create the configuration file in the same directory as the manifest that declares the component. The name of the configuration file matches the component name with a .cnf extension.

  4. Populate the configuration file with parameters for the chosen component. The required parameters depend on the component.

  5. Set the file ownership of the manifest file and the configuration file to the mysql user.

  6. Restrict read access on the configuration file to the mysql user. The configuration file may contain credentials.

  7. Start the mysqld process.

Verify the installation

Connect to the server and run the following query:

SELECT * FROM performance_schema.keyring_component_status;

The query returns one row per status field. The Component_status field reports Active when the component loaded. The Component_status field reports Disabled when the component failed to load. Inspect the server error log to identify the cause of any failure.

Operate the keyring

Rotate the master key

The InnoDB master key wraps the tablespace keys that protect data on disk. Rotate the master key on a scheduled cadence to limit the volume of data wrapped by any single master key.

Run the following statement to rotate the master key:

ALTER INSTANCE ROTATE INNODB MASTER KEY;

The statement generates a master key and stores the key in the keyring. The server then wraps subsequent tablespace keys with the stored key.

The rotation operation is fast. The server does not re-encrypt the tablespace data.

Earlier master keys must remain in the keyring. Deleting an earlier master key prevents the server from reading data that was wrapped with that key.

For required privileges and the full rotation procedure, see Rotate the master encryption key.

Monitor the keyring

Use the query in Verify the installation to inspect the keyring at runtime. Configure alerts on Component_status transitions to Disabled. Watch the server error log for keyring-related errors. The server writes manifest parse errors, configuration parse errors, and backend connection errors to the error log.

Back up and restore

A backup of an encrypted tablespace requires keyring access on the destination host. The destination host must read the same key material that encrypted the backup.

The following table summarizes the keyring requirement for each backend:

Backend Backup requirement
component_keyring_file Copy the keyring file alongside the data backup. The destination host must read the same key material.
component_keyring_vault Configure the destination host to authenticate with the same Vault server and access the same secret_mount_point.
component_keyring_kmip Configure the destination host to authenticate with the same KMIP server and access the same key namespace.
component_keyring_kms Configure the destination host with credentials and permissions for the same cloud KMS keys.

Test a restore on a separate host before relying on the backup. A restore that runs on the original host can succeed even when the keyring configuration is incorrect.

Replace or remove a keyring component

To replace a keyring component:

  1. Stop the mysqld process.

  2. Edit the manifest file to reference the replacement component.

  3. Create the configuration file for the replacement component.

  4. Start the mysqld process.

To remove the keyring:

  1. Confirm that no encrypted tablespaces exist on the server. A server with encrypted tablespaces cannot start without a keyring.

  2. Stop the mysqld process.

  3. Delete the manifest file.

  4. Start the mysqld process.

Warning

Configure exactly one keyring per server instance. Percona Server does not support multiple keyring plugins, multiple keyring components, or any combination of plugin and component. Such configurations risk data loss.

An example of a manifest and a configuration file is the following:

An example of ./bin/mysqld.my:

{
    "components": "file://component_keyring_file"
}

An example of /lib/plugin/component_keyring_file.cnf:

{
    "path": "/var/lib/mysql-keyring/keyring_file", "read_only": false
}

Verify the keyring component

After you start Percona Server for MySQL, confirm that the keyring component loaded and initialized. Query the performance_schema.keyring_component_status table:

SELECT * FROM performance_schema.keyring_component_status;

The query returns one row for each status key that the loaded component reports. The Component_status row indicates the result of initialization:

Component_status value Meaning Recovery
Active The component loaded and initialized successfully None
Disabled The component loaded but failed to initialize, typically due to a configuration error Review the server error log, correct the configuration file, then run ALTER INSTANCE RELOAD KEYRING

If a keyring component fails to load entirely, the server does not start. Check the server error log for diagnostic messages.

For the complete list of status keys that each component reports, see keyring_component_status Table in the MySQL Reference Manual.

Use the keyring vault component

The keyring_vault component connects Percona Server to a HashiCorp Vault server. The component stores and retrieves the encryption keys that protect data at rest.

Install a keyring component through a manifest file. During startup, the server reads the manifest. Each component reads a corresponding configuration file during initialization.

Do not load keyring components with either of the following methods:

Method Why it fails
--early-plugin-load option Loads plugins only, not components
INSTALL COMPONENT statement Registers components in the mysql.component table, which the server loads after InnoDB initialization

Components that InnoDB requires at startup must load earlier.

Create a global manifest file named mysqld.my in the installation directory. Optionally, create a local manifest file with the same name in a data directory.

To install a keyring component, complete the following steps:

  1. Write a manifest in valid JSON format

Each component uses the same manifest mechanism. Each component reads its own configuration file. The name of the configuration file matches the component name with a .cnf extension. For example, component_keyring_vault reads component_keyring_vault.cnf.

Manifest files

Manifest fields

The manifest is a JavaScript Object Notation (JSON) object with the following fields:

Field Type Required Description
components string Conditional URL-style locator for the component to load. The format is file://<component_name>. Required when the manifest declares the component directly.
read_local_manifest boolean No When true, the global manifest delegates to the local manifest in the data directory. When false or absent, the global manifest declares the component directly.

Manifest examples

Replace <component_name> in the following examples with one of the available components.

The following example shows a global manifest that loads a component directly:

{
  "components": "file://<component_name>"
}

The following example shows a global manifest that delegates to the local manifest:

{
  "read_local_manifest": true
}

The following example shows a local manifest:

{
  "components": "file://<component_name>"
}

Manifest locations

Percona Server reads two manifest files at startup:

Manifest Location Purpose
Global manifest The directory that contains the mysqld binary Default manifest for the server installation
Local manifest Data directory Per-instance override for hosts that run multiple instances with different keyring components

Both files use the name mysqld.my. The global manifest declares the component directly or delegates to the local manifest.

Why the manifest is the only supported load path

The keyring must load before InnoDB opens an encrypted page. Any mechanism that depends on the Structured Query Language (SQL) layer therefore loads the keyring too late.

A mysqld startup proceeds in the following order:

  1. mysqld parses startup configuration and reads the manifest file from the server installation directory.

  2. The server loads each component named in the manifest.

  3. InnoDB initializes, replays the redo log, and opens tablespaces.

  4. The SQL layer accepts connections.

The keyring must load between steps 1 and 3. The following table explains why each alternative mechanism fails to load the keyring within this window:

Mechanism Reason for failure
INSTALL COMPONENT The statement runs as SQL and cannot execute until step 4. The registration record lives in mysql.component, an InnoDB table that the server reads only after InnoDB initializes. The system tablespace cannot decrypt without the keyring, which creates a circular dependency. Crash recovery also runs before the SQL layer, so the encrypted redo log must remain readable without SQL.
--early-plugin-load The flag applies to legacy keyring plugins, not components. Plugins and components load through separate subsystems. The flag cannot locate component entry points.

A component registered through INSTALL COMPONENT on a running server does not persist across restarts. On the next restart, InnoDB cannot unwrap tablespace keys because no manifest file exists on disk. A missing or malformed mysqld.my therefore prevents startup for any instance with encrypted tablespaces.

Install a keyring component

To install a keyring component:

  1. Stop the mysqld process.

  2. Create the manifest file at the chosen location. Use one of the examples in Manifest examples as a template.

  3. Create the configuration file in the same directory as the manifest that declares the component. The name of the configuration file matches the component name with a .cnf extension.

  4. Populate the configuration file with parameters for the chosen component. The required parameters depend on the component.

  5. Set the file ownership of the manifest file and the configuration file to the mysql user.

  6. Restrict read access on the configuration file to the mysql user. The configuration file may contain credentials.

  7. Start the mysqld process.

Verify the installation

Connect to the server and run the following query:

SELECT * FROM performance_schema.keyring_component_status;

The query returns one row per status field. The Component_status field reports Active when the component loaded. The Component_status field reports Disabled when the component failed to load. Inspect the server error log to identify the cause of any failure.

Operate the keyring

Rotate the master key

The InnoDB master key wraps the tablespace keys that protect data on disk. Rotate the master key on a scheduled cadence to limit the volume of data wrapped by any single master key.

Run the following statement to rotate the master key:

ALTER INSTANCE ROTATE INNODB MASTER KEY;

The statement generates a master key and stores the key in the keyring. The server then wraps subsequent tablespace keys with the stored key.

The rotation operation is fast. The server does not re-encrypt the tablespace data.

Earlier master keys must remain in the keyring. Deleting an earlier master key prevents the server from reading data that was wrapped with that key.

For required privileges and the full rotation procedure, see Rotate the master encryption key.

Monitor the keyring

Use the query in Verify the installation to inspect the keyring at runtime. Configure alerts on Component_status transitions to Disabled. Watch the server error log for keyring-related errors. The server writes manifest parse errors, configuration parse errors, and backend connection errors to the error log.

Back up and restore

A backup of an encrypted tablespace requires keyring access on the destination host. The destination host must read the same key material that encrypted the backup.

The following table summarizes the keyring requirement for each backend:

Backend Backup requirement
component_keyring_file Copy the keyring file alongside the data backup. The destination host must read the same key material.
component_keyring_vault Configure the destination host to authenticate with the same Vault server and access the same secret_mount_point.
component_keyring_kmip Configure the destination host to authenticate with the same KMIP server and access the same key namespace.
component_keyring_kms Configure the destination host with credentials and permissions for the same cloud KMS keys.

Test a restore on a separate host before relying on the backup. A restore that runs on the original host can succeed even when the keyring configuration is incorrect.

Replace or remove a keyring component

To replace a keyring component:

  1. Stop the mysqld process.

  2. Edit the manifest file to reference the replacement component.

  3. Create the configuration file for the replacement component.

  4. Start the mysqld process.

To remove the keyring:

  1. Confirm that no encrypted tablespaces exist on the server. A server with encrypted tablespaces cannot start without a keyring.

  2. Stop the mysqld process.

  3. Delete the manifest file.

  4. Start the mysqld process.

Warning

Configure exactly one keyring per server instance. Percona Server does not support multiple keyring plugins, multiple keyring components, or any combination of plugin and component. Such configurations risk data loss.

Configure the keyring vault component

The configuration settings live in either a global configuration file or a local configuration file.

The component connects to the HashiCorp Vault server over Hypertext Transfer Protocol Secure (HTTPS) for production deployments. The component also accepts plain http:// URLs for development. Prepare the certificate and key files for the secure connection. Each Vault server instance requires three artifacts:

  • An organizational Certificate Authority (CA)

  • A private vault key

  • A server certificate signed by the CA

You can generate the artifacts with OpenSSL or reuse existing files. The key files contain sensitive material. Store the key files and the password for each key in a secure location.

You can also build a CA inside Vault and then issue a Vault server certificate from that CA.

Configuration parameters

The component_keyring_vault.cnf file contains required and optional parameters.

Required parameters

  • secret_mount_point — the mount point name where keyring_vault stores the keys.

  • token — a token issued by the Vault server.

  • vault_url — the address of the Vault server. The address must start with http:// or https://. Use https:// for production traffic. Use http:// only for development. The component follows HTTP redirects returned by Vault. This behavior supports cluster setups where a load balancer redirects clients to the active Vault node.

Optional parameters

The component checks the value of read_local_config before any other parameter.

  • read_local_config — Default: false. Declares whether the component reads the local configuration file. Use this parameter only in the global configuration file. The allowed values are true or false.

    false — the component processes the other parameters in the global configuration file and ignores the local configuration file.

    true — the component ignores the other parameters in the global configuration file and reads the local configuration file.

  • secret_mount_point_version — Default: AUTO. The Key-Value (KV) Secrets Engine version (kv or kv-v2). The allowed values are AUTO, 1, and 2.

  • timeout — Default: 15. The duration in seconds applied to both the connection phase and the total operation. The allowed range is 0 through 86400. Set the value to 0 to wait indefinitely.

  • vault_ca — Default: unset (fallback to the host trust store). The path to the CA certificate that signed the certificate of the Vault server. Set this parameter when the host does not trust the CA of the Vault server. The component always verifies the peer certificate and the hostname when vault_url uses https://.

The component trims leading and trailing whitespace from each parameter value before parsing.

Run SELECT * FROM performance_schema.keyring_component_status to inspect the loaded configuration. The view reports each parameter value and the Active or Disabled status. The view also reports the component identity fields: Component_name, Author, License, Implementation_name, and Version.

Example configuration

The following example shows a component_keyring_vault.cnf file in JSON format with sample values for each parameter.

{
  "timeout": 15,
  "vault_url": "https://vault.public.com:8202",
  "secret_mount_point": "secret",
  "secret_mount_point_version": "AUTO",
  "token": "{randomly-generated-alphanumeric-string}",
  "vault_ca": "/data/keyring_vault_confs/vault_ca.crt"
}

Warning

Each secret_mount_point must serve only one Percona Server instance. Multiple servers that share a secret_mount_point write to the same Vault namespace. The shared namespace exposes the following risks:

Risk Mechanism
Permanent data loss Key writes from one server can overwrite key writes from another server. Overwritten keys cannot decrypt previously encrypted data. Cloned servers that retain a source server_uuid collide on InnoDB master-key names. User-named keys created through the keyring UDF surface collide on any duplicate name.
Unauthorized key disclosure Every server with access to the shared mount point can read every key stored by every other server. The component applies no per-server scoping inside the mount.
Iteration noise The keyring metadata iterator returns all keys under the mount, including keys owned by other servers. Rotation and audit tooling sees foreign keys.

Assign a unique secret_mount_point to each Percona Server instance.

The component fetches the key type and data from the Vault server on the first request for that key.

Validation rules

The component validates each parameter at startup. The following table lists the rejected configurations and their effects:

Condition Result
secret_mount_point ends with / The component fails to initialize.
secret_mount_point starts with / The component fails to initialize.
timeout exceeds 86400 The component fails to initialize.
vault_ca is set together with an http:// vault_url The component fails to initialize.
vault_url does not start with http:// or https:// The component fails to initialize.
vault_url uses https:// but vault_ca is unset The component logs an error and falls back to the host trust store for certificate validation.

secret_mount_point_version values

The secret_mount_point_version parameter accepts one of the following values:

Value Description
1 Works with KV Secrets Engine - Version 1 (kv). The component uses secret_mount_point directly when forming key operation URLs. For example, the URL for a key named skey is <vault_url>/v1/<secret_mount_point>/skey.
2 Works with KV Secrets Engine - Version 2 (kv-v2). Initialization splits the secret_mount_point parameter into two parts:
  • mount_point_path — the mount path under which the Vault server secret was created
  • directory_path — a virtual directory suffix that creates virtual namespaces under the same real mount point
The component uses both parts to form key access URLs, for example <vault_url>/v1/<mount_point_path>/data/<directory_path>/skey.
AUTO The component probes the secrets engine to detect kv or kv-v2. The component then either uses secret_mount_point directly or splits the parameter into a mount point path and a directory path. See Auto-detection algorithm.
Not listed The component behaves as if secret_mount_point_version is set to AUTO.

The component fails to initialize for any other value. Numeric values other than 1 or 2 produce an error. Non-numeric values other than AUTO produce a separate error.

A version mismatch causes one of the following failures:

Configured value Actual engine version Result
2 kv (Version 1) The component runs auto-detection during initialization to confirm the configured value. The component fails to initialize when the probe finds no kv-v2 mount. The error message reads Auto-detected mount point version is not the same as specified in 'secret_mount_point_version'.
1 kv-v2 (Version 2) The component initializes, but every keyring operation fails.

Auto-detection algorithm

The component runs the auto-detection probe during initialization in two cases:

  • secret_mount_point_version is set to AUTO.

  • secret_mount_point_version is set to 2.

The probe walks secret_mount_point from the longest prefix to the shortest. The probe queries the Vault metadata configuration endpoint for each prefix. The first prefix that returns a kv-v2 response defines the mount point path. The remainder of secret_mount_point becomes the directory path. The component falls back to kv when no prefix returns kv-v2.

The probe writes one informational message per prefix to the server log. The messages identify successful matches, unreadable responses, and rejected prefixes.

Upgrade from Vault Secrets Engine Version 1 to Version 2

Use either of the following methods to upgrade from Version 1 to Version 2:

  • Set secret_mount_point_version to AUTO in the keyring_vault configuration file on every Percona Server, or omit the parameter. The AUTO value triggers autodetection during component initialization.

  • Set secret_mount_point_version to 2 to ensure each component initializes only after the upgrade from kv to kv-v2 completes.

Note

The keyring_vault component does not use the built-in key versioning of kv-v2. The component encodes each version into the keyring key name.

Services exposed by the component

The keyring_vault component implements the following services for keyring operations. Other components and the server core consume these services to manage keys backed by Vault:

Service Purpose
keyring_aes Performs Advanced Encryption Standard (AES) encryption and decryption with keys stored in Vault.
keyring_component_metadata_query Reports the component metadata, including the loaded configuration values.
keyring_component_status Reports the component status.
keyring_generator Generates keys on demand and stores the keys in Vault.
keyring_keys_metadata_iterator Iterates over key metadata for inspection.
keyring_load Loads the keyring during server startup and reloads on demand.
keyring_reader_with_status Reads keys and exposes the read status.
keyring_writer Writes keys to Vault.

The keyring_aes service supports server-side encryption at rest. The service also supports the AES user-defined functions (UDFs) when paired with the encryption UDF component.

Operate the vault component

This shared section covers rotation, monitoring, and backup procedures that apply to every keyring component. The following notes describe behaviors that are specific to keyring_vault.

Master key rotation

Each ALTER INSTANCE ROTATE INNODB MASTER KEY statement writes a new master key entry to Vault under the configured secret_mount_point. The previous key remains in Vault history when secret_mount_point_version is 2 or when the AUTO probe identifies a kv-v2 mount. The component contacts Vault during the rotation, so a slow or unreachable Vault delays statement completion. Schedule rotation during a low-load window.

Monitoring fields

Compare the values returned by performance_schema.keyring_component_status against component_keyring_vault.cnf to detect drift in vault_url, secret_mount_point, vault_ca, and secret_mount_point_version. Watch the server error log for HTTP 403 (token expiration) and HTTP 503 (sealed Vault). Configure alerts on both patterns.

Backup and restore considerations

A Vault-backed restore requires the following on the destination host:

  • Network reachability to the same Vault server, with valid name resolution and certificate trust.

  • A valid token with read access to the configured secret_mount_point.

  • A server_uuid distinct from the source host. A duplicated server_uuid triggers the data-loss case described in Configure the keyring vault component.

  • kv-v2 history retention that covers the time between backup and restore. A backup taken before a master-key rotation cannot decrypt tablespaces written after the rotation unless the post-rotation key remains in Vault history.

See also

Verify the keyring component

After you start Percona Server for MySQL, confirm that the keyring component loaded and initialized. Query the performance_schema.keyring_component_status table:

SELECT * FROM performance_schema.keyring_component_status;

The query returns one row for each status key that the loaded component reports. The Component_status row indicates the result of initialization:

Component_status value Meaning Recovery
Active The component loaded and initialized successfully None
Disabled The component loaded but failed to initialize, typically due to a configuration error Review the server error log, correct the configuration file, then run ALTER INSTANCE RELOAD KEYRING

If a keyring component fails to load entirely, the server does not start. Check the server error log for diagnostic messages.

For the complete list of status keys that each component reports, see keyring_component_status Table in the MySQL Reference Manual.

See also

Hashicorp Documentation: Installing Vault

Hashicorp Documentation: Production Hardening

Maintain the Vault connection

Installation of the keyring vault component is the first step. The most common production failure of keyring_vault is a lost connection to Vault, not a misconfigured component. Two conditions cause the loss:

  • The token used by the component expires.

  • The Vault server seals.

Either condition prevents Percona Server from fetching master keys.

The keyring_vault component reads the token value from the configuration file at startup. The component then uses the stored token for the lifetime of the mysqld process. The component does not renew the token and does not reread the configuration file at runtime. Renewal and rotation must run outside MySQL, on the Vault side or through a helper process.

Renew the Vault token

Every Vault token has a time-to-live (TTL). When the TTL elapses, Vault revokes the token. Any subsequent Application Programming Interface (API) call returns HTTP 403, including the master-key fetch performed by keyring_vault.

The following symptoms indicate an expired or revoked token:

  • mysqld startup fails after a restart with keyring component initialization or fetch errors.

  • The statement ALTER INSTANCE ROTATE INNODB MASTER KEY fails.

  • Opening an encrypted table after a restart fails with a keyring error.

  • The row in performance_schema.keyring_component_status shows the component as loaded but unable to reach Vault.

Because the component does not renew tokens, choose one of the following patterns to keep the configured token valid:

A periodic token has no maximum TTL. The token continues to work indefinitely as long as a renewal occurs within each period. This renewal model matches the lifetime of a long-running database server.

Create a Vault role that issues periodic tokens, then mint the token referenced by component_keyring_vault.cnf:

vault write auth/token/roles/percona-keyring \
    allowed_policies="percona-keyring-policy" \
    period="24h" \
    renewable=true

vault token create -role=percona-keyring -format=json

Store the issued token in the token field of the keyring_vault configuration file. Pair the periodic token with a renewer (see Option 2) so a renewal occurs at least once per period.

Option 2: Sidecar renewer

A sidecar process must call vault token renew before each expiry. The requirement applies whether the token is periodic or has a fixed maximum TTL. The following table compares two common implementations:

Implementation Description
vault agent Configure vault agent with an auto_auth block. The agent authenticates, writes the issued token to a sink, and renews the token automatically. Configure component_keyring_vault.cnf to read a token issued by the same auth method, or generate the configuration with agent templating.
systemd timer or cron job Schedule vault token renew at an interval shorter than the token TTL, for example hourly for a 24-hour token. Run the job under a dedicated service account, not as root, and forward renewal failures to the alerting system.

Configure alerts on renewal failure. Without alerts, a failed renewer behaves the same as no renewer at all. The failure surfaces only at the next mysqld restart, when startup fails.

Operational checklist

  • Treat the token value as a secret equal in sensitivity to the Vault unseal keys. Restrict file permissions on component_keyring_vault.cnf to the mysql user.

  • Monitor the token TTL with vault token lookup. Alert when the remaining TTL drops below a safe threshold.

  • Rotate the token only during a maintenance window. Update the configuration file, then restart mysqld. The component reads the file only at startup.

  • Assign one token to one server. The one-token-per-server rule complements the secret_mount_point warning in Configure the keyring vault component.

Handle a sealed Vault

The keyring_vault component calls the Vault HTTP API at startup and during every key operation. A sealed Vault returns HTTP 503 for any request to the secrets engine. The component cannot read or write keys against a sealed Vault, even with a valid token.

Vault becomes sealed in the following situations:

  • A host reboot or package upgrade restarts the Vault process.

  • An operator runs vault operator seal.

  • Vault seals itself in response to a detected integrity issue.

Percona Server behavior during a sealed Vault depends on timing:

State Behavior
mysqld running, keys cached in memory Reads and writes against the open encrypted tables continue to succeed.
mysqld running, additional key fetch required Operations that require an additional key fetch fail. Examples include opening an encrypted table that was not already open, rotating the master key, and creating an encrypted tablespace.
mysqld restarted while Vault remains sealed InnoDB cannot unwrap tablespace keys. Startup fails or encrypted tables remain inaccessible until Vault is unsealed.

  • Unseal Vault before starting or restarting Percona Server. Confirm that vault status reports Sealed as false.

  • Automate the unseal step with auto-unseal . Use a cloud KMS, Hardware Security Module (HSM), or Transit secret engine as the backing store. A Vault restart then completes without manual intervention. A Vault reboot outside business hours blocks every Percona Server that depends on the sealed Vault.

  • Order the startup dependencies. Configure mysqld to depend on Vault being unsealed, not only on Vault being reachable. Apply the dependency when both processes run on the same host or under the same orchestrator.

  • Monitor the /v1/sys/health endpoint of Vault. The endpoint reports seal status. Alert on sealed=true independently of database telemetry, so operators can respond before the next mysqld restart.

  • Keep unseal keys and recovery keys offline and distributed across custodians. The custody practice belongs to the Vault domain rather than Percona. Lost unseal keys produce permanent data loss for every Percona Server that depends on the sealed Vault.

See also

Percona Server documentation:

HashiCorp documentation:

Use the Key Management Interoperability Protocol (KMIP)

Percona Server for MySQL supports the OASIS Key Management Interoperability Protocol (KMIP) . Percona has validated the KMIP implementation against the following servers:

KMIP enables communication between key management systems and the database server. The protocol provides the following capabilities:

  • Centralized storage and lifecycle management for encryption keys

  • Standardized key exchange between databases and key management systems

Component installation

Install a keyring component through a manifest file. During startup, the server reads the manifest. Each component reads a corresponding configuration file during initialization.

Do not load keyring components with either of the following methods:

Method Why it fails
--early-plugin-load option Loads plugins only, not components
INSTALL COMPONENT statement Registers components in the mysql.component table, which the server loads after InnoDB initialization

Components that InnoDB requires at startup must load earlier.

Create a global manifest file named mysqld.my in the installation directory. Optionally, create a local manifest file with the same name in a data directory.

To install a keyring component, complete the following steps:

  1. Write a manifest in valid JSON format

Each component uses the same manifest mechanism. Each component reads its own configuration file. The name of the configuration file matches the component name with a .cnf extension. For example, component_keyring_vault reads component_keyring_vault.cnf.

Manifest files

Manifest fields

The manifest is a JavaScript Object Notation (JSON) object with the following fields:

Field Type Required Description
components string Conditional URL-style locator for the component to load. The format is file://<component_name>. Required when the manifest declares the component directly.
read_local_manifest boolean No When true, the global manifest delegates to the local manifest in the data directory. When false or absent, the global manifest declares the component directly.

Manifest examples

Replace <component_name> in the following examples with one of the available components.

The following example shows a global manifest that loads a component directly:

{
  "components": "file://<component_name>"
}

The following example shows a global manifest that delegates to the local manifest:

{
  "read_local_manifest": true
}

The following example shows a local manifest:

{
  "components": "file://<component_name>"
}

Manifest locations

Percona Server reads two manifest files at startup:

Manifest Location Purpose
Global manifest The directory that contains the mysqld binary Default manifest for the server installation
Local manifest Data directory Per-instance override for hosts that run multiple instances with different keyring components

Both files use the name mysqld.my. The global manifest declares the component directly or delegates to the local manifest.

Why the manifest is the only supported load path

The keyring must load before InnoDB opens an encrypted page. Any mechanism that depends on the Structured Query Language (SQL) layer therefore loads the keyring too late.

A mysqld startup proceeds in the following order:

  1. mysqld parses startup configuration and reads the manifest file from the server installation directory.

  2. The server loads each component named in the manifest.

  3. InnoDB initializes, replays the redo log, and opens tablespaces.

  4. The SQL layer accepts connections.

The keyring must load between steps 1 and 3. The following table explains why each alternative mechanism fails to load the keyring within this window:

Mechanism Reason for failure
INSTALL COMPONENT The statement runs as SQL and cannot execute until step 4. The registration record lives in mysql.component, an InnoDB table that the server reads only after InnoDB initializes. The system tablespace cannot decrypt without the keyring, which creates a circular dependency. Crash recovery also runs before the SQL layer, so the encrypted redo log must remain readable without SQL.
--early-plugin-load The flag applies to legacy keyring plugins, not components. Plugins and components load through separate subsystems. The flag cannot locate component entry points.

A component registered through INSTALL COMPONENT on a running server does not persist across restarts. On the next restart, InnoDB cannot unwrap tablespace keys because no manifest file exists on disk. A missing or malformed mysqld.my therefore prevents startup for any instance with encrypted tablespaces.

Install a keyring component

To install a keyring component:

  1. Stop the mysqld process.

  2. Create the manifest file at the chosen location. Use one of the examples in Manifest examples as a template.

  3. Create the configuration file in the same directory as the manifest that declares the component. The name of the configuration file matches the component name with a .cnf extension.

  4. Populate the configuration file with parameters for the chosen component. The required parameters depend on the component.

  5. Set the file ownership of the manifest file and the configuration file to the mysql user.

  6. Restrict read access on the configuration file to the mysql user. The configuration file may contain credentials.

  7. Start the mysqld process.

Verify the installation

Connect to the server and run the following query:

SELECT * FROM performance_schema.keyring_component_status;

The query returns one row per status field. The Component_status field reports Active when the component loaded. The Component_status field reports Disabled when the component failed to load. Inspect the server error log to identify the cause of any failure.

Operate the keyring

Rotate the master key

The InnoDB master key wraps the tablespace keys that protect data on disk. Rotate the master key on a scheduled cadence to limit the volume of data wrapped by any single master key.

Run the following statement to rotate the master key:

ALTER INSTANCE ROTATE INNODB MASTER KEY;

The statement generates a master key and stores the key in the keyring. The server then wraps subsequent tablespace keys with the stored key.

The rotation operation is fast. The server does not re-encrypt the tablespace data.

Earlier master keys must remain in the keyring. Deleting an earlier master key prevents the server from reading data that was wrapped with that key.

For required privileges and the full rotation procedure, see Rotate the master encryption key.

Monitor the keyring

Use the query in Verify the installation to inspect the keyring at runtime. Configure alerts on Component_status transitions to Disabled. Watch the server error log for keyring-related errors. The server writes manifest parse errors, configuration parse errors, and backend connection errors to the error log.

Back up and restore

A backup of an encrypted tablespace requires keyring access on the destination host. The destination host must read the same key material that encrypted the backup.

The following table summarizes the keyring requirement for each backend:

Backend Backup requirement
component_keyring_file Copy the keyring file alongside the data backup. The destination host must read the same key material.
component_keyring_vault Configure the destination host to authenticate with the same Vault server and access the same secret_mount_point.
component_keyring_kmip Configure the destination host to authenticate with the same KMIP server and access the same key namespace.
component_keyring_kms Configure the destination host with credentials and permissions for the same cloud KMS keys.

Test a restore on a separate host before relying on the backup. A restore that runs on the original host can succeed even when the keyring configuration is incorrect.

Replace or remove a keyring component

To replace a keyring component:

  1. Stop the mysqld process.

  2. Edit the manifest file to reference the replacement component.

  3. Create the configuration file for the replacement component.

  4. Start the mysqld process.

To remove the keyring:

  1. Confirm that no encrypted tablespaces exist on the server. A server with encrypted tablespaces cannot start without a keyring.

  2. Stop the mysqld process.

  3. Delete the manifest file.

  4. Start the mysqld process.

Warning

Configure exactly one keyring per server instance. Percona Server does not support multiple keyring plugins, multiple keyring components, or any combination of plugin and component. Such configurations risk data loss.

For more information, see Installing and Uninstalling Components .

The following example shows a global manifest file that does not use local manifests:

{
 "read_local_manifest": false,
 "components": "file://component_keyring_kmip"
}

The following example shows a global manifest file that references a local manifest file:

{
 "read_local_manifest": true
}

The following example shows a local manifest file:

{
 "components": "file://component_keyring_kmip"
}

Both global and local configuration files use the same settings. The following example shows a configuration file:

{
 "server_addr": "127.0.0.1",
 "server_port": "5696",
 "client_ca": "client_certificate.pem",
 "client_key": "client_key.pem",
 "server_ca": "root_certificate.pem",
 "object_group": "",
 "max_objects": 65535,
 "kmip_timeout_ms": 5000,
 "tls_peer_verification": false,
 "tls_hostname_verification": false
}

Configuration options

The configuration file for the KMIP keyring supports the following options:

Option Type Required Default Description
server_addr string Yes — Hostname or IP address of the KMIP server
server_port string Yes — TCP port of the KMIP server, for example "5696"
client_ca string Yes — Path to the client certificate presented to the KMIP server, in Privacy-Enhanced Mail (PEM) format
client_key string Yes — Path to the client private key that matches client_ca, in PEM format
server_ca string Yes — Path to the certificate authority (CA) certificate that verifies the KMIP server, in PEM format
object_group string No "" KMIP object group used to register and enumerate objects; an empty value disables grouping
max_objects integer No 65535 Maximum objects retrieved per object type in one locate operation. Applies to symmetric keys and secret data
kmip_timeout_ms integer No 5000 KMIP connection timeout in milliseconds
tls_peer_verification boolean No false When true, verifies the Transport Layer Security (TLS) certificate of the KMIP server against server_ca
tls_hostname_verification boolean No false When true, verifies server_addr against the Subject Alternative Name (SAN) or Common Name (CN) of the KMIP server certificate

TLS verification

Both tls_peer_verification and tls_hostname_verification default to false for backward compatibility. Enable both options in production deployments. The KMIP keyring backend then rejects connections to KMIP servers that present an untrusted or mismatched certificate.

To enable peer verification, set server_ca to the certificate authority that signed the KMIP server certificate. To enable hostname verification, set server_addr to match the SAN or CN of the KMIP server certificate.

KMIP object state requirements

The KMIP keyring backend loads only objects in the ACTIVE state. Loadable objects include symmetric keys and secret data. The backend ignores objects in the PRE_ACTIVE, DEACTIVATED, COMPROMISED, or DESTROYED states. The backend activates each key or secret during registration.

Important

Activate any PRE_ACTIVE keys on the KMIP server before you start Percona Server for MySQL. Otherwise, Percona Server for MySQL cannot load those keys.

Use the KMIP Activate operation through the management interface of your KMIP server. Supported interfaces include a web console, a command-line tool, or a KMIP client library. For exact steps, see the documentation for your KMIP server.

Supported AES key sizes

The KMIP keyring backend accepts these Advanced Encryption Standard (AES) key sizes:

AES key size Length in bytes
128 bits 16
192 bits 24
256 bits 32

The KMIP keyring backend rejects AES keys of any other length. The keyring_udf plugin returns ER_KEYRING_UDF_KEYRING_SERVICE_ERROR for requests that specify a non-standard key size.

Important

Review any scripts or tooling that use non-standard AES key sizes before you upgrade.

Verify the keyring component

After you start Percona Server for MySQL, confirm that the keyring component loaded and initialized. Query the performance_schema.keyring_component_status table:

SELECT * FROM performance_schema.keyring_component_status;

The query returns one row for each status key that the loaded component reports. The Component_status row indicates the result of initialization:

Component_status value Meaning Recovery
Active The component loaded and initialized successfully None
Disabled The component loaded but failed to initialize, typically due to a configuration error Review the server error log, correct the configuration file, then run ALTER INSTANCE RELOAD KEYRING

If a keyring component fails to load entirely, the server does not start. Check the server error log for diagnostic messages.

For the complete list of status keys that each component reports, see keyring_component_status Table in the MySQL Reference Manual.

Use the Amazon Key Management Service (AWS KMS)

Percona Server for MySQL supports the Amazon Key Management Service (AWS KMS) . Percona Server generates the keyring keys. Amazon Web Services (AWS) encrypts the keyring data.

The AWS KMS lets you create and manage cryptographic keys across AWS services. For more information, see the AWS Key Management Service Documentation .

Prerequisites

Complete the following before you install the AWS KMS component:

  • Create an AWS user with an access key and a secret key.

  • Create a KMS key. Reference the key by its identifier, alias, or Amazon Resource Name (ARN). Each key can have any number of aliases.

Install a keyring component through a manifest file. During startup, the server reads the manifest. Each component reads a corresponding configuration file during initialization.

Do not load keyring components with either of the following methods:

Method Why it fails
--early-plugin-load option Loads plugins only, not components
INSTALL COMPONENT statement Registers components in the mysql.component table, which the server loads after InnoDB initialization

Components that InnoDB requires at startup must load earlier.

Create a global manifest file named mysqld.my in the installation directory. Optionally, create a local manifest file with the same name in a data directory.

To install a keyring component, complete the following steps:

  1. Write a manifest in valid JSON format

Each component uses the same manifest mechanism. Each component reads its own configuration file. The name of the configuration file matches the component name with a .cnf extension. For example, component_keyring_vault reads component_keyring_vault.cnf.

Manifest files

Manifest fields

The manifest is a JavaScript Object Notation (JSON) object with the following fields:

Field Type Required Description
components string Conditional URL-style locator for the component to load. The format is file://<component_name>. Required when the manifest declares the component directly.
read_local_manifest boolean No When true, the global manifest delegates to the local manifest in the data directory. When false or absent, the global manifest declares the component directly.

Manifest examples

Replace <component_name> in the following examples with one of the available components.

The following example shows a global manifest that loads a component directly:

{
  "components": "file://<component_name>"
}

The following example shows a global manifest that delegates to the local manifest:

{
  "read_local_manifest": true
}

The following example shows a local manifest:

{
  "components": "file://<component_name>"
}

Manifest locations

Percona Server reads two manifest files at startup:

Manifest Location Purpose
Global manifest The directory that contains the mysqld binary Default manifest for the server installation
Local manifest Data directory Per-instance override for hosts that run multiple instances with different keyring components

Both files use the name mysqld.my. The global manifest declares the component directly or delegates to the local manifest.

Why the manifest is the only supported load path

The keyring must load before InnoDB opens an encrypted page. Any mechanism that depends on the Structured Query Language (SQL) layer therefore loads the keyring too late.

A mysqld startup proceeds in the following order:

  1. mysqld parses startup configuration and reads the manifest file from the server installation directory.

  2. The server loads each component named in the manifest.

  3. InnoDB initializes, replays the redo log, and opens tablespaces.

  4. The SQL layer accepts connections.

The keyring must load between steps 1 and 3. The following table explains why each alternative mechanism fails to load the keyring within this window:

Mechanism Reason for failure
INSTALL COMPONENT The statement runs as SQL and cannot execute until step 4. The registration record lives in mysql.component, an InnoDB table that the server reads only after InnoDB initializes. The system tablespace cannot decrypt without the keyring, which creates a circular dependency. Crash recovery also runs before the SQL layer, so the encrypted redo log must remain readable without SQL.
--early-plugin-load The flag applies to legacy keyring plugins, not components. Plugins and components load through separate subsystems. The flag cannot locate component entry points.

A component registered through INSTALL COMPONENT on a running server does not persist across restarts. On the next restart, InnoDB cannot unwrap tablespace keys because no manifest file exists on disk. A missing or malformed mysqld.my therefore prevents startup for any instance with encrypted tablespaces.

Install a keyring component

To install a keyring component:

  1. Stop the mysqld process.

  2. Create the manifest file at the chosen location. Use one of the examples in Manifest examples as a template.

  3. Create the configuration file in the same directory as the manifest that declares the component. The name of the configuration file matches the component name with a .cnf extension.

  4. Populate the configuration file with parameters for the chosen component. The required parameters depend on the component.

  5. Set the file ownership of the manifest file and the configuration file to the mysql user.

  6. Restrict read access on the configuration file to the mysql user. The configuration file may contain credentials.

  7. Start the mysqld process.

Verify the installation

Connect to the server and run the following query:

SELECT * FROM performance_schema.keyring_component_status;

The query returns one row per status field. The Component_status field reports Active when the component loaded. The Component_status field reports Disabled when the component failed to load. Inspect the server error log to identify the cause of any failure.

Operate the keyring

Rotate the master key

The InnoDB master key wraps the tablespace keys that protect data on disk. Rotate the master key on a scheduled cadence to limit the volume of data wrapped by any single master key.

Run the following statement to rotate the master key:

ALTER INSTANCE ROTATE INNODB MASTER KEY;

The statement generates a master key and stores the key in the keyring. The server then wraps subsequent tablespace keys with the stored key.

The rotation operation is fast. The server does not re-encrypt the tablespace data.

Earlier master keys must remain in the keyring. Deleting an earlier master key prevents the server from reading data that was wrapped with that key.

For required privileges and the full rotation procedure, see Rotate the master encryption key.

Monitor the keyring

Use the query in Verify the installation to inspect the keyring at runtime. Configure alerts on Component_status transitions to Disabled. Watch the server error log for keyring-related errors. The server writes manifest parse errors, configuration parse errors, and backend connection errors to the error log.

Back up and restore

A backup of an encrypted tablespace requires keyring access on the destination host. The destination host must read the same key material that encrypted the backup.

The following table summarizes the keyring requirement for each backend:

Backend Backup requirement
component_keyring_file Copy the keyring file alongside the data backup. The destination host must read the same key material.
component_keyring_vault Configure the destination host to authenticate with the same Vault server and access the same secret_mount_point.
component_keyring_kmip Configure the destination host to authenticate with the same KMIP server and access the same key namespace.
component_keyring_kms Configure the destination host with credentials and permissions for the same cloud KMS keys.

Test a restore on a separate host before relying on the backup. A restore that runs on the original host can succeed even when the keyring configuration is incorrect.

Replace or remove a keyring component

To replace a keyring component:

  1. Stop the mysqld process.

  2. Edit the manifest file to reference the replacement component.

  3. Create the configuration file for the replacement component.

  4. Start the mysqld process.

To remove the keyring:

  1. Confirm that no encrypted tablespaces exist on the server. A server with encrypted tablespaces cannot start without a keyring.

  2. Stop the mysqld process.

  3. Delete the manifest file.

  4. Start the mysqld process.

Warning

Configure exactly one keyring per server instance. Percona Server does not support multiple keyring plugins, multiple keyring components, or any combination of plugin and component. Such configurations risk data loss.

For more information, see Installing and uninstalling components .

Configuration settings

The configuration settings live in either a global configuration file or a local configuration file.

Configuration parameters

The component_keyring_kms.cnf file contains the following parameters:

  • read_local_config [optional] — declares whether the component reads the local configuration file. Use this parameter only in the global configuration file. The allowed values are true or false.

  • path — the path to the keyring file on the local host. The file uses JSON format.

  • read_only [optional] — when true, the component prevents runtime changes to the keyring. The default value is false.

  • kms_key — the identifier of an AWS KMS master key. Create the key before you create the manifest file. The identifier accepts one of the following formats:

    • Universally Unique Identifier (UUID)

    • Alias

    • Amazon Resource Name (ARN)

    For more information, see Finding the key ID and key ARN .

  • region — the AWS region that hosts the KMS key. Each Hypertext Transfer Protocol (HTTP) request connects to this region.

  • auth_key — the access key for the AWS user. The user must have permission to access the KMS key.

  • secret_access_key — the secret key for the AWS user.

Warning

The configuration file contains authentication credentials. Restrict read access on the file to the mysql user.

Example configuration

The following example shows a component_keyring_kms.cnf file in JSON format with placeholder values. Replace each placeholder with the value for the target AWS KMS deployment.

{
  "read_local_config": false,
  "path": "/usr/local/mysql/keyring-mysql/aws-keyring-data",
  "region": "<AWS_REGION>",
  "kms_key": "<KMS_KEY_IDENTIFIER>",
  "auth_key": "<AWS_USER_ACCESS_KEY>",
  "secret_access_key": "<AWS_USER_SECRET_ACCESS_KEY>"
}

Verify the keyring component

After you start Percona Server for MySQL, confirm that the keyring component loaded and initialized. Query the performance_schema.keyring_component_status table:

SELECT * FROM performance_schema.keyring_component_status;

The query returns one row for each status key that the loaded component reports. The Component_status row indicates the result of initialization:

Component_status value Meaning Recovery
Active The component loaded and initialized successfully None
Disabled The component loaded but failed to initialize, typically due to a configuration error Review the server error log, correct the configuration file, then run ALTER INSTANCE RELOAD KEYRING

If a keyring component fails to load entirely, the server does not start. Check the server error log for diagnostic messages.

For the complete list of status keys that each component reports, see keyring_component_status Table in the MySQL Reference Manual.

Encrypt File-Per-Table Tablespace

The file_per_table tablespace inherits the default schema encryption setting. Use the ENCRYPTION clause in the in CREATE TABLE statement to explicitly set the encryption.

CREATE TABLE ... ENCRYPTION='Y';
To change the encryption setting for an existing file_per_table tablespace, add the ENCRYPTION clause. The ALTER TABLE statement without the ENCRYPTION clause does not change the encryption state.

ALTER TABLE ... ENCRYPTION='Y';

If the table_encryption_privilege_check is enabled, the TABLE_ENCRYPTION_ADMIN privilige is required to change the file_per_table encryption setting from the default schema encryption.

Encrypt schema or general tablespace

Percona Server for MySQL uses the same encryption architecture as MySQL, a two-tier system consisting of a master key and tablespace keys. The master key can be changed, or rotated in the keyring, as needed. Each tablespace key, when decrypted, remains the same.

The feature requires the keyring plugin.

Set the default for schemas and general tablespace encryption

The tables in a general tablespace are either all encrypted or all unencrypted. A tablespace cannot contain a mixture of encrypted tables and unencrypted tables.

The encryption of a schema or a general tablespace is determined by the default_table_encryption variable unless you specify the ENCRYPTION clause in the CREATE SCHEMA or CREATE TABLESPACE statement.

You can set the default_table_encryption variable in an individual connection.

SET default_table_encryption=ON;

default_table_encryption

Option Description
Command-line default-table-encryption
Scope Session
Dynamic Yes
Data type Text
Default OFF

Defines the default encryption setting for schemas and general tablespaces. The variable allows you to create or alter schemas or tablespaces without specifying the ENCRYPTION clause. The default encryption setting applies only to schemas and general tablespaces and is not applied to the MySQL system tablespace.

The variable has the following possible options:

Value Description
ON New tables are encrypted. Add ENCRYPTION="N" to the CREATE TABLE or ALTER TABLE statement to create unencrypted tables.
OFF By default, new tables are unencrypted. Add ENCRYPTION="Y" to the CREATE TABLE or ALTER TABLE statement to create encrypted tables.

Note

The ALTER TABLE statement changes the current encryption mode only if you use the ENCRYPTION clause.

innodb_encrypt_online_alter_logs

Option Description
Command-line –innodb-encrypt-online-alter-logs
Scope Global
Dynamic Yes
Data type Boolean
Default OFF

This variable simultaneously turns on the encryption of files used by InnoDB for full-text search using parallel sorting, building indexes using merge sort, and online DDL logs created by InnoDB for online DDL. Encryption is available for file merges used in queries and backend processes.

Use ENCRYPTION

If you do not set the default encryption setting, you can create general tablespaces with the ENCRYPTION setting.

CREATE TABLESPACE tablespace_name ENCRYPTION='Y';

All tables contained in the tablespace are either encrypted or not encrypted. You cannot encrypt only some of the tables in a general tablespace. This feature extends the CREATE TABLESPACE statement to accept the ENCRYPTION='Y/N' option.

The option is a tablespace attribute and is not allowed with the CREATE TABLE or SHOW CREATE TABLE statement except with file-per-table tablespaces.

In an encrypted general tablespace, an attempt to create an unencrypted table generates the following error:

CREATE TABLE t3 (a INT, b TEXT) TABLESPACE foo ENCRYPTION='N';
Expected output 
ERROR 1478 (HY0000): InnoDB: Tablespace 'foo' can contain only ENCRYPTED tables.

The server diagnoses an attempt to create or move tables, including partitioned ones, to a general tablespace with an incompatible encryption setting and aborts the process.

If you must move tables between incompatible tablespaces, create tables with the same structure in another tablespace and run INSERT INTO SELECT from each of the source tables into the destination tables.

Export an encrypted general tablespace

You can only export encrypted file-per-table tablespaces

Encrypt system tablespace

By default, the system tablespace, which contains the system database and the data dictionary tables, is unencrypted. To change the encryption requires the CREATE TABLESPACE privilege on all tables in the instance.

In an ALTER TABLESPCE statement, add the ENCRYPTION option with the tablespace name to enable encryption.

ALTER TABLESPACE mysql ENCRYPTION='Y';
Disable encryption by setting the ENCRYPTION option to ‘N”.

ALTER TABLESPACE mysql ENCRYPTION='N';

Encrypt doublewrite file pages

InnoDB encrypts doublewrite file pages associated with encrypted tablespaces automatically. Doublewrite files can contain the following page types:

  • Unencrypted
  • Uncompressed
  • Encrypted
  • Compressed

Encrypt temporary files

For InnoDB user-created temporary tables are created in a temporary tablespace file and use the innodb_temp_tablespace_encrypt variable.

The CREATE TEMPORARY TABLE does not support the ENCRYPTION clause. The TABLESPACE clause cannot be set to innodb_temporary.

The global temporary tablespace datafile ibtmp1 contains the temporary table undo logs while intrinsic temporary tables and user-created temporary tables are located in the encrypted session temporary tablespace.

To create new temporary tablespaces unencrypted, the following variables must be set to OFF at runtime:

  • innodb_temp_tablespace_encrypt

  • default_table_encryption

Any existing encrypted user-created temporary files and intrinsic temporary tables remain in an encrypted session.

Temporary tables are only destroyed when the session is disconnected.

The default_table_encryption setting in my.cnf determines if a temporary table is encrypted.

If the innodb_temp_tablespace_encrypt = “OFF” and the default_table_encryption =”ON”, the user-created temporary tables are encrypted. The temporary tablespace data file ibtmp1, which contains undo logs, is not encrypted.

If the innodb_temp_tablespace_encrypt is “ON” for the system tablespace, InnoDB generates an encryption key and encrypts the system’s temporary tablespace. If you reset the encryption to “OFF”, all subsequent pages are written to an unencrypted tablespace. Any generated keys are not erased to allow encrypted tables and undo data to be decrypted.

For each temporary file, an encryption key has the following attributes:

  • Generated locally

  • Maintained in memory for the lifetime of the temporary file

  • Discarded with the temporary file

encrypt_tmp_files

Option Description
Command-line –encrypt_tmp_files
Scope Global
Dynamic No
Data type Boolean
Default OFF

This variable turns “ON” the encryption of temporary files created by the Percona Server for MySQL. The default value is OFF.

innodb_temp_tablespace_encrypt

Option Description
Command-line innodb-temp-tablespace-encrypt
Scope Global
Dynamic Yes
Data type Boolean
Default OFF

When this variable is set to ON, the server encrypts the global temporary tablespace and has the .ibtmp1 file extension and the session temporary tablespace and has the .ibt file extension.

The variable does not enforce the encryption of currently open temporary files and does not rebuild the system’s temporary tablespace to encrypt data that has already been written.

Log encryption

Describes the redo log encryption and the undo log encryption.

Redo Log encryption

Use the innodb_redo_log_encrypt option to enable or disable redo log data encryption. By default, the encryption of the redo log is disabled.

InnoDB uses the tablespace encryption key to encrypt the redo log data. If the encryption is enabled, when the server encrypts and writes the redo log data to the disk. When the server reads the redo log data from disk, the data is decrypted.

Changing the encryption does not change existing redo log pages. Setting the option to ON, any existing redo log pages remain unencrypted; writing new pages to disk encrypts them. Setting the option to OFF, any existing encrypted pages remain encrypted; writing new pages to disk are unencrypted.

The metadata for the redo log encryption includes the tablespace encryption key and is stored the in redo log file header. Removing the encryption metadata disables the redo log encryption.

Without the keyring component or the encryption key, a normal restart is not possible. InnoDB scans the redo log pages during startup. If the encryption options are not available, InnoDB cannot scan these pages. A forced startup without the redo logs is possible.

Option

innodb_redo_log_encrypt

Variable Description
Command-line --innodb-redo-log-encrypt[= {ON \| OFF}]
Dynamic Yes
Scope Global
Data type Boolean
Default OFF
Option Description
ON This option is a compatibility alias for the master_key. Any existing redo log pages remain unencrypted; new pages are encrypted when written to disk.
OFF Any existing encrypted pages remain encrypted; new pages are unencrypted.

Determines the encryption for the table redo log data. The default option for the variable is OFF.

Undo Log encryption

Use the innodb_undo_log_encrypt option to enable or disable undo log data encryption. By default, the option to encrypt the undo log data is disabled.

InnoDB uses the tablespace encryption key to encrypt the undo log data. If the encryption is enabled, when the server encrypts and writes the undo log data to the disk. When the server reads the undo log data from disk, the data is decrypted.

Changing the encryption does not change existing undo log pages. Setting the option to ON, any existing pages remain unencrypted; writing new pages to disk encrypts them. Setting the option to OFF, any existing encrypted pages remain encrypted; writing new pages to disk are unencrypted.

The metadata for the redo log encryption includes the tablespace encryption key and is stored the in undo log file header.

The server requires the keyring component used to encrypt log data until that data is truncated, even if the current option setting is OFF. When the undo tablespace is truncated, the encryption header is removed.

innodb_undo_log_encrypt

Option Description
Command-line --innodb-undo-log-encrypt[= {ON \| OFF}]
Scope Global
Dynamic Yes
Data type Boolean
Default OFF

This system variable defines the encryption status for the undo log data. The default setting is OFF, which disables the encryption.

Rotate the master encryption key

Rotate the master encryption key periodically and if the key has been compromised.

Rotating the master encryption key changes that key and tablespace keys are re-encrypted and updated in the tablespace headers. The rotation only succeeds if all operations are successful. If the rotation is interrupted, the operation is rolled forward when the server restarts.

The rotation operation does not affect tablespace data. To change a tablespace key, disable and then re-enable encryption for that tablespace.

The ENCRYPTION_KEY_ADMIN privilege is required to rotate the master encryption key.

InnoDB reads the encryption data from the tablespace header, if certain tablespace keys have been encrypted with the prior master key, InnoDB retrieves the master key from the keyring to decrypt the tablespace key. InnoDB re-encrypts the tablespace key with the new Master key.

Rotate the master encryption key with following statement:

ALTER INSTANCE ROTATE INNODB MASTER KEY;

The rotation operation must complete before any tablespace encryption operation can begin.

Verify the encryption for tables, tablespaces, and schemas

If a general tablespace contains tables, check the table information to see if the table is encrypted. When the general tablespace contains no tables, you may verify if the tablespace is encrypted or not.

For single tablespaces, verify the ENCRYPTION option using INFORMATION_SCHEMA.TABLES and the CREATE OPTIONS settings.

SELECT TABLE_SCHEMA, TABLE_NAME, CREATE_OPTIONS FROM
       INFORMATION_SCHEMA.TABLES WHERE CREATE_OPTIONS LIKE '%ENCRYPTION%';
Expected output 
+----------------------+-------------------+------------------------------+
| TABLE_SCHEMA         | TABLE_NAME        | CREATE_OPTIONS               |
+----------------------+-------------------+------------------------------+
|sample                | t1                | ENCRYPTION="Y"               |
+----------------------+-------------------+------------------------------+

A flag field in the INFORMATION_SCHEMA.INNODB_TABLESPACES has bit number 13 set if the tablespace is encrypted. This bit can be checked with the flag & 8192 expression in the following way:

SELECT space, name, flag, (flag & 8192) != 0 AS encrypted FROM
INFORMATION_SCHEMA.INNODB_TABLESPACES WHERE name in ('foo', 'test/t2', 'bar',
'noencrypt');

The encrypted table metadata is contained in the INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION table. You must have the Process privilege to view the table information.

Note

This table is in tech preview and may change in future releases.

   DESCRIBE INNODB_TABLESPACES_ENCRYPTION;
Expected output 
+-----------------------------+--------------------+-----+----+--------+------+
| Field                       | Type               | Null| Key| Default| Extra|
+-----------------------------+--------------------+-----+----+--------+------+
| SPACE                       | int(11) unsigned   | NO  |    |        |      |
| NAME                        | varchar(655)       | YES |    |        |      |
| ENCRYPTION_SCHEME           | int(11) unsigned   | NO  |    |        |      |
| KEYSERVER_REQUESTS          | int(11) unsigned   | NO  |    |        |      |
| MIN_KEY_VERSION             | int(11) unsigned   | NO  |    |        |      |
| CURRENT_KEY_VERSION         | int(11) unsigned   | NO  |    |        |      |
| KEY_ROTATION_PAGE_NUMBER    | bigint(21) unsigned| YES |    |        |      |
| KEY_ROTATION_MAX_PAGE_NUMBER| bigint(21) unsigned| YES |    |        |      |
| CURRENT_KEY_ID              | int(11) unsigned   | NO  |    |        |      |
| ROTATING_OR_FLUSHING        | int(1) unsigned    | NO  |    |        |      |
+-----------------------------+--------------------+-----+----+--------+------+

To identify encryption-enabled schemas, query the INFORMATION_SCHEMA.SCHEMATA table:

SELECT SCHEMA_NAME, DEFAULT_ENCRYPTION FROM
INFORMATION_SCHEMA.SCHEMATA WHERE DEFAULT_ENCRYPTION='YES';
Expected output 
+------------------------------+---------------------------------+
| SCHEMA_NAME                  | DEFAULT_ENCRYPTION              |
+------------------------------+---------------------------------+
| samples                      | YES                             |
+------------------------------+---------------------------------+

The SHOW CREATE SCHEMA statement returns the DEFAULT ENCRYPTION clause.

Replicate

Manage group replication flow control

Group Replication flow control signals the cluster to reduce queue growth on members that fall behind. Flow control does not prevent members from lagging behind the group. Flow control does not stop writes when queues remain above the configured limits.

Each member reports statistics to the group. Flow control compares queue sizes against thresholds on the certification queue and the applier queue. When a threshold is exceeded, the group asks writer members to slow down so the delayed member can reduce its queues toward the limit. If the member cannot catch up, the cluster continues to run. Flow control never blocks the group indefinitely.

For a deeper discussion of failover behavior when queues grow, see The failover brownout: rethinking high availability in MySQL Group Replication.

How flow control behaves

Flow control runs asynchronously. The group monitors throughput and queue sizes on each member. When thresholds are exceeded, the group throttles writers to match the capacity of the slowest member.

The cluster aims to bring the applier queue back under the configured limit. The cluster does not guarantee that result. Members can remain at different stages of applied data. Secondary members may and will hold a different view of applied data than the primary.

What to expect when members lag

When flow control cannot keep queues under the limit, the following effects are common:

  • The certification buffer grows on the primary and on secondary members.

  • After failover, the group elects a new primary but keeps the new primary in read-only mode until the applier queue is processed.

  • Pending transactions accumulate in the relay log on lagging members.

Plan capacity, monitoring, and failover testing with these limits in mind. Do not treat flow control as a substitute for right-sized hardware, parallel apply tuning, or application design.

Tune parallel apply with flow control

Flow control works with the replication applier. The replica_parallel_workers variable controls how many applier threads process the relay log.

Set replica_parallel_workers to approximately 2.5 times the number of CPU cores on the member. For example, a host with 8 cores can use replica_parallel_workers = 20. Parallel apply helps the member drain the applier queue more efficiently. The setting does not guarantee that flow control prevents lag.

Review workload patterns, commit ordering requirements, and any known limitations before raising replica_parallel_workers in production.

Configure flow control variables

The following system variables control flow control behavior for Group Replication:

Enable or disable flow control with group_replication_flow_control_mode. You can apply thresholds on the certifier queue, the applier queue, or both. Percona Server for MySQL also adds a MAJORITY mode. For details, see Group replication system variables.

Group replication system variables

variable name
group_replication_auto_evict_timeout
group_replication_certification_loop_chunk_size
group_replication_certification_loop_sleep_time
group_replication_flow_control_mode

group_replication_auto_evict_timeout

The variable is in tech preview mode. Before using the variable in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.

Option Description
Command-line –group-replication-auto-evict-timeout
Dynamic Yes
Scope Global
Type Integer
Default value 0
Maximum Value 65535
Unit seconds

The value can be changed while Group Replication is running. The change takes effect immediately. Every node in the group can have a different timeout value, but, to avoid unexpected exits, we recommend that all nodes have the same value.

The variable specifies a period of time in seconds before a node is automatically evicted if the node exceeds the flow control threshold. The default value is 0, which disables the eviction. To set the timeout, change the value with a number higher than zero.

In single-primary mode, the primary server ignores the timeout.

group_replication_certification_loop_chunk_size

Option Description
Command-line –group-replication-certification-loop-chunk-size
Dynamic Yes
Scope Global
Data type ulong
Default value 0

Defines the size of the chunk that must be processed during the certifier garbage collection phase after which the client transactions are allowed to interleave. The default value is 0.

The minimum value is 0. The maximum value is 4294967295.

group_replication_certification_loop_sleep_time

Option Description
Command-line –group-replication-certification-loop-sleep-time
Dynamic Yes
Scope Global
Data type ulong
Default value 0

Defines the sleep time in microseconds that the certifier garbage collection loop allows client transactions to interleave. The default value is 0.

The minimum value is 0. The maximum value is 1000000.

group_replication_flow_control_mode

Option Description
Command-line –group_replication_flow_control_mode
Dynamic Yes
Scope Global
Data type Enumeration
Default value Quota
Valid values DISABLED
QUOTA
MAJORITY

The “MAJORITY” value is in tech preview mode. Before using the variable in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.

The variable specifies the mode use for flow control.

Percona Server for MySQL adds the “MAJORITY” value to the group_replication_flow_control_mode variable. In “MAJORITY” mode, flow control is activated only if the majority, more than half the number of members, exceed the flow control threshold. The other values are not changed.

Replication Security

Encrypt binary log files and relay log files

Encrypt the binary log files and the relay log files to protect them from unauthorized viewing. The encryption uses the Advanced Encryption Standard (AES) symmetric block cipher algorithm. Instead of bits, this algorithm works on bytes.

Any supported encryption keyring plugin or component must be installed and configured to use encryption.

Enabling binary log encryption at system start, the server generates a binary log encryption key before iniatializing the binary log and relay logs. The key encrypts a file password for each binary log, if binary logging is enabled, and relay log. Keys generated from the file passwords encrypt the data in the files.

When used by the server, the binary log encryption key is called the binary log master key. This key can be rotated as needed and only the file password for each file is re-encrypted.

The binary log index file and relay log index file is never encrypted. Relay log files are encrypted for all channels.

To review if a binary log file is encrypted or not, use the SHOW BINARY LOGS statement.

If the server is running, the BINLOG_ENCRYPTION_ADMIN privilege is required to enable or disable encryption.

binlog_encryption

Option Description
Command-line --binlog-encryption[= {ON \| OFF}]
Scope Global
Dynamic Yes
Data type Boolean
Default OFF

This system variable enables binary log file encryption and relay log file encryption on the server. The default value is OFF. You can enable encryption for relay log files on a replica without a binary log.

If you set the binlog_encryption to OFF, the server immediately rotates the binary log file and relay log files and all logging is not encrypted. For any previously encrypted files, the server can still read them and they are not decrypted.

Monitor

Audit log plugin

Deprecation notice

The audit log plugin is deprecated in Percona Server for MySQL 8.4 and will be removed in a future release. This deprecation is due to the availability of the audit log filter component, which is the recommended replacement. Users should migrate to this component, which provides equivalent functionality with enhanced flexibility, performance, and filtering capabilities, ensuring continued support for auditing and compliance requirements.

See Migrate to the audit log filter component for a step-by-step mapping of audit_log_* system variables, audit_log_policy, and include/exclude lists to filter JSON and component variables.

This deprecation affects all installations that rely on the audit log plugin for event logging, compliance auditing, or activity tracking. The plugin will continue to function, but no further development or maintenance is planned.

The audit log plugin and the audit log filter component use different configuration variables and options.

  • Do not attempt to use audit log filter variables, options, or configuration syntax with the deprecated audit log plugin. Doing so can lead to startup failures, unexpected behavior, or data loss.

  • Do not install both audit log plugin and audit log filter component simultaneously.

The audit‑log entries may look different from the 8.0 entries. The audit log plugin itself has not changed, but other server components have, and those changes affect the log output. For example, 8.4 logs a SELECT $$ statement each time a client connects because the client supports “dollar‑quoted” strings. This feature did not exist in 8.0.

Percona does not plan to modify 8.4 audit logs to match the format or content of 8.0 logs.

Percona Audit Log Plugin provides monitoring and logging of connection and query activity that were performed on specific server. Information about the activity is stored in a log file.

Install the plugin

The audit Log plugin is installed, but, by default, is not enabled when you install Percona Server for MySQL. To check if the plugin is enabled run the following command. This command searches for plugins with names containing the word “audit” in the information_schema.PLUGINS table. 

SELECT * FROM information_schema.PLUGINS WHERE PLUGIN_NAME LIKE '%audit%';

The empty result suggests that no such plugins are installed or loaded.

Expected output 
Empty set (0.00 sec)

This command checks for system variables whose names start with “audit.” 

SHOW variables LIKE 'audit%';

The empty result means that no such system variables exist or are defined.

Expected output 
Empty set (0.01 sec)

This command lists system variables with names starting with “plugin.” As seen in the example output, it displays the plugin_dir variable, which specifies the directory path where MySQL plugins are stored.

SHOW variables LIKE 'plugin%';
Expected output 
+---------------+------------------------+
| Variable_name | Value                  |
+---------------+------------------------+
| plugin_dir    | /usr/lib/mysql/plugin/ |
+---------------+------------------------+
1 row in set (0.00 sec)

Note

The location of the MySQL plugin directory depends on the operating system and may be different on your system.

The following command enables the plugin:

INSTALL PLUGIN audit_log SONAME 'audit_log.so';

Run the following command to verify if the plugin was installed correctly:

SELECT * FROM information_schema.PLUGINS WHERE PLUGIN_NAME LIKE '%audit%'\G
Expected output 
*************************** 1. row ***************************
        PLUGIN_NAME: audit_log
        PLUGIN_VERSION: 0.2
        PLUGIN_STATUS: ACTIVE
        PLUGIN_TYPE: AUDIT
PLUGIN_TYPE_VERSION: 4.1
        PLUGIN_LIBRARY: audit_log.so
PLUGIN_LIBRARY_VERSION: 1.7
        PLUGIN_AUTHOR: Percona LLC and/or its affiliates.
    PLUGIN_DESCRIPTION: Audit log
        PLUGIN_LICENSE: GPL
        LOAD_OPTION: ON
1 row in set (0.00 sec)

You can review the audit log variables with the following command:

SHOW variables LIKE 'audit%';
Expected output 
+-----------------------------+---------------+
| Variable_name               | Value         |
+-----------------------------+---------------+
| audit_log_buffer_size       | 1048576       |
| audit_log_exclude_accounts  |               |
| audit_log_exclude_commands  |               |
| audit_log_exclude_databases |               |
| audit_log_file              | audit.log     |
| audit_log_flush             | OFF           |
| audit_log_format            | OLD           |
| audit_log_handler           | FILE          |
| audit_log_include_accounts  |               |
| audit_log_include_commands  |               |
| audit_log_include_databases |               |
| audit_log_policy            | ALL           |
| audit_log_rotate_on_size    | 0             |
| audit_log_rotations         | 0             |
| audit_log_strategy          | ASYNCHRONOUS  |
| audit_log_syslog_facility   | LOG_USER      |
| audit_log_syslog_ident      | percona-audit |
| audit_log_syslog_priority   | LOG_INFO      |
+-----------------------------+---------------+
18 rows in set (0.00 sec)

Log format

The plugin supports the following log formats: OLD, NEW, JSON, and CSV. The OLD format and the NEW format are based on XML. The OLD format defines each log record with XML attributes. The NEW format defines each log record with XML tags. The information logged is the same for all four formats. The audit_log_format variable controls the log format choice.

Format examples

The following formats are available:

<AUDIT_RECORD
NAME="Query"
RECORD="3_2021-06-30T11:56:53"
TIMESTAMP="2021-06-30T11:57:14 UTC"
COMMAND_CLASS="select"
CONNECTION_ID="3"
STATUS="0"
SQLTEXT="select * from information_schema.PLUGINS where PLUGIN_NAME like '%audit%'"
USER="root[root] @ localhost []"
HOST="localhost"
OS_USER=""
IP=""
DB=""
/>

<AUDIT_RECORD>
<NAME>Query</NAME>
<RECORD>16684_2021-06-30T16:07:41</RECORD>
<TIMESTAMP>2021-06-30T16:08:06 UTC</TIMESTAMP>
<COMMAND_CLASS>select</COMMAND_CLASS>
<CONNECTION_ID>2</CONNECTION_ID>
<STATUS>0</STATUS>
<SQLTEXT>select id, holder from one</SQLTEXT>
<USER>root[root] @ localhost []</USER>
<HOST>localhost</HOST>
<OS_USER></OS_USER>
<IP></IP>
<DB></DB>

{"audit_record":{"name":"Query","record":"13149_2021-06-30T15:03:11","timestamp":"2021-06-30T15:07:58 UTC","command_class":"show_databases","connection_id":"2","status":0,"sqltext":"show databases","user":"root[root] @ localhost []","host":"localhost","os_user":"","ip":"","db":""}}

"Query","22567_2021-06-30T16:10:09","2021-06-30T16:19:00 UTC","select","2",0,"select count(*) from one","root[root] @ localhost []","localhost","","",""

Audit log events

The audit Log plugin generates a log of following events.

Audit event indicates that audit logging started or finished. NAME field will be Audit when logging started and NoAudit when logging finished. Audit record also includes server version and command-line arguments.

??? example "Audit event"

    ```{.text .no-copy}
    <AUDIT_RECORD
    NAME="Audit"
    RECORD="1_2021-06-30T11:56:53"
    TIMESTAMP="2021-06-30T11:56:53 UTC"
    MYSQL_VERSION="5.7.34-37"
    STARTUP_OPTIONS="--daemonize --pid-file=/var/run/mysqld/mysqld.pid"
    OS_VERSION="x86_64-debian-linux-gnu"
    />
    ```

Connect record event will have NAME field Connect when user logged in or login failed, or Quit when connection is closed.

The additional fields for this event are the following:

* `CONNECTION_ID`

* `STATUS`

* `USER`

* `PRIV_USER`

* `OS_LOGIN`

* `PROXY_USER`

* `HOST`

* `IP`

The value for STATUS is 0 for successful logins and non-zero for failed logins.

Disconnect event 
<AUDIT_RECORD
NAME="Quit"
RECORD="5_2021-06-29T19:33:03"
TIMESTAMP="2021-06-29T19:34:38Z"
CONNECTION_ID="14"
STATUS="0"
USER="root"
PRIV_USER="root"
OS_LOGIN=""
PROXY_USER=""
HOST="localhost"
IP=""
DB=""
/>

Additional fields for this event are: COMMAND_CLASS (values come from the com_status_vars array in the `sql/mysqld.cc`` file in a MySQL source distribution.

Examples are select, alter_table, create_table, etc.), CONNECTION_ID, STATUS (indicates an error when the vaule is non-zero), SQLTEXT (text of SQL-statement), USER, HOST, OS_USER, IP.

The possible values for the NAME name field for this event are Query, Prepare, Execute, Change user, etc.

Query event 
<AUDIT_RECORD
NAME="Query"
RECORD="4_2021-06-29T19:33:03"
TIMESTAMP="2021-06-29T19:33:34Z"
COMMAND_CLASS="show_variables"
CONNECTION_ID="14"
STATUS="0"
SQLTEXT="show variables like 'audit%'"
USER="root[root] @ localhost []"
HOST="localhost"
OS_USER=""
IP=""
DB=""
/>

Stream the audit log to syslog

To stream the audit log to syslog you’ll need to set audit_log_handler variable to SYSLOG. To control the syslog file handler, the following variables can be used: audit_log_syslog_ident, audit_log_syslog_facility, and audit_log_syslog_priority These variables have the same meaning as appropriate parameters described in the syslog(3) manual.

Note

The actions for the variables: audit_log_strategy, audit_log_buffer_size, audit_log_rotate_on_size, audit_log_rotations are captured only with FILE handler.

Filter methods

You can filter the results by the following methods.

The filtering by user feature adds two new global variables: audit_log_include_accounts and audit_log_exclude_accounts to specify which user accounts should be included or excluded from audit logging.

Only one of these variables can contain a list of users to be either included or excluded, while the other must be NULL. If one of the variables is set to be not NULL (contains a list of users), the attempt to set another one fails. An empty string means an empty list.

Changes of audit_log_include_accounts and audit_log_exclude_accounts do not apply to existing server connections.

The filtering by SQL command type adds two new global variables: audit_log_include_commands and audit_log_exclude_commands to specify which command types should be included or excluded from audit logging.

Only one of these variables can contain a list of command types to be either included or excluded, while the other needs to be NULL. If one of the variables is set to be not NULL (contains a list of command types), the attempt to set another one will fail. An empty string is defined as an empty list.

If both the audit_log_exclude_commands variable and the audit_log_include_commands variable are NULL, all commands are logged.

The filtering by an SQL database is implemented by two global variables: audit_log_include_databases and audit_log_exclude_databases to specify which databases should be included or excluded from audit logging.

Only one of these variables can contain a list of databases to be either included or excluded, while the other needs to be NULL. If one of the variables is set to be not NULL (contains a list of databases), the attempt to set another one will fail. Empty string means an empty list.

If query is accessing any of databases listed in audit_log_include_databases, the query will be logged. If query is accessing only databases listed in audit_log_exclude_databases, the query will not be logged. CREATE TABLE statements are logged unconditionally.

Changes of audit_log_include_databases and audit_log_exclude_databases do not apply to existing server connections.

Filter examples

The following are examples of the different filters.

The following example adds users who will be monitored:

SET GLOBAL audit_log_include_accounts = 'user1@localhost,root@localhost';
Expected output 
Query OK, 0 rows affected (0.00 sec)

If you try to add users to both the include list and the exclude list, the server returns the following error:

SET GLOBAL audit_log_exclude_accounts = 'user1@localhost,root@localhost';
Expected output 
ERROR 1231 (42000): Variable 'audit_log_exclude_accounts' can't be set to the value of 'user1@localhost,root@localhost'

To switch from filtering by included user list to the excluded user list or back, first set the active filtering variable to NULL:

SET GLOBAL audit_log_include_accounts = NULL;
Expected output 
Query OK, 0 rows affected (0.00 sec)

SET GLOBAL audit_log_exclude_accounts = 'user1@localhost,root@localhost';
Expected output 
Query OK, 0 rows affected (0.00 sec)

SET GLOBAL audit_log_exclude_accounts = "'user'@'host'";
Expected output 
Query OK, 0 rows affected (0.00 sec)

SET GLOBAL audit_log_exclude_accounts = '''user''@''host''';
Expected output 
Query OK, 0 rows affected (0.00 sec)

SET GLOBAL audit_log_exclude_accounts = '\'user\'@\'host\'';
Expected output 
Query OK, 0 rows affected (0.00 sec)

To see which user accounts have been added to the exclude list, run the following command:

SELECT @@audit_log_exclude_accounts;
Expected output 
+------------------------------+
| @@audit_log_exclude_accounts |
+------------------------------+
| 'user'@'host'                |
+------------------------------+
1 row in set (0.00 sec)

Account names from mysql.user table are logged in the audit log. For example when you create a user:

CREATE USER 'user1'@'%' IDENTIFIED BY '111';
Expected output 
Query OK, 0 rows affected (0.00 sec)

When user1 connects from localhost, the user is listed:

<AUDIT_RECORD
NAME="Connect"
RECORD="2_2021-06-30T11:56:53"
TIMESTAMP="2021-06-30T11:56:53 UTC"
CONNECTION_ID="6"
STATUS="0"
USER="user1" ;; this is a 'user' part of account
PRIV_USER="user1"
OS_LOGIN=""
PROXY_USER=""
HOST="localhost" ;; this is a 'host' part of account
IP=""
DB=""
/>

To exclude user1 from logging in Percona Server for MySQL 8.4, set:

SET GLOBAL audit_log_exclude_accounts = 'user1@%';

The value can be NULL or comma separated list of accounts in form user@host or 'user'@'host' (if user or host contains comma).

The available command types can be listed by running:

SELECT name FROM performance_schema.setup_instruments WHERE name LIKE "statement/sql/%" ORDER BY name;
Expected output 
+------------------------------------------+
| name                                     |
+------------------------------------------+
| statement/sql/alter_db                   |
| statement/sql/alter_db_upgrade           |
| statement/sql/alter_event                |
| statement/sql/alter_function             |
| statement/sql/alter_procedure            |
| statement/sql/alter_server               |
| statement/sql/alter_table                |
| statement/sql/alter_tablespace           |
| statement/sql/alter_user                 |
| statement/sql/analyze                    |
| statement/sql/assign_to_keycache         |
| statement/sql/begin                      |
| statement/sql/binlog                     |
| statement/sql/call_procedure             |
| statement/sql/change_db                  |
| statement/sql/change_master              |
...
| statement/sql/xa_rollback                |
| statement/sql/xa_start                   |
+------------------------------------------+
145 rows in set (0.00 sec)

You can add commands to the include filter by running:

SET GLOBAL audit_log_include_commands= 'set_option,create_db';

Create a database with the following command:

CREATE DATABASE sample;
Expected output 
<AUDIT_RECORD>
<NAME>Query</NAME>
<RECORD>24320_2021-06-30T17:44:46</RECORD>
<TIMESTAMP>2021-06-30T17:45:16 UTC</TIMESTAMP>
<COMMAND_CLASS>create_db</COMMAND_CLASS>
<CONNECTION_ID>2</CONNECTION_ID>
<STATUS>0</STATUS>
<SQLTEXT>CREATE DATABASE sample</SQLTEXT>
<USER>root[root] @ localhost []</USER>
<HOST>localhost</HOST>
<OS_USER></OS_USER>
<IP></IP>
<DB></DB>
</AUDIT_RECORD>

To switch the command type filtering type from included type list to the excluded list or back, first reset the active list to NULL:

SET GLOBAL audit_log_include_commands = NULL;
Expected output 
Query OK, 0 rows affected (0.00 sec)

SET GLOBAL audit_log_exclude_commands= 'set_option,create_db';
Expected output 
Query OK, 0 rows affected (0.00 sec)

A stored procedure has the call_procedure command type. All the statements executed within the procedure have the same type call_procedure as well.

To add databases to be monitored, run:

SET GLOBAL audit_log_include_databases = 'test,mysql,db1';
Expected output 
Query OK, 0 rows affected (0.00 sec)

SET GLOBAL audit_log_include_databases= 'db1','db3';
Expected output 
Query OK, 0 rows affected (0.00 sec)

If you you try to add databases to both include and exclude lists server will show you the following error:

SET GLOBAL audit_log_exclude_databases = 'test,mysql,db1';
Error message 
ERROR 1231 (42000): Variable 'audit_log_exclude_databases can't be set to the value of 'test,mysql,db1'

To switch from filtering by included database list to the excluded one or back, first set the active filtering variable to NULL:

SET GLOBAL audit_log_include_databases = NULL;
Expected output 
Query OK, 0 rows affected (0.00 sec)

SET GLOBAL audit_log_exclude_databases = 'test,mysql,db1';
Expected output 
Query OK, 0 rows affected (0.00 sec)

System variables

audit_log_strategy

Option Description
Command Line: Yes
Scope: Global
Dynamic: No
Data type String
Default value ASYNCHRONOUS
Allowed values ASYNCHRONOUS, PERFORMANCE, SEMISYNCHRONOUS, SYNCHRONOUS

This variable is used to specify the audit log strategy, possible values are:

  • ASYNCHRONOUS - (default) log using memory buffer, do not drop messages if buffer is full

  • PERFORMANCE - log using memory buffer, drop messages if buffer is full

  • SEMISYNCHRONOUS - log directly to file, do not flush and sync every event

  • SYNCHRONOUS - log directly to file, flush and sync every event

This variable has effect only when audit_log_handler is set to FILE.

audit_log_file

Option Description
Command Line: Yes
Scope: Global
Dynamic: No
Data type String
Default value audit.log

This variable is used to specify the filename that’s going to store the audit log. It can contain the path relative to the datadir or absolute path.

audit_log_flush

Option Description
Command Line: Yes
Scope: Global
Dynamic: Yes
Data type String
Default value OFF

When this variable is set to ON log file will be closed and reopened.

audit_log_buffer_size

Option Description
Command Line: Yes
Scope: Global
Dynamic: No
Data type Numeric
Default value 1 Mb

This variable can be used to specify the size of memory buffer used for logging, used when audit_log_strategy variable is set to ASYNCHRONOUS or PERFORMANCE values. This variable has effect only when audit_log_handler is set to FILE.

audit_log_exclude_accounts

Option Description
Command Line: Yes
Scope: Global
Dynamic: Yes
Data type String

This variable is used to specify the list of users for which Filtering by user is applied. The value can be NULL or comma separated list of accounts in form user@host or 'user'@'host' (if user or host contains comma). If this variable is set, then audit_log_include_accounts must be unset, and the reverse also applies.

audit_log_exclude_commands

Option Description
Command Line: Yes
Scope: Global
Dynamic: Yes
Data type String

This variable is used to specify the list of commands for which Filtering by SQL command type is applied. The value can be NULL or comma separated list of commands. If this variable is set, then audit_log_include_commands must be unset, and the reverse also applies.

audit_log_exclude_databases

Option Description
Command Line: Yes
Scope: Global
Dynamic: Yes
Data type String

Use this variable to specify the databases to be filtered. The value can be NULL or a comma-separated list of databases. If you set this variable, you must unset audit_log_include_databases. The reverse also applies.

audit_log_format

Option Description
Command Line: Yes
Scope: Global
Dynamic: No
Data type String
Default value OLD
Allowed values OLD, NEW, CSV, JSON

This variable is used to specify the audit log format. The audit log plugin supports four log formats: OLD, NEW, JSON, and CSV. OLD and NEW formats are based on XML, where the former outputs log record properties as XML attributes and the latter as XML tags. Information logged is the same in all four formats.

audit_log_include_accounts

Option Description
Command Line: Yes
Scope: Global
Dynamic: Yes
Data type String

This variable is used to specify the list of users for which Filtering by user is applied. The value can be NULL or comma separated list of accounts in form user@host or 'user'@'host' (if user or host contains comma). If this variable is set, then audit_log_exclude_accounts must be unset, and the reverse also applies.

audit_log_include_commands

Option Description
Command Line: Yes
Scope: Global
Dynamic: Yes
Data type String

This variable is used to specify the list of commands for which Filtering by SQL command type is applied. The value can be NULL or comma separated list of commands. If this variable is set, then audit_log_exclude_commands must be unset, and the reverse also applies.

audit_log_include_databases

Option Description
Command Line: Yes
Scope: Global
Dynamic: Yes
Data type String

This variable defines the list of databases to be filtered. You can set the value to NULL or a comma-separated list of databases. If you set this variable, you must unset audit_log_exclude_databases; the opposite is true.

audit_log_policy

Option Description
Command Line: Yes
Scope: Global
Dynamic: Yes
Data type String
Default ALL
Allowed values ALL, LOGINS, QUERIES, NONE

This variable is used to specify which events should be logged. Possible values are:

  • ALL - all events will be logged

  • LOGINS - only logins will be logged

  • QUERIES - only queries will be logged

  • NONE - no events will be logged

audit_log_rotate_on_size

Option Description
Command Line: Yes
Scope: Global
Dynamic: Yes
Data type Numeric
Default value 0

This variable is measured in bytes and specifies the maximum size of the audit log file. Upon reaching this size, the audit log will be rotated. The rotated log files are present in the same directory as the current log file. The sequence number is appended to the log file name upon rotation.

If the value is set to 0 (the default), the audit log files won’t rotate.

Set the audit_log_handler to FILE to enable this variable.

audit_log_rotations

Option Description
Command Line: Yes
Scope: Global
Dynamic: Yes
Data type Numeric
Default value 0

This variable is used to specify how many log files should be kept when audit_log_rotate_on_size variable is set to non-zero value. This variable has effect only when audit_log_handler is set to FILE.

audit_log_handler

Option Description
Command Line: Yes
Scope: Global
Dynamic: No
Data type String
Default value FILE
Allowed values FILE, SYSLOG

This variable is used to configure where the audit log will be written. If it is set to FILE, the log will be written into a file specified by audit_log_file variable. If it is set to SYSLOG, the audit log will be written to syslog.

audit_log_syslog_ident

Option Description
Command Line: Yes
Scope: Global
Dynamic: No
Data type String
Default value percona-audit

This variable is used to specify the ident value for syslog. This variable has the same meaning as the appropriate parameter described in the syslog(3) manual.

audit_log_syslog_facility

Option Description
Command Line: Yes
Scope: Global
Dynamic: No
Data type String
Default value LOG_USER

This variable is used to specify the facility value for syslog. This variable has the same meaning as the appropriate parameter described in the syslog(3) manual.

audit_log_syslog_priority

Option Description
Command Line: Yes
Scope: Global
Dynamic: No
Data type String
Default value LOG_INFO
Allowed values LOG_EMERG, LOG_ALERT, LOG_CRIT, LOG_ERR, LOG_WARNING, LOG_NOTICE, LOG_INFO, LOG_DEBUG

This variable is used to specify the severity level for syslog. The audit_log_syslog_priority variable does not include the facility; it only selects the severity level (LOG_EMERG … LOG_DEBUG).

The full syslog priority that syslog() receives is built internally by OR-ing the configured facility (audit_log_syslog_facility) with this level.

The default LOG_INFO means “ordinary informational messages”; you can raise or lower the level as needed, while the facility stays at its default unless you change it explicitly.

For more details about syslog priority levels, see the syslog(3) manual.

Status Variables

Audit_log_buffer_size_overflow

Option Description
Scope: Global
Data type Numeric

The number of times an audit log entry was either dropped or written directly to the file due to its size being bigger than audit_log_buffer_size variable.

Additional reading

Jemalloc memory allocation profiling

Percona Server for MySQL can take advantage of the memory-profiling ability of the jemalloc allocator. This ability provides a method to investigate memory-related issues.

Requirements

This memory-profiling requires jemalloc_detected. This read-only variable returns true if jemalloc with the profiling-enabled option is being used by Percona Server for MySQL.

As root, customize jemalloc with the following flags:

Option Description
–enable-stats Enables statistics-gathering ability
–enable-prof Enables heap profiling and the ability to detect leaks.

Using LD_PRELOAD. Build the library, configure the malloc configuration with the prof:true string, and then use LD_PRELOAD to preload the libjemalloc.so library. Percona Server for MySQL detects jemalloc with profiling enabled automatically when properly configured.

The following is an example of the required commands:

./configure --enable-stats --enable-prof && make && make install
export MALLOC_CONF=prof:true
export LD_PRELOAD=/usr/lib/libjemalloc.so

Use Percona Server for MySQL with jemalloc with profiling enabled

To detect if jemalloc is set, run the following command:

SELECT @@jemalloc_detected;

To enable jemalloc profiling in a MySQL client, run the following command:

set global jemalloc_profiling=on;

The malloc_stats_totals table provides statistics, in bytes, of the memory usage.

The following example commands display this result:

use performance_schema;

SELECT * FROM malloc_stats_totals;
Expected output 
+------------+------------+------------+------------+-------------+------------+
| ALLOCATED  | ACTIVE     | MAPPED     | RESIDENT   | RETAINED    | METADATA   |
+------------+------------+------------+------------+-------------+------------+
| 390977528  | 391012352  | 405291008  | 520167424  | 436813824   | 9933744    |
+------------+------------+------------+------------+-------------+------------+
1 row in set (0.00 sec)

The malloc_stats table provides cumulative totals, in bytes, of several statistics per allocation size type (small, large, and huge).

The following example commands display this result:

use performance_schema;

SELECT * FROM malloc_stats ORDER BY TYPE DESC LIMIT 3;
Expected output 
+--------+-------------+-------------+-------------+-------------+
| TYPE   | ALLOCATED   | NMALLOC     | NDALLOC     | NRESQUESTS  |
+--------+-------------+-------------+-------------+-------------+
| small  | 23578872    | 586156      | 0           | 2649417     |
| large  | 367382528   | 2218        | 0           | 6355        |
| huge   | 0           | 0           | 0           | 0 |
+--------+-------------+-------------+-------------+-------------+
3 rows in set (0.00 sec)

Dumping the profile

The profiling samples the malloc() calls and stores the sampled stack traces in a separate location in memory. These samples can be dumped into the filesystem. A dump returns a detailed view of the state of the memory.

The process is global; therefore, only a single concurrent run is available. Each dump creates a new file with a unique timestamp, and previous dumps are retained unless manually deleted.

Use the following command to create a profile dump file:

flush memory profile;

The generated memory profile dumps are written to the /tmp directory.

You can analyze the dump files with jeprof program, which must be installed on the host system and available in the system PATH. This program is a C program that post-processes the dump files in their raw format. While jeprof can work with dumps from different jemalloc versions, compatibility issues may occur with significantly different versions.

To verify the dump, run the following command:

ls /tmp/jeprof_mysqld*
Expected output 
/tmp/jeprof_mysqld.1.0.170013202213

Then analyze the profile:

jeprof --show_bytes /usr/sbin/mysqld /tmp/jeprof_mysqld.1.0.170013202213

You can also access the memory profile to plot a graph of the memory use. This ability requires that jeprof and dot are available in the system PATH. For the graph to display useful information, the binary file must contain debug symbol information.

Important considerations

  • Ensure the MySQL process has write permissions to /tmp

  • Profile dumps can be large; monitor available disk space

  • Profiling has performance overhead and should be used judiciously

Run the following command:

jeprof --dot /usr/sbin/mysqld /tmp/jeprof_mysqld.1.0.170013202213 > /tmp/jeprof1.dot
dot -Tpng /tmp/jeprof1.dot > /tmp/jeprof1.png

Note

An example of allocation graph .

PERFORMANCE_SCHEMA tables

The following tables are implemented to retrieve memory allocation statistics for a running instance or return the cumulative number of allocations requested or allocations returned for a running instance.

More information about the stats that are returned can be found in jemalloc .

malloc_stats_totals

The current stats for allocations. All measurements are in bytes.

Column Name Description
ALLOCATED The total amount the application allocated
ACTIVE The total amount allocated by the application of active pages. A multiple of the page size and this value is greater than or equal to the stats.allocated value. The sum does not include allocator metadata pages and stats.arenas.[i].pdirty or stats.arenas.[i].pmuzzy.
MAPPED The total amount in chunks that are mapped by the allocator in active extents. This value does not include inactive chunks. The value is at least as large as the stats.active and is a multiple of the chunk size.
RESIDENT A maximum number the allocator has mapped in physically resident data pages. All allocator metadata pages and unused dirty pages are included in this value. Pages may not be physically resident if they correspond to demand-zeroed virtual memory that has not yet been touched. This value is a maximum rather than a precise value and is a multiple of the page size. The value is greater than the stats.active.
RETAINED The amount retained by the virtual memory mappings of the operating system. This value does not include any returned mappings. This type of memory, usually de-committed, untouched, or purged. The value is associated with physical memory and is excluded from mapped memory statistics.
METADATA The total amount dedicated to metadata. This value contains the base allocations which are used for bootstrap-sensitive allocator metadata structures. Transparent huge pages usage is not included.

malloc_stats

The cumulative statistics for allocations and deallocations for a running instance.

Column Name Description
Type The type of object: small, large, and huge
ALLOCATED The number of bytes that are currently allocated to the application.
NMALLOC A cumulative number of times an allocation was requested from the arena’s bins. The number includes times when the allocation satisfied an allocation request or filled a relevant tcache if opt.tcache is enabled.
NDALLOC A cumulative number of times an allocation was returned to the arena’s bins. The number includes times when the allocation was deallocated or flushed the relevant tcache if opt.tcache is enabled.
NREQUESTS The cumulative number of allocation requests satisfied.

System variables

The following variables have been added:

jemalloc_detected

Description: This read-only variable returns true if jemalloc with profiling enabled is detected. The following options are required:

  • Jemalloc is installed and compiled with profiling enabled

  • Percona Server for MySQL is configured to use jemalloc by using the environment variable LD_PRELOAD.

  • The environment variable MALLOC_CONF is set to prof:true.

Properties:

  • Scope: Global

  • Variable Type: Boolean

  • Default Value: false

jemalloc_profiling

Description: Enables jemalloc profiling. The variable requires jemalloc_detected.

  • Command Line: –jemalloc_profiling[=(OFF|ON)]

  • Config File: Yes

  • Scope: Global

  • Dynamic: Yes

  • Variable Type: Boolean

  • Default Value: OFF

Disable profiling

To disable jemalloc profiling, in a MySQL client, run the following command:

set global jemalloc_profiling=off;

User statistics

This feature adds several INFORMATION_SCHEMA tables, several commands, and the userstat variable. The tables and commands can be used to understand the server activity better and identify the source of the load.

The functionality is disabled by default and must be enabled by setting userstat to ON. It works by keeping several hash tables in memory. To avoid contention over global mutexes, each connection has its own local statistics, which are occasionally merged into the global statistics, and the local statistics are then reset to 0.

System variables

userstat

Option Description
Command-line Yes
Config file Yes
Scope Global
Dynamic Yes
Data type BOOLEAN
Default OFF
Range ON/OFF

Enables or disables collection of statistics. The default is OFF, meaning no statistics are gathered. This is to ensure that the statistics collection doesn’t cause any extra load on the server unless desired.

thread_statistics

Option Description
Command-line Yes
Config file Yes
Scope Global
Dynamic Yes
Data type BOOLEAN
Default OFF
Range ON/OFF

Enables or disables collection of thread statistics. The default is OFF, meaning no thread statistics are gathered. This is to ensure that the statistics collection doesn’t cause any extra load on the server unless desired. The variable userstat must be enabled as well in order for thread statistics to be collected.

INFORMATION_SCHEMA Tables

INFORMATION_SCHEMA.CLIENT_STATISTICS

Column Name Description
‘CLIENT’ ‘The IP address or hostname from which the connection originated.’
‘TOTAL_CONNECTIONS’ ‘The number of connections created for this client.’
‘CONCURRENT_CONNECTIONS’ ‘The number of concurrent connections for this client.’
‘CONNECTED_TIME’ ‘The cumulative number of seconds elapsed while there were connections from this client.’
‘BUSY_TIME’ ‘The cumulative number of seconds there was activity on connections from this client.’
‘CPU_TIME’ ‘The cumulative CPU time elapsed, in seconds, while servicing this client’s connections.’
‘BYTES_RECEIVED’ ‘The number of bytes received from this client’s connections.’
‘BYTES_SENT’ ‘The number of bytes sent to this client’s connections.’
‘BINLOG_BYTES_WRITTEN’ ‘The number of bytes written to the binary log from this client’s connections.’
‘ROWS_FETCHED’ ‘The number of rows fetched by this client’s connections.’
‘ROWS_UPDATED’ ‘The number of rows updated by this client’s connections.’
‘TABLE_ROWS_READ’ ‘The number of rows read from tables by this client’s connections. (It may be different from ROWS_FETCHED.)’
‘SELECT_COMMANDS’ ‘The number of SELECT commands executed from this client’s connections.’
‘UPDATE_COMMANDS’ ‘The number of UPDATE commands executed from this client’s connections.’
‘OTHER_COMMANDS’ ‘The number of other commands executed from this client’s connections.’
‘COMMIT_TRANSACTIONS’ ‘The number of COMMIT commands issued by this client’s connections.’
‘ROLLBACK_TRANSACTIONS’ ‘The number of ROLLBACK commands issued by this client’s connections.’
‘DENIED_CONNECTIONS’ ‘The number of connections denied to this client.’
‘LOST_CONNECTIONS’ ‘The number of this client’s connections that were terminated uncleanly.’
‘ACCESS_DENIED’ ‘The number of times this client’s connections issued commands that were denied.’
‘EMPTY_QUERIES’ ‘The number of times this client’s connections sent empty queries to the server.’

This table holds statistics about client connections. The Percona version of the feature restricts this table’s visibility to users who have the SUPER or PROCESS privilege.

For example:

SELECT * FROM INFORMATION_SCHEMA.CLIENT_STATISTICS\G
Expected output 
*************************** 1. row ***************************
                CLIENT: 10.1.12.30
     TOTAL_CONNECTIONS: 20
CONCURRENT_CONNECTIONS: 0
        CONNECTED_TIME: 0
             BUSY_TIME: 93
              CPU_TIME: 48
        BYTES_RECEIVED: 5031
            BYTES_SENT: 276926
   BINLOG_BYTES_WRITTEN: 217
          ROWS_FETCHED: 81
          ROWS_UPDATED: 0
       TABLE_ROWS_READ: 52836023
       SELECT_COMMANDS: 26
       UPDATE_COMMANDS: 1
        OTHER_COMMANDS: 145
   COMMIT_TRANSACTIONS: 1
 ROLLBACK_TRANSACTIONS: 0
    DENIED_CONNECTIONS: 0
      LOST_CONNECTIONS: 0
         ACCESS_DENIED: 0
         EMPTY_QUERIES: 0

INFORMATION_SCHEMA tables

INFORMATION_SCHEMA.INDEX_STATISTICS

Column Name Description
‘TABLE_SCHEMA’ ‘The schema (database) name.’
‘TABLE_NAME’ ‘The table name.’
‘INDEX_NAME’ ‘The index name (as visible in SHOW CREATE TABLE).’
‘ROWS_READ’ ‘The number of rows read from this index.’

This table shows statistics on index usage. An older version of the feature contained a single column that had the TABLE_SCHEMA, TABLE_NAME, and INDEX_NAME columns concatenated together. The Percona version of the feature separates these into three columns. Users can see entries only for tables to which they have SELECT access.

This table makes it possible to do many things that were difficult or impossible previously. For example, you can use it to find unused indexes and generate DROP commands to remove them.

Example:

SELECT * FROM INFORMATION_SCHEMA.INDEX_STATISTICS WHERE TABLE_NAME='tables_priv';
Expected output 
+--------------+-----------------------+--------------------+-----------+
| TABLE_SCHEMA | TABLE_NAME            | INDEX_NAME         | ROWS_READ |
+--------------+-----------------------+--------------------+-----------+
| mysql        | tables_priv           | PRIMARY            |         2 |
+--------------+-----------------------+--------------------+-----------+

Note

The current implementation of index statistics doesn’t support partitioned tables.

INFORMATION_SCHEMA.TABLE_STATISTICS

Column Name Description
‘TABLE_SCHEMA’ ‘The schema (database) name.’
‘TABLE_NAME’ ‘The table name.’
‘ROWS_READ’ ‘The number of rows read from the table.’
‘ROWS_CHANGED’ ‘The number of rows changed in the table.’
‘ROWS_CHANGED_X_INDEXES’ ‘The number of rows changed in the table, multiplied by the number of indexes changed.’

This table is similar in function to the INDEX_STATISTICS table.

For example:

SELECT * FROM INFORMATION_SCHEMA.TABLE_STATISTICS WHERE TABLE_NAME=``tables_priv``;
Expected output 
+--------------+-------------------------------+-----------+--------------+------------------------+
| TABLE_SCHEMA | TABLE_NAME                    | ROWS_READ | ROWS_CHANGED | ROWS_CHANGED_X_INDEXES |
+--------------+-------------------------------+-----------+--------------+------------------------+
| mysql        | tables_priv                   |         2 |            0 |                      0 |
+--------------+-------------------------------+-----------+--------------+------------------------+

Note

The current implementation of table statistics doesn’t support partitioned tables.

INFORMATION_SCHEMA.THREAD_STATISTICS

Column Name Description
‘THREAD_ID’ ‘Thread ID’
‘TOTAL_CONNECTIONS’ ‘The number of connections created from this thread.’
‘CONNECTED_TIME’ ‘The cumulative number of seconds elapsed while there were connections from this thread.’
‘BUSY_TIME’ ‘The cumulative number of seconds there was activity from this thread.’
‘CPU_TIME’ ‘The cumulative CPU time elapsed while servicing this thread.’
‘BYTES_RECEIVED’ ‘The number of bytes received from this thread.’
‘BYTES_SENT’ ‘The number of bytes sent to this thread.’
‘BINLOG_BYTES_WRITTEN’ ‘The number of bytes written to the binary log from this thread.’
‘ROWS_FETCHED’ ‘The number of rows fetched by this thread.’
‘ROWS_UPDATED’ ‘The number of rows updated by this thread.’
‘TABLE_ROWS_READ’ ‘The number of rows read from tables by this tread.’
‘SELECT_COMMANDS’ ‘The number of SELECT commands executed from this thread.’
‘UPDATE_COMMANDS’ ‘The number of UPDATE commands executed from this thread.’
‘OTHER_COMMANDS’ ‘The number of other commands executed from this thread.’
‘COMMIT_TRANSACTIONS’ ‘The number of COMMIT commands issued by this thread.’
‘ROLLBACK_TRANSACTIONS’ ‘The number of ROLLBACK commands issued by this thread.’
‘DENIED_CONNECTIONS’ ‘The number of connections denied to this thread.’
‘LOST_CONNECTIONS’ ‘The number of thread connections that were terminated uncleanly.’
‘ACCESS_DENIED’ ‘The number of times this thread issued commands that were denied.’
‘EMPTY_QUERIES’ ‘The number of times this thread sent empty queries to the server.’
‘TOTAL_SSL_CONNECTIONS’ ‘The number of thread connections that used SSL.’

In order for this table to be populated with statistics, the additional variable thread_statistics should be set to ON.

INFORMATION_SCHEMA.USER_STATISTICS

Column Name Description
‘USER’ ‘The username. The value #mysql_system_user# appears when there is no username (such as for the replica SQL thread).’
‘TOTAL_CONNECTIONS’ ‘The number of connections created from this user.’
‘CONCURRENT_CONNECTIONS’ ‘The number of concurrent connections for this user.’
‘CONNECTED_TIME’ ‘The cumulative number of seconds elapsed while there were connections from this user.’
‘BUSY_TIME’ ‘The cumulative number of seconds there was activity on connections from this user.’
‘CPU_TIME’ ‘The cumulative CPU time elapsed, in seconds, while servicing this user’s connections.’
‘BYTES_RECEIVED’ ‘The number of bytes received from this user’s connections.’
‘BYTES_SENT’ ‘The number of bytes sent to this user’s connections.’
‘BINLOG_BYTES_WRITTEN’ ‘The number of bytes written to the binary log from this user’s connections.’
‘ROWS_FETCHED’ ‘The number of rows fetched by this user’s connections.’
‘ROWS_UPDATED’ ‘The number of rows updated by this user’s connections.’
‘TABLE_ROWS_READ’ ‘The number of rows read from tables by this user’s connections. (It may be different from ROWS_FETCHED.)’
‘SELECT_COMMANDS’ ‘The number of SELECT commands executed from this user’s connections.’
‘UPDATE_COMMANDS’ ‘The number of UPDATE commands executed from this user’s connections.’
‘OTHER_COMMANDS’ ‘The number of other commands executed from this user’s connections.’
‘COMMIT_TRANSACTIONS’ ‘The number of COMMIT commands issued by this user’s connections.’
‘ROLLBACK_TRANSACTIONS’ ‘The number of ROLLBACK commands issued by this user’s connections.’
‘DENIED_CONNECTIONS’ ‘The number of connections denied to this user.’
‘LOST_CONNECTIONS’ ‘The number of this user’s connections that were terminated uncleanly.’
‘ACCESS_DENIED’ ‘The number of times this user’s connections issued commands that were denied.’
‘EMPTY_QUERIES’ ‘The number of times this user’s connections sent empty queries to the server.’

This table contains information about user activity. The Percona version of the patch restricts this table’s visibility to users who have the SUPER or PROCESS privilege.

The table gives answers to questions such as which users cause the most load, and whether any users are being abusive. It also lets you measure how close to capacity the server may be. For example, you can use it to find out whether replication is likely to start falling behind.

Example:

SELECT * FROM INFORMATION_SCHEMA.USER_STATISTICS\G
Expected output 
*************************** 1. row ***************************
                  USER: root
     TOTAL_CONNECTIONS: 5592
 CONCURRENT_CONNECTIONS: 0
         CONNECTED_TIME: 6844
             BUSY_TIME: 179
              CPU_TIME: 72
        BYTES_RECEIVED: 603344
            BYTES_SENT: 15663832
  BINLOG_BYTES_WRITTEN: 217
          ROWS_FETCHED: 9793
          ROWS_UPDATED: 0
       TABLE_ROWS_READ: 52836023
       SELECT_COMMANDS: 9701
       UPDATE_COMMANDS: 1
        OTHER_COMMANDS: 2614
   COMMIT_TRANSACTIONS: 1
 ROLLBACK_TRANSACTIONS: 0
    DENIED_CONNECTIONS: 0
      LOST_CONNECTIONS: 0
         ACCESS_DENIED: 0
         EMPTY_QUERIES: 0

Commands Provided

  • FLUSH CLIENT_STATISTICS

  • FLUSH INDEX_STATISTICS

  • FLUSH TABLE_STATISTICS

  • FLUSH THREAD_STATISTICS

  • FLUSH USER_STATISTICS

These commands discard the specified type of stored statistical information.

  • SHOW CLIENT_STATISTICS

  • SHOW INDEX_STATISTICS

  • SHOW TABLE_STATISTICS

  • SHOW THREAD_STATISTICS

  • SHOW USER_STATISTICS

These commands are another way to display the information you can get from the INFORMATION_SCHEMA tables. The commands accept WHERE clauses. They also accept but ignore LIKE clauses.

Status Variables

Com_show_client_statistics

Option Description
Scope Global/Session
Data type numeric

The Com_show_client_statistics statement counter variable indicates the number of times the statement SHOW CLIENT_STATISTICS has been executed.

Com_show_index_statistics

Option Description
Scope Global/Session
Data type numeric

The Com_show_index_statistics statement counter variable indicates the number of times the statement SHOW INDEX_STATISTICS has been executed.

Com_show_table_statistics

Option Description
Scope Global/Session
Data type numeric

The Com_show_table_statistics statement counter variable indicates the number of times the statement SHOW TABLE_STATISTICS has been executed.

Com_show_thread_statistics

Option Description
Scope Global/Session
Data type numeric

The Com_show_thread_statistics statement counter variable indicates the number of times the statement SHOW THREAD_STATISTICS has been executed.

Com_show_user_statistics

Option Description
Scope Global/Session
Data type numeric

The Com_show_user_statistics statement counter variable indicates the number of times the statement SHOW USER_STATISTICS has been executed.

Slow query log

This feature adds microsecond time resolution and additional statistics to the slow query log output. It lets you enable or disable the slow query log at runtime, adds logging for the replica SQL thread, and adds fine-grained control over what and how much to log into the slow query log.

You can use Percona-Toolkit’s pt-query-digest tool to aggregate similar queries together and report on those that consume the most execution time.

System Variables

log_slow_filter

Option Description
Command-line Yes
Config file Yes
Scope Global, Session
Dynamic Yes

Filters the slow log by the query’s execution plan. The value is a comma-delimited string, and can contain any combination of the following values:

  • full_scan: The query performed a full table scan.

  • full_join: The query performed a full join (a join without indexes).

  • tmp_table: The query created an implicit internal temporary table.

  • tmp_table_on_disk: The query’s temporary table was stored on disk.

  • filesort: The query used a filesort.

  • filesort_on_disk: The filesort was performed on disk.

Values are OR’ed together. If the string is empty, then the filter is disabled. If it is not empty, then queries will only be logged to the slow log if their execution plan matches one of the types of plans present in the filter.

For example, to log only queries that perform a full table scan, set the value to full_scan. To log only queries that use on-disk temporary storage for intermediate results, set the value to tmp_table_on_disk,filesort_on_disk.

log_slow_rate_type

Option Description
Command-line Yes
Config file Yes
Scope Global
Dynamic yes
Data type Enumerated
Default session, query

Specifies semantic of log_slow_rate_limit - session or query.

log_slow_rate_limit

Option Description
Command-line Yes
Config file Yes
Scope Global, session
Dynamic yes
Default 1
Range 1-1000

Behavior of this variable depends on the selected log_slow_rate_type.

Specifies that only a fraction of session/query should be logged. Logging is enabled for every nth session/query. By default, n is 1, so logging is enabled for every session/query. Please note: when log_slow_rate_type is session rate limiting is disabled for the replication thread.

Logging all queries might consume I/O bandwidth and cause the log file to grow large.

  • When log_slow_rate_type is session, this option lets you log full sessions, so you have complete records of sessions for later analysis; but you can rate-limit the number of sessions that are logged. Note that this feature will not work well if your application uses any type of connection pooling or persistent connections. Note that you change log_slow_rate_limit in session mode, you should reconnect for get effect.

  • When log_slow_rate_type is query, this option lets you log just some queries for later analysis. For example, if you set the value to 100, then one percent of queries will be logged.

Note that every query has global unique query_id and every connection can has it own (session) log_slow_rate_limit. Decision “log or no” calculated in following manner:

  • if log_slow_rate_limit is 1 - log every query

  • If log_slow_rate_limit 1 - randomly log every 1/log_slow_rate_limit query.

This allows flexible setup logging behavior.

For example, if you set the value to 100, then one percent of sessions/queries will be logged. In Percona Server for MySQL information about the log_slow_rate_limit has been added to the slow query log. This means that if the log_slow_rate_limit is effective it will be reflected in the slow query log for each written query.

Expected output 
Log_slow_rate_type: query  Log_slow_rate_limit: 10

log_slow_sp_statements

Option Description
Command-line Yes
Config file Yes
Scope Global
Dynamic Yes
Data type Enumerated
Default session
Range session, query

If TRUE, statements executed by stored procedures are logged to the slow if it is open.

Percona Server for MySQL implemented improvements for logging of stored procedures to the slow query log:

  • Each query from a stored procedure is now logged to the slow query log individually

  • CALL itself isn’t logged to the slow query log anymore as this would be counting twice for the same query which would lead to incorrect results

  • Queries that were called inside of stored procedures are annotated in the slow query log with the stored procedure name in which they run.

Example of the improved stored procedure slow query log entry:

mysqlDELIMITER //
mysqlCREATE PROCEDURE improved_sp_log()
       BEGIN
        SELECT * FROM City;
        SELECT * FROM Country;
       END//
mysqlDELIMITER ;
mysqlCALL improved_sp_log();

When we check the slow query log after running the stored procedure, with log_slow_sp_statements set to TRUE, it should look like this:

Expected output 
# Time: 150109 11:38:55
# User@Host: root[root] @ localhost []
# Thread_id: 40  Schema: world  Last_errno: 0  Killed: 0
# Query_time: 0.012989  Lock_time: 0.000033  Rows_sent: 4079  Rows_examined: 4079  Rows_affected: 0  Rows_read: 4079
# Bytes_sent: 161085
# Stored routine: world.improved_sp_log
SET timestamp=1420803535;
SELECT * FROM City;
# User@Host: root[root] @ localhost []
# Thread_id: 40  Schema: world  Last_errno: 0  Killed: 0
# Query_time: 0.001413  Lock_time: 0.000017  Rows_sent: 4318  Rows_examined: 4318  Rows_affected: 0  Rows_read: 4318
# Bytes_sent: 194601
# Stored routine: world.improved_sp_log
SET timestamp=1420803535;

If variable log_slow_sp_statements is set to FALSE:

  • Entry is added to a slow-log for a CALL statement only and not for any of the individual statements run in that stored procedure

  • Execution time is reported for the CALL statement as the total execution time of the CALL including all its statements

If we run the same stored procedure with the log_slow_sp_statements is set to FALSE slow query log should look like this:

Expected output 
# Time: 150109 11:51:42
# User@Host: root[root] @ localhost []
# Thread_id: 40  Schema: world  Last_errno: 0  Killed: 0
# Query_time: 0.013947  Lock_time: 0.000000  Rows_sent: 4318  Rows_examined: 4318  Rows_affected: 0  Rows_read: 4318
# Bytes_sent: 194612
SET timestamp=1420804302;
CALL improved_sp_log();

Note

Support for logging stored procedures doesn’t involve triggers, so they won’t be logged even if this feature is enabled.

log_slow_verbosity

Option Description
Command-line Yes
Config file Yes
Scope Global, session
Dynamic Yes

Specifies how much information to include in your slow log. The value is a comma-delimited string, and can contain any combination of the following values:

  • microtime: Log queries with microsecond precision.

  • query_plan: Log information about the query’s execution plan.

  • innodb: Log InnoDB statistics.

  • minimal: Equivalent to enabling just microtime.

  • standard: Equivalent to enabling microtime,query_plan.

  • full: Equivalent to all other values OR’ed together without the profiling and profiling_use_getrusage options.

  • profiling: Enables profiling of all queries in all connections.

  • profiling_use_getrusage: Enables usage of the getrusage function.

  • query_info: Enables printing Query_tables and Query_digest into the slow query log. These fields are disabled by default.

Values are OR’ed together.

For example, to enable microsecond query timing and InnoDB statistics, set this option to microtime,innodb or standard. To turn all options on, set the option to full.

slow_query_log_use_global_control

Option Description
Command-line Yes
Config file Yes
Scope Global
Dynamic Yes
Default None

Specifies which variables have global scope instead of local. For such variables, the global variable value is used in the current session, but without copying this value to the session value. Value is a “flag” variable - you can specify multiple values separated by commas

  • none: All variables use local scope

  • log_slow_filter: Global variable log_slow_filter has effect (instead of local)

  • log_slow_rate_limit: Global variable log_slow_rate_limit has effect (instead of local)

  • log_slow_verbosity: Global variable log_slow_verbosity has effect (instead of local)

  • long_query_time: Global variable long_query_time has effect (instead of local)

  • min_examined_row_limit: Global variable min_examined_row_limit has effect (instead of local)

  • all Global variables has effect (instead of local)

slow_query_log_always_write_time

Option Description
Command-line Yes
Config file Yes
Scope Global
Dynamic Yes
Default 10

This variable can be used to specify the query execution time after which the query will be written to the slow query log. It can be used to specify an additional execution time threshold for the slow query log, that, when exceeded, will cause a query to be logged unconditionally, that is, log_slow_rate_limit will not apply to it.

Other information

Changes to the log format

The feature adds more information to the slow log output.

Expected output 
# Time: 130601  8:01:06.058915
# User@Host: root[root] @ localhost []  Id:    42
# Schema: imdb  Last_errno: 0  Killed: 0
# Query_time: 7.725616  Lock_time: 0.000328  Rows_sent: 4  Rows_examined: 1543720  Rows_affected: 0
# Bytes_sent: 272  Tmp_tables: 0  Tmp_disk_tables: 0  Tmp_table_sizes: 0
# Full_scan: Yes  Full_join: No  Tmp_table: No  Tmp_table_on_disk: No
# Filesort: No  Filesort_on_disk: No  Merge_passes: 0
SET timestamp=1370073666;
SELECT id,title,production_year FROM title WHERE title = 'Bambi';

Another example (log_slow_verbosity =profiling):

Expected output 
# Time: 130601  8:03:20.700441
# User@Host: root[root] @ localhost []  Id:    43
# Schema: imdb  Last_errno: 0  Killed: 0
# Query_time: 7.815071  Lock_time: 0.000261  Rows_sent: 4  Rows_examined: 1543720  Rows_affected: 0
# Bytes_sent: 272
# Profile_starting: 0.000125 Profile_starting_cpu: 0.000120
Profile_checking_permissions: 0.000021 Profile_checking_permissions_cpu: 0.000021
Profile_Opening_tables: 0.000049 Profile_Opening_tables_cpu: 0.000048 Profile_init: 0.000048
Profile_init_cpu: 0.000049 Profile_System_lock: 0.000049 Profile_System_lock_cpu: 0.000048
Profile_optimizing: 0.000024 Profile_optimizing_cpu: 0.000024 Profile_statistics: 0.000036 
Profile_statistics_cpu: 0.000037 Profile_preparing: 0.000029 Profile_preparing_cpu: 0.000029
Profile_executing: 0.000012 Profile_executing_cpu: 0.000012 Profile_Sending_data: 7.814583
Profile_Sending_data_cpu: 7.811634 Profile_end: 0.000013 Profile_end_cpu: 0.000012
Profile_query_end: 0.000014 Profile_query_end_cpu: 0.000014 Profile_closing_tables: 0.000023
Profile_closing_tables_cpu: 0.000023 Profile_freeing_items: 0.000051
Profile_freeing_items_cpu: 0.000050 Profile_logging_slow_query: 0.000006
Profile_logging_slow_query_cpu: 0.000006
# Profile_total: 7.815085 Profile_total_cpu: 7.812127
SET timestamp=1370073800;
SELECT id,title,production_year FROM title WHERE title = 'Bambi';

Notice that the Killed: \` keyword is followed by zero when the query successfully completes. If the query was killed, the ``Killed:` keyword is followed by a number other than zero:

Killed Numeric Code Exception
0 NOT_KILLED
1 KILL_BAD_DATA
1053 ER_SERVER_SHUTDOWN (see MySQL Documentation)
1317 ER_QUERY_INTERRUPTED (see MySQL Documentation)
3024 ER_QUERY_TIMEOUT (see MySQL Documentation)
Any other number KILLED_NO_VALUE (Catches all other cases)

Connection and Schema Identifier

Each slow log entry now contains a connection identifier, so you can trace all the queries coming from a single connection. This is the same value that is shown in the Id column in SHOW FULL PROCESSLIST or returned from the CONNECTION_ID() function.

Each entry also contains a schema name, so you can trace all the queries whose default database was set to a particular schema.

Expected output 
# Id: 43  Schema: imdb

Microsecond time resolution and extra row information

This is the original functionality offered by the microslow feature. Query_time and Lock_time are logged with microsecond resolution.

The feature also adds information about how many rows were examined for SELECT queries, and how many were analyzed and affected for UPDATE, DELETE, and INSERT queries,

Expected output 
# Query_time: 0.962742  Lock_time: 0.000202  Rows_sent: 4  Rows_examined: 1543719  Rows_affected: 0

Values and context:

  • Rows_examined: Number of rows scanned - SELECT

  • Rows_affected: Number of rows changed - UPDATE, DELETE, INSERT

Memory footprint

The feature provides information about the amount of bytes sent for the result of the query and the number of temporary tables created for its execution - differentiated by whether they were created on memory or on disk - with the total number of bytes used by them.

Expected output 
# Bytes_sent: 8053  Tmp_tables: 1  Tmp_disk_tables: 0  Tmp_table_sizes: 950528

Values and context:

  • Bytes_sent: The amount of bytes sent for the result of the query

  • Tmp_tables: Number of temporary tables created on memory for the query

  • Tmp_disk_tables: Number of temporary tables created on disk for the query

  • Tmp_table_sizes: Total Size in bytes for all temporary tables used in the query

Query plan information

Each query can be executed in various ways. For example, it may use indexes or do a full table scan, or a temporary table may be needed. These are the things that you can usually see by running EXPLAIN on the query. The feature will now allow you to see the most important facts about the execution in the log file.

Expected output 
# Full_scan: Yes  Full_join: No  Tmp_table: No  Tmp_table_on_disk: No
# Filesort: No  Filesort_on_disk: No  Merge_passes: 0

The values and their meanings are documented with the log_slow_filter option.

InnoDB usage information

The final part of the output is the InnoDB usage statistics. MySQL currently shows many per-session statistics for operations with SHOW SESSION STATUS, but that does not include those of InnoDB, which are always global and shared by all threads. This feature lets you see those values for a given query.

Expected output 
#   InnoDB_IO_r_ops: 6415  InnoDB_IO_r_bytes: 105103360  InnoDB_IO_r_wait: 0.001279
#   InnoDB_rec_lock_wait: 0.000000  InnoDB_queue_wait: 0.000000
#   InnoDB_pages_distinct: 6430

Values:

  • innodb_IO_r_ops: Counts the number of page read operations scheduled. The actual number of read operations may be different, but since this can be done asynchronously, there is no good way to measure it.

  • innodb_IO_r_bytes: Similar to innodb_IO_r_ops, but the unit is bytes.

  • innodb_IO_r_wait: Shows how long (in seconds) it took InnoDB to actually read the data from storage.

  • innodb_rec_lock_wait: Shows how long (in seconds) the query waited for row locks.

  • innodb_queue_wait: Shows how long (in seconds) the query spent either waiting to enter the InnoDB queue or inside that queue waiting for execution.

  • innodb_pages_distinct: Counts approximately the number of unique pages the query accessed. The approximation is based on a small hash array representing the entire buffer pool, because it could take a lot of memory to map all the pages. The inaccuracy grows with the number of pages accessed by a query, because there is a higher probability of hash collisions.

If the query did not use InnoDB tables, that information is written into the log instead of the above statistics.

Log output destination

The log_output variable sets the destination for the slow query log and the general query log.

log_output

Option Description
Command-line Yes
Config file Yes
Scope Global
Dynamic Yes
Data type Set
Default FILE

Valid values:

  • FILE - write logs to files

  • TABLE - write logs to tables in the mysql database

  • NONE - disable log output

  • Comma-separated values such as FILE,TABLE

With FILE, the server writes the slow query log to slow_query_log_file. With TABLE, the server writes the slow query log to the mysql.slow_log table.

The default is FILE.

log_output does not enable logging. Set slow_query_log or general_log separately.

Set log_output in the configuration file, at server startup, or at runtime.

Configuration file:

[mysqld]
log_output = FILE

Runtime:

SET GLOBAL log_output = 'FILE';

Check the current value:

SELECT @@log_output;

PMM Query Analytics

Percona Monitoring and Management (PMM) Query Analytics (QAN) uses these data sources:

  • Slow query log file

  • Performance Schema

QAN ignores these logs:

  • General query log

  • Slow log with log_output set to TABLE

QAN reads the slow query log file when log_output includes FILE.

Prepared statements

  • Most client drivers send server-side prepared statements

  • Performance Schema shows placeholder parameters (?)

  • Performance Schema omits literal values

  • EXPLAIN may be unavailable for these queries

  • MySQL applies these limits upstream of PMM

  • PMM reports the data that MySQL exposes

  • The slow query log file records literal values

Process list

This page describes Percona changes to both the standard MySQL SHOW PROCESSLIST command and the standard MySQL INFORMATION_SCHEMA table PROCESSLIST.

INFORMATION_SCHEMA Tables

INFORMATION_SCHEMA.PROCESSLIST

This table implements modifications to the standard MySQL INFORMATION_SCHEMA table PROCESSLIST.

Column Name Description
‘ID’ ‘The connection identifier.’
‘USER’ ‘The MySQL user who issued the statement.’
‘HOST’ ‘The host name of the client issuing the statement.’
‘DB’ ‘The default database, if one is selected, otherwise NULL.’
‘COMMAND’ ‘The type of command the thread is executing.’
‘TIME’ ‘The time in seconds that the thread has been in its current state.’
‘STATE’ ‘An action, event, or state that indicates what the thread is doing.’
‘INFO’ ‘The statement that the thread is executing, or NULL if it is not executing any statement.’
‘TIME_MS’ ‘The time in milliseconds that the thread has been in its current state.’
‘ROWS_EXAMINED’ ‘The number of rows examined by the statement being executed (NOTE: This column is not updated for each examined row so it does not necessarily show an up-to-date value while the statement is executing. It only shows a correct value after the statement has completed.).’
‘ROWS_SENT’ ‘The number of rows sent by the statement being executed.’
‘TID’ ‘The Linux Thread ID. For Linux, this corresponds to light-weight process ID (LWP ID) and can be seen in the ps -L output. In case when Thread Pool is enabled, “TID” is not null for only currently executing statements and statements received via “extra” connection.’

Example output

Table PROCESSLIST:

SELECT * FROM INFORMATION_SCHEMA.PROCESSLIST;
Expected output 
+----+------+-----------+--------------------+---------+------+-----------+---------------------------+---------+-----------+---------------+
| ID | USER | HOST      | DB                 | COMMAND | TIME | STATE     | INFO                      | TIME_MS | ROWS_SENT | ROWS_EXAMINED |
+----+------+-----------+--------------------+---------+------+-----------+---------------------------+---------+-----------+---------------+
| 12 | root | localhost | information_schema | Query   |    0 | executing | select * from processlist |       0 |         0 |             0 |
+----+------+-----------+--------------------+---------+------+-----------+---------------------------+---------+-----------+---------------+

Additional INFORMATION_SCHEMA tables

This document lists the additional INFORMATION_SCHEMA tables provided by Percona Server for MySQL that are not documented elsewhere in this documentation.

Temporary tables

Note

This feature implementation is considered tech preview quality.

Only the temporary tables that were explicitly created with CREATE TEMPORARY TABLE or ALTER TABLE are shown, and not the ones created to process complex queries.

INFORMATION_SCHEMA.GLOBAL_TEMPORARY_TABLES

Column Name Description
‘SESSION_ID’ ‘MySQL connection id’
‘TABLE_SCHEMA’ ‘Schema in which the temporary table is created’
‘TABLE_NAME’ ‘Name of the temporary table’
‘ENGINE’ ‘Engine of the temporary table’
‘NAME’ ‘Internal name of the temporary table’
‘TABLE_ROWS’ ‘Number of rows of the temporary table’
‘AVG_ROW_LENGTH’ ‘Average row length of the temporary table’
‘DATA_LENGTH’ ‘Size of the data (Bytes)’
‘INDEX_LENGTH’ ‘Size of the indexes (Bytes)’
‘CREATE_TIME’ ‘Date and time of creation of the temporary table’
‘UPDATE_TIME’ ‘Date and time of the latest update of the temporary table’

This table holds information on the temporary tables that exist for all connections. No special privileges are required to query this table.

INFORMATION_SCHEMA.TEMPORARY_TABLES

Column Name Description
‘SESSION_ID’ ‘MySQL connection id’
‘TABLE_SCHEMA’ ‘Schema in which the temporary table is created’
‘TABLE_NAME’ ‘Name of the temporary table’
‘ENGINE’ ‘Engine of the temporary table’
‘NAME’ ‘Internal name of the temporary table’
‘TABLE_ROWS’ ‘Number of rows of the temporary table’
‘AVG_ROW_LENGTH’ ‘Average row length of the temporary table’
‘DATA_LENGTH’ ‘Size of the data (Bytes)’
‘INDEX_LENGTH’ ‘Size of the indexes (Bytes)’
‘CREATE_TIME’ ‘Date and time of creation of the temporary table’
‘UPDATE_TIME’ ‘Date and time of the latest update of the temporary table’

This table holds information on the temporary tables existing for the running connection.

Additional PERFORMANCE_SCHEMA tables

This document lists the additional PERFORMANCE_SCHEMA tables provided by Percona Server for MySQL that are not documented elsewhere.

performance_schema.account_failed_login_lock_status

The performance_schema.account_failed_login_lock_status table is available in Percona Server for MySQL 8.4.7-7 and later. The table exposes information about temporary account locking from the MySQL internal Access Control List (ACL) cache.

This table is read-only. Users need SELECT privilege on the performance_schema database or on this specific table to access its contents.

Note

If an account is locked using the CREATE USER or ALTER USER statements with the ACCOUNT LOCK clause, the ACCOUNT LOCK status is not captured in the account_failed_login_lock_status table. The server records the ACCOUNT LOCK state in the mysql.user table.

Column Name Data Type Description
USER CHAR(..) The MySQL user name
HOST CHAR(..) The MySQL host name
IS_TRACKING_ACTIVE enum('YES','NO') Indicates whether failed login tracking is enabled for the account
MAX_ATTEMPTS INTEGER Maximum number of failed login attempts allowed before account is locked (corresponds to FAILED_LOGIN_ATTEMPTS clause value in CREATE USER statement)
PASSWORD_LOCK_DAYS INTEGER Number of days for which account will be temporarily locked after exceeding the MAX_ATTEMPTS limit. Set to -1 if account is locked forever (corresponds to PASSWORD_LOCK_TIME clause value in CREATE USER)
IS_LOCKED BOOLEAN Indicates if account is temporarily locked by failed login lock tracking. NULL if tracking is not enabled for account
REMAINING_ATTEMPTS INTEGER Number of failed login attempts remaining before account will be locked. NULL if tracking is not enabled for account
REMAINING_DAYS_LOCKED INTEGER Number of days for which account is locked due to failed login lock tracking. -1 means that account is locked “forever” (until server restart/FLUSH PRIVILEGES or specific account unlock). NULL if tracking is not enabled for account

Troubleshoot

Troubleshooting overview

Use this section when something goes wrong or you need to diagnose an issue with Percona Server for MySQL. The topics below point you to the right guide.

Review Get help from Percona for ways that we can work with you.

Topics

Topic When to use it
Use PMM Advisors Run automated checks on your database settings and get recommendations.
Too many connections warning Diagnose and address connection limit messages in the log.
Handle corrupted tables React when InnoDB encounters a corrupted table without crashing the server.
Thread-based profiling Profile queries and understand where time is spent.
Stack trace Capture and interpret stack traces for debugging.
Core dumps (libcoredumper) Capture and analyze core dumps for crash diagnosis.

Troubleshooting by topic

These guides live under their topic sections but are useful when you are troubleshooting:

Topic Guide
SELinux Troubleshoot SELinux issues
AppArmor Troubleshoot AppArmor profiles
SQL Troubleshoot SQL code

What to do next

Use Percona Monitoring and Management (PMM) Advisors

Percona Monitoring and Management (PMM) provides several categories of Advisors. Each Advisor contains a set of automated checks. These checks investigate your database settings for a specific range of possible issues.

The PMM Server automatically downloads the Advisors if the Advisors and Telemetry options are enabled in Configuration > Settings > Advanced Settings. Both options are enabled by default.

Need Additional Guidance?

If you encounter complex advisor recommendations or require expert interpretation, Percona Support offers professional assistance to help you optimize your database configuration and resolve any identified issues.

See also

PMM documentation - Advisor checks

Too many connections warning

If the log_error_verbosity system variable is set to 2 or higher, this feature generates the Too many connections warning in the log.

Troubleshooting Connection Issues

For comprehensive guidance on managing database connections and performance, Percona Support can provide expert assistance in diagnosing and resolving connection management challenges.

Handle corrupted tables

When a server subsystem tries to access a corrupted table, the server may crash. If this outcome is not desirable when a corrupted table is encountered, set the new system innodb_corrupt_table_action variable to a value which allows the ongoing operation to continue without crashing the server.

The server error log registers attempts to access corrupted table pages.

Interacting with the innodb_force_recovery variable

The innodb_corrupt_table_action variable may work in conjunction with the innodb_force_recovery variable which considerably reduces the effect of InnoDB subsystems running in the background.

If the innodb_force_recovery option is <4, corrupted pages are lost and the server may continue to run due to the innodb_corrupt_table_action variable having a non-default value.

Complex Database Recovery Scenarios

Navigating table corruption and recovery can be challenging. Percona Support offers expert guidance for complex database recovery and optimization strategies.

For more information about the innodb_force_recovery variable, see Forcing InnoDB Recovery from the MySQL Reference Manual.

This feature adds a system variable.

System variables

innodb_corrupt_table_action

Option Description
Command-line Yes
Config file Yes
Scope Global
Dynamic Yes
Data type ULONG
Default assert
Range assert, warn, salvage

  • Enabling innodb_file_per_table and using the assert value creates an assertion failure which causes XtraDB to intentionally crash the server. This action is expected when detecting corrupted data in a single-table tablespace.

  • Enabling innodb_file_per_table and using the warn value causes XtraDB to pass the table corruption as corrupt table instead of crashing the server. Detecting the file as corrupt also disables the file I/O for that data file, except for the deletion operation.

  • Enabling innodb_file_per_table and using the salvage value causes XtraDB to allow read access to the corrupted tablespace but ignores any corrupted pages.

Thread based profiling

Percona Server for MySQL now uses thread based profiling by default, instead of process based profiling. This was implemented because with process based profiling, threads on the server, other than the one being profiled, can affect the profiling information.

Thread based profiling is using the information provided by the kernel getrusage function. Since the 2.6.26 kernel version, thread based resource usage is available with the RUSAGE_THREAD. This means that the thread based profiling will be used if you’re running the 2.6.26 kernel or newer, or if the RUSAGE_THREAD has been ported back.

This feature is enabled by default if your system supports it, in other cases it uses process based profiling.

Performance Profiling Challenges?

For advanced performance tuning and profiling strategies, Percona Support provides expert consultation to optimize your database performance.

Stack trace

Developers use the stack trace in the debug process, either an interactive investigation or during the post-mortem. No configuration is required to generate a stack trace.

Stack trace adds the following:

Name Description
Prints binary BuildID The Strip utility removes unneeded sections and debugging
information to reduce the size. This method is standard with
containers where the image size is essential. The BuildID lets
you resolve the stack trace when the Strip utility removes the
binary symbols table.
Print the server version information The version information establishes the starting point for
analysis. Some applications, such as MySQL, only print this
information to a log on startup, and when the crash occurs, the
log may be large, rotated, or truncated.

Need Help Interpreting Stack Traces?

Percona Support offers expert debugging and performance analysis to help you understand complex stack trace diagnostics.

Using libcoredumper

A core dump file is the documented moment of a computer when either the computer or an application exits. Developers examine the dump as one of the tasks when searching for the cause of a failure.

The libcoredumper is a free and Open Source fork of google-coredumper, enhanced to work on newer Linux versions, and GCC and CLANG.

ARM64 not supported

The libcoredumper is not supported on ARM64 (aarch64) platforms. Use standard core dumps (for example, core-file) or kernel core handling if you need crash dumps on ARM64.

Need Help Interpreting Stack Traces?

Percona Support offers expert debugging and performance analysis to help you understand complex stack trace diagnostics.

Enabling the libcoredumper

Enable core dumps for troubleshooting purposes on supported platforms.

To enable the libcoredumper, add the coredumper variable to the mysqld section of my.cnf. This variable is independent of the older core-file variable.

The variable can have the following possible values:

Value Description
Blank The core dump is saved under MySQL datadir and named core.
A path ending with / The core dump is saved under the specified directory and named core.
Full path with a filename The core dump is saved under the specified directory and filename

Restart the server.

Verifying the libcoredumper is active

MySQL writes to the log when generating a core file and delegates the core dump operation to the Linux kernel. 

Writing a core file

MySQL using the libcoredumper to generate the file creates the following message in the log:

Writing a core file using lib coredumper

Every core file adds a crash timestamp instead of a PID for the following reasons:

  • Correlates the core file with the crash. MySQL prints a UTC timestamp on the crash log.
10:02:09 UTC - mysqld got signal 11;
  • Stores multiple core files.

Note

For example, operators and containers run as the process id of PID 1. If the process ID is used to identify the core file, each container crash generates a core dump that overwrites the previous core file.

Complex Core Dump Analysis?

Percona Support offers expert debugging and in-depth analysis of core dump diagnostics to help you resolve critical system issues.

Disabling the libcoredumper

You can disable the libcoredumper. A core file may contain sensitive data and takes disk space.

To disable the libcoredumper you must do the following:

  1. In the mysqld section of my.cnf, remove the libcoredumper variable.

  2. Restart the server.

Storage engines

Percona XtraDB storage engine

The Percona XtraDB storage engine

Percona XtraDB is an enhanced version of the InnoDB storage engine, designed to better scale on modern hardware. It also includes a variety of other features useful in high-performance environments. It is fully backward compatible, and so can be used as a drop-in replacement for standard InnoDB.

Percona XtraDB includes all of InnoDB ‘s robust, reliable ACID-compliant design and advanced MVCC architecture, and builds on that solid foundation with more features, more tunability, more metrics, and more scalability. In particular, it is designed to scale better on many cores, use memory more efficiently, and be more convenient and useful. The new features are specially designed to alleviate some of InnoDB’s limitations. We choose features and fixes based on customer requests and on our best judgment of real-world needs as a high-performance consulting company.

Percona XtraDB engine will not have further binary releases, it is distributed as part of the Percona Server for MySQL.

Improved MEMORY storage engine

A Fixed Row Format (FRF) is used in the MEMORY storage engine. The fixed row format imposes restrictions on the type of columns as it assigns in advance a limited amount of memory per row. This renders a VARCHAR field in a CHAR field in practice and makes it impossible to have a TEXT or BLOB field with that engine implementation.

To overcome this limitation, the Improved MEMORY Storage Engine is introduced in this release for supporting true VARCHAR, VARBINARY, TEXT, and BLOB fields in the MEMORY tables.

This implementation is based on the Dynamic Row Format (DFR) introduced by the mysql-heap-dynamic-rows patch.

DFR is used to store column values in a variable-length form, thus helping to decrease the memory footprint of those columns and making possible BLOB and TEXT fields and real VARCHAR and VARBINARY.

Unlike the fixed implementation, each column value in DRF uses only as much space as required. Variable-length values can use up to 4 bytes to store the actual value length, and only the necessary number of blocks is used to store the value.

Rows in DFR are represented internally by multiple memory blocks, which means that a single row can consist of multiple blocks organized into one set. Each row occupies at least one block, there can not be multiple rows within a single block. Block size can be configured when creating a table (see below).

This DFR implementation has two caveats regarding ordering and indexes.

Caveats

Ordering of rows

In the absence of ORDER BY, records may be returned in a different order than the previous MEMORY implementation.

This is not a bug. Any application relying on a specific order without an ORDER BY clause may deliver unexpected results. A specific order without ORDER BY is a side effect of a storage engine and query optimizer implementation which may and will change between minor MySQL releases.

Indexing

It is currently impossible to use indexes on BLOB columns due to some limitations of the Dynamic Row Format. Trying to create such an index will fail with the following error:

Expected output 
BLOB column '<name>' can't be used in key specification with the used table type.

Restrictions

For performance reasons, a mixed solution is implemented: the fixed format is used at the beginning of the row, while the dynamic one is used for the rest of it.

The size of the fixed-format portion of the record is chosen automatically on CREATE TABLE and cannot be changed later. This, in particular, means that no indexes can be created later with CREATE INDEX or ALTER TABLE when the dynamic row format is used.

All values for columns used in indexes are stored in fixed format at the first block of the row, then the following columns are handled with DRF.

This sets two restrictions to tables:

  • the order of the fields and therefore,

  • the minimum size of the block used in the table.

Ordering of columns

The columns used in fixed format must be defined before the dynamic ones in the CREATE TABLE statement. If this requirement is not met, the engine will not be able to add blocks to the set for these fields and they will be treated as fixed.

Minimum block size

The block size has to be big enough to store all fixed-length information in the first block. If not, the CREATE TABLE or ALTER TABLE statements will fail (see below).

Limitations

MyISAM tables are still used for query optimizer internal temporary tables where the MEMORY tables could be used now instead: for temporary tables containing large VARCHAR\`s, ``BLOB, andTEXT` columns.

Setting row format

Taking the restrictions into account, the Improved MEMORY Storage Engine will choose DRF over FRF at the moment of creating the table according to following criteria:

  • There is an implicit request of the user in the column types OR

  • There is an explicit request of the user AND the overhead incurred by DFR is beneficial.

Implicit request

The implicit request by the user is taken when there is at least one BLOB or TEXT column in the table definition. If there are none of these columns and no relevant option is given, the engine will choose FRF.

For example, this will yield the use of the dynamic format:

CREATE TABLE t1 (f1 VARCHAR(32), f2 TEXT, PRIMARY KEY (f1)) ENGINE=HEAP;

While this will not:

CREATE TABLE t1 (f1 VARCHAR(16), f2 VARCHAR(16), PRIMARY KEY (f1)) ENGINE=HEAP;

Explicit request

The explicit request is set with one of the following options in the CREATE TABLE statement:

  • KEY_BLOCK_SIZE = <value>

  • Requests the DFR with the specified block size (in bytes)

Despite its name, the KEY_BLOCK_SIZE option refers to a block size used to store data rather then indexes. The reason for this is that an existing CREATE TABLE option is reused to avoid introducing new ones.

The Improved MEMORY Engine checks whether the specified block size is large enough to keep all key column values. If it is too small, table creation will abort with an error.

After DRF is requested explicitly and there are no BLOB or TEXT columns in the table definition, the Improved MEMORY Engine will check if using the dynamic format provides any space saving benefits as compared to the fixed one:

  • if the fixed row length is less than the dynamic block size (plus the dynamic row overhead - platform dependent) OR

  • there isn’t any variable-length columns in the table or VARCHAR fields are declared with length 31 or less,

the engine will revert to the fixed format as it is more space efficient in such case. The row format being used by the engine can be checked using SHOW TABLE STATUS.

Examples

CREATE TABLE t1 (f1 VARCHAR(32), f2 VARCHAR(32), f3 VARCHAR(32), f4 VARCHAR(32), PRIMARY KEY (f1)) KEY_BLOCK_SIZE=124 ENGINE=HEAP;

SHOW TABLE STATUS LIKE 't1';
Expected output 
Name  Engine  Version    Rows Avg_row_length  Data_length     Max_data_length Index_length    Data_free       Auto_increment  Create_time     Update_time     Check_time      Collation       Checksum        Create_options  Comment
t1    MEMORY  10         X    0       X       0       0       NULL    NULL    NULL    NULL    latin1_swedish_ci       NULL    KEY_BLOCK_SIZE=124

Implementation details

MySQL MEMORY tables keep data in arrays of fixed-size chunks. These chunks are organized into two groups of HP_BLOCK structures:

  • group1 contains indexes, with one HP_BLOCK per key (part of HP_KEYDEF),

  • group2 contains record data, with a single HP_BLOCK for all records.

While columns used in indexes are usually small, other columns in the table may need to accommodate larger data. Typically, larger data is placed into VARCHAR or BLOB columns.

The Improved MEMORY Engine implements the concept of dataspace, HP_DATASPACE, which incorporates the HP_BLOCK structures for the record data, adding more information for managing variable-sized records.

Variable-size records are stored in multiple “chunks”, which means that a single record of data (a database “row”) can consist of multiple chunks organized into one “set”, contained in HP_BLOCK structures.

In variable-size format, one record is represented as one or many chunks depending on the actual data, while in fixed-size mode, one record is always represented as one chunk. The index structures would always point to the first chunk in the chunkset.

Variable-size records are necessary only in the presence of variable-size columns. The Improved Memory Engine will be looking for BLOB or VARCHAR columns with a declared length of 32 or more. If no such columns are found, the table will be switched to the fixed-size format. You should always put such columns at the end of the table definition in order to use the variable-size format.

Whenever data is being inserted or updated in the table, the Improved Memory Engine will calculate how many chunks are necessary.

For INSERT operations, the engine only allocates new chunksets in the recordspace. For UPDATE operations it will modify the length of the existing chunkset if necessary, unlinking unnecessary chunks at the end, or allocating and adding more if a larger length is needed.

When writing data to chunks or copying data back to a record, fixed-size columns are copied in their full format, while VARCHAR and BLOB columns are copied based on their actual length, skipping any NULL values.

When allocating a new chunkset of N chunks, the engine will try to allocate chunks one-by-one, linking them as they become allocated. For allocating a single chunk, it will attempt to reuse a deleted (freed) chunk. If no free chunks are available, it will try to allocate a new area inside a HP_BLOCK.

When freeing chunks, the engine will place them at the front of a free list in the dataspace, each one containing a reference to the previously freed chunk.

The allocation and contents of the actual chunks varies between fixed and variable-size modes:

  • Format of a fixed-size chunk:

    • uchar[] * With sizeof=chunk_dataspace_length, but at least sizeof(uchar\*) bytes. It keeps actual data or pointer to the next deleted chunk, where chunk_dataspace_length equals to full record length

    • uchar * Status field (1 means “in use”, 0 means “deleted”)

  • Format of a variable-size chunk:

    • uchar[] * With sizeof=chunk_dataspace_length, but at least sizeof(uchar\*) bytes. It keeps actual data or pointer to the next deleted chunk, where chunk_dataspace_length is set according to table’s key_block_size

    • uchar\* * Pointer to the next chunk in this chunkset, or NULL for the last chunk

    • uchar * Status field (1 means “first”, 0 means “deleted”, 2 means “linked”)

Total chunk length is always aligned to the next sizeof(uchar\*).

See also

Dynamic row format for MEMORY tables

Improved InnoDB I/O scalability

Because InnoDB is a complex storage engine it must be configured properly in order to perform at its best. Some points are not configurable in standard InnoDB. The goal of this feature is to provide a more exhaustive set of options for XtraDB.

System variables

innodb_flush_method

Option Description
Command-line Yes
Config file Yes
Scope Global
Dynamic No
Data type Enumeration
Default NULL
Allowed values fsync, O_DSYNC, O_DIRECT, O_DIRECT_NO_FSYNC, littlesync, nosync

The following values are allowed:

  • fdatasync: use fsync() to flush data, log, and parallel doublewrite files.

  • O_SYNC: use O_SYNC to open and flush the log and parallel doublewrite files; use fsync() to flush the data files. Do not use fsync() to flush the parallel doublewrite file.

  • O_DIRECT: use O_DIRECT to open the data files and fsync() system call to flush data, log, and parallel doublewrite files.

  • O_DIRECT_NO_FSYNC: use O_DIRECT to open the data files and parallel doublewrite files, but does not use the fsync() system call to flush the data files, log files, and parallel doublewrite files. Do not use this option for the XFS file system.

Note

On an ext4 filesystem, set innodb_log_write_ahead_size to match the filesystem’s write-ahead block size. This variable avoids unaligned AIO/DIO warnings.

Status variables

The following information has been added to SHOW ENGINE INNODB STATUS to confirm the checkpointing activity:

The current checkpoint age target
The current age of the oldest page modification which has not been flushed to disk yet.
The current age of the last checkpoint
...
---
LOG
---
Log sequence number 0 1059494372
Log flushed up to   0 1059494372
Last checkpoint at  0 1055251010
Max checkpoint age  162361775
Checkpoint age target 104630090
Modified age        4092465
Checkpoint age      4243362
0 pending log writes, 0 pending chkp writes
...

Enforcing storage engine

Percona Server for MySQL implements a variable to enforce the use of a specific storage engine.

When this variable is specified and a user tries to create a table using an explicit storage engine that is not the specified enforced engine, the user will get either an error if the NO_ENGINE_SUBSTITUTION SQL mode is enabled or a warning if NO_ENGINE_SUBSTITUTION is disabled and the table will be created anyway using the enforced engine (this is consistent with the default MySQL way of creating the default storage engine if other engines are not available unless NO_ENGINE_SUBSTITUTION is set).

In case a user tries to enable enforce_storage_engine with an engine that isn’t available, the system will not start.

Note

If you’re using enforce_storage_engine, you must either disable it before doing mysql_upgrade or perform mysql_upgrade with server started with --skip-grants-tables.

System variables

enforce_storage_engine

Option Description
Command Line: Yes
Config file Yes
Scope: Global
Dynamic: No
Data type String
Default value NULL

This variable is not case sensitive.

Example

Adding following option to my.cnf will start the server with InnoDB as enforced storage engine.

enforce_storage_engine=InnoDB

Extended show engine InnoDB status

This feature reorganizes the output of SHOW ENGINE INNODB STATUS to improve readability and to provide additional information. The variable innodb_show_locks_held controls the umber of locks held to print for each InnoDB transaction.

This feature modified the SHOW ENGINE INNODB STATUS command as follows:

  • Added extended information about InnoDB internal hash table sizes (in bytes) in the BUFFER POOL AND MEMORY section; also added buffer pool size in bytes.

  • Added additional LOG section information.

Other information

System variables

innodb_show_locks_held

Option Description
Command-line Yes
Config file Yes
Scope Global
Dynamic Yes
Data type ULONG
Default 10
Range 0 - 1000

Specifies the number of locks held to print for each InnoDB transaction in SHOW ENGINE INNODB STATUS.

innodb_print_lock_wait_timeout_info

Option Description
Command-line Yes
Config file Yes
Scope Global
Dynamic Yes
Data type Boolean
Default OFF

Makes InnoDB to write information about all lock wait timeout errors into the log file.

This allows to find out details about the failed transaction, and, most importantly, the blocking transaction. Query string can be obtained from EVENTS_STATEMENTS_CURRENT table, based on the PROCESSLIST_ID field, which corresponds to thread_id from the log output.

Taking into account that blocking transaction is often a multiple statement one, following query can be used to obtain blocking thread statements history:

SELECT s.SQL_TEXT FROM performance_schema.events_statements_history s
INNER JOIN performance_schema.threads t ON t.THREAD_ID = s.THREAD_ID
WHERE t.PROCESSLIST_ID = %d
UNION
SELECT s.SQL_TEXT FROM performance_schema.events_statements_current s
INNER JOIN performance_schema.threads t ON t.THREAD_ID = s.THREAD_ID
WHERE t.PROCESSLIST_ID = %d;

The PROCESSLIST_ID in this example is exactly the thread id from error log output.

Status variables

The status variables here contain information available in the output of SHOW ENGINE INNODB STATUS, organized by the sections SHOW ENGINE INNODB STATUS displays. If you are familiar with the output of SHOW ENGINE INNODB STATUS, you will probably already recognize the information these variables contain.

BACKGROUND THREAD

The following variables contain information in the BACKGROUND THREAD section of the output from SHOW ENGINE INNODB STATUS.

Expected output 
-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 1 srv_active, 0 srv_shutdown, 11844 srv_idle
srv_master_thread log flush and writes: 11844

InnoDB has a source thread which performs background tasks depending on the server state, once per second. If the server is under workload, the source thread runs the following: performs background table drops; performs change buffer merge, adaptively; flushes the redo log to disk; evicts tables from the dictionary cache if needed to satisfy its size limit; makes a checkpoint. If the server is idle: performs background table drops, flushes and/or checkpoints the redo log if needed due to the checkpoint age; performs change buffer merge at full I/O capacity; evicts tables from the dictionary cache if needed; and makes a checkpoint.

Innodb_background_log_sync

Option Description
Scope Global
Data type Numeric

This variable shows the number of times the InnoDB source thread has written and flushed the redo log.

SEMAPHORES

The following variables contain information in the SEMAPHORES section of the output from SHOW ENGINE INNODB STATUS. An example of that output is:

Expected output 
----------
SEMAPHORES
----------
OS WAIT ARRAY INFO: reservation count 9664, signal count 11182
Mutex spin waits 20599, rounds 223821, OS waits 4479
RW-shared spins 5155, OS waits 1678; RW-excl spins 5632, OS waits 2592
Spin rounds per wait: 10.87 mutex, 15.01 RW-shared, 27.19 RW-excl

INSERT BUFFER AND ADAPTIVE HASH INDEX

The following variables contain information in the INSERT BUFFER AND ADAPTIVE HASH INDEX section of the output from SHOW ENGINE INNODB STATUS. An example of that output is:

Expected output 
-------------------------------------
INSERT BUFFER AND ADAPTIVE HASH INDEX
-------------------------------------
Ibuf: size 1, free list len 6089, seg size 6091,
44497 inserts, 44497 merged recs, 8734 merges
0.00 hash searches/s, 0.00 non-hash searches/s

Innodb_ibuf_free_list

Option Description
Scope Global
Data type Numeric

Innodb_ibuf_segment_size

Option Description
Scope Global
Data type Numeric

LOG

The following variables contain information in the LOG section of the output from SHOW ENGINE INNODB STATUS. An example of that output is:

Expected output 
LOG
---
Log sequence number 10145937666
Log flushed up to   10145937666
Pages flushed up to 10145937666
Last checkpoint at  10145937666
Max checkpoint age    80826164
Checkpoint age target 78300347
Modified age          0
Checkpoint age        0
0 pending log writes, 0 pending chkp writes
9 log i/o's done, 0.00 log i/o's/second
Log tracking enabled
Log tracked up to   10145937666
Max tracked LSN age 80826164

Innodb_lsn_current

Option Description
Scope Global
Data type Numeric

This variable shows the current log sequence number.

Innodb_lsn_flushed

Option Description
Scope Global
Data type Numeric

This variable shows the current maximum LSN that has been written and flushed to disk.

Innodb_lsn_last_checkpoint

Option Description
Scope Global
Data type Numeric

This variable shows the LSN of the latest completed checkpoint.

Innodb_checkpoint_age

Option Description
Scope Global
Data type Numeric

This variable shows the current InnoDB checkpoint age, i.e., the difference between the current LSN and the LSN of the last completed checkpoint.

BUFFER POOL AND MEMORY

The following variables contain information in the BUFFER POOL AND MEMORY section of the output from SHOW ENGINE INNODB STATUS. An example of that output is:

Expected output 
----------------------
BUFFER POOL AND MEMORY
----------------------
Total memory allocated 137363456; in additional pool allocated 0
Total memory allocated by read views 88
Internal hash tables (constant factor + variable factor)
    Adaptive hash index 2266736         (2213368 + 53368)
    Page hash           139112 (buffer pool 0 only)
    Dictionary cache    729463  (554768 + 174695)
    File system         824800  (812272 + 12528)
    Lock system         333248  (332872 + 376)
    Recovery system     0       (0 + 0)
Dictionary memory allocated 174695
Buffer pool size        8191
Buffer pool size, bytes 134201344
Free buffers            7481
Database pages          707
Old database pages      280
Modified db pages       0
Pending reads 0
Pending writes: LRU 0, flush list 0 single page 0
Pages made young 0, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 707, created 0, written 1
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
No buffer pool page gets since the last printout
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 707, unzip_LRU len: 0

Innodb_mem_adaptive_hash

Option Description
Scope Global
Data type Numeric

This variable shows the current size, in bytes, of the adaptive hash index.

Innodb_mem_dictionary

Option Description
Scope Global
Data type Numeric

This variable shows the current size, in bytes, of the InnoDB in-memory data dictionary info.

Innodb_mem_total

Option Description
Scope Global
Data type Numeric

This variable shows the total amount of memory, in bytes, InnoDB has allocated in the process heap memory.

Innodb_buffer_pool_pages_LRU_flushed

Option Description
Scope Global
Data type Numeric

This variable shows the total number of buffer pool pages which have been flushed from the LRU list, i.e., too old pages which had to be flushed in order to make buffer pool room to read in new data pages.

Innodb_buffer_pool_pages_made_not_young

Option Description
Scope Global
Data type Numeric

This variable shows the number of times a buffer pool page was not marked as accessed recently in the LRU list because of innodb_old_blocks_time variable setting.

Innodb_buffer_pool_pages_made_young

Option Description
Scope Global
Data type Numeric

This variable shows the number of times a buffer pool page was moved to the young end of the LRU list due to its access, to prevent its eviction from the buffer pool.

Innodb_buffer_pool_pages_old

Option Description
Scope Global
Data type Numeric

This variable shows the total number of buffer pool pages which are considered to be old according to the Making the Buffer Pool Scan Resistant manual page.

TRANSACTIONS

The following variables contain information in the TRANSACTIONS section of the output from SHOW INNODB STATUS. An example of that output is:

Expected output 
------------
TRANSACTIONS
------------
Trx id counter F561FD
Purge done for trx's n:o < F561EB undo n:o < 0
History list length 19
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 0, not started, process no 993, OS thread id 140213152634640
mysql thread id 15933, query id 32109 localhost root
show innodb status
---TRANSACTION F561FC, ACTIVE 29 sec, process no 993, OS thread id 140213152769808 updating or deleting
mysql tables in use 1, locked 1

Innodb_max_trx_id

Option Description
Scope Global
Data type Numeric

This variable shows the next free transaction id number.

Innodb_oldest_view_low_limit_trx_id

Option Description
Scope Global
Data type Numeric

This variable shows the highest transaction id, above which the current oldest open read view does not see any transaction changes. Zero if there is no open view.

Innodb_purge_trx_id

Option Description
Scope Global
Data type Numeric

This variable shows the oldest transaction id whose records have not been purged yet.

Innodb_purge_undo_no

Option Description
Scope Global
Data type Numeric

INFORMATION_SCHEMA Tables

The following table contains information about the oldest active transaction in the system.

INFORMATION_SCHEMA.XTRADB_READ_VIEW

The data type for the following columns is BIGINT UNSIGNED. The columns contain 64-bit integers.

Column Name Description
‘READ_VIEW_LOW_LIMIT_TRX_NUMBER’ This is the highest transactions number at the time the view was created.
‘READ_VIEW_UPPER_LIMIT_TRX_ID’ This is the highest transactions ID at the time the view was created. This means that it should not see newer transactions with IDs bigger than or equal to that value.
‘READ_VIEW_LOW_LIMIT_TRX_ID’ This is the latest committed transaction ID at the time the oldest view was created. This means that it should see all transactions with IDs smaller than or equal to that value.

The following table contains information about the memory usage for InnoDB/XtraDB hash tables.

INFORMATION_SCHEMA.XTRADB_INTERNAL_HASH_TABLES

Column Name Description
‘INTERNAL_HASH_TABLE_NAME’ Hash table name
‘TOTAL_MEMORY’ Total amount of memory
‘CONSTANT_MEMORY’ Constant memory
‘VARIABLE_MEMORY’ Variable memory

Other reading

Show storage engines

This feature changes the comment field displayed when the SHOW STORAGE ENGINES command is executed and XtraDB is the storage engine.

Before the Change:

show storage engines;
Expected output 
+------------+---------+----------------------------------------------------------------+--------------+------+------------+
| Engine     | Support | Comment                                                        | Transactions | XA   | Savepoints |
+------------+---------+----------------------------------------------------------------+--------------+------+------------+
| InnoDB     | YES     | Supports transactions, row-level locking, and foreign keys     | YES          | YES  | YES        |
...
+------------+---------+----------------------------------------------------------------+--------------+------+------------+

After the Change:

show storage engines;
Expected output 
+------------+---------+----------------------------------------------------------------------------+--------------+------+------------+
| Engine     | Support | Comment                                                                    | Transactions |   XA | Savepoints |
+------------+---------+----------------------------------------------------------------------------+--------------+------+------------+
| InnoDB     | YES     | Percona-XtraDB, Supports transactions, row-level locking, and foreign keys |          YES | YES  | YES        |
...
+------------+---------+----------------------------------------------------------------------------+--------------+------+------------+

Compressed columns with dictionaries

The per-column compression feature is a data type modifier, independent from user-level SQL and InnoDB data compression, that causes the data stored in the column to be compressed on writing to storage and decompressed on reading. For all other purposes, the data type is identical to the one without the modifier, i.e. no new data types are created. Compression is done by using the zlib library.

Additionally, it is possible to pre-define a set of strings for each compressed column to achieve a better compression ratio on relatively small individual data items.

This feature provides:

  • a better compression ratio for text data which consists of a large number of predefined words (e.g. JSON or XML) using compression methods with static dictionaries

  • a way to select columns in the table to compress (in contrast to the InnoDB row compression method). This feature is based on a patch provided by Weixiang Zhai.

Specifications

The feature is limited to InnoDB/XtraDB storage engine and to columns of the following data types:

  • BLOB (including TINYBLOB, MEDIUMBLOB, LONGBLOG)

  • TEXT (including TINYTEXT, MEDUUMTEXT, LONGTEXT)

  • VARCHAR (including NATIONAL VARCHAR)

  • VARBINARY

  • JSON

A compressed column is declared by using the syntax that extends the existing COLUMN_FORMAT modifier: COLUMN_FORMAT COMPRESSED. If this modifier is applied to an unsupported column type or storage engine, an error is returned.

The compression can be specified:

  • when creating a table: CREATE TABLE ... (..., foo BLOB COLUMN_FORMAT COMPRESSED, ...);

  • when altering a table and modifying a column to the compressed format: ALTER TABLE ... MODIFY [COLUMN] ... COLUMN_FORMAT COMPRESSED, or ALTER TABLE ... CHANGE [COLUMN] ... COLUMN_FORMAT COMPRESSED.

Unlike Oracle MySQL, compression is applicable to generated stored columns. Use this syntax extension as follows:

CREATE TABLE t1(
       id INT,
       a BLOB,
       b JSON COLUMN_FORMAT COMPRESSED,
       g BLOB GENERATED ALWAYS AS (a) STORED COLUMN_FORMAT COMPRESSED WITH COMPRESSION_DICTIONARY numbers
     ) ENGINE=InnoDB;

To decompress a column, specify a value other than COMPRESSED to COLUMN_FORMAT: FIXED, DYNAMIC, or DEFAULT. If there is a column compression/decompression request in an ALTER TABLE, it is forced to the COPY algorithm.

Two new variables: innodb_compressed_columns_zip_level and innodb_compressed_columns_threshold have been implemented.

Compression dictionary support

To achieve a better compression ratio on relatively small individual data items, it is possible to predefine a compression dictionary, which is a set of strings for each compressed column.

Compression dictionaries can be represented as a list of words in the form of a string (a comma or any other character can be used as a delimiter although not required). In other words, a, bb, ccc, a bb ccc, and abbccc will have the same effect. However, the latter is more compact. The Quote symbol quoting is handled by regular SQL quoting. The maximum supported dictionary length is 32506 bytes (zlib limitation).

The compression dictionary is stored in a new system InnoDB table. As this table is of the data dictionary kind, concurrent reads are allowed, but writes are serialized, and reads are blocked by writes. Table read through old read views are not supported, similar to InnoDB internal DDL transactions.

Interaction with innodb_force_recovery variable

Compression dictionary operations are treated like DDL operations with the exception when innodb_force_value is set to 3: with values less than 3, compression dictionary operations are allowed, and with values >= 3, they are forbidden.

Example

In order to use the compression dictionary, you need to create it. This can be done by running:

SET @dictionary_data = 'one' 'two' 'three' 'four';
Expected output 
Query OK, 0 rows affected (0.00 sec)

CREATE COMPRESSION_DICTIONARY numbers (@dictionary_data);
Expected output 
Query OK, 0 rows affected (0.00 sec)

To create a table that has both compression and compressed dictionary support you should run:

CREATE TABLE t1(
        id INT,
        a BLOB COLUMN_FORMAT COMPRESSED,
        b BLOB COLUMN_FORMAT COMPRESSED WITH COMPRESSION_DICTIONARY numbers
      ) ENGINE=InnoDB;

The following example shows how to insert a sample of JSON data into the table:

SET @json_value =
'[\n'
' {\n'
' "one" = 0,\n'
' "two" = 0,\n'
' "three" = 0,\n'
' "four" = 0\n'
' },\n'
' {\n'
' "one" = 0,\n'
' "two" = 0,\n'
' "three" = 0,\n'
' "four" = 0\n'
' },\n'
' {\n'
' "one" = 0,\n'
' "two" = 0,\n'
' "three" = 0,\n'
' "four" = 0\n'
' },\n'
' {\n'
' "one" = 0,\n'
' "two" = 0,\n'
' "three" = 0,\n'
' "four" = 0\n'
' }\n'
']\n'
;

INSERT INTO t1 VALUES(0, @json_value, @json_value);
Query OK, 1 row affected (0.01 sec)

INFORMATION_SCHEMA Tables

This feature implements two new INFORMATION_SCHEMA tables.

INFORMATION_SCHEMA.COMPRESSION_DICTIONARY

Column Name Description
‘BIGINT(21)_UNSIGNED dict_version’ ‘dictionary version’
‘VARCHAR(64) dict_name’ ‘dictionary name’
‘BLOB dict_data’ ‘compression dictionary string’

This table provides a view of the internal compression dictionary. The SUPER privilege is required to query it.

INFORMATION_SCHEMA.COMPRESSION_DICTIONARY_TABLES

Column Name Description
‘BIGINT(21)_UNSIGNED table_schema’ ‘table schema’
‘BIGINT(21)_UNSIGNED table_name’ ‘table ID from INFORMATION_SCHEMA.INNODB_SYS_TABLES’
‘BIGINT(21)_UNSIGNED column_name’ ‘column position (starts from 0 as in INFORMATION_SCHEMA.INNODB_SYS_COLUMNS)’
‘BIGINT(21)_UNSIGNED dict_name’ ‘dictionary ID’

This table provides a view over the internal table that stores the mapping between the compression dictionaries and the columns using them. The SUPER privilege is require to query it.

Limitations

Compressed columns cannot be used in indices (neither on their own nor as parts of composite keys).

Note

CREATE TABLE t2 AS SELECT \* FROM t1 will create a new table with a compressed column, whereas CREATE TABLE t2 AS SELECT CONCAT(a,'') AS a FROM t1 will not create compressed columns.

At the same time, after executing the CREATE TABLE t2 LIKE t1 statement, t2.a will have the COMPRESSED attribute.

ALTER TABLE ... DISCARD/IMPORT TABLESPACE is not supported for tables with compressed columns. To export and import tablespaces with compressed columns, you uncompress them first with: ALTER TABLE ... MODIFY ... COLUMN_FORMAT DEFAULT.

mysqldump command line parameters

By default, with no additional options, mysqldump will generate a MySQL compatible SQL output.

All /\*!50633 COLUMN_FORMAT COMPRESSED \*/ and /\*!50633 COLUMN_FORMAT COMPRESSED WITH COMPRESSION_DICTIONARY <dictionary> \*/ won’t be in the dump.

When a new option enable-compressed-columns is specified, all /\*!50633 COLUMN_FORMAT COMPRESSED \*/ will be left intact and all /\*!50633 COLUMN_FORMAT COMPRESSED WITH COMPRESSION_DICTIONARY <dictionary> \*/ will be transformed into /\*!50633 COLUMN_FORMAT COMPRESSED \*/. In this mode, the dump will contain the necessary SQL statements to create compressed columns, but without dictionaries.

When a new enable-compressed-columns-with-dictionaries option is specified, dump will contain all compressed column attributes and compression dictionary.

Moreover, the following dictionary creation fragments will be added before CREATE TABLE statements which are going to use these dictionaries for the first time.

/*!50633 DROP COMPRESSION_DICTIONARY IF EXISTS <dictionary>; */
/*!50633 CREATE COMPRESSION_DICTIONARY <dictionary>(...); */

Two new options add-drop-compression-dictionary and skip-add-drop-compression-dictionary will control if /\*!50633 DROP COMPRESSION_DICTIONARY IF EXISTS <dictionary> \*/ part from previous paragraph will be skipped or not. By default, add-drop-compression-dictionary the mode will be used.

When both enable-compressed-columns-with-dictionaries and --tab=<dir> (separate file for each table) options are specified, necessary compression dictionaries will be created in each output file using the following fragment (regardless of the values of add-drop-compression-dictionary and skip-add-drop-compression-dictionary options).

/*!50633 CREATE COMPRESSION_DICTIONARY IF NOT EXISTS <dictionary>(...); */

System variables

innodb_compressed_columns_zip_level

Option Description
Command-line Yes
Config file Yes
Scope Global
Dynamic Yes
Data type Numeric
Default 6
Range 0-9

This variable is used to specify the compression level used for compressed columns. Specifying 0 will use no compression, 1 the fastest, and 9 the best compression. The default value is 6.

innodb_compressed_columns_threshold

Option Description
Command-line Yes
Config file Yes
Scope Global
Dynamic Yes
Data type Numeric
Default 96
Range 1 - 2^64-1 (or 2^32-1 for 32-bit release)

By default, a value being inserted will be compressed if its length exceeds innodb_compressed_columns_threshold bytes. Otherwise, it will be stored in the raw (uncompressed) form.

Please also note that because of the nature of some data, the compressed representation can be longer than the original value. In this case, it does not make sense to store such values in compressed form as Percona Server for MySQL would have to waste both memory space and CPU resources for unnecessary decompression. Therefore, even if the length of such non-compressible values exceeds innodb_compressed_columns_threshold, they will be stored in an uncompressed form (however, an attempt to compress them will still be made).

This parameter can be tuned to skip unnecessary attempts of data compression for values that are known in advance by the user to have a bad compression ratio of their first N bytes.

InnoDB full-text search improvements

Ignoring stopword list

By default, all Full-Text Search indexes check the stopwords list , to see if any indexed elements contain words on that list.

Using this list for n-gram indexes isn’t always suitable, for example, any item that contains a or i will be ignored. Another word that can’t be searched is east, this one will find no matches because a is on the FTS stopword list.

To resolve this issue, Percona Server for MySQL has the innodb_ft_ignore_stopwords variable to control whether InnoDB Full-Text Search should ignore the stopword list.

Although this variable is introduced to resolve n-gram issues, it affects all Full-Text Search indexes as well.

Being a stopword doesn’t just mean being one of the predefined words from the list. Tokens shorter than innodb_ft_min_token_size or longer than innodb_ft_max_token_size are also considered stopwords. Therefore, when innodb_ft_ignore_stopwords is set to ON even for non-ngram FTS, innodb_ft_min_token_size / innodb_ft_max_token_size will be ignored meaning that in this case very short and very long words will also be indexed.

System variables

innodb_ft_ignore_stopwords

Option Description
Command-line Yes
Config file Yes
Scope Session, Global
Dynamic Yes
Data type Boolean
Default OFF

When enabled, this variable will instruct InnoDB Full Text Search parser to ignore the stopword list when building/updating an FTS index.

XtraDB performance improvements for I/O-bound highly-concurrent workloads

Priority refill for the buffer pool free list

Under heavy concurrent I/O load, the buffer pool free list can fall behind demand.

The following issues can occur:

  • Query and purge threads consume free pages faster than the Least Recently Used (LRU) cleaner thread refills the free list

  • Empty free lists force query and purge threads to poll, sleep, or run single-page LRU flushes

  • Many waiting threads increase contention on the buffer pool free list mutex

  • Mutex contention delays the return of freed pages to the free list

Percona Server for MySQL addresses these issues with the following changes:

  • The LRU manager thread handles all LRU flushes

  • Query threads avoid page eviction and single-page LRU flushes

  • A backoff algorithm reduces mutex pressure on empty free lists

Configure this behavior with innodb_empty_free_list_algorithm.

innodb_empty_free_list_algorithm

Option Description
Command-line: Yes
Config file: Yes
Scope: Global
Dynamic: Yes
Data type: Enumeration
Default: legacy

Values

Value Description
legacy Uses the upstream algorithm. Default value
backoff Uses the Percona Server for MySQL algorithm

Adaptive page cleaner flushing

InnoDB page cleaners flush dirty buffer pool pages to disk. This frees redo log space.

The adaptive flushing algorithm sets the flush rate from checkpoint age. Checkpoint age is the gap between the current Log Sequence Number (LSN) and the LSN of the last completed checkpoint. A larger checkpoint age triggers more aggressive flushing.

Percona Server for MySQL uses a different age-factor formula than upstream MySQL. The innodb_cleaner_lsn_age_factor variable selects the formula.

The default value is high_checkpoint. Flushing starts slower at low checkpoint ages. Flushing accelerates as checkpoint age grows. The server can keep more dirty pages in the buffer pool. Write throughput improves on sustained write-heavy workloads.

The legacy value uses the upstream MySQL formula. Flushing starts earlier for a given checkpoint age. Use legacy when checkpoint age spikes cause flush storms or write pauses with high_checkpoint.

Monitor checkpoint age with these status variables:

You can also use the InnoDB Checkpoint Age graph in Percona Monitoring and Management (PMM). Keep checkpoint age high without flush storms or write pauses near the maximum checkpoint age.

Adaptive flushing also uses these variables:

  • innodb_adaptive_flushing

  • innodb_adaptive_flushing_lwm

  • innodb_io_capacity

  • innodb_io_capacity_max

See the following posts for more detail on variable interaction:

innodb_cleaner_lsn_age_factor

Option Description
Command-line: Yes
Config file: Yes
Scope: Global
Dynamic: Yes
Data type: Enumeration
Default: high_checkpoint

Values

Value Description
high_checkpoint Uses the Percona Server for MySQL age-factor formula. Flushing pressure rises slowly at low checkpoint ages and faster at higher ages. The server keeps more dirty pages. Default value
legacy Uses the upstream MySQL age-factor formula. Flushing starts earlier for a given checkpoint age. Use when checkpoint age spikes cause flush storms or write pauses with high_checkpoint

Change the value at runtime:

SET GLOBAL innodb_cleaner_lsn_age_factor = 'legacy';

Add a persistent setting in the option file:

[mysqld]
innodb_cleaner_lsn_age_factor = high_checkpoint

Multi-threaded LRU flusher

Percona Server for MySQL restores the InnoDB LRU threads from earlier releases.

These threads flush pages by LRU.

These threads also reclaim free pages and move the pages to the free list.

Earlier releases always ran these threads.

Earlier releases had no option to enable or disable the threads.

The innodb_lru_threads variable enables or disables the threads.

Disable the threads for low-concurrency setups.

Enable the threads for high-concurrency setups.

The default value is OFF.

When innodb_lru_threads is ON, each buffer pool instance has a dedicated LRU manager thread.

That thread flushes and evicts pages to refill the instance free list.

When innodb_lru_threads is OFF, page cleaners flush pages by LRU.

innodb_lru_threads

Option Description
Command-line: Yes
Config file: Yes
Scope: Global
Dynamic: No
Data type: Boolean
Default: OFF

Add a persistent setting in the option file:

[mysqld]
innodb_lru_threads=ON

innodb_sched_priority_master

Option Description
Command-line: Yes
Config file: Yes
Scope: Global
Dynamic: Yes
Data type: Boolean

Add this variable to the configuration file.

Lazy buffer pool latch initialization

innodb_buffer_pool_lazy_latch_init

Option Description
Command-line: Yes
Config file: Yes
Scope: Global
Dynamic: No
Data type: Boolean
Default: OFF

Enables speeding up MySQL startup time by deferring some initialization cost (latches used by individual blocks belonging to the buffer pool are not initialized on startup, but later on the first use of each page). By default: OFF.

Multiple page asynchronous I/O requests

The I/O unit size in InnoDB is only one page, even if the server doing read ahead. A 16KB I/O unit size is too small for sequential reads, and less efficient than a larger I/O unit size. InnoDB uses Linux asynchronous I/O (aio) by default. By submitting multiple, consecutive 16KB read requests at the same time, Linux internally merges the requests and reads more efficiently.

This feature is able to submit multiple page I/O requests and works in the background. You can manage the feature with the [linear read-ahead technique]. This technique adds pages to the buffer pool based on the buffer pool pages being accessed sequentially. The innodb_read_ahead_threshold configuration parameter controls this operation.

On a HDD RAID 1+0 environment , more than 1000MB/s disk reads can be achieved by submitting 64 consecutive pages requests at once, while only 160MB/s disk reads is shown by submitting single page request.

Status variables

Innodb_buffered_aio_submitted

Option Description
Scope: Global
Data type: Numeric

This variable shows the number of submitted buffered asynchronous I/O requests.

Other reading

Prefix index queries optimization

Percona Server for MySQL has ported Prefix Index Queries Optimization feature from Facebook patch for MySQL.

Prior to this InnoDB would always fetch the clustered index for all prefix columns in an index, even when the value of a particular record was smaller than the prefix length. This implementation optimizes that case to use the record from the secondary index and avoid the extra lookup.

Status variables

Innodb_secondary_index_triggered_cluster_reads

Option Description
Scope: Global
Data type: Numeric

This variable shows the number of times secondary index lookup triggered cluster lookup.

Innodb_secondary_index_triggered_cluster_reads_avoided

Option Description
Scope: Global
Data type: Numeric

This variable shows the number of times prefix optimization avoided triggering cluster lookup.

Limit the estimation of records in a Query

Important

This feature is a tech preview. Before using this feature in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.

This page describes an alternative when running queries against a large number of table partitions. When a query runs, InnoDB estimates the records in each partition. This process can result in more pages read and more disk I/O, if the buffer pool must fetch the pages from disk. This process increases the query time if there are a large number of partitions.

The addition of two variables makes it possible to override records_in_range which effectively bypasses the process.

Warning

The use of these variables may result in improper index selection by the optimizer.

innodb_records_in_range

Option Description
Command-line: --innodb-records-in-range
Scope: Global
Dynamic: Yes
Data type: Numeric
Default 0

Important

This feature is a tech preview. Before using this feature in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.

The variable provides a method to limit the number of records estimated for a query.

SET @@GLOBAL.innodb_records_in_range=100;
100

innodb_force_index_records_in_range

Option Description
Command-line: --innodb-force-index-records-in-range
Scope: Global
Dynamic: Yes
Data type: Numeric
Default 0

Important

This feature is a tech preview. Before using this feature in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.

This variable provides a method to override the records_in_range result when a FORCE INDEX is used in a query.

SET @@GLOBAL.innodb_force_index_records_in_range=100;
100

Using the favor_range_scan optimizer switch

Important

This feature is a tech preview. Before using this feature in production, we recommend that you test restoring production from physical backups in your environment, and also use the alternative backup method for redundancy.

In specific scenarios, the optimizer chooses to scan a table instead of using a range scan. The conditions are the following:

  • Table with an extremely large number of rows

  • Compound primary keys made of two or more columns

  • WHERE clause contains multiple range conditions

The optimizer_switch controls the optimizer behavior. The favor_range_scan switch arbitrarily lowers the cost of a range scan by a factor of 10.

The available values are:

  • ON

  • OFF (Default)

  • DEFAULT

SET optimizer_switch='favor_range_scan=on';

InnoDB page fragmentation counters

InnoDB page fragmentation is caused by random insertion or deletion from a secondary index. This means that the physical ordering of the index pages on the disk is not same as the index ordering of the records on the pages. As a consequence this means that some pages take a lot more space and that queries which require a full table scan can take a long time to finish.

To provide more information about the InnoDB page fragmentation Percona Server for MySQL now provides the following counters as status variables: Innodb_scan_pages_contiguous, Innodb_scan_pages_disjointed, Innodb_scan_data_size, Innodb_scan_deleted_recs_size, and Innodb_scan_pages_total_seek_distance.

Status variables

Innodb_scan_pages_contiguous

Option Description
Scope Session
Data type Numeric

This variable shows the number of contiguous page reads inside a query.

Innodb_scan_pages_disjointed

Option Description
Scope Session
Data type Numeric

This variable shows the number of disjointed page reads inside a query.

Innodb_scan_data_size

Option Description
Scope Session
Data type Numeric

This variable shows the size of data in all InnoDB pages read inside a query (in bytes) - calculated as the sum of page_get_data_size(page) for every page scanned.

Innodb_scan_deleted_recs_size

Option Description
Scope Session
Data type Numeric

This variable shows the size of deleted records (marked as deleted in page_delete_rec_list_end()) in all InnoDB pages read inside a query (in bytes) - calculated as the sum of page_header_get_field(page, PAGE_GARBAGE) for every page scanned.

Innodb_scan_pages_total_seek_distance

Option Description
Scope Session
Data type Numeric

This variable shows the total seek distance when moving between pages.

Percona MyRocks

Percona MyRocks introduction

MyRocks is a storage engine for MySQL based on RocksDB , an embeddable, persistent key-value store. Percona MyRocks is an implementation for Percona Server for MySQL .

The RocksDB store is based on the log-structured merge-tree (or LSM tree). It is optimized for fast storage and combines outstanding space and write efficiency with acceptable read performance. As a result, MyRocks has the following advantages compared to other storage engines, if your workload uses fast storage, such as SSD:

  • Requires less storage space

  • Provides more storage endurance

  • Ensures better IO capacity

Important

The latest MyRocks storage engine is based on recent RocksDB source code, with minor modifications applied by Percona.

To check the RocksDB version currently used by MyRocks, run the following command:

$ head storage/rocksdb/rocksdb/HISTORY.md
Expected output 
...
## 8.5.1 (08/31/2023)
...

Percona MyRocks Installation Guide

MyRocks Limitations

Differences between Percona MyRocks and Facebook MyRocks

MyRocks Column Families

MyRocks Server Variables

MyRocks Information Schema Tables

Performance Schema MyRocks changes

Percona MyRocks installation guide

Percona MyRocks is distributed as a separate package that can be enabled as a plugin for Percona Server for MySQL 8.4 and later versions.

Note

File formats across different MyRocks variants may not be compatible. Percona Server for MySQL supports only Percona MyRocks. Migrating from one variant to another requires a logical data dump and reload.

Install Percona MyRocks

It is recommended to install Percona software from official repositories:

  1. Configure Percona repositories as described in Percona Software Repositories Documentation .

  2. Install Percona MyRocks using the corresponding package manager:

    $ sudo apt install percona-server-rocksdb

    $ sudo dnf install percona-server-rocksdb

After installation, you should see the following output:

Expected output 
* This release of |Percona Server| is distributed with RocksDB storage engine.
* Run the following script to enable the RocksDB storage engine in Percona Server:

$ ps-admin --enable-rocksdb -u <mysql_admin_user> -p[mysql_admin_pass] [-S <socket>] [-h <host> -P <port>]

Enable MyRocks with ps-admin

Run the ps-admin script as system root user or with sudo and provide the MySQL root user credentials to properly enable the RocksDB (MyRocks) storage engine:

$ sudo ps-admin --enable-rocksdb -u root -pPassw0rd
Expected output 
Checking if RocksDB plugin is available for installation ...
INFO: ha_rocksdb.so library for RocksDB found at /usr/lib64/mysql/plugin/ha_rocksdb.so.

Checking RocksDB engine plugin status...
INFO: RocksDB engine plugin is not installed.

Installing RocksDB engine...
INFO: Successfully installed RocksDB engine plugin.

Note

When you use the ps-admin script to enable Percona MyRocks, it performs the following:

  • Disables Transparent huge pages

  • Installs and enables the RocksDB plugin

If the script returns no errors, Percona MyRocks should be successfully enabled on the server. You can verify it as follows:

SHOW ENGINES;
Expected output 
+---------+---------+----------------------------------------------------------------------------+--------------+------+------------+
| Engine  | Support | Comment                                                                    | Transactions | XA   | Savepoints |
+---------+---------+----------------------------------------------------------------------------+--------------+------+------------+
| ROCKSDB | YES     | RocksDB storage engine                                                     | YES          | YES  | YES        |
...
| InnoDB  | DEFAULT | Percona-XtraDB, Supports transactions, row-level locking, and foreign keys | YES          | YES  | YES        |
+---------+---------+----------------------------------------------------------------------------+--------------+------+------------+
10 rows in set (0.00 sec)

Note that the RocksDB engine is not set to be default, new tables will still be created using the InnoDB (XtraDB) storage engine. To make RocksDB storage engine default, set default-storage-engine=rocksdb in the [mysqld] section of my.cnf and restart Percona Server for MySQL.

Alternatively, you can add ENGINE=RocksDB after the CREATE TABLE statement for every table that you create.

Install MyRocks plugins

You can install MyRocks manually with a series of INSTALL PLUGIN statements. You must have the INSERT privilege for the mysql.plugin system table.

The following statements install MyRocks:

INSTALL PLUGIN ROCKSDB SONAME 'ha_rocksdb.so';
INSTALL PLUGIN ROCKSDB_CFSTATS SONAME 'ha_rocksdb.so';
INSTALL PLUGIN ROCKSDB_DBSTATS SONAME 'ha_rocksdb.so';
INSTALL PLUGIN ROCKSDB_PERF_CONTEXT SONAME 'ha_rocksdb.so';
INSTALL PLUGIN ROCKSDB_PERF_CONTEXT_GLOBAL SONAME 'ha_rocksdb.so';
INSTALL PLUGIN ROCKSDB_CF_OPTIONS SONAME 'ha_rocksdb.so';
INSTALL PLUGIN ROCKSDB_GLOBAL_INFO SONAME 'ha_rocksdb.so';
INSTALL PLUGIN ROCKSDB_COMPACTION_HISTORY SONAME 'ha_rocksdb.so';
INSTALL PLUGIN ROCKSDB_COMPACTION_STATS SONAME 'ha_rocksdb.so';
INSTALL PLUGIN ROCKSDB_ACTIVE_COMPACTION_STATS SONAME 'ha_rocksdb.so';
INSTALL PLUGIN ROCKSDB_DDL SONAME 'ha_rocksdb.so';
INSTALL PLUGIN ROCKSDB_INDEX_FILE_MAP SONAME 'ha_rocksdb.so';
INSTALL PLUGIN ROCKSDB_LOCKS SONAME 'ha_rocksdb.so';
INSTALL PLUGIN ROCKSDB_TRX SONAME 'ha_rocksdb.so';
INSTALL PLUGIN ROCKSDB_DEADLOCK SONAME 'ha_rocksdb.so';

Remove Percona MyRocks

It will not be possible to access tables created using the RocksDB engine with another storage engine after you remove Percona MyRocks. If you need this data, alter the tables to another storage engine. For example, to alter the City table to InnoDB, run the following:

ALTER TABLE City ENGINE=InnoDB;

To disable and uninstall the RocksDB engine plugins, use the ps-admin script as follows:

$ sudo ps-admin --disable-rocksdb -u root -pPassw0rd
Expected output 
Checking RocksDB engine plugin status...
INFO: RocksDB engine plugin is installed.

Uninstalling RocksDB engine plugin...
INFO: Successfully uninstalled RocksDB engine plugin.

After the engine plugins have been uninstalled, remove the Percona MyRocks package:

=== "Debian or Ubuntu"

    ```shell
    $ sudo apt remove percona-server-rocksdb-8.4
    ```

=== "RHEL or derivatives"

    ```shell
    $ sudo yum remove percona-server-rocksdb-80.x86_64
    ```

Finally, remove all the MyRocks Server Variables from the configuration file (my.cnf) and restart Percona Server for MySQL.

Uninstall MyRocks plugins

You can uninstall the plugins for MyRocks. You must have the DELETE privilege for the mysql.plugin system table.

The following statements remove the MyRocks plugins:

UNINSTALL PLUGIN ROCKSDB;
UNINSTALL PLUGIN ROCKSDB_CFSTATS;
UNINSTALL PLUGIN ROCKSDB_DBSTATS;
UNINSTALL PLUGIN ROCKSDB_PERF_CONTEXT;
UNINSTALL PLUGIN ROCKSDB_PERF_CONTEXT_GLOBAL;
UNINSTALL PLUGIN ROCKSDB_CF_OPTIONS;
UNINSTALL PLUGIN ROCKSDB_GLOBAL_INFO;
UNINSTALL PLUGIN ROCKSDB_COMPACTION_HISTORY;
UNINSTALL PLUGIN ROCKSDB_COMPACTION_STATS;
UNINSTALL PLUGIN ROCKSDB_ACTIVE_COMPACTION_STATS;
UNINSTALL PLUGIN ROCKSDB_DDL;
UNINSTALL PLUGIN ROCKSDB_INDEX_FILE_MAP;
UNINSTALL PLUGIN ROCKSDB_LOCKS;
UNINSTALL PLUGIN ROCKSDB_TRX;
UNINSTALL PLUGIN ROCKSDB_DEADLOCK;

Updated supported features

The following is a list of the latest supported features:

  • Percona Server for MySQLsupports SELECT FOR UPDATE SKIP LOCKED/NOWAIT. The transaction isolation level must be READ COMMITTED.

  • Percona Server for MySQL adds the ability to cancel ongoing manual compactions. The cancel methods are the following:

    • Using either Control+C (from a session) or KILL (from another session) for client sessions running manual compactions by SET GLOBAL rocksdb_compact_cf (variable).

    • Using a global variable rocksdb_cancel_manual_compactions to cancel all ongoing manual compactions.

  • Percona Server for MySQL adds supported for Generated Columns and index are supported.

  • Percona Server for MySQL adds support for explicit DEFAULT value expressions .

MyRocks limitations

Online DDL limitations

MyRocks has limited support for Online DDL operations due to the lack of atomic DDL. As a result the schema changes are more restricted compared to InnoDB.

Traditional MyRocks DDL behavior

Operation type Examples ALGORITHM
Index operations ADD INDEX, DROP INDEX, RENAME INDEX INPLACE (always)
Column changes ADD COLUMN, DROP COLUMN, MODIFY COLUMN COPY (full table rebuild)
Metadata changes RENAME TABLE, some RENAME COLUMN operations May be INSTANT

Note: MyRocks does not support atomic DDL. Even metadata-only operations may require a full table rebuild, depending on the nature of the change.

Partition management support

MyRocks supports INPLACE partition management for certain operations:

ALTER TABLE t1 DROP PARTITION p1, ALGORITHM=INPLACE;
ALTER TABLE t1 ADD PARTITION (PARTITION p2 VALUES LESS THAN (MAXVALUE)), ALGORITHM=INPLACE;
These operations does not require a full table rebuild. However, operations that modify partitioning schemes, such as changing VALUES LESS THAN, fall back to the COPY algorithm.

Note: Dropping a partition permanently deletes any data stored in it unless that data is reassigned to another partition.

Instant DDL support

MyRocks provides limited Instant DDL support that is disabled by default, and you can activate the specific instant operations you need by setting the appropriate configuration variables.

Configuration variable Enables Instant DDL for
rocksdb_enable_instant_ddl_for_append_column=ON ALTER TABLE ... ADD COLUMN
rocksdb_enable_instant_ddl_for_column_default_changes=ON ALTER/MODIFY COLUMN … DEFAULT
rocksdb_enable_instant_ddl_for_drop_index_changes=ON ALTER TABLE ... DROP INDEX
rocksdb_enable_instant_ddl_for_table_comment_changes=ON ALTER TABLE ... COMMENT

Note: Instant DDL in MyRocks is applied only when both of the following conditions are met:

  • The configuration variable is set to ON.
  • The ALTER TABLE statement explicitly includes ALGORITHM=INSTANT.

For example:

SET GLOBAL rocksdb_enable_instant_ddl_for_table_comment_changes = ON;
ALTER TABLE my_table COMMENT = 'New comment', ALGORITHM=INSTANT;

If either condition is missing:

  • When the variable is ON but ALGORITHM=INSTANT is omitted, MyRocks falls back to the default (non‑instant) algorithm.
  • When the variable is OFF, any ALTER TABLE … ALGORITHM=INSTANT statement fails with an error.

Unsupported InnoDB features in MyRocks

You should also consider the following:

  • All collations are supported on CHAR and VARCHAR indexed columns. By default, MyRocks prevents creating indexes with non-binary collations (including latin1). You can optionally use it by setting rocksdb_strict_collation_exceptions to t1 (table names with regex format), but non-binary covering indexes other than latin1 (excluding german1) still require a primary key lookup to return the CHAR or VARCHAR column.

  • Either ORDER BY DESC or ORDER BY ASC is slow. This is because of “Prefix Key Encoding” feature in RocksDB. See https://www.slideshare.net/matsunobu/myrocks-deep-dive/58 for details. By default, ascending scan is faster and descending scan is slower. If the “reverse column family” is configured, then descending scan will be faster and ascending scan will be slower. Note that InnoDB also imposes a cost when the index is scanned in the opposite order.

  • When converting from large MyISAM/InnoDB tables, either by using the ALTER or INSERT INTO SELECT statements it’s recommended that you check the Data loading documentation and create MyRocks tables as below (in case the table is sufficiently big it will cause the server to consume all the memory and then be terminated by the OOM killer):

 SET session sql_log_bin=0;
 SET session rocksdb_bulk_load=1;
 ALTER TABLE large_myisam_table ENGINE=RocksDB;
 SET session rocksdb_bulk_load=0;
Expected output 
.. warning::

   If you are loading large data without enabling :ref:`rocksdb_bulk_load`
   or :ref:`rocksdb_commit_in_the_middle`, please make sure transaction
   ize is small enough. All modifications of the ongoing transactions are
   kept in memory.

  • With partitioned tables that use the MyRocks storage engine, the upgrade only works with native partitioning.

    See also

    MySQL Documentation: Preparing Your Installation for Upgrade

  • Percona Server for MySQL 8.4 and Unicode 9.0.0 standards have defined a change in the handling of binary collations. These collations are handled as NO PAD, trailing spaces are included in key comparisons. A binary collation comparison may result in two unique rows inserted and does not generate a`DUP_ENTRY` error. MyRocks key encoding and comparison does not account for this character set attribute.

Not supported on MyRocks

MyRocks does not support the following:

  • Operating as either a source or a replica in any replication topology that is not exclusively row-based. Statement-based and mixed-format binary logging is not supported. For more information, see Replication Formats .

  • Using multi-valued indexes . InnoDB supports this feature.

  • Using spatial data types .

  • Using the Clone Plugin and the Clone Plugin API. InnoDB supports either these features.

  • Using encryption in tables. At this time, during an ALTER TABLE operation, MyRocks mistakenly detects all InnoDB tables as encrypted. Therefore, any attempt to ALTER an InnoDB table to MyRocks fails.

    As a workaround, we recommend a manual move of the table. The following steps are the same as the ALTER TABLE ... ENGINE=... process:

    • Use SHOW CREATE TABLE ... to return the InnoDB table definition.

    • With the table definition as the source, perform a CREATE TABLE ... ENGINE=RocksDB.

    • In the new table, use INSERT INTO <new table> SELECT \* FROM <old table>.

    Note

    With MyRocks and with large tables, it is recommended to set the session variable rocksdb_bulk_load=1 during the load to prevent running out of memory. This recommendation is because of the MyRocks large transaction limitation. For more information, see MyRocks Data Loading

Differences between Percona MyRocks and Facebook MyRocks

The original MyRocks was developed by Facebook and works with their implementation of MySQL. Percona MyRocks is a branch of MyRocks for Percona Server for MySQL and includes the following differences from the original implementation:

Storage Engine Transaction isolation level
READ COMMITTED REPEATABLE READ
InnoDB Success Success
Facebook MyRocks Fail Success (MyRocks engine only; read-only, as all MyRocks engine snapshots)
Percona MyRocks Fail with any DML which would violate the read-only snapshot constraint Success (read-only snapshots independent of the engines in use)
  • Percona MyRocks includes the lz4 and zstd statically linked libraries.

MyRocks Information Schema tables

When you install the MyRocks plugin for MySQL, the Information Schema is extended to include the following tables:

ROCKSDB_GLOBAL_INFO

Columns

Column Name Type
TYPE varchar(513)
NAME varchar(513)
VALUE varchar(513)

ROCKSDB_CFSTATS

Columns

Column Name Type
CF_NAME varchar(193)
STAT_TYPE varchar(193)
VALUE bigint(8)

ROCKSDB_TRX

This table stores mappings of RocksDB transaction identifiers to MySQL client identifiers to enable associating a RocksDB transaction with a MySQL client operation.

Columns

Column Name Type
TRANSACTION_ID bigint(8)
STATE varchar(193)
NAME varchar(193)
WRITE_COUNT bigint(8)
LOCK_COUNT bigint(8)
TIMEOUT_SEC int(4)
WAITING_KEY varchar(513)
WAITING_COLUMN_FAMILY_ID int(4)
IS_REPLICATION int(4)
SKIP_TRX_API int(4)
READ_ONLY int(4)
HAS_DEADLOCK_DETECTION int(4)
NUM_ONGOING_BULKLOAD int(4)
THREAD_ID int(8)
QUERY varchar(193)

ROCKSDB_CF_OPTIONS

Columns

Column Name Type
CF_NAME varchar(193)
OPTION_TYPE varchar(193)
VALUE varchar(193)

ROCKSDB_ACTIVE_COMPACTION_STATS

Columns

Column Name Type
THREAD_ID bigint
CF_NAME varchar(193)
INPUT_FILES varchar(513)
OUTPUT_FILES varchar(513)
COMPACTION_REASON varchar(513)

ROCKSDB_COMPACTION_HISTORY

Columns

Column Name Type
THREAD_ID bigint
CF_NAME varchar(513)
INPUT_LEVEL integer
OUTPUT_LEVEL integer
INPUT_FILES varchar(513)
OUTPUT_FILES varchar(513)
COMPACTION_REASON varchar(513)
START_TIMESTAMP bigint
END_TIMESTAMP bigint

ROCKSDB_COMPACTION_STATS

Columns

Column Name Type
CF_NAME varchar(193)
LEVEL varchar(513)
TYPE varchar(513)
VALUE double

ROCKSDB_DBSTATS

Columns

Column Name Type
STAT_TYPE varchar(193)
VALUE bigint(8)

ROCKSDB_DDL

Columns

Column Name Type
TABLE_SCHEMA varchar(193)
TABLE_NAME varchar(193)
PARTITION_NAME varchar(193)
INDEX_NAME varchar(193)
COLUMN_FAMILY int(4)
INDEX_NUMBER int(4)
INDEX_TYPE smallint(2)
KV_FORMAT_VERSION smallint(2)
TTL_DURATION bigint(8)
INDEX_FLAGS bigint(8)
CF varchar(193)
AUTO_INCREMENT bigint(8) unsigned

ROCKSDB_INDEX_FILE_MAP

Columns

Column Name Type
COLUMN_FAMILY int(4)
INDEX_NUMBER int(4)
SST_NAME varchar(193)
NUM_ROWS bigint(8)
DATA_SIZE bigint(8)
ENTRY_DELETES bigint(8)
ENTRY_SINGLEDELETES bigint(8)
ENTRY_MERGES bigint(8)
ENTRY_OTHERS bigint(8)
DISTINCT_KEYS_PREFIX varchar(400)

ROCKSDB_LIVE_FILES_METADATA

Column Name Type
CF_NAME varchar(193)
LEVEL varchar(513)
NAME varchar(513)
DB_PATH varchar(513)
FILE_NUMBER bigint
FILE_TYPE varchar(193)
SIZE bigint
RELATIVE_FILENAME varchar(193)
DIRECTORY varchar(513)
TEMPERATURE varchar(193)
FILE_CHECKSUM varchar(513)
FILE_CHECKSUM_FUNC_NAME varchar(193)
SMALLEST_SEQNO bigint
LARGEST_SEQNO bigint
SMALLEST_KEY varchar(513)
LARGEST_KEY varchar(513)
NUM_READS_SAMPLED bigint
BEING_COMPACTED tinyint
NUM_ENTRIES bigint
NUM_DELETIONS bigint
OLDEST_BLOB_FILE_NUMBER bigint
OLDEST_ANCESTER_TIME bigint
FILE_CREATION_TIME bigint

ROCKSDB_LOCKS

This table contains the set of locks granted to MyRocks transactions.

Columns

Column Name Type
COLUMN_FAMILY_ID int(4)
TRANSACTION_ID bigint
KEY varchar(513)
MODE varchar(32)

ROCKSDB_PERF_CONTEXT

Columns

Column Name Type
TABLE_SCHEMA varchar(193)
TABLE_NAME varchar(193)
PARTITION_NAME varchar(193)
STAT_TYPE varchar(193)
VALUE bigint(8)

ROCKSDB_PERF_CONTEXT_GLOBAL

Columns

Column Name Type
STAT_TYPE varchar(193)
VALUE bigint(8)

ROCKSDB_DEADLOCK

This table records information about deadlocks.

Columns

Column Name Type
DEADLOCK_ID bigint(8)
TRANSACTION_ID bigint(8)
CF_NAME varchar(193)
WAITING_KEY varchar(513)
LOCK_TYPE varchar(193)
INDEX_NAME varchar(193)
TABLE_NAME varchar(193)
ROLLED_BACK bigint(8)

MyRocks server variables

The MyRocks server variables expose configuration of the underlying RocksDB engine. There several ways to set these variables:

  • For production deployments, you should have all variables defined in the configuration file.

  • Dynamic variables can be changed at runtime using the SET statement.

  • If you want to test things out, you can set some of the variables when starting mysqld using corresponding command-line options.

If a variable was not set in either the configuration file or as a command-line option, the default value is used.

Also, all variables can exist in one or both of the following scopes:

  • Global scope defines how the variable affects overall server operation.

  • Session scope defines how the variable affects operation for individual client connections.

Variable table

Variable Name
rocksdb_access_hint_on_compaction_start
rocksdb_advise_random_on_open
rocksdb_allow_concurrent_memtable_write
rocksdb_allow_mmap_reads
rocksdb_allow_mmap_writes
rocksdb_allow_to_start_after_corruption
rocksdb_allow_unsafe_alter
rocksdb_alter_column_default_inplace
rocksdb_alter_table_comment_inplace
rocksdb_base_background_compactions
rocksdb_blind_delete_primary_key
rocksdb_block_cache_numshardbits
rocksdb_block_cache_size
rocksdb_block_restart_interval
rocksdb_block_size
rocksdb_block_size_deviation
rocksdb_bulk_load
rocksdb_bulk_load_allow_sk
rocksdb_bulk_load_allow_unsorted
rocksdb_bulk_load_compression_parallel_threads
rocksdb_bulk_load_enable_unique_key_check
rocksdb_bulk_load_fail_if_not_bottommost_level
rocksdb_bulk_load_partial_index
rocksdb_bulk_load_size
rocksdb_bulk_load_use_sst_partitioner
rocksdb_bytes_per_sync
rocksdb_cache_dump
rocksdb_cache_high_pri_pool_ratio
rocksdb_cache_index_and_filter_blocks
rocksdb_cache_index_and_filter_with_high_priority
rocksdb_cancel_manual_compactions
rocksdb_charge_memory
rocksdb_check_iterate_bounds
rocksdb_checksums_pct
rocksdb_collect_sst_properties
rocksdb_column_default_value_as_expression
rocksdb_commit_in_the_middle
rocksdb_commit_time_batch_for_recovery
rocksdb_compact_cf
rocksdb_compact_lzero_now
rocksdb_compaction_readahead_size
rocksdb_compaction_sequential_deletes
rocksdb_compaction_sequential_deletes_count_sd
rocksdb_compaction_sequential_deletes_file_size
rocksdb_compaction_sequential_deletes_window
rocksdb_concurrent_prepare
rocksdb_converter_record_cached_length
rocksdb_corrupt_data_action
rocksdb_create_checkpoint
rocksdb_create_if_missing
rocksdb_create_missing_column_families
rocksdb_create_temporary_checkpoint
rocksdb_datadir
rocksdb_db_write_buffer_size
rocksdb_deadlock_detect
rocksdb_deadlock_detect_depth
rocksdb_debug_cardinality_multiplier
rocksdb_debug_manual_compaction_delay
rocksdb_debug_optimizer_no_zero_cardinality
rocksdb_debug_skip_bloom_filter_check_on_iterator_bounds
rocksdb_debug_ttl_ignore_pk
rocksdb_debug_ttl_read_filter_ts
rocksdb_debug_ttl_rec_ts
rocksdb_debug_ttl_snapshot_ts
rocksdb_default_cf_options
rocksdb_delayed_write_rate
rocksdb_delete_cf
rocksdb_delete_obsolete_files_period_micros
rocksdb_disable_file_deletions
rocksdb_disable_instant_ddl
rocksdb_enable_bulk_load_api
rocksdb_enable_delete_range_for_drop_index
rocksdb_enable_insert_with_update_caching
rocksdb_enable_instant_ddl
rocksdb_enable_instant_ddl_for_append_column
rocksdb_enable_instant_ddl_for_column_default_changes
rocksdb_enable_instant_ddl_for_drop_index_changes
rocksdb_enable_instant_ddl_for_table_comment_changes
rocksdb_enable_iterate_bounds
rocksdb_enable_pipelined_write
rocksdb_enable_remove_orphaned_dropped_cfs
rocksdb_enable_thread_tracking
rocksdb_enable_ttl
rocksdb_enable_ttl_read_filtering
rocksdb_enable_udt_in_mem
rocksdb_enable_write_thread_adaptive_yield
rocksdb_error_if_exists
rocksdb_error_on_suboptimal_collation
rocksdb_file_checksums
rocksdb_flush_log_at_trx_commit
rocksdb_flush_memtable_on_analyze
rocksdb_force_compute_memtable_stats
rocksdb_force_compute_memtable_stats_cachetime
rocksdb_force_flush_memtable_and_lzero_now
rocksdb_force_flush_memtable_now
rocksdb_force_index_records_in_range
rocksdb_hash_index_allow_collision
rocksdb_ignore_unknown_options
rocksdb_index_type
rocksdb_info_log_level
rocksdb_invalid_create_option_action
rocksdb_io_error_action
rocksdb_is_fd_close_on_exec
rocksdb_keep_log_file_num
rocksdb_large_prefix
rocksdb_lock_scanned_rows
rocksdb_lock_wait_timeout
rocksdb_log_file_time_to_roll
rocksdb_manifest_preallocation_size
rocksdb_manual_compaction_bottommost_level
rocksdb_manual_compaction_threads
rocksdb_manual_wal_flush
rocksdb_master_skip_tx_api
rocksdb_max_background_compactions
rocksdb_max_background_flushes
rocksdb_max_background_jobs
rocksdb_max_bottom_pri_background_compactions
rocksdb_max_compaction_history
rocksdb_max_file_opening_threads
rocksdb_max_latest_deadlocks
rocksdb_max_log_file_size
rocksdb_max_manifest_file_size
rocksdb_max_manual_compactions
rocksdb_max_open_files
rocksdb_max_row_locks
rocksdb_max_subcompactions
rocksdb_max_total_wal_size
rocksdb_merge_buf_size
rocksdb_merge_combine_read_size
rocksdb_merge_tmp_file_removal_delay_ms
rocksdb_new_table_reader_for_compaction_inputs
rocksdb_no_block_cache
rocksdb_no_create_column_family
rocksdb_override_cf_options
rocksdb_paranoid_checks
rocksdb_partial_index_blind_delete
rocksdb_partial_index_ignore_killed
rocksdb_partial_index_sort_max_mem
rocksdb_pause_background_work
rocksdb_perf_context_level
rocksdb_persistent_cache_path
rocksdb_persistent_cache_size_mb
rocksdb_pin_l0_filter_and_index_blocks_in_cache
rocksdb_print_snapshot_conflict_queries
rocksdb_protection_bytes_per_key
rocksdb_rate_limiter_bytes_per_sec
rocksdb_read_free_rpl
rocksdb_read_free_rpl_tables
rocksdb_records_in_range
rocksdb_reset_stats
rocksdb_rollback_on_timeout
rocksdb_rpl_skip_tx_api
rocksdb_seconds_between_stat_computes
rocksdb_signal_drop_index_thread
rocksdb_sim_cache_size
rocksdb_skip_bloom_filter_on_read
rocksdb_skip_fill_cache
rocksdb_skip_locks_if_skip_unique_check
rocksdb_sst_mgr_rate_bytes_per_sec
rocksdb_stats_dump_period_sec
rocksdb_stats_level
rocksdb_stats_recalc_rate
rocksdb_store_row_debug_checksums
rocksdb_strict_collation_check
rocksdb_strict_collation_exceptions
rocksdb_table_cache_numshardbits
rocksdb_table_stats_background_thread_nice_value
rocksdb_table_stats_max_num_rows_scanned
rocksdb_table_stats_recalc_threshold_count
rocksdb_table_stats_recalc_threshold_pct
rocksdb_table_stats_sampling_pct
rocksdb_table_stats_skip_system_cf
rocksdb_table_stats_use_table_scan
rocksdb_tmpdir
rocksdb_trace_block_cache_access
rocksdb_trace_queries
rocksdb_trace_sst_api
rocksdb_track_and_verify_wals_in_manifest
rocksdb_two_write_queues
rocksdb_unsafe_for_binlog
rocksdb_update_cf_options
rocksdb_use_adaptive_mutex
rocksdb_use_default_sk_cf
rocksdb_use_direct_io_for_flush_and_compaction
rocksdb_use_direct_reads
rocksdb_use_fsync
rocksdb_use_hyper_clock_cache
rocksdb_use_io_uring
rocksdb_use_write_buffer_manager
rocksdb_validate_tables
rocksdb_verify_row_debug_checksums
rocksdb_wal_bytes_per_sync
rocksdb_wal_dir
rocksdb_wal_recovery_mode
rocksdb_wal_size_limit_mb
rocksdb_wal_ttl_seconds
rocksdb_whole_key_filtering
rocksdb_write_batch_flush_threshold
rocksdb_write_batch_max_bytes
rocksdb_write_disable_wal
rocksdb_write_ignore_missing_column_families
rocksdb_write_policy

Variable definitions

rocksdb_access_hint_on_compaction_start

Option Description
Command-line –rocksdb-access-hint-on-compaction-start
Dynamic No
Scope Global
Data type String or numeric
Default NORMAL or 1

[Removed: 8.4.5-5] As of Percona Server for MySQL 8.4.5-5, the rocksdb-access-hint-on-compaction-start variable has been removed.

Specifies the file access pattern once a compaction is started, applied to all input files of a compaction. Possible values are:

  • 0 = NONE

  • 1 = NORMAL (default)

  • 2 = SEQUENTIAL

  • 3 = WILLNEED

rocksdb_advise_random_on_open

Option Description
Command-line –rocksdb-advise-random-on-open
Dynamic No
Scope Global
Data type Boolean
Default ON

Specifies whether to hint the underlying file system that the file access pattern is random, when a data file is opened. Enabled by default.

rocksdb_allow_concurrent_memtable_write

Option Description
Command-line –rocksdb-allow-concurrent-memtable-write
Dynamic No
Scope Global
Data type Boolean
Default OFF

Specifies whether to allow multiple writers to update memtables in parallel. Disabled by default.

rocksdb_allow_mmap_reads

Option Description
Command-line –rocksdb-allow-mmap-reads
Dynamic No
Scope Global
Data type Boolean
Default OFF

Specifies whether to allow the OS to map a data file into memory for reads. Disabled by default. If you enable this, make sure that rocksdb_use_direct_reads is disabled.

rocksdb_allow_mmap_writes

Option Description
Command-line –rocksdb-allow-mmap-writes
Dynamic No
Scope Global
Data type Boolean
Default OFF

Specifies whether to allow the OS to map a data file into memory for writes. Disabled by default.

rocksdb_allow_to_start_after_corruption

Option Description
Command-line –rocksdb_allow_to_start_after_corruption
Dynamic No
Scope Global
Data type Boolean
Default OFF

Specifies whether to allow server to restart once MyRocks reported data corruption. Disabled by default.

Once corruption is detected server writes marker file (named ROCKSDB_CORRUPTED) in the data directory and aborts. If marker file exists, then mysqld exits on startup with an error message. The restart failure will continue until the problem is solved or until mysqld is started with this variable turned on in the command line.

Note

Not all memtables support concurrent writes.

rocksdb_allow_unsafe_alter

Option Description
Command-line –rocksdb-allow-unsafe-alter
Dynamic No
Scope Global
Data type Boolean
Default OFF

Enables crash unsafe INPLACE ADD|DROP partition.

rocksdb_alter_column_default_inplace

Option Description
Command-line –rocksdb-alter-column-default-inplace
Dynamic Yes
Scope Global
Data type Boolean
Default ON

Allows an inplace alter for the ALTER COLUMN default operation.

rocksdb_alter_table_comment_inplace

Option Description
Command-line –rocksdb_alter_table_comment_inplace
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

Allows changing ALTER TABLE COMMENT inplace.

This variable is disabled (OFF) by default.

rocksdb_base_background_compactions

Option Description
Command-line –rocksdb-base-background-compactions
Dynamic No
Scope Global
Data type Numeric
Default 1

Specifies the suggested number of concurrent background compaction jobs, submitted to the default LOW priority thread pool in RocksDB. The default is 1. The allowed range of values is from -1 to 64. The maximum value depends on the rocksdb_max_background_compactions variable. This variable was replaced with rocksdb_max_background_jobs, which automatically decides how many threads to allocate toward flush/compaction.

rocksdb_blind_delete_primary_key

Option Description
Command-line –rocksdb-blind-delete-primary-key
Dynamic Yes
Scope Global, Session
Data type Boolean
Default OFF

Skips verifying if rows exists before executing deletes. The following conditions must be met:

  • The variable is enabled

  • Only a single table listed in the DELETE statement

  • The table has only a primary key with no secondary keys

rocksdb_block_cache_numshardbits

Option Description
Command-line –rocksdb-block-cache-numshardbits
Dynamic No
Scope Global
Data type Numeric
Default -1

This variable specifies the number of shards ,numShardBits, for the block cache in RocksDB. The cache is sharded into 2^numShardBits shards by the key hash.

The default value is -1. The -1 value means that RocksDB automatically determines the number of shards for the block cache based on the cache capacity.

The minimum value is -1 and the maximum value is 8.

rocksdb_block_cache_size

Option Description
Command-line –rocksdb-block-cache-size
Dynamic Yes
Scope Global
Data type Numeric
Default 536870912

This variable sets the RocksDB LRU block cache size. This memory is reserved for the block cache and supplements any filesystem caching.

The minimum value is 1024, representing the size of a single block.

The default value is 536870912.

The maximum value is 9223372036854775807.

rocksdb_block_restart_interval

Option Description
Command-line –rocksdb-block-restart-interval
Dynamic No
Scope Global
Data type Numeric
Default 16

Specifies the number of keys for each set of delta encoded data. Default value is 16. Allowed range is from 1 to 2147483647.

rocksdb_block_size

Option Description
Command-line –rocksdb-block-size
Dynamic No
Scope Global
Data type Numeric
Default 16 KB

Specifies the size of the data block for reading RocksDB data files. The default value is 16 KB. The allowed range is from 1024 to 18446744073709551615 bytes.

Version changes

In Percona Server for MySQL 8.4.7-7, the maximum value remains 18446744073709551615 bytes.

rocksdb_block_size_deviation

Option Description
Command-line –rocksdb-block-size-deviation
Dynamic No
Scope Global
Data type Numeric
Default 10

Specifies the threshold for free space allowed in a data block (see rocksdb_block_size). If there is less space remaining, close the block (and write to new block). Default value is 10, meaning that the block is not closed until there is less than 10 bits of free space remaining.

Allowed range is from 1 to 2147483647.

rocksdb_bulk_load

Option Description
Command-line –rocksdb-bulk-load
Dynamic Yes
Scope Global, Session
Data type Boolean
Default OFF

Specifies whether to use bulk load: MyRocks will ignore checking keys for uniqueness or acquiring locks during transactions. Disabled by default. Enable this only if you are certain that there are no row conflicts, for example, when setting up a new MyRocks instance from a MySQL dump.

When the rocksdb_bulk_load variable is enabled, it behaves as if the variable rocksdb_commit_in_the_middle is enabled, even if the variable rocksdb_commit_in_the_middle is disabled.

rocksdb_bulk_load_allow_sk

Option Description
Command-line –rocksdb-bulk-load-allow-sk
Dynamic Yes
Scope Global, Session
Data type Boolean
Default OFF

Enabling this variable allows secondary keys to be added using the bulk loading feature. This variable can be enabled or disabled only when the rocksdb_bulk_load is OFF.

rocksdb_bulk_load_allow_unsorted

Option Description
Command-line –rocksdb-bulk-load-allow-unsorted
Dynamic Yes
Scope Global, Session
Data type Boolean
Default OFF

By default, the bulk loader requires its input to be sorted in the primary key order. If enabled, unsorted inputs are allowed too, which are then sorted by the bulkloader itself, at a performance penalty.

rocksdb_bulk_load_compression_parallel_threads

Option Description
Dynamic Yes
Scope Global, Session
Data type Numeric
Default 1
Minimum 1
Maximum 1024

Added in Percona Server for MySQL 8.4.5-5

Specifies the number of parallel worker threads used to compress SST data blocks during bulk load.

rocksdb_bulk_load_enable_unique_key_check

Option Description
Dynamic Yes
Scope Global, Session
Data type Bool
Default OFF

Added in Percona Server for MySQL 8.4.5-5.

Controls whether the unique key constraint is checked during bulk loading. This setting can only be changed when bulk loading is disabled.

rocksdb_bulk_load_fail_if_not_bottommost_level

Option Description
Command-line –rocksdb_bulk_load_fail_if_not_bottommost_level
Dynamic Yes
Scope Global, Session
Data type Boolean
Default OFF

When this variable is enabled, the bulk load fails if an sst file created during bulk load cannot be placed to the bottommost level in the rocksdb.

This variable can be enabled or disabled only when the rocksdb_bulk_load is OFF.

This variable is disabled (OFF) by default.

Warning

When rocksdb_bulk_load_fail_if_not_bottommost_level is disabled, it may cause severe performance impact.

rocksdb_bulk_load_partial_index

Option Description
Command-line –rocksdb-bulk-load-partial-index
Dynamic Yes
Scope Local
Data type Boolean
Default ON

Materializes partial index during bulk load instead of leaving the index empty.

rocksdb_bulk_load_size

Option Description
Command-line –rocksdb-bulk-load-size
Dynamic Yes
Scope Global, Session
Data type Numeric
Default 1000

Specifies the number of keys to accumulate before committing them to the storage engine when bulk load is enabled (see rocksdb_bulk_load). Default value is 1000, which means that a batch can contain up to 1000 records before they are implicitly committed. Allowed range is from 1 to 1073741824.

rocksdb_bulk_load_use_sst_partitioner

Option Description
Command-line –rocksdb_bulk_load_use_sst_partitioner
Dynamic Yes
Scope Global, Session
Data type Boolean
Default OFF

If enabled, this variable uses sst partitioner to split sst files to ensure bulk load sst files can be ingested to bottommost level.

This variable is disabled (OFF) by default.

rocksdb_bytes_per_sync

Option Description
Command-line –rocksdb-bytes-per-sync
Dynamic Yes
Scope Global
Data type Numeric
Default 0

Specifies how often should the OS sync files to disk as they are being written, asynchronously, in the background. This operation can be used to smooth out write I/O over time. Default value is 0 meaning that files are never synced. Allowed range is up to 18446744073709551615.

rocksdb_cache_dump

Option Description
Command-line –rocksdb-cache-dump
Dynamic No
Scope Global
Data type Boolean
Default ON

Includes RocksDB block cache content in core dump. This variable is enabled by default.

rocksdb_cache_high_pri_pool_ratio

Option Description
Command-line –rocksdb-cache-high-pri-pool-ratio
Dynamic No
Scope Global
Data type Double
Default 0.0

This variable specifies the size of the block cache high-pri pool. The default value and minimum value is 0.0. The maximum value is 1.0.

rocksdb_cache_index_and_filter_blocks

Option Description
Command-line –rocksdb-cache-index-and-filter-blocks
Dynamic No
Scope Global
Data type Boolean
Default ON

Specifies whether RocksDB should use the block cache for caching the index and bloomfilter data blocks from each data file. Enabled by default. If you disable this feature, RocksDB allocates additional memory to maintain these data blocks.

rocksdb_cache_index_and_filter_with_high_priority

Option Description
Command-line –rocksdb-cache-index-and-filter-with-high-priority
Dynamic No
Scope Global
Data type Boolean
Default ON

Specifies whether RocksDB should use the block cache with high priority for caching the index and bloomfilter data blocks from each data file. Enabled by default. If you disable this feature, RocksDB allocates additional memory to maintain these data blocks.

rocksdb_cancel_manual_compactions

Option Description
Command-line –rocksdb-cancel-manual-compactions
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

Cancels all ongoing manual compactions.

rocksdb_charge_memory

Option Description
Command-line –rocksdb_charge_memory
Dynamic No
Scope Global
Data type Boolean
Default OFF

This variable is tech preview and may be removed in the future releases.

Turns on RocksDB memory-charging related features (BlockBasedTableOptions::cache_usage_options.options.charged) from cnf files. This variable is related to rocksdb_use_write_buffer_manager.

This variable is disabled (OFF) by default.

rocksdb_check_iterate_bounds

Option Description
Command-line –rocksdb-check-iterate-bounds
Dynamic Yes
Scope Global, Session
Data type Boolean
Default ON

This variable enables checking the upper and lower bounds of the RocksDB iterator during iteration. The default value in ON which means this variable is enabled.

rocksdb_checksums_pct

Option Description
Command-line –rocksdb-checksums-pct
Dynamic Yes
Scope Global, Session
Data type Numeric
Default 100

Specifies the percentage of rows to be checksummed. Default value is 100 (checksum all rows). Allowed range is from 0 to 100.

rocksdb_collect_sst_properties

Option Description
Command-line –rocksdb-collect-sst-properties
Dynamic No
Scope Global
Data type Boolean
Default ON

Specifies whether to collect statistics on each data file to improve optimizer behavior. Enabled by default.

rocksdb_column_default_value_as_expression

Option Description
Command-line –rocksdb_column_default_value_as_expression
Dynamic Yes
Scope Global
Data type Boolean
Default ON

Allows to set a function as the default value for a column.

This variable is enabled (ON) by default.

rocksdb_commit_in_the_middle

Option Description
Command-line –rocksdb-commit-in-the-middle
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

Specifies whether to commit rows implicitly when a batch contains more than the value of rocksdb_bulk_load_size.

This option should only be enabled at the time of data import because it may cause locking errors.

This variable is disabled by default. When the rocksdb_bulk_load variable is enabled, it behaves as if the variable rocksdb_commit_in_the_middle is enabled, even if the variable rocksdb_commit_in_the_middle is disabled.

rocksdb_commit_time_batch_for_recovery

Option Description
Command-line –rocksdb-commit-time-batch-for-recovery
Dynamic Yes
Scope Global, Session
Data type Boolean
Default OFF

Specifies whether to write the commit time write batch into the database or not.

Note

If the commit time write batch is only useful for recovery, then writing to WAL is enough.

rocksdb_compact_cf

Option Description
Command-line –rocksdb-compact-cf
Dynamic Yes
Scope Global
Data type String
Default

Specifies the name of the column family to compact.

rocksdb_compact_lzero_now

Option Description
Command-line –rocksdb-compact-lzero-now
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

This variable acts as a trigger. Set the variable to ON, rocksdb-compact-lzero-now=ON, to immediately compact all the Level 0 (L0) files. After all the L0 files are compacted, the variable value automatically switches to OFF.

rocksdb_compaction_readahead_size

[Changed in 8.4.5-5] The default value of rocksdb_compaction_readahead_size has changed from 0 to 2097152.

Option Description
Command-line –rocksdb-compaction-readahead-size
Dynamic Yes
Scope Global
Data type Numeric
Default 2097152 (Prior to 8.4.5-5, the default was 0)

Specifies the size of reads to perform ahead of compaction. The default value is now 2097152. Set this to at least 2 megabytes (16777216) when using MyRocks with spinning disks to ensure sequential reads instead of random. Maximum allowed value is 18446744073709551615.

Note

If you set this variable to a non-zero value, rocksdb_new_table_reader_for_compaction_inputs is enabled.

rocksdb_compaction_sequential_deletes

Option Description
Command-line –rocksdb-compaction-sequential-deletes
Dynamic Yes
Scope Global
Data type Numeric
Default 149999

Specifies the threshold to trigger compaction on a file if it has more than this number of sequential delete markers.

The default value is 149999.

Maximum allowed value is 2000000 (two million delete markers).

Note

Depending on workload patterns, MyRocks can potentially maintain large numbers of delete markers, which increases latency of queries. This compaction feature will reduce latency, but may also increase the MyRocks write rate. Use this variable together with rocksdb_compaction_sequential_deletes_file_size to only perform compaction on large files.

rocksdb_compaction_sequential_deletes_count_sd

Option Description
Command-line –rocksdb-compaction-sequential-deletes-count-sd
Dynamic Yes
Scope Global
Data type Boolean
Default ON

Specifies whether to count single deletes as delete markers recognized by rocksdb_compaction_sequential_deletes.

The default value is ON which means the variable is enabled.

rocksdb_compaction_sequential_deletes_file_size

Option Description
Command-line –rocksdb-compaction-sequential-deletes-file-size
Dynamic Yes
Scope Global
Data type Numeric
Default 0

Specifies the minimum file size required to trigger compaction on it by rocksdb_compaction_sequential_deletes. Default value is 0, meaning that compaction is triggered regardless of file size. Allowed range is from -1 to 9223372036854775807.

rocksdb_compaction_sequential_deletes_window

Option Description
Command-line –rocksdb-compaction-sequential-deletes-window
Dynamic Yes
Scope Global
Data type Numeric
Default 150000

Specifies the size of the window for counting delete markers by rocksdb_compaction_sequential_deletes. Default value is 150000.

Allowed range is up to 2000000 (two million).

rocksdb_concurrent_prepare

Option Description
Command-line –rocksdb-concurrent_prepare
Dynamic No
Scope Global
Data type Boolean
Default ON

When enabled this variable allows/encourages threads that are using two-phase commit to prepare in parallel. This variable was renamed in upstream to rocksdb_two_write_queues.

rocksdb_converter_record_cached_length

Option Description
Command-line –rocksdb_converter_record_cached_length
Dynamic Yes
Scope Global
Data type Numeric
Default 0

Specifies the maximum number of bytes to cache on table handler for encoding table record data.

If the used memory exceeds rocksdb_converter_record_cached_length, the memory is released when the handler is returned to the table handler cache.

The minimum value is 0 (zero) that means there is no limit. The maximum value is UINT64_MAX (0xffffffffffffffff).

The default value is 0(zero) that means there is no limit.

rocksdb_corrupt_data_action

Option Description
Command-line –rocksdb_corrupt_data_action
Dynamic Yes
Scope Global
Data type enum { ERROR = 0, ABORT_SERVER, WARNING };
Default ERROR

This variable controls the behavior when hitting the data corruption in MyRocks.

You can select one of the following actions:

  • ERROR - fail the query with the error HA_ERR_ROCKSDB_CORRUPT_DATA

  • ABORT_SERVER - crash the server

  • WARNING - pass the query with warning

The default value is ERROR that means the query fails with the error HA_ERR_ROCKSDB_CORRUPT_DATA.

rocksdb_create_checkpoint

Option Description
Command-line –rocksdb-create-checkpoint
Dynamic Yes
Scope Global
Data type String
Default

Specifies the directory where MyRocks should create a checkpoint. Empty by default.

rocksdb_create_if_missing

Option Description
Command-line –rocksdb-create-if-missing
Dynamic No
Scope Global
Data type Boolean
Default ON

Specifies whether MyRocks should create its database if it does not exist. Enabled by default.

rocksdb_create_missing_column_families

Option Description
Command-line –rocksdb-create-missing-column-families
Dynamic No
Scope Global
Data type Boolean
Default OFF

Specifies whether MyRocks should create new column families if they do not exist. Disabled by default.

rocksdb_create_temporary_checkpoint

Option Description
Command-line –rocksdb-create-temporary-checkpoint
Dynamic Yes
Scope Session
Data type String

When specified it will create a temporary RocksDB ‘checkpoint’ or ‘snapshot’ in the datadir. If the session ends with an existing checkpoint, or if the variable is reset to another value, the checkpoint will get removed. This variable should be used by backup tools. Prolonged use or other misuse can have serious side effects to the server instance.

rocksdb_datadir

Option Description
Command-line –rocksdb-datadir
Dynamic No
Scope Global
Data type String
Default ./.rocksdb

Specifies the location of the MyRocks data directory. By default, it is created in the current working directory.

rocksdb_db_write_buffer_size

Option Description
Command-line –rocksdb-db-write-buffer-size
Dynamic No
Scope Global
Data type Numeric
Default 0

Specifies the maximum size of all memtables used to store writes in MyRocks across all column families. When this size is reached, the data is flushed to persistent media. The default value is 0. The allowed range is up to 18446744073709551615.

rocksdb_deadlock_detect

Option Description
Command-line –rocksdb-deadlock-detect
Dynamic Yes
Scope Global, Session
Data type Boolean
Default OFF

Specifies whether MyRocks should detect deadlocks. Disabled by default.

rocksdb_deadlock_detect_depth

Option Description
Command-line –rocksdb-deadlock-detect-depth
Dynamic Yes
Scope Global, Session
Data type Numeric
Default 50

Specifies the number of transactions deadlock detection will traverse through before assuming deadlock.

rocksdb_debug_cardinality_multiplier

Option Description
Command-line –rocksdb-debug-cardinality-multiplier
Dynamic Yes
Scope Global
Data type UINT
Default 2

The cardinality multiplier used in tests. The minimum value is 0. The maximum value is 2147483647 (INT_MAX).

Version changes

In Percona Server for MySQL 8.4.7-7, the minimum value was changed to 1.

rocksdb_debug_manual_compaction_delay

Option Description
Command-line –rocksdb-debug-manual-compaction-delay
Dynamic Yes
Scope Global
Data type UINT
Default 0

Only use this variable when debugging.

This variable specifies a sleep, in seconds, to simulate long-running compactions. The minimum value is 0. The maximum value is 4292967295 (UINT_MAX).

rocksdb_debug_optimizer_no_zero_cardinality

Option Description
Command-line –rocksdb-debug-optimizer-no-zero-cardinality
Dynamic Yes
Scope Global
Data type Boolean
Default ON

Specifies whether MyRocks should prevent zero cardinality by always overriding it with some value.

rocksdb_debug_skip_bloom_filter_check_on_iterator_bounds

Option Description
Command-line –rocksdb-debug-skip-bloom-filter-check-on-iterator-bounds
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

Added in Percona Server for MySQL 8.4.5-5

Allows setting iterator bounds in RocksDB even when the query range conditions would normally enable Bloom filter usage.

rocksdb_debug_ttl_ignore_pk

Option Description
Command-line –rocksdb-debug-ttl-ignore-pk
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

For debugging purposes only. If true, compaction filtering will not occur on Primary Key TTL data. This variable is a no-op in non-debug builds.

rocksdb_debug_ttl_read_filter_ts

Option Description
Command-line –rocksdb_debug-ttl-read-filter-ts
Dynamic Yes
Scope Global
Data type Numeric
Default 0

For debugging purposes only. Overrides the TTL read filtering time to time + debug_ttl_read_filter_ts. A value of 0 denotes that the variable is not set. This variable is a no-op in non-debug builds.

rocksdb_debug_ttl_rec_ts

Option Description
Command-line –rocksdb-debug-ttl-rec-ts
Dynamic Yes
Scope Global
Data type Numeric
Default 0

For debugging purposes only. Overrides the TTL of records to now() + debug_ttl_rec_ts. The value can be ± to simulate a record inserted in the past vs a record inserted in the future . A value of 0 denotes that the variable is not set. This variable is a no-op in non-debug builds.

rocksdb_debug_ttl_snapshot_ts

Option Description
Command-line –rocksdb-debug-ttl-snapshot-ts
Dynamic Yes
Scope Global
Data type Numeric
Default 0

For debugging purposes only. Sets the snapshot during compaction to now() + rocksdb_debug_set_ttl_snapshot_ts.

The value can be ± to simulate a snapshot in the past vs a snapshot created in the future . A value of 0 denotes that the variable is not set. This variable is a no-op in non-debug builds.

rocksdb_default_cf_options

Option Description
Command-line –rocksdb-default-cf-options
Dynamic No
Scope Global
Data type String

The rocksdb_default_cf_options variable defines the settings for the default column family. MyRocks stores data in this column family unless a table or index uses a dedicated one.

How the option works

MyRocks does not expose every RocksDB tuning knob as a separate MySQL variable. Instead, the server accepts a semicolon-separated list of parameters in RocksDB shorthand and passes them to the engine.

These settings apply to every table that uses the default column family. For example, write_buffer_size=64M;target_file_size_base=32M configures memtable size and SST file size.

On startup, the server applies this option to all existing column families. The option is read-only at runtime.

Which parameters are commonly tuned

The following parameters control memory, compaction, and storage behavior:

  • block_based_table_factory — Nested settings for blocks, including Bloom filters, index types, and block cache behavior.

  • compression_per_level — Compression algorithm per level, such as LZ4 or ZSTD, to balance CPU and disk space.

  • level0_file_num_compaction_trigger — Number of L0 (level 0) files that trigger a compaction.

  • max_bytes_for_level_base — Total size limit for level 1 of the LSM (Log-Structured Merge) tree. The level-1 limit influences how large subsequent levels become.

  • max_write_buffer_number — Maximum number of memtables that can accumulate in memory, with one active and the others waiting to flush. Raising max_write_buffer_number helps absorb bursts of writes.

  • target_file_size_base — Target size for a single SST file at level 1. Combined with level size limits, target_file_size_base affects how many files exist per level.

  • write_buffer_size — Size of a single memtable. When the limit is reached, MyRocks freezes the memtable and schedules a flush to an SST (Sorted String Table) file.

When to tune the option

Adjusting the rocksdb_default_cf_options string for the hardware, such as SSD versus HDD, is the primary way to optimize MyRocks throughput. The string provides centralized control over compaction style, memory, and I/O (input/output) parallelism.

The default varies by MyRocks version but balances LZ4 compression with moderate buffer sizes, such as 64 MB memtables. The default value is:

block_based_table_factory={cache_index_and_filter_blocks=1;filter_policy=bloomfilter:10:false;whole_key_filtering=1};level_compaction_dynamic_level_bytes=true;optimize_filters_for_hits=true;compaction_pri=kMinOverlappingRatio;compression=kLZ4Compression;bottommost_compression=kLZ4Compression;

What each component of the default value does

The default value combines four groups of settings:

  1. Block-based table options control how data is laid out and cached inside SST (Sorted String Table) files:

    • cache_index_and_filter_blocks=1 forces the index and Bloom filter data into the RocksDB block cache instead of pinning them outside the cache, for better control of total memory.

    • filter_policy=bloomfilter:10:false configures a Bloom filter with 10 bits per key. The false value refers to use_block_based_builder and selects the modern, more efficient Full Filter format.

    • whole_key_filtering=1 hashes the entire key in the Bloom filter for the fastest performance on point lookups.

  2. Compaction and layout settings shape how levels grow. level_compaction_dynamic_level_bytes=true adjusts per-level byte limits from the bottom level, reducing space amplification and making sizing more self-tuning. compaction_pri=kMinOverlappingRatio prefers compactions that free the most space relative to bytes written.

  3. Read optimization reduces CPU work on lookups. optimize_filters_for_hits=true skips Bloom filter checks on the bottommost level where hits are statistically more likely, saving CPU (central processing unit) time.

  4. Compression settings reduce disk usage. compression=kLZ4Compression and bottommost_compression=kLZ4Compression use LZ4 for low CPU overhead and solid general-purpose compression.

rocksdb_delayed_write_rate

Option Description
Command-line –rocksdb-delayed-write-rate
Dynamic Yes
Scope Global
Data type Numeric
Default 16777216

Specifies the write rate in bytes per second, which should be used if MyRocks hits a soft limit or threshold for writes. Default value is 16777216 (16 MB/sec). Allowed range is from 0 to 18446744073709551615.

rocksdb_delete_cf

Option Description
Command-line –rocksdb-delete-cf
Dynamic Yes
Scope Global
Data type String
Default “”

Deletes the column family by name. The default value is “” , an empty string.

For example:

SET @@global.ROCKSDB_DELETE_CF = 'cf_primary_key';

rocksdb_delete_obsolete_files_period_micros

Option Description
Command-line –rocksdb-delete-obsolete-files-period-micros
Dynamic No
Scope Global
Data type Numeric
Default 21600000000

Specifies the period in microseconds to delete obsolete files regardless of files removed during compaction. Default value is 21600000000 (6 hours). Allowed range is up to 9223372036854775807.

rocksdb_disable_file_deletions

Option Description
Command-line –rocksdb-disable-file-deletions
Dynamic Yes
Scope Session
Data type Boolean
Default OFF

It allows a client to temporarily disable RocksDB deletion of old WAL and .sst files for the purposes of making a consistent backup. If the client session terminates for any reason after disabling deletions and has not re-enabled deletions, they will be explicitly re-enabled. This variable should be used by backup tools. Prolonged use or other misuse can have serious side effects to the server instance.

rocksdb_disable_instant_ddl

Option Description
Command-line –rocksdb_disable_instant_ddl
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

[Deprecated: 8.4.5-5] The rocksdb_disable_instant_ddl variable is deprecated and will be removed in a future version. Its default value has changed to OFF as of this version.

Disables Instant DDL during ALTER TABLE operations.

Prior to Percona Server for MySQL 8.4.5-5, this variable was enabled (ON) by default.

rocksdb_enable_bulk_load_api

Option Description
Command-line –rocksdb-enable-bulk-load-api
Dynamic No
Scope Global
Data type Boolean
Default ON

Specifies whether to use the SSTFileWriter feature for bulk loading, This feature bypasses the memtable, but requires keys to be inserted into the table in either ascending or descending order. Enabled by default. If disabled, bulk loading uses the normal write path via the memtable and does not require keys to be inserted in any order.

rocksdb_enable_delete_range_for_drop_index

Option Description
Command-line –rocksdb_enable_delete_range_for_drop_index
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

Enables drop table / index by calling the DeleteRange.

This option is disabled (OFF) by default.

rocksdb_enable_insert_with_update_caching

Option Description
Command-line –rocksdb-enable-insert-with-update-caching
Dynamic Yes
Scope Global
Data type Boolean
Default ON

Specifies whether to enable optimization where the read is cached from a failed insertion attempt in INSERT ON DUPLICATE KEY UPDATE.

rocksdb_enable_instant_ddl

Option Description
Command-line –rocksdb-enable-instant-ddl
Dynamic Yes
Scope Global
Data type Boolean
Default ON

Added in Percona Server for MySQL 8.4.5-5.

Enables Instant DDL during ALTER TABLE operations when possible. If set to OFF, no DDL operations can be executed as instant.

rocksdb_enable_instant_ddl_for_append_column

Option Description
Command-line –rocksdb-enable-instant-ddl-for-append-column
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

Added in Percona Server for MySQL 8.4.5-5.

Enables Instant DDL specifically for appending columns during ALTER TABLE operations.

rocksdb_enable_instant_ddl_for_column_default_changes

Option Description
Command-line –rocksdb-enable-instant-ddl-for-column-default-changes
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

Added in Percona Server for MySQL 8.4.5-5.

Enables Instant DDL for changes to column defaults during ALTER TABLE operations.

rocksdb_enable_instant_ddl_for_drop_index_changes

Option Description
Command-line –rocksdb-enable-instant-ddl-for-drop-index-changes
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

Added in Percona Server for MySQL 8.4.5-5.

Enables Instant DDL for dropping indexes during ALTER TABLE operations.

rocksdb_enable_instant_ddl_for_table_comment_changes

Option Description
Command-line –rocksdb-enable-instant-ddl-for-table-comment-changes
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

Added in Percona Server for MySQL 8.4.5-5.

Enables Instant DDL for changes to table comments during ALTER TABLE operations.

rocksdb_enable_iterate_bounds

Option Description
Command-line –rocksdb-enable-iterate-bounds
Dynamic Yes
Scope Global, Local
Data type Boolean
Default ON

Enables the rocksdb iterator upper bounds and lower bounds in read options.

rocksdb_enable_pipelined_write

Option Description
Command-line –rocksdb-enable-pipelined-write
Dynamic No
Scope Global
Data type Boolean
Default OFF

DBOptions::enable_pipelined_write for RocksDB.

If enable_pipelined_write is ON, a separate write thread is maintained for WAL write and memtable write. A write thread first enters the WAL writer queue and then the memtable writer queue. A pending thread on the WAL writer queue only waits for the previous WAL write operations but does not wait for memtable write operations. Enabling the feature may improve write throughput and reduce latency of the prepare phase of a two-phase commit.

rocksdb_enable_remove_orphaned_dropped_cfs

Option Description
Command-line –rocksdb-enable-remove-orphaned-dropped-cfs
Dynamic Yes
Scope Global
Data type Boolean
Default ON

Enables the removal of dropped column families (cfs) from metadata if the cfs do not exist in the cf manager.

The default value is ON.

rocksdb_enable_thread_tracking

Option Description
Command-line –rocksdb-enable-thread-tracking
Dynamic No
Scope Global
Data type Boolean
Default OFF

Specifies whether to enable tracking the status of threads accessing the database. Disabled by default. If enabled, thread status will be available via GetThreadList().

rocksdb_enable_ttl

Option Description
Command-line –rocksdb-enable-ttl
Dynamic No
Scope Global
Data type Boolean
Default ON

Specifies whether to keep expired TTL records during compaction. Enabled by default. If disabled, expired TTL records will be dropped during compaction.

rocksdb_enable_ttl_read_filtering

Option Description
Command-line –rocksdb-enable-ttl-read-filtering
Dynamic Yes
Scope Global
Data type Boolean
Default ON

For tables with TTL, expired records are skipped/filtered out during processing and in query results. Disabling this will allow these records to be seen, but as a result rows may disappear in the middle of transactions as they are dropped during compaction. Use with caution.

rocksdb_enable_udt_in_mem

Option Description
Command-line –rocksdb-enable-udt-in-mem
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

Added in Percona Server for MySQL 8.4.5-5.

[Not yet implemented.]

Enables the user-defined timestamp in memtable feature to support Hybrid Logical Clock (HLC) snapshot reads in MyRocks.

rocksdb_enable_write_thread_adaptive_yield

Option Description
Command-line –rocksdb-enable-write-thread-adaptive-yield
Dynamic No
Scope Global
Data type Boolean
Default OFF

Specifies whether the MyRocks write batch group leader should wait up to the maximum allowed time before blocking on a mutex. Disabled by default. Enable it to increase throughput for concurrent workloads.

rocksdb_error_if_exists

Option Description
Command-line –rocksdb-error-if-exists
Dynamic No
Scope Global
Data type Boolean
Default OFF

Specifies whether to report an error when a database already exists. Disabled by default.

rocksdb_error_on_suboptimal_collation

Option Description
Command-line –rocksdb-error-on-suboptimal-collation
Dynamic No
Scope Global
Data type Boolean
Default ON

Specifies whether to report an error instead of a warning if an index is created on a char field where the table has a sub-optimal collation (case insensitive). Enabled by default.

rocksdb_file_checksums

[Changed in 8.4.5-5] The rocksdb_file_checksums variable’s default value changed from OFF to CHECKSUMS_OFF, and its data type changed to Enum= CHECKSUMS_OFF, CHECKSUMS_WRITE_ONLY, CHECKSUMS_WRITE_AND_VERIFY.

Option Description
Command-line –rocksdb-file-checksums
Dynamic No
Scope Global
Data type Enum= CHECKSUMS_OFF, CHECKSUMS_WRITE_ONLY, CHECKSUMS_WRITE_AND_VERIFY
Default CHECKSUMS_OFF (Prior to 8.4.5-5, the default was OFF)

This variable controls whether to write and check RocksDB file-level checksums. The possible values are:

  • CHECKSUMS_OFF: Disables checksums.

  • CHECKSUMS_WRITE_ONLY: Writes checksums but skips verification on database open.

  • CHECKSUMS_WRITE_AND_VERIFY: Writes checksums and verifies them when the database is opened.

Prior to Percona Server for MySQL 8.4.5-5, the default value was OFF, and the data type was a Boolean (where OFF equated to disabled).

rocksdb_flush_log_at_trx_commit

Option Description
Command-line –rocksdb-flush-log-at-trx-commit
Dynamic Yes
Scope Global, Session
Data type Numeric
Default 1

This variable controls whether the RocksDB Write-Ahead Log (WAL) is synchronized to disk on every transaction commit. The behavior is similar to innodb_flush_log_at_trx_commit .

The default value is 1, which ensures ACID compliance. Committed transactions remain durable after a crash. Less strict values improve performance at the cost of durability.

Which value should you choose

The variable accepts the values 0, 1, or 2. The following sections describe each value, the trade-offs, and the operational outcomes.

Value 0: do not sync on commit

Setting 0 does not flush or sync the WAL on commit. The setting removes commit-time I/O, so throughput is highest and commit latency is lowest. The trade-off is the weakest durability of the three values. After a crash, recently committed work may be missing or the database may be inconsistent. The risk window is wider than the roughly one-second window associated with 2 and far beyond what 1 allows.

The setting produces the following outcomes:

  • Leaves the WAL unflushed and unsynced on transaction commit.

  • Minimizes commit-time I/O relative to 1 and 2.

  • Risks extensive data loss or inconsistency after a crash compared with stricter settings.

Value 1: sync on every commit (default)

Setting 1 requires every commit to wait until the WAL is durably on disk before the commit returns. The sync is typically a full sync such as fsync. Use this value when a successful commit must survive a crash. The setting provides the strongest durability and ACID guarantees of the three values. The trade-off is the most synchronous disk work per commit, so commit latency and sustained write throughput are lower than with 0 or 2 when commits are frequent or disk sync is slow.

The setting produces the following outcomes:

  • Writes and syncs the WAL to disk at each transaction commit.

  • Ensures full durability and ACID compliance for committed work.

  • Incurs the highest per-commit I/O and the slowest commits of the three values.

Value 2: sync in background, typically once per second

Setting 2 writes the WAL on each commit, but the session does not wait for the durable sync. A background thread performs syncs on a schedule, for example about once per second. Individual commits return faster than with 1 because they skip the per-commit sync wait. The trade-off is the possible loss of up to one second of commits after a crash.

The setting produces the following outcomes:

  • Records each commit in the WAL without blocking the commit on a full durable sync.

  • Balances performance and durability.

  • Risks the loss of up to about one second of committed transactions after a crash.

rocksdb_flush_memtable_on_analyze

Option Description
Command-line –rocksdb-flush-memtable-on-analyze
Dynamic Yes
Scope Global, Session
Data type Boolean
Default ON

Specifies whether to flush the memtable when running ANALYZE on a table. Enabled by default. This ensures accurate cardinality by including data in the memtable for calculating stats.

rocksdb_force_compute_memtable_stats

Option Description
Command-line –rocksdb-force-compute-memtable-stats
Dynamic Yes
Scope Global
Data type Boolean
Default ON

Specifies whether data in the memtables should be included for calculating index statistics used by the query optimizer. Enabled by default. This provides better accuracy, but may reduce performance.

rocksdb_force_compute_memtable_stats_cachetime

Option Description
Command-line –rocksdb-force-compute-memtable-stats-cachetime
Dynamic Yes
Scope Global
Data type Numeric
Default 60000000 (60 seconds)

This variable controls how long, in microseconds, MyRocks caches memtable statistics for the query optimizer. The optimizer needs row-count estimates to plan queries. Data not yet flushed to disk requires scanning memtables for accurate statistics.

How the cache works

To avoid the CPU cost of rescanning memtables for every query, MyRocks stores the statistics in a cache. This variable defines the expiration time for the cached value. The default is 60000000 microseconds, or 60 seconds.

The cached value is reused on every query plan analysis until the timer expires.

When to raise or lower the value

A higher value, such as several minutes, improves performance in high-query-rate environments by reducing how often statistics collection runs. The optimizer may use stale data if the table changes rapidly.

A lower value, such as one second, gives the optimizer a near-real-time view of the data. The setting can yield better plans on volatile workloads, at the cost of more CPU use during query optimization.

rocksdb_force_flush_memtable_and_lzero_now

Option Description
Command-line –rocksdb-force-flush-memtable-and-lzero-now
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

Works similar to rocksdb_force_flush_memtable_now but also flushes all L0 files.

rocksdb_force_flush_memtable_now

Option Description
Command-line –rocksdb-force-flush-memtable-now
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

Forces MyRocks to immediately flush all memtables out to data files.

Warning

Use with caution! Write requests will be blocked until all memtables are flushed.

rocksdb_force_index_records_in_range

Option Description
Command-line –rocksdb-force-index-records-in-range
Dynamic Yes
Scope Global, Session
Data type Numeric
Default 1

Specifies the value used to override the number of rows returned to query optimizer when FORCE INDEX is used. Default value is 1. Allowed range is from 0 to 2147483647. Set to 0 if you do not want to override the returned value.

rocksdb_hash_index_allow_collision

Option Description
Command-line –rocksdb-hash-index-allow-collision
Dynamic No
Scope Global
Data type Boolean
Default ON

Specifies whether hash collisions are allowed. Enabled by default, which uses less memory. If disabled, full prefix is stored to prevent hash collisions.

rocksdb_ignore_unknown_options

Option Description
Command-line
Dynamic No
Scope Global
Data type Boolean
Default ON

When enabled, it allows RocksDB to receive unknown options and not exit.

rocksdb_index_type

Option Description
Command-line –rocksdb-index-type
Dynamic No
Scope Global
Data type Enum
Default kBinarySearch

Specifies the type of indexing used by MyRocks:

  • kBinarySearch: Binary search (default).

  • kHashSearch: Hash search.

rocksdb_info_log_level

Option Description
Command-line –rocksdb-info-log-level
Dynamic Yes
Scope Global
Data type Enum
Default error_level

Specifies the level for filtering messages written by MyRocks to the mysqld log.

  • debug_level: Maximum logging (everything including debugging log messages)

  • info_level

  • warn_level

  • error_level (default)

  • fatal_level: Minimum logging (only fatal error messages logged)

rocksdb_invalid_create_option_action

Option Description
Command-line –rocksdb-invalid-create-option-action
Dynamic Yes
Scope Global
Data type Enum: LOG, PUSH_WARNING, PUSH_ERROR
Default LOG

Added in Percona Server for MySQL 8.4.5-5.

Controls the behavior when creating a table encounters an error related to RocksDB options. You can choose to only log the error, pass the query and give users a warning, or fail the query.

rocksdb_io_error_action

Option Description
Command-line –rocksdb-io-error-action
Dynamic Yes
Scope Global
Data type Enum: ABORT_SERVER, IGNORE_ERROR
Default ABORT_SERVER

Added in Percona Server for MySQL 8.4.5-5.

Controls the behavior when an I/O error occurs within RocksDB. By default, MyRocks aborts the server and refuses to start. Setting this option to IGNORE_ERROR suppresses the error instead.

rocksdb_is_fd_close_on_exec

Option Description
Command-line –rocksdb-is-fd-close-on-exec
Dynamic No
Scope Global
Data type Boolean
Default ON

Specifies whether child processes should inherit open file jandles. Enabled by default.

rocksdb_keep_log_file_num

Option Description
Command-line –rocksdb-keep-log-file-num
Dynamic No
Scope Global
Data type Numeric
Default 1000

Specifies the maximum number of info log files to keep. Default value is 1000. Allowed range is from 1 to 18446744073709551615.

rocksdb_large_prefix

Option Description
Command-line –rocksdb-large-prefix
Dynamic Yes
Scope Global
Data type Boolean
Default ON

[Removed: 8.4.5-5] As of Percona Server for MySQL 8.4.5-5, the rocksdb_large_prefix variable has been removed.

This variable is deprecated in Percona Server for MySQL 8.3.0-1 and will be removed in a future release.

When enabled, this option allows index key prefixes longer than 767 bytes (up to 3072 bytes). The values for rocksdb_large_prefix should be the same between source and replica.

rocksdb_lock_scanned_rows

Option Description
Command-line –rocksdb-lock-scanned-rows
Dynamic Yes
Scope Global, Session
Data type Boolean
Default OFF

Specifies whether to hold the lock on rows that are scanned during UPDATE and not actually updated. Disabled by default.

rocksdb_lock_wait_timeout

Option Description
Command-line –rocksdb-lock-wait-timeout
Dynamic Yes
Scope Global, Session
Data type Numeric
Default 1

Specifies the number of seconds MyRocks should wait to acquire a row lock before aborting the request. Default value is 1. Allowed range is up to 1073741824.

rocksdb_log_file_time_to_roll

Option Description
Command-line –rocksdb-log-file-time-to-roll
Dynamic No
Scope Global
Data type Numeric
Default 0

Specifies the period (in seconds) for rotating the info log files. Default value is 0, meaning that the log file is not rotated. Allowed range is up to 18446744073709551615.

rocksdb_manifest_preallocation_size

Option Description
Command-line –rocksdb-manifest-preallocation-size
Dynamic No
Scope Global
Data type Numeric
Default 0

Specifies the number of bytes to preallocate for the MANIFEST file used by MyRocks to store information about column families, levels, active files, etc. Default value is 0. Allowed range is up to 18446744073709551615.

Note

A value of 4194304 (4 MB) is reasonable to reduce random I/O on XFS.

rocksdb_manual_compaction_bottommost_level

Option Description
Command-line –rocksdb-manual-compaction-bottommost-level
Dynamic Yes
Scope Local
Data type Enum
Default kForceOptimized

Option for bottommost level compaction during manual compaction:

  • kSkip - Skip bottommost level compaction

  • kIfHaveCompactionFilter - Only compact bottommost level if there is a compaction filter

  • kForce - Always compact bottommost level

  • kForceOptimized - Always compact bottommost level but in bottommost level avoid double-compacting files created in the same compaction

rocksdb_manual_compaction_threads

Option Description
Command-line –rocksdb-manual-compaction-threads
Dynamic Yes
Scope Local
Data type INT
Default 0

The variable defines the number of RocksDB threads to run for a manual compaction. The minimum value is 0. The maximum value is 120.

rocksdb_manual_wal_flush

Option Description
Command-line –rocksdb-manual-wal-flush
Dynamic No
Scope Global
Data type Boolean
Default ON

This variable can be used to disable automatic/timed WAL flushing and instead rely on the application to do the flushing.

rocksdb_master_skip_tx_api

Option Description
Command-line
Dynamic Yes
Scope Global, Session
Data type Boolean
Default OFF

When enabled, uses the WriteBatch API, which is faster. The session does not hold any lock on row access. This variable is not effective on replica.

Note

Due to the disabled row locks, improper use of the variable can cause data corruption or inconsistency.

rocksdb_max_background_compactions

Option Description
Command-line –rocksdb-max-background-compactions
Dynamic Yes
Scope Global
Data type Numeric
Default -1

Sets DBOptions:: max_background_compactions for RocksDB. The default value is -1 The allowed range is -1 to 64. This variable was replaced by rocksdb_max_background_jobs, which automatically decides how many threads to allocate towards flush/compaction.

rocksdb_max_background_flushes

Option Description
Command-line –rocksdb-max-background-flushes
Dynamic No
Scope Global
Data type Numeric
Default -1

Sets DBOptions:: max_background_flushes for RocksDB. The default value is -1. The allowed range is -1 to 64. This variable has been replaced by rocksdb_max_background_jobs, which automatically decides how many threads to allocate towards flush/compaction.

rocksdb_max_background_jobs

Option Description
Command-line –rocksdb-max-background-jobs
Dynamic Yes
Scope Global
Data type Numeric
Default 2

This variable replaced rocksdb_base_background_compactions, rocksdb_max_background_compactions, and rocksdb_max_background_flushes variables. This variable specifies the maximum number of background jobs. It automatically decides how many threads to allocate towards flush/compaction. It was implemented to reduce the number of (confusing) options users and can tweak and push the responsibility down to RocksDB level.

rocksdb_max_bottom_pri_background_compactions

Option Description
Command-line –rocksdb_max_bottom_pri_background_compactions
Dynamic No
Data type Unsigned integer
Default 0

Creates a specified number of threads, sets a lower CPU priority, and letting compactions use them. The maximum compaction concurrency is capped by rocksdb_max_background_compactions or rocksdb_max_background_jobs

The minimum value is 0 and the maximum value is 64.

rocksdb_max_compaction_history

Option Description
Command-line –rocksdb-max-compaction-history
Dynamic Yes
Scope Global
Data type Unsigned integer
Default 64

The minimum value is 0 and the maximum value is UINT64_MAX.

Tracks the history for at most rockdb_mx_compaction_history completed compactions. The history is in the INFORMATION_SCHEMA.ROCKSDB_COMPACTION_HISTORY table.

rocksdb_max_file_opening_threads

Option Description
Command-line –rocksdb-max-file-opening-threads
Dynamic No
Scope Global
Data type Numeric
Default 16

This variable sets DBOptions::max_file_opening_threads for RocksDB. The default value is 16. The minimum value is 1 and the maximum value is 2147483647 (INT_MAX).

Version changes

In Percona Server for MySQL 8.4.7-7, the maximum value was changed to 262144.

rocksdb_max_latest_deadlocks

Option Description
Command-line –rocksdb-max-latest-deadlocks
Dynamic Yes
Scope Global
Data type Numeric
Default 5

Specifies the maximum number of recent deadlocks to store.

rocksdb_max_log_file_size

Option Description
Command-line –rocksdb-max-log-file-size
Dynamic No
Scope Global
Data type Numeric
Default 0

Specifies the maximum size for info log files, after which the log is rotated. Default value is 0, meaning that only one log file is used. Allowed range is up to 18446744073709551615.

Also see rocksdb_log_file_time_to_roll.

rocksdb_max_manifest_file_size

Option Description
Command-line –rocksdb-manifest-log-file-size
Dynamic No
Scope Global
Data type Numeric
Default 18446744073709551615

Specifies the maximum size of the MANIFEST data file, after which it is rotated. Default value is also the maximum, making it practically unlimited: only one manifest file is used.

rocksdb_max_manual_compactions

Option Description
Command-line –rocksdb-max-manual-compactions
Dynamic Yes
Scope Global
Data type UINT
Default 10

The variable defines the maximum number of pending plus ongoing manual compactions. The default value and the minimum value is 0. The maximum value is 4294967295 (UNIT_MAX).

rocksdb_max_open_files

Option Description
Command-line –rocksdb-max-open-files
Dynamic No
Scope Global
Data type Numeric
Default 1000

Specifies the maximum number of file handles opened by MyRocks. Values in the range between 0 and open_files_limit are taken as they are. If rocksdb_max_open_files value is greater than open_files_limit, it will be reset to ½ of open_files_limit, and a warning will be emitted to the mysqld error log. A value of -2 denotes auto tuning: just sets rocksdb_max_open_files value to ½ of open_files_limit. Finally, -1 means no limit, i.e. an infinite number of file handles.

Warning

Setting rocksdb_max_open_files to -1 is dangerous, as the server may quickly run out of file handles in this case.

rocksdb_max_row_locks

Option Description
Command-line –rocksdb-max-row-locks
Dynamic Yes
Scope Global
Data type Numeric
Default 1048576

Specifies the limit on the maximum number of row locks a transaction can have before it fails. Default value is also the maximum, making it practically unlimited: transactions never fail due to row locks.

rocksdb_max_subcompactions

Option Description
Command-line –rocksdb-max-subcompactions
Dynamic No
Scope Global
Data type Numeric
Default 1

Specifies the maximum number of threads allowed for each compaction job. Default value of 1 means no subcompactions (one thread per compaction job). Allowed range is up to 64.

rocksdb_max_total_wal_size

Option Description
Command-line –rocksdb-max-total-wal-size
Dynamic No
Scope Global
Data type Numeric
Default 2 GB

This variable limits the total disk space consumed by Write-Ahead Log (WAL) files across all column families. The limit helps prevent log files from exhausting disk capacity. When the combined size exceeds the threshold, MyRocks flushes memtables to SST (Sorted String Table) files.

The default value is 2 GB. The allowed range is up to 9223372036854775807.

How the limit works

When the combined size of all WAL files exceeds the threshold, RocksDB identifies the oldest logs. RocksDB then forces a flush of their associated memtables to SST files.

After the data is in an SST file, RocksDB deletes or archives the corresponding WAL files. Total usage returns under the limit.

When to raise or lower the limit

A higher limit improves write performance by allowing larger, less frequent flushes. Disk usage increases and recovery time after a crash lengthens, because more log data must be replayed.

A lower limit keeps the disk footprint small and recovery fast. The setting may cause frequent forced flushes, which can throttle write throughput.

rocksdb_merge_buf_size

Option Description
Command-line –rocksdb-merge-buf-size
Dynamic Yes
Scope Global
Data type Numeric
Default 67108864

Specifies the size (in bytes) of the merge-sort buffers used to accumulate data during secondary key creation. New entries are written directly to the lowest level in the database, instead of updating indexes through the memtable and L0. These values are sorted using merge-sort, with buffers set to 64 MB by default (67108864). Allowed range is from 100 to 18446744073709551615.

rocksdb_merge_combine_read_size

Option Description
Command-line –rocksdb-merge-combine-read-size
Dynamic Yes
Scope Global
Data type Numeric
Default 1073741824

Specifies the size (in bytes) of the merge-combine buffer used for the merge-sort algorithm as described in rocksdb_merge_buf_size. Default size is 1 GB (1073741824). Allowed range is from 100 to 18446744073709551615.

rocksdb_merge_tmp_file_removal_delay_ms

Option Description
Command-line –rocksdb_merge_tmp_file_removal_delay_ms
Dynamic Yes
Scope Global, Session
Data type Numeric
Default 0

Fast secondary index creation creates merge files when needed. After finishing secondary index creation, merge files are removed. By default, the file removal is done without any sleep, so removing GBs of merge files within <1s may happen, which will cause trim stalls on Flash. This variable can be used to rate limit the delay in milliseconds.

rocksdb_new_table_reader_for_compaction_inputs

Option Description
Command-line –rocksdb-new-table-reader-for-compaction-inputs
Dynamic No
Scope Global
Data type Boolean
Default OFF

Specifies whether MyRocks should create a new file descriptor and table reader for each compaction input. Disabled by default. Enabling this may increase memory consumption, but will also allow pre-fetch options to be specified for compaction input files without impacting table readers used for user queries.

rocksdb_no_block_cache

Option Description
Command-line –rocksdb-no-block-cache
Dynamic No
Scope Global
Data type Boolean
Default OFF

Specifies whether to disable the block cache for column families. Variable is disabled by default, meaning that using the block cache is allowed.

rocksdb_no_create_column_family

Option Description
Command-line –rocksdb-no-create-column-family
Dynamic No
Scope Global
Data type Boolean
Default ON

Controls the processing of the column family name given in the COMMENT clause in the CREATE TABLE or ALTER TABLE statement in case the column family name does not refer to an existing column family.

If rocksdb_no_create_column_family is set to NO, a new column family will be created and the new index will be placed into it.

If rocksdb_no_create_column_family is set to YES, no new column family will be created and the index will be placed into the default column family. A warning is issued in this case informing that the specified column family does not exist and cannot be created.

rocksdb_override_cf_options

Option Description
Command-line –rocksdb-override-cf-options
Dynamic No
Scope Global
Data type String
Default

Specifies option overrides for each column family. Empty by default.

rocksdb_paranoid_checks

Option Description
Command-line –rocksdb-paranoid-checks
Dynamic No
Scope Global
Data type Boolean
Default ON

Specifies whether MyRocks should re-read the data file as soon as it is created to verify correctness. Enabled by default.

rocksdb_partial_index_blind_delete

Option Description
Command-line –rocksdb_partial_index_blind_delete
Dynamic Yes
Scope Global
Data type Boolean
Default ON

If enabled, the server does not read from the partial index to check if the key exists before deleting the partial index and the delete marker is unconditionally written.

If the variable is disabled (OFF), the server always reads from partial index to check if key exists before deleting the partial index.

This variable is enabled (ON) by default.

rocksdb_partial_index_ignore_killed

Option Description
Command-line –rocksdb-partial-index-ignore-killed
Dynamic Yes
Scope Global
Data type Boolean
Default ON

If this variable is set to ON, the partial index materialization ignores the killed flag and continues materialization until completion. If queries are killed during materialization due to timeout, the work done so far is wasted, and the killed query will likely be retried later, hitting the same issue.

The default value is ON which means this variable is enabled.

rocksdb_partial_index_sort_max_mem

Option Description
Command-line –rocksdb-partial-index-sort-max-mem
Dynamic Yes
Scope Local
Data type Unsigned Integer
Default 0

This variable sets the memory threshold, in bytes, for MyRocks to perform an in-memory sort when a query is only partially satisfied by an index.

How the default behaves

The default value is 0, which removes the memory limit. MyRocks may use as much RAM (random-access memory) as needed to perform the sort in memory.

The default produces the following effects:

  • Delivers maximum performance for partial index scans by avoiding slow disk-based filesorts.

  • Risks consuming all available system memory when a single large query, or many concurrent queries, run at once. The condition can lead to an out-of-memory (OOM) crash.

When to set a memory cap

A non-zero value, such as 16777216 for 16 MB, introduces a safety governor.

The cap produces the following effects:

  • Enables MyRocks to use the optimized in-memory sort path only when the result set fits within the defined memory budget.

  • Forces a fallback to a standard filesort when a sort requires more than the cap. The fallback avoids unbounded memory use and protects server stability. Affected queries take longer to complete because sorting uses disk or temporary files instead of memory.

rocksdb_pause_background_work

Option Description
Command-line –rocksdb-pause-background-work
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

Specifies whether MyRocks should pause all background operations. Disabled by default. There is no practical reason for a user to ever use this variable because it is intended as a test synchronization tool for the MyRocks MTR test suites.

Warning

If someone were to set a rocksdb_force_flush_memtable_now to 1 while rocksdb_pause_background_work is set to 1, the client that issued the rocksdb_force_flush_memtable_now=1 will be blocked indefinitely until rocksdb_pause_background_work is set to 0.

rocksdb_perf_context_level

Option Description
Command-line –rocksdb-perf-context-level
Dynamic Yes
Scope Global, Session
Data type Numeric
Default 0

Specifies the level of information to capture with the Perf Context plugins. The default value is 0. The allowed range is up to 5.

Value Description
1 Disable perf stats
2 Enable only count stats
3 Enable count stats and time stats except for mutexes
4 Enable count stats and time stats, except for wall time or CPU time for mutexes
5 Enable all count stats and time stats

rocksdb_persistent_cache_path

Option Description
Command-line –rocksdb-persistent-cache-path
Dynamic No
Scope Global
Data type String
Default

Specifies the path to the persistent cache. Set this together with rocksdb_persistent_cache_size_mb.

rocksdb_persistent_cache_size_mb

Option Description
Command-line –rocksdb-persistent-cache-size-mb
Dynamic No
Scope Global
Data type Numeric
Default 0

Specifies the size of the persisten cache in megabytes. Default is 0 (persistent cache disabled). Allowed range is up to 18446744073709551615. Set this together with rocksdb_persistent_cache_path.

rocksdb_pin_l0_filter_and_index_blocks_in_cache

Option Description
Command-line –rocksdb-pin-l0-filter-and-index-blocks-in-cache
Dynamic No
Scope Global
Data type Boolean
Default ON

Specifies whether MyRocks pins the filter and index blocks in the cache if rocksdb_cache_index_and_filter_blocks is enabled. Enabled by default.

rocksdb_print_snapshot_conflict_queries

Option Description
Command-line –rocksdb-print-snapshot-conflict-queries
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

Specifies whether queries that generate snapshot conflicts should be logged to the error log. Disabled by default.

rocksdb_protection_bytes_per_key

Option Description
Command-line –rocksdb_protection_bytes_per_key
Dynamic Yes
Scope Global, Session
Data type Numeric
Default 0

This variable is used to configure WriteOptions::protection_bytes_per_key. The default value is 0 (disabled). When this variable is set to 1, 2, 4, or 8, it uses that number of bytes per key value to protect entries in the WriteBatch.

The minimum value is 0.

The maximum value is ULONG_MAX (0xFFFFFFFF).

rocksdb_rate_limiter_bytes_per_sec

Option Description
Command-line –rocksdb-rate-limiter-bytes-per-sec
Dynamic Yes
Scope Global
Data type Numeric
Default 0

Specifies the maximum rate at which MyRocks can write to media via memtable flushes and compaction. Default value is 0 (write rate is not limited). Allowed range is up to 9223372036854775807.

rocksdb_read_free_rpl

Option Description
Command-line –rocksdb-read-free-rpl
Dynamic Yes
Scope Global
Data type Enum
Default OFF

Uses read-free replication, which allows no row lookup during replication, on the replica.

The options are the following:

  • OFF - Disables the variable

  • PK_SK - Enables the variable on all tables with a primary key

  • PK_ONLY - Enables the variable on tables where the only key is the primary key

rocksdb_read_free_rpl_tables

Option Description
Command-line –rocksdb-read-free-rpl-tables
Dynamic Yes
Scope Global, Session
Data type String
Default

We recommend that you use rocksdb_read_free_rpl instead of this variable.

This variable lists tables (as a regular expression) that should use read-free replication on the replica (that is, replication without row lookups). Empty by default.

rocksdb_records_in_range

Option Description
Command-line –rocksdb-records-in-range
Dynamic Yes
Scope Global, Session
Data type Numeric
Default 0

Specifies the value to override the result of records_in_range(). Default value is 0. Allowed range is up to 2147483647.

rocksdb_reset_stats

Option Description
Command-line –rocksdb-reset-stats
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

Resets MyRocks internal statistics dynamically (without restarting the server).

rocksdb_rollback_on_timeout

Option Description
Command-line –rocksdb-rollback-on-timeout
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

By default, only the last statement on a transaction is rolled back. If --rocksdb-rollback-on-timeout=ON, a transaction timeout causes a rollback of the entire transaction.

rocksdb_rpl_skip_tx_api

Option Description
Command-line –rocksdb-rpl-skip-tx-api
Dynamic No
Scope Global
Data type Boolean
Default OFF

Specifies whether write batches should be used for replication thread instead of the transaction API. Disabled by default.

There are two conditions which are necessary to use it: row replication format and replica operating in super read only mode.

rocksdb_seconds_between_stat_computes

Option Description
Command-line –rocksdb-seconds-between-stat-computes
Dynamic Yes
Scope Global
Data type Numeric
Default 3600

Specifies the number of seconds to wait between recomputation of table statistics for the optimizer. During that time, only changed indexes are updated. Default value is 3600. Allowed is from 0 to 4294967295.

rocksdb_signal_drop_index_thread

Option Description
Command-line –rocksdb-signal-drop-index-thread
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

Signals the MyRocks drop index thread to wake up.

rocksdb_sim_cache_size

Option Description
Command-line –rocksdb-sim-cache-size
Dynamic No
Scope Global
Data type Numeric
Default 0

Enables the simulated cache, which allows us to figure out the hit/miss rate with a specific cache size without changing the real block cache.

rocksdb_skip_bloom_filter_on_read

Option Description
Command-line –rocksdb-skip-bloom-filter-on_read
Dynamic Yes
Scope Global, Session
Data type Boolean
Default OFF

Specifies whether bloom filters should be skipped on reads. Disabled by default (bloom filters are not skipped).

rocksdb_skip_fill_cache

Option Description
Command-line –rocksdb-skip-fill-cache
Dynamic Yes
Scope Global, Session
Data type Boolean
Default OFF

Specifies whether to skip caching data on read requests. Disabled by default (caching is not skipped).

rocksdb_skip_locks_if_skip_unique_check

Option Description
Command-line rocksdb_skip_locks_if_skip_unique_check
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

Skip row locking when unique checks are disabled.

rocksdb_sst_mgr_rate_bytes_per_sec

Option Description
Command-line –rocksdb-sst-mgr-rate-bytes-per-sec
Dynamic Yes
Scope Global, Session
Data type Numeric
Default 0

Specifies the maximum rate for writing to data files. Default value is 0. This option is not effective on HDD. Allowed range is from 0 to 18446744073709551615.

rocksdb_stats_dump_period_sec

Option Description
Command-line –rocksdb-stats-dump-period-sec
Dynamic No
Scope Global
Data type Numeric
Default 600

Specifies the period in seconds for performing a dump of the MyRocks statistics to the info log. Default value is 600. Allowed range is up to 2147483647.

rocksdb_stats_level

Option Description
Command-line –rocksdb-stats-level
Dynamic Yes
Scope Global
Data type Numeric
Default 0

Controls the RocksDB statistics level. The default value is “0” (kExceptHistogramOrTimers), which is the fastest level. The maximum value is “4”.

rocksdb_stats_recalc_rate

Option Description
Command-line –rocksdb-stats-recalc-rate
Dynamic No
Scope Global
Data type Numeric
Default 0

Specifies the number of indexes to recalculate per second. Recalculating index statistics periodically ensures it to match the actual sum from SST files. Default value is 0. Allowed range is up to 4294967295.

rocksdb_store_row_debug_checksums

Option Description
Command-line –rocksdb-store-row-debug-checksums
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

Specifies whether to include checksums when writing index or table records. Disabled by default.

rocksdb_strict_collation_check

Option Description
Command-line –rocksdb-strict-collation-check
Dynamic Yes
Scope Global
Data type Boolean
Default ON

[Removed: 8.4.5-5] As of Percona Server for MySQL 8.4.5-5, the rocksdb_strict_collation_check variable has been removed.

Specifies whether to check and verify that table indexes have proper collation settings. Enabled by default.

rocksdb_strict_collation_exceptions

Option Description
Command-line –rocksdb-strict-collation-exceptions
Dynamic Yes
Scope Global
Data type String
Default

[Removed: 8.4.5-5] As of Percona Server for MySQL 8.4.5-5, the rocksdb_strict_collation_exceptions variable has been removed.

Lists tables (as a regular expression) that should be excluded from verifying case-sensitive collation enforced by rocksdb_strict_collation_check. Empty by default.

rocksdb_table_cache_numshardbits

Option Description
Command-line –rocksdb-table-cache-numshardbits
Dynamic No
Scope Global
Data type Numeric
Default 6

Specifies the number if table caches. The default value is 6. The allowed range is from 0 to 19.

rocksdb_table_stats_background_thread_nice_value

Option Description
Command-line –rocksdb-table-stats-background-thread-nice-value
Dynamic Yes
Scope Global
Data type Numeric
Default 19

The nice value for index stats. The minimum = -20 (THREAD_PRIO_MIN) The maximum = 19 (THREAD_PRIO_MAX)

rocksdb_table_stats_max_num_rows_scanned

Option Description
Command-line –rocksdb-table-stats-max-num-rows-scanned
Dynamic Yes
Scope Global
Data type Numeric
Default 0

The maximum number of rows to scan in a table scan based on a cardinality calculation. The minimum is 0 (every modification triggers a stats recalculation). The maximum is 18,446,744,073,709,551,615.

rocksdb_table_stats_recalc_threshold_count

Option Description
Command-line –rocksdb-table-stats-recalc-threshold-count
Dynamic Yes
Scope Global
Data type Numeric
Default 100

The number of modified rows to trigger a stats recalculation. This is a dependent variable for stats recalculation. The minimum is 0. The maximum is 18,446,744,073,709,551,615.

rocksdb_table_stats_recalc_threshold_pct

Option Description
Command-line –rocksdb-table-stats-recalc-threshold-pct
Dynamic Yes
Scope Global
Data type Numeric
Default 10

The percentage of the number of modified rows over the total number of rows to trigger stats recalculations. This is a dependent variable for stats recalculation. The minimum value is 0 The maximum value is 100 (RDB_TBL_STATS_RECALC_THRESHOLD_PCT_MAX).

rocksdb_table_stats_sampling_pct

Option Description
Command-line –rocksdb-table-stats-sampling-pct
Dynamic Yes
Scope Global
Data type Numeric
Default 10

Specifies the percentage of entries to sample when collecting statistics about table properties. Default value is 10. Allowed range is from 0 to 100.

rocksdb_table_stats_skip_system_cf

Option Description
Command-line –rocksdb-table-stats-skip-system-cf
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

Added in Percona Server for MySQL 8.4.5-5.

Determines whether to skip recording table statistics for the system column family.

rocksdb_table_stats_use_table_scan

Option Description
Command-line –rocksdb-table-stats-use-table-scan
Dynamic Yes
Scope Global
Data type Boolean
Default OFF.

Enables table-scan-based index calculations. The default value is OFF.

rocksdb_tmpdir

Option Description
Command-line –rocksdb-tmpdir
Dynamic Yes
Scope Global, Session
Data type String
Default

Specifies the path to the directory for temporary files during DDL operations.

rocksdb_trace_block_cache_access

Option Description
Command-line –rocksdb-trace-block-cache-access
Dynamic Yes
Scope Global
Data type String
Default ""

Defines the block cache trace option string. The format is sampling frequency: max_trace_file_size:trace_file_name. The sampling frequency value and max_trace_file_size value are positive integers. The block accesses are saved to the rocksdb_datadir/block_cache_traces/trace_file_name. The default value is an empty string.

rocksdb_trace_queries

Option Description
Command-line –rocksdb-trace-queries
Dynamic Yes
Scope Global
Data type String
Default ""

This variable is a trace option string. The format is sampling_frequency:max_trace_file_size:trace_file_name. The sampling_frequency and max_trace_file_size are positive integers. The queries are saved to the rocksdb_datadir/queries_traces/trace_file_name.

rocksdb_trace_sst_api

Option Description
Command-line –rocksdb-trace-sst-api
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

Specifies whether to generate trace output in the log for each call to SstFileWriter. Disabled by default.

rocksdb_track_and_verify_wals_in_manifest

Option Description
Command-line –rocksdb-track-and-verify-wals-in-manifest
Dynamic No
Scope Global
Data type Boolean
Default ON

DBOptions::track_and_verify_wals_in_manifest for RocksDB.

rocksdb_two_write_queues

Option Description
Command-line –rocksdb-track-and-verify-wals-in-manifest
Dynamic No
Scope Global
Data type Boolean
Default ON

When enabled this variable allows/encourages threads that are using two-phase commit to prepare in parallel.

rocksdb_unsafe_for_binlog

Option Description
Command-line –rocksdb-unsafe-for-binlog
Dynamic Yes
Scope Global, Session
Data type Boolean
Default OFF

Specifies whether to allow statement-based binary logging which may break consistency. Disabled by default.

rocksdb_update_cf_options

Option Description
Command-line –rocksdb-update-cf-options
Dynamic No
Scope Global
Data type String
Default

Specifies option updates for each column family. Empty by default.

rocksdb_use_adaptive_mutex

Option Description
Command-line –rocksdb-use-adaptive-mutex
Dynamic No
Scope Global
Data type Boolean
Default OFF

Specifies whether to use adaptive mutex which spins in user space before resorting to the kernel. Disabled by default.

rocksdb_use_default_sk_cf

Option Description
Command-line –rocksdb-use-default-sk-cf
Dynamic No
Scope Global
Data type Boolean
Default OFF

Use default_sk column family for secondary keys.

rocksdb_use_direct_io_for_flush_and_compaction

Option Description
Command-line –rocksdb-use-direct-io-for-flush-and-compaction
Dynamic No
Scope Global
Data type Boolean
Default OFF

Specifies whether to write to data files directly, without caches or buffers. Disabled by default.

rocksdb_use_direct_reads

Option Description
Command-line –rocksdb-use-direct-reads
Dynamic No
Scope Global
Data type Boolean
Default OFF

Specifies whether to read data files directly, without caches or buffers. Disabled by default. If you enable this, make sure that rocksdb_allow_mmap_reads is disabled.

rocksdb_use_fsync

Option Description
Command-line –rocksdb-use-fsync
Dynamic No
Scope Global
Data type Boolean
Default OFF

Specifies whether MyRocks should use fsync instead of fdatasync when requesting a sync of a data file. Disabled by default.

rocksdb_use_hyper_clock_cache

Option Description
Command-line –rocksdb_use_hyper_clock_cache
Dynamic No
Scope Global
Data type Boolean
Default OFF

This variable replaces the standard LRU (Least Recently Used) block cache with a lock-free HyperClockCache implementation. When enabled, MyRocks uses HyperClockCache instead of the default LRUCache for RocksDB. The default value is OFF.

Key benefits

The HyperClockCache provides the following benefits:

  • High concurrency on many-core systems with 16 or more cores. The cache reduces the global lock bottleneck found in traditional LRU caches.

  • CPU efficiency through a clock algorithm rather than a linked list. The algorithm avoids expensive memory writes and synchronization on every cache hit.

Trade-offs

Enabling HyperClockCache produces the following trade-offs:

  • Memory overhead. The cache uses a fixed-size hash table, which has slightly higher per-entry memory overhead than a standard LRU cache.

  • Approximate LRU ordering. Eviction precision is lower than with a traditional LRU cache, but faster to maintain.

  • Throughput improvement. Heavy read or scan workloads can see significantly higher throughput.

When to enable HyperClockCache

Enable HyperClockCache when CPU profiling shows high mutex contention within the RocksDB block cache, or when running on high core-count servers.

rocksdb_use_io_uring

Option Description
Command-line –rocksdb-use-io-uring
Dynamic Yes
Scope Global
Data type Boolean
Default OFF

Added in Percona Server for MySQL 8.4.5-5.

Enables the use of io_uring for RocksDB.

rocksdb_use_write_buffer_manager

Option Description
Command-line –rocksdb_use_write_buffer_manager
Dynamic No
Scope Global
Data type Boolean
Default OFF

This variable is tech preview and may be removed in the future releases.

Allows to turn on the write buffer manager (WriteBufferManager) from cnf files. This variable is related to rocksdb_charge_memory.

rocksdb_validate_tables

Option Description
Command-line –rocksdb-validate-tables
Dynamic No
Scope Global
Data type Numeric
Default 1

Specifies whether to verify that MySQL data dictionary is equal to the MyRocks data dictionary.

  • 0: do not verify.

  • 1: verify and fail on error (default).

  • 2: verify and continue with error.

rocksdb_verify_row_debug_checksums

Option Description
Command-line –rocksdb-verify-row-debug-checksums
Dynamic Yes
Scope Global, Session
Data type Boolean
Default OFF

Specifies whether to verify checksums when reading index or table records. Disabled by default.

rocksdb_wal_bytes_per_sync

Option Description
Command-line –rocksdb-wal-bytes-per-sync
Dynamic Yes
Scope Global
Data type Numeric
Default 0

Specifies how often should the OS sync WAL (write-ahead log) files to disk as they are being written, asynchronously, in the background. This operation can be used to smooth out write I/O over time. Default value is 0, meaning that files are never synced. Allowed range is up to 18446744073709551615.

rocksdb_wal_dir

Option Description
Command-line –rocksdb-wal-dir
Dynamic No
Scope Global
Data type String
Default

Specifies the path to the directory where MyRocks stores WAL files.

rocksdb_wal_recovery_mode

Option Description
Command-line –rocksdb-wal-recovery-mode
Dynamic Yes
Scope Global
Data type Numeric
Default 2

Specifies the level of tolerance when recovering write-ahead logs (WAL) files after a system crash.

The following are the options:

  • 0: if the last WAL entry is corrupted, truncate the entry and either start the server normally or refuse to start.

  • 1: if a WAL entry is corrupted, the server fails to start and does not recover from the crash.

  • 2 (default): if a corrupted WAL entry is detected, truncate all entries after the detected corrupted entry. You can select this setting for replication replicas.

  • 3: If a corrupted WAL entry is detected, skip only the corrupted entry and continue the apply WAL entries. This option can be dangerous.

rocksdb_wal_size_limit_mb

Option Description
Command-line –rocksdb-wal-size-limit-mb
Dynamic No
Scope Global
Data type Numeric
Default 0

Specifies the maximum size of all WAL files in megabytes before attempting to flush memtables and delete the oldest files. Default value is 0 (never rotated). Allowed range is up to 9223372036854775807.

rocksdb_wal_ttl_seconds

Option Description
Command-line –rocksdb-wal-ttl-seconds
Dynamic No
Scope Global
Data type Numeric
Default 0

Specifies the timeout in seconds before deleting archived WAL files. Default is 0 (archived WAL files are never deleted). Allowed range is up to 9223372036854775807.

rocksdb_whole_key_filtering

Option Description
Command-line –rocksdb-whole-key-filtering
Dynamic No
Scope Global
Data type Boolean
Default ON

The rocksdb_whole_key_filtering variable controls whether the Bloom filter stores a hash of the entire key or only the prefix. The option is part of RocksDB BlockBasedTableOptions. The default value is ON.

When the variable is enabled, ensure that lookups use the whole key for matching.

How the filter behaves

The two states produce the following behavior:

  • Enabled (default): MyRocks adds both the whole key and the prefix to the Bloom filter. Storing both yields the most accurate filtering for point lookups, such as WHERE pk = 10. The engine can skip SST (Sorted String Table) files that do not contain the key.

  • Disabled: MyRocks adds only the prefix to the Bloom filter. Because fewer unique prefixes exist than unique keys, Bloom filters are smaller and save significant memory.

When to disable whole-key filtering

Disabling whole-key filtering suits memory-constrained environments or workloads dominated by prefix scans. Point lookups see a higher false-positive rate. The database may read from disk because the prefix matched, even though the full key did not.

rocksdb_write_batch_flush_threshold

Option Description
Command-line –rocksdb-write-batch-flush-threshold
Dynamic Yes
Scope Local
Data type Integer
Default 0

This variable specifies the maximum size of the write batch in bytes before flushing. Only valid if rockdb_write_policy is WRITE_UNPREPARED. There is no limit if the variable is set to the default setting.

rocksdb_write_batch_max_bytes

Option Description
Command-line –rocksdb-write-batch-max-bytes
Dynamic Yes
Scope Global
Data type Numeric
Default 0

Specifies the maximum size of a RocksDB write batch in bytes. 0 means no limit. In case user exceeds the limit following error will be shown: ERROR HY000: Status error 10 received from RocksDB: Operation aborted: Memory limit reached.

rocksdb_write_disable_wal

Option Description
Command-line –rocksdb-write-disable-wal
Dynamic Yes
Scope Global, Session
Data type Boolean
Default OFF

Lets you temporarily disable writes to WAL files, which can be useful for bulk loading.

rocksdb_write_ignore_missing_column_families

Option Description
Command-line –rocksdb-write-ignore-missing-column-families
Dynamic Yes
Scope Global, Session
Data type Boolean
Default OFF

Specifies whether to ignore writes to column families that do not exist. Disabled by default (writes to non-existent column families are not ignored).

rocksdb_write_policy

Option Description
Command-line –rocksdb-write-policy
Dynamic No
Scope Global
Data type String
Default write_committed

Specifies when two-phase commit data are written into the database. Allowed values are write_committed, write_prepared, and write_unprepared.

Value Description
write_committed Data written at commit time
write_prepared Data written after the prepare phase of a two-phase transaction
write_unprepared Data written before the prepare phase of a two-phase transaction

MyRocks status variables

MyRocks status variables provide details about the inner workings of the storage engine and they can be useful in tuning the storage engine to a particular environment.

You can view these variables and their values by running:

SHOW STATUS LIKE 'rocksdb%';

The following global status variables are available:

Name Var Type
rocksdb_rows_deleted Numeric
rocksdb_rows_inserted Numeric
rocksdb_rows_read Numeric
rocksdb_rows_unfiltered_no_snapshot Numeric
rocksdb_rows_updated Numeric
rocksdb_rows_expired Numeric
rocksdb_system_rows_deleted Numeric
rocksdb_system_rows_inserted Numeric
rocksdb_system_rows_read Numeric
rocksdb_system_rows_updated Numeric
rocksdb_memtable_total Numeric
rocksdb_memtable_unflushed Numeric
rocksdb_queries_point Numeric
rocksdb_queries_range Numeric
rocksdb_covered_secondary_key_lookups Numeric
rocksdb_additional_compactions_trigger Numeric
rocksdb_block_cache_add Numeric
rocksdb_block_cache_add_failures Numeric
rocksdb_block_cache_bytes_read Numeric
rocksdb_block_cache_bytes_write Numeric
rocksdb_block_cache_data_add Numeric
rocksdb_block_cache_data_bytes_insert Numeric
rocksdb_block_cache_data_hit Numeric
rocksdb_block_cache_data_miss Numeric
rocksdb_block_cache_filter_add Numeric
rocksdb_block_cache_filter_bytes_evict Numeric
rocksdb_block_cache_filter_bytes_insert Numeric
rocksdb_block_cache_filter_hit Numeric
rocksdb_block_cache_filter_miss Numeric
rocksdb_block_cache_hit Numeric
rocksdb_block_cache_index_add Numeric
rocksdb_block_cache_index_bytes_evict Numeric
rocksdb_block_cache_index_bytes_insert Numeric
rocksdb_block_cache_index_hit Numeric
rocksdb_block_cache_index_miss Numeric
rocksdb_block_cache_miss Numeric
rocksdb_block_cache_compressed_hit Numeric
rocksdb_block_cache_compressed_miss Numeric
rocksdb_bloom_filter_prefix_checked Numeric
rocksdb_bloom_filter_prefix_useful Numeric
rocksdb_bloom_filter_useful Numeric
rocksdb_bytes_read Numeric
rocksdb_bytes_written Numeric
rocksdb_compact_read_bytes Numeric
rocksdb_compact_write_bytes Numeric
rocksdb_compaction_key_drop_new Numeric
rocksdb_compaction_key_drop_obsolete Numeric
rocksdb_compaction_key_drop_user Numeric
rocksdb_flush_write_bytes Numeric
rocksdb_get_hit_l0 Numeric
rocksdb_get_hit_l1 Numeric
rocksdb_get_hit_l2_and_up Numeric
rocksdb_get_updates_since_calls Numeric
rocksdb_iter_bytes_read Numeric
rocksdb_memtable_hit Numeric
rocksdb_memtable_miss Numeric
rocksdb_no_file_closes Numeric
rocksdb_no_file_errors Numeric
rocksdb_no_file_opens Numeric
rocksdb_num_iterators Numeric
rocksdb_number_block_not_compressed Numeric
rocksdb_number_db_next Numeric
rocksdb_number_db_next_found Numeric
rocksdb_number_db_prev Numeric
rocksdb_number_db_prev_found Numeric
rocksdb_number_db_seek Numeric
rocksdb_number_db_seek_found Numeric
rocksdb_number_deletes_filtered Numeric
rocksdb_number_keys_read Numeric
rocksdb_number_keys_updated Numeric
rocksdb_number_keys_written Numeric
rocksdb_number_merge_failures Numeric
rocksdb_number_multiget_bytes_read Numeric
rocksdb_number_multiget_get Numeric
rocksdb_number_multiget_keys_read Numeric
rocksdb_number_reseeks_iteration Numeric
rocksdb_number_sst_entry_delete Numeric
rocksdb_number_sst_entry_merge Numeric
rocksdb_number_sst_entry_other Numeric
rocksdb_number_sst_entry_put Numeric
rocksdb_number_sst_entry_singledelete Numeric
rocksdb_number_stat_computes Numeric
rocksdb_number_superversion_acquires Numeric
rocksdb_number_superversion_cleanups Numeric
rocksdb_number_superversion_releases Numeric
rocksdb_rate_limit_delay_millis Numeric
rocksdb_row_lock_deadlocks Numeric
rocksdb_row_lock_wait_timeouts Numeric
rocksdb_snapshot_conflict_errors Numeric
rocksdb_stall_l0_file_count_limit_slowdowns Numeric
rocksdb_stall_locked_l0_file_count_limit_slowdowns Numeric
rocksdb_stall_l0_file_count_limit_stops Numeric
rocksdb_stall_locked_l0_file_count_limit_stops Numeric
rocksdb_stall_pending_compaction_limit_stops Numeric
rocksdb_stall_pending_compaction_limit_slowdowns Numeric
rocksdb_stall_memtable_limit_stops Numeric
rocksdb_stall_memtable_limit_slowdowns Numeric
rocksdb_stall_total_stops Numeric
rocksdb_stall_total_slowdowns Numeric
rocksdb_stall_micros Numeric
rocksdb_wal_bytes Numeric
rocksdb_wal_group_syncs Numeric
rocksdb_wal_synced Numeric
rocksdb_write_other Numeric
rocksdb_write_self Numeric
rocksdb_write_timedout Numeric
rocksdb_write_wal Numeric

rocksdb_rows_deleted

This variable shows the number of rows that were deleted from MyRocks tables.

rocksdb_rows_inserted

This variable shows the number of rows that were inserted into MyRocks tables.

rocksdb_rows_read

This variable shows the number of rows that were read from MyRocks tables.

rocksdb_rows_unfiltered_no_snapshot

This variable shows how many reads need TTL and have no snapshot timestamp.

rocksdb_rows_updated

This variable shows the number of rows that were updated in MyRocks tables.

rocksdb_rows_expired

This variable shows the number of expired rows in MyRocks tables.

rocksdb_system_rows_deleted

This variable shows the number of rows that were deleted from MyRocks system tables.

rocksdb_system_rows_inserted

This variable shows the number of rows that were inserted into MyRocks system tables.

rocksdb_system_rows_read

This variable shows the number of rows that were read from MyRocks system tables.

rocksdb_system_rows_updated

This variable shows the number of rows that were updated in MyRocks system tables.

rocksdb_memtable_total

This variable shows the memory usage, in bytes, of all memtables.

rocksdb_memtable_unflushed

This variable shows the memory usage, in bytes, of all unflushed memtables.

rocksdb_queries_point

This variable shows the number of single row queries.

rocksdb_queries_range

This variable shows the number of multi/range row queries.

rocksdb_covered_secondary_key_lookups

This variable shows the number of lookups via the secondary index that returned all fields requested directly from the secondary index.

rocksdb_additional_compactions_trigger

This variable shows the number of triggered additional compactions. MyRocks triggers an additional compaction if (number of deletions / number of entries) > (rocksdb_compaction_sequential_deletes / rocksdb_compaction_sequential_deletes_window) in the SST file.

rocksdb_block_cache_add

This variable shows the number of blocks added to block cache.

rocksdb_block_cache_add_failures

This variable shows the number of failures when adding blocks to block cache.

rocksdb_block_cache_bytes_read

This variable shows the number of bytes read from cache.

rocksdb_block_cache_bytes_write

This variable shows the number of bytes written into cache.

rocksdb_block_cache_data_add

This variable shows the number of data blocks added to block cache.

rocksdb_block_cache_data_bytes_insert

This variable shows the number of bytes of data blocks inserted into cache.

rocksdb_block_cache_data_hit

This variable shows the number of cache hits when accessing the data block from the block cache.

rocksdb_block_cache_data_miss

This variable shows the number of cache misses when accessing the data block from the block cache.

rocksdb_block_cache_filter_add

This variable shows the number of filter blocks added to block cache.

rocksdb_block_cache_filter_bytes_evict

This variable shows the number of bytes of bloom filter blocks removed from cache.

rocksdb_block_cache_filter_bytes_insert

This variable shows the number of bytes of bloom filter blocks inserted into cache.

rocksdb_block_cache_filter_hit

This variable shows the number of times cache hit when accessing filter block from block cache.

rocksdb_block_cache_filter_miss

This variable shows the number of times cache miss when accessing filter block from block cache.

rocksdb_block_cache_hit

This variable shows the total number of block cache hits.

rocksdb_block_cache_index_add

This variable shows the number of index blocks added to block cache.

rocksdb_block_cache_index_bytes_evict

This variable shows the number of bytes of index block erased from cache.

rocksdb_block_cache_index_bytes_insert

This variable shows the number of bytes of index blocks inserted into cache.

rocksdb_block_cache_index_hit

This variable shows the total number of block cache index hits.

rocksdb_block_cache_index_miss

This variable shows the number of times cache hit when accessing index block from block cache.

rocksdb_block_cache_miss

This variable shows the total number of block cache misses.

rocksdb_block_cache_compressed_hit

This variable shows the number of hits in the compressed block cache.

rocksdb_block_cache_compressed_miss

This variable shows the number of misses in the compressed block cache.

rocksdb_bloom_filter_prefix_checked

This variable shows the number of times bloom was checked before creating iterator on a file.

rocksdb_bloom_filter_prefix_useful

This variable shows the number of times the check was useful in avoiding iterator creation (and thus likely IOPs).

rocksdb_bloom_filter_useful

This variable shows the number of times bloom filter has avoided file reads.

rocksdb_bytes_read

This variable shows the total number of uncompressed bytes read. It could be either from memtables, cache, or table files.

rocksdb_bytes_written

This variable shows the total number of uncompressed bytes written.

rocksdb_compact_read_bytes

This variable shows the number of bytes read during compaction

rocksdb_compact_write_bytes

This variable shows the number of bytes written during compaction.

rocksdb_compaction_key_drop_new

This variable shows the number of key drops during compaction because it was overwritten with a newer value.

rocksdb_compaction_key_drop_obsolete

This variable shows the number of key drops during compaction because it was obsolete.

rocksdb_compaction_key_drop_user

This variable shows the number of key drops during compaction because user compaction function has dropped the key.

rocksdb_flush_write_bytes

This variable shows the number of bytes written during flush.

rocksdb_get_hit_l0

This variable shows the number of Get() queries served by L0.

rocksdb_get_hit_l1

This variable shows the number of Get() queries served by L1.

rocksdb_get_hit_l2_and_up

This variable shows the number of Get() queries served by L2 and up.

rocksdb_get_updates_since_calls

This variable shows the number of calls to GetUpdatesSince function. Useful to keep track of transaction log iterator refreshes

rocksdb_iter_bytes_read

This variable shows the number of uncompressed bytes read from an iterator. It includes size of key and value.

rocksdb_memtable_hit

This variable shows the number of memtable hits.

rocksdb_memtable_miss

This variable shows the number of memtable misses.

rocksdb_no_file_closes

This variable shows the number of time file were closed.

rocksdb_no_file_errors

This variable shows number of errors trying to read in data from an sst file.

rocksdb_no_file_opens

This variable shows the number of time file were opened.

rocksdb_num_iterators

This variable shows the number of currently open iterators.

rocksdb_number_block_not_compressed

This variable shows the number of uncompressed blocks.

rocksdb_number_db_next

This variable shows the number of calls to next.

rocksdb_number_db_next_found

This variable shows the number of calls to next that returned data.

rocksdb_number_db_prev

This variable shows the number of calls to prev.

rocksdb_number_db_prev_found

This variable shows the number of calls to prev that returned data.

rocksdb_number_db_seek

This variable shows the number of calls to seek.

rocksdb_number_db_seek_found

This variable shows the number of calls to seek that returned data.

rocksdb_number_deletes_filtered

This variable shows the number of deleted records that were not required to be written to storage because key did not exist.

rocksdb_number_keys_read

This variable shows the number of keys read.

rocksdb_number_keys_updated

This variable shows the number of keys updated, if inplace update is enabled.

rocksdb_number_keys_written

This variable shows the number of keys written to the database.

rocksdb_number_merge_failures

This variable shows the number of failures performing merge operator actions in RocksDB.

rocksdb_number_multiget_bytes_read

This variable shows the number of bytes read during RocksDB MultiGet() calls.

rocksdb_number_multiget_get

This variable shows the number MultiGet() requests to RocksDB.

rocksdb_number_multiget_keys_read

This variable shows the keys read via MultiGet().

rocksdb_number_reseeks_iteration

This variable shows the number of times reseek happened inside an iteration to skip over large number of keys with same userkey.

rocksdb_number_sst_entry_delete

This variable shows the total number of delete markers written by MyRocks.

rocksdb_number_sst_entry_merge

This variable shows the total number of merge keys written by MyRocks.

rocksdb_number_sst_entry_other

This variable shows the total number of non-delete, non-merge, non-put keys written by MyRocks.

rocksdb_number_sst_entry_put

This variable shows the total number of put keys written by MyRocks.

rocksdb_number_sst_entry_singledelete

This variable shows the total number of single delete keys written by MyRocks.

rocksdb_number_stat_computes

This variable isn’t used anymore and will be removed in future releases.

rocksdb_number_superversion_acquires

This variable shows the number of times the superversion structure has been acquired in RocksDB, this is used for tracking all of the files for the database.

rocksdb_number_superversion_cleanups

rocksdb_number_superversion_releases

rocksdb_rate_limit_delay_millis

This variable was removed in Percona Server for MySQL Percona Server 5.7.23-23 .

rocksdb_row_lock_deadlocks

This variable shows the total number of deadlocks that have been detected since the instance was started.

rocksdb_row_lock_wait_timeouts

This variable shows the total number of row lock wait timeouts that have been detected since the instance was started.

rocksdb_snapshot_conflict_errors

This variable shows the number of snapshot conflict errors occurring during write transactions that forces the transaction to rollback.

rocksdb_stall_l0_file_count_limit_slowdowns

This variable shows the slowdowns in write due to L0 being close to full.

rocksdb_stall_locked_l0_file_count_limit_slowdowns

This variable shows the slowdowns in write due to L0 being close to full and compaction for L0 is already in progress.

rocksdb_stall_l0_file_count_limit_stops

This variable shows the stalls in write due to L0 being full.

rocksdb_stall_locked_l0_file_count_limit_stops

This variable shows the stalls in write due to L0 being full and compaction for L0 is already in progress.

rocksdb_stall_pending_compaction_limit_stops

This variable shows the stalls in write due to hitting limits set for max number of pending compaction bytes.

rocksdb_stall_pending_compaction_limit_slowdowns

This variable shows the slowdowns in write due to getting close to limits set for max number of pending compaction bytes.

rocksdb_stall_memtable_limit_stops

This variable shows the stalls in write due to hitting max number of memTables allowed.

rocksdb_stall_memtable_limit_slowdowns

This variable shows the slowdowns in writes due to getting close to max number of memtables allowed.

rocksdb_stall_total_stops

This variable shows the total number of write stalls.

rocksdb_stall_total_slowdowns

This variable shows the total number of write slowdowns.

rocksdb_stall_micros

This variable shows how long (in microseconds) the writer had to wait for compaction or flush to finish.

rocksdb_wal_bytes

This variables shows the number of bytes written to WAL.

rocksdb_wal_group_syncs

This variable shows the number of group commit WAL file syncs that have occurred.

rocksdb_wal_synced

This variable shows the number of times WAL sync was done.

rocksdb_write_other

This variable shows the number of writes processed by another thread.

rocksdb_write_self

This variable shows the number of writes that were processed by a requesting thread.

rocksdb_write_timedout

This variable shows the number of writes ending up with timed-out.

rocksdb_write_wal

This variable shows the number of Write calls that request WAL.

Gap locks detection

The Gap locks detection is based on a Facebook MySQL patch.

If a transactional storage engine does not support gap locks (for example MyRocks) and a gap lock is being attempted while the transaction isolation level is either REPEATABLE READ or SERIALIZABLE, the following SQL error will be returned to the client and no actual gap lock will be taken on the effected rows.

Error message 
ERROR HY000: Using Gap Lock without full unique key in multi-table or multi-statement transactions is not allowed. You need to either rewrite queries to use all unique key columns in WHERE equal conditions, or rewrite to single-table, single-statement transaction.

Data loading

By default, MyRocks configurations are optimized for short transactions, and not for data loading. MyRocks has a couple of special session variables to speed up data loading dramatically.

Sorted bulk loading

If your data is guaranteed to be loaded in primary key order, then this method is recommended. This method works by dropping any secondary keys first, loading data into your table in primary key order, and then restoring the secondary keys via Fast Secondary Index Creation.

Creating secondary indexes

When loading data into empty tables, it is highly recommended to drop all secondary indexes first, then loading data, and adding all secondary indexes after finishing loading data. MyRocks has a feature called Fast Secondary Index Creation. Fast Secondary Index Creation is automatically used when executing CREATE INDEX or ALTER TABLE ... ADD INDEX. With Fast Secondary Index Creation, the secondary index entries are directly written to bottommost RocksDB levels and bypassing compaction. This significantly reduces total write volume and CPU time for decompressing and compressing data on higher levels.

Loading data

As described above, loading data is highly recommended for tables with primary key only (no secondary keys), with all secondary indexes added after loading data.

When loading data into MyRocks tables, there are two recommended session variables:

SET session sql_log_bin=0;
SET session rocksdb_bulk_load=1;

When converting from large MyISAM/InnoDB tables, either by using the ALTER or INSERT INTO SELECT statements it’s recommended that you create MyRocks tables as below (in case the table is sufficiently big it will cause the server to consume all the memory and then be terminated by the OOM killer):

SET session sql_log_bin=0;
SET session rocksdb_bulk_load=1;
ALTER TABLE large_myisam_table ENGINE=RocksDB;
SET session rocksdb_bulk_load=0;

Using sql_log_bin=0 avoids writing to binary logs.

With rocksdb_bulk_load set to 1, MyRocks enters special mode to write all inserts into bottommost RocksDB levels, and skips writing data into MemTable and the following compactions. This is very efficient way to load data.

The rocksdb_bulk_load mode operates with a few conditions:

  • None of the data being bulk loaded can overlap with existing data in the table. The easiest way to ensure this is to always bulk load into an empty table, but the mode will allow loading some data into the table, doing other operations, and then returning and bulk loading addition data if there is no overlap between what is being loaded and what already exists.

  • The data may not be visible until bulk load mode is ended (i.e. the rocksdb_bulk_load is set to zero again). The method that is used is building up SST files which will later be added as-is to the database. Until a particular SST has been added the data will not be visible to the rest of the system, thus issuing a SELECT on the table currently being bulk loaded will only show older data and will likely not show the most recently added rows. Ending the bulk load mode will cause the most recent SST file to be added. When bulk loading multiple tables, starting a new table will trigger the code to add the most recent SST file to the system – as a result, it is inadvisable to interleave INSERT statements to two or more tables during bulk load mode.

By default, the rocksdb_bulk_load mode expects all data be inserted in primary key order (or reversed order). If the data is in the reverse order (i.e. the data is descending on a normally ordered primary key or is ascending on a reverse ordered primary key), the rows are cached in chunks to switch the order to match the expected order.

Inserting one or more rows out of order will result in an error and may result in some of the data being inserted in the table and some not. To resolve the problem, one can either fix the data order of the insert, truncate the table, and restart.

Unsorted bulk loading

If your data is not ordered in primary key order, then this method is recommended. With this method, secondary keys do not need to be dropped and restored. However, writing to the primary key no longer goes directly to SST files, and are written to temporary files for sorted first, so there is extra cost to this method.

To allow for loading unsorted data:

SET session sql_log_bin=0;
SET session rocksdb_bulk_load_allow_unsorted=1;
SET session rocksdb_bulk_load=1;
...
SET session rocksdb_bulk_load=0;
SET session rocksdb_bulk_load_allow_unsorted=0;

Note that rocksdb_bulk_load_allow_unsorted can only be changed when rocksdb_bulk_load is disabled (set to 0). In this case, all input data will go through an intermediate step that writes the rows to temporary SST files, sorts them rows in the primary key order, and then writes to final SST files in the correct order.

Other approaches

If rocksdb_commit_in_the_middle is enabled, MyRocks implicitly commits every rocksdb_bulk_load_size records (default is 1,000) in the middle of your transaction. If your data loading fails in the middle of the statement (LOAD DATA or bulk INSERT), rows are not entirely rolled back, but some of rows are stored in the table. To restart data loading, you’ll need to truncate the table and loading data again.

Warning

If you are loading large data without enabling rocksdb_bulk_load or rocksdb_commit_in_the_middle, please make sure transaction size is small enough. All modifications of the ongoing transactions are kept in memory.

Other reading

Reference

Percona Server for MySQL 8.4 variables

System variables

Name Cmd-Line Option File Var Scope Dynamic
csv_mode Yes Yes Both Yes
enforce_storage_engine Yes Yes Global No
expand_fast_index_creation Yes No Both Yes
extra_max_connections Yes Yes Global Yes
extra_port Yes Yes Global No
have_backup_locks Yes No Global No
have_backup_safe_binlog_info Yes No Global No
have_snapshot_cloning Yes No Global No
innodb_cleaner_lsn_age_factor Yes Yes Global Yes
innodb_corrupt_table_action Yes Yes Global Yes
innodb_buffer_pool_lazy_latch_init Yes Yes Global No
innodb_empty_free_list_algorithm Yes Yes Global Yes
innodb_encrypt_online_alter_logs Yes Yes Global Yes
innodb_encrypt_tables Yes Yes Global Yes
innodb_kill_idle_transaction Yes Yes Global Yes
innodb_lru_threads Yes Yes Global No
innodb_max_bitmap_file_size Yes Yes Global Yes
innodb_max_changed_pages Yes Yes Global Yes
innodb_print_lock_wait_timeout_info Yes Yes Global Yes
innodb_show_locks_held Yes Yes Global Yes
innodb_temp_tablespace_encrypt Yes Yes Global No
innodb_track_changed_pages Yes Yes Global No
keyring_vault_config Yes Yes Global Yes
keyring_vault_timeout Yes Yes Global Yes
log_slow_filter Yes Yes Both Yes
log_slow_rate_limit Yes Yes Both Yes
log_slow_rate_type Yes Yes Global Yes
log_slow_sp_statements Yes Yes Global Yes
log_slow_verbosity Yes Yes Both Yes
log_warnings_suppress Yes Yes Global Yes
proxy_protocol_networks Yes Yes Global No
query_response_time_flush Yes No Global No
query_response_time_range_base Yes Yes Global Yes
query_response_time_stats Yes Yes Global Yes
secure_log_path Yes Yes Global No
slow_query_log_always_write_time Yes Yes Global Yes
slow_query_log_use_global_control Yes Yes Global Yes
thread_pool_high_prio_mode Yes Yes Both Yes
thread_pool_high_prio_tickets Yes Yes Both Yes
thread_pool_idle_timeout Yes Yes Global Yes
thread_pool_max_threads Yes Yes Global Yes
thread_pool_oversubscribe Yes Yes Global Yes
thread_pool_size Yes Yes Global Yes
thread_pool_stall_limit Yes Yes Global No
thread_statistics Yes Yes Global Yes
userstat Yes Yes Global Yes
version_comment Yes Yes Global Yes
version_suffix Yes Yes Global Yes

Status variables

Name Var Type Var Scope
Binlog_snapshot_file String Global
Binlog_snapshot_position Numeric Global
Com_lock_binlog_for_backup Numeric Both
Com_lock_tables_for_backup Numeric Both
Com_show_client_statistics Numeric Both
Com_show_index_statistics Numeric Both
Com_show_table_statistics Numeric Both
Com_show_thread_statistics Numeric Both
Com_show_user_statistics Numeric Both
Com_unlock_binlog Numeric Both
Innodb_background_log_sync Numeric Global
Innodb_buffer_pool_pages_LRU_flushed Numeric Global
Innodb_buffer_pool_pages_made_not_young Numeric Global
Innodb_buffer_pool_pages_made_young Numeric Global
Innodb_buffer_pool_pages_old Numeric Global
Innodb_checkpoint_age Numeric Global
Innodb_checkpoint_max_age Numeric Global
Innodb_ibuf_free_list Numeric Global
Innodb_ibuf_segment_size Numeric Global
Innodb_lsn_current Numeric Global
Innodb_lsn_flushed Numeric Global
Innodb_lsn_last_checkpoint Numeric Global
Innodb_max_trx_id Numeric Global
Innodb_mem_adaptive_hash Numeric Global
Innodb_mem_dictionary Numeric Global
Innodb_oldest_view_low_limit_trx_id Numeric Global
Innodb_purge_trx_id Numeric Global
Innodb_purge_undo_no Numeric Global
Threadpool_average_hp_queue_wait_us String Global
Threadpool_average_queue_wait_us String Global
Threadpool_idle_threads Numeric Global
Threadpool_requests_starved_in_queue Numeric Global
Threadpool_requests_waiting_in_hp_queue Numeric Global
Threadpool_requests_waiting_in_queue Numeric Global
Threadpool_threads Numeric Global

List of features available in Percona Server for MySQL releases

Percona Server for MySQL 5.7 Percona Server for MySQL 8.0
Improved Buffer Pool Scalability Improved Buffer Pool Scalability
Improved InnoDB I/O Scalability Improved InnoDB I/O Scalability
Multiple Adaptive Hash Search Partitions Multiple Adaptive Hash Search Partitions
Atomic write support for Fusion-io devices Atomic write support for Fusion-io devices
Query Cache Enhancements Feature not implemented
Improved NUMA support Improved NUMA support
Thread Pool Thread Pool
Suppress Warning Messages Suppress Warning Messages
Ability to change the database for mysqlbinlog Ability to change the database for mysqlbinlog
Fixed Size for the Read Ahead Area Fixed Size for the Read Ahead Area
Improved MEMORY Storage Engine Improved MEMORY Storage Engine
Restricting the number of binlog files Restricting the number of binlog files
Ignoring missing tables in mysqldump Ignoring missing tables in mysqldump
Too Many Connections Warning Too Many Connections Warning
Handle Corrupted Tables Handle Corrupted Tables
Lock-Free SHOW SLAVE STATUS Lock-Free SHOW REPLICA STATUS
Expanded Fast Index Creation Expanded Fast Index Creation
Percona Toolkit UDFs Percona Toolkit UDFs
Support for Fake Changes Support for Fake Changes
Kill Idle Transactions Kill Idle Transactions
XtraDB changed page tracking XtraDB changed page tracking
Enforcing Storage Engine Replaced with upstream implementation
Utility user Utility user
Extending the secure-file-priv server option Extending the secure-file-priv server option
Expanded Program Option Modifiers Feature not implemented
PAM Authentication Plugin PAM Authentication Plugin
Log Archiving for XtraDB Log Archiving for XtraDB
User Statistics User Statistics
Slow Query Log Slow Query Log
Count InnoDB Deadlocks Count InnoDB Deadlocks
Log All Client Commands (syslog) Log All Client Commands (syslog)
Response Time Distribution Feature not implemented
Show Storage Engines Show Storage Engines
Show Lock Names Show Lock Names
Process List Process List
Misc. INFORMATION_SCHEMA Tables Misc. INFORMATION_SCHEMA Tables
Extended Show Engine InnoDB Status Extended Show Engine InnoDB Status
Thread Based Profiling Thread Based Profiling
XtraDB Performance Improvements for I/O-Bound Highly-Concurrent Workloads XtraDB Performance Improvements for I/O-Bound Highly-Concurrent Workloads
Page cleaner thread tuning Page cleaner thread tuning
Statement Timeout Statement Timeout
Extended SELECT INTO OUTFILE/DUMPFILE Extended SELECT INTO OUTFILE/DUMPFILE
Per-query variable statement Per-query variable statement
Extended mysqlbinlog Extended mysqlbinlog
Slow Query Log Rotation and Expiration Slow Query Log Rotation and Expiration
Metrics for scalability measurement Feature not implemented
Audit Log Audit Log
Backup Locks Backup Locks
CSV engine mode for a standard-compliant quote and comma parsing CSV engine mode for a standard-compliant quote and comma parsing
Super read-only Super read-only

Other reading

Percona Server for MySQL feature comparison

Percona Server for MySQL is a free, fully compatible, enhanced, and open source drop-in replacement for any MySQL database. It provides superior performance, scalability, and instrumentation.

Percona Server for MySQL is trusted by thousands of enterprises to provide better performance and concurrency for their most demanding workloads. It delivers higher value to MySQL server users with optimized performance, greater performance scalability and availability, enhanced backups, and increased visibility.

We provide these benefits by significantly enhancing Percona Server for MySQL as compared to the standard MySQL database server:

Features Percona Server for MySQL 8.4 MySQL 8.4
Open Source Yes Yes
ACID Compliance Yes Yes
Multi-Version Concurrency Control Yes Yes
Row-Level Locking Yes Yes
Automatic Crash Recovery Yes Yes
Table Partitioning Yes Yes
Views Yes Yes
Subqueries Yes Yes
Triggers Yes Yes
Stored Procedures Yes Yes
Foreign Keys Yes Yes
Window Functions Yes Yes
Common Table Expressions Yes Yes
Geospatial Features (GIS, SRS) Yes Yes
GTID Replication Yes Yes
Group Replication Yes Yes
MyRocks Storage Engine Yes No
Improvements for Developers Percona Server for MySQL 8.4 MySQL 8.4
NoSQL Socket-Level Interface Yes Yes
X API Support Yes Yes
JSON Functions Yes Yes
InnoDB Full-Text Search Improvements Yes No
Extra Hash/Digest Functions Yes No
Instrumentation and Troubleshooting Features Percona Server for MySQL 8.4 MySQL 8.4
INFORMATION_SCHEMA Tables 95 65
Global Performance and Status Counters 853 434
Optimizer Histograms Yes Yes
Per-Table Performance Counters Yes No
Per-Index Performance Counters Yes No
Per-User Performance Counters Yes No
Per-Client Performance Counters Yes No
Per-Thread Performance Counters Yes No
Global Query Response Time Statistics Yes No
Enhanced SHOW INNODB ENGINE STATUS Yes No
Undo Segment Information Yes No
Temporary Tables Information Yes No
Extended Slow Query Logging Yes No
User Statistics Yes No
Performance and Scalability Features Percona Server for MySQL 8.4 MySQL 8.4
InnoDB Resource Groups Yes Yes
Configurable Page Sizes Yes Yes
Contention-Aware Transaction Scheduling Yes Yes
Improved Scalability By Splitting Mutexes Yes No
Improved MEMORY Storage Engine Yes No
Improved Flushing Yes No
Parallel Doublewrite Buffer Yes Yes
Configurable Fast Index Creation Yes No
Per-Column Compression for VARCHAR/BLOB and JSON Yes No
Compressed Columns with Dictionaries Yes No
Security Features Percona Server for MySQL 8.4 MySQL 8.4
SQL Roles Yes Yes
SHA-2 Based Password Hashing Yes Yes
Password Rotation Policy Yes Yes
PAM Authentication Plugin Yes Enterprise-Only
Encryption Features Percona Server for MySQL 8.4 MySQL 8.4
Storing Keyring in a File Yes Yes
Storing Keyring in Hashicorp Vault Yes Enterprise Only
Encrypt InnoDB Data Yes Yes
Encrypt InnoDB Logs Yes Yes
Encrypt Built-In InnoDB Tablespaces (General, System, Undo, Temp) Yes Yes
Encrypt Binary Logs Yes No
Encrypt Temporary Files Yes No
Enforce Encryption Yes No
Operational Improvements Percona Server for MySQL 8.4 MySQL 8.4
Atomic DDL Yes Yes
Transactional Data Dictionary Yes Yes
Instant DDL Yes Yes
SET PERSIST Yes Yes
Invisible Indexes Yes Yes
Threadpool Yes Enterprise-Only
Backup Locks Yes No
Extended SHOW GRANTS Yes No
Improved Handling of Corrupted Tables Yes No
Ability to Kill Idle Transactions Yes No
Improvements to START TRANSACTION WITH CONSISTENT SNAPSHOT Yes No
Features for Running Database as a Service (DBaaS) Percona Server for MySQL 8.4 MySQL 8.4
Enforce a Specific Storage Engine Yes Yes

Understand version numbers

A version number identifies the innovation product release. The product contains the latest features, improvements, and bug fixes at the time of that release.

8.4.2 -2
Base version Minor build version

Percona uses semantic version numbering, which follows the pattern of base version and build version. Percona assigns unique, non-negative integers in increasing order for each version release. The version number combines the base MySQL 8.4 version number and the minor build version.

The version numbers for Percona Server for MySQL 8.4.11-11 define the following information:

  • Base version - the leftmost numbers indicate MySQL 8.4 version used as a base.

  • Minor build version - an internal number that increases by one every time Percona Server for MySQL is released.

Development of Percona Server for MySQL

Percona Server for MySQL is an open source project to produce a distribution of the MySQL Server with improved performance, scalability and diagnostics.

Submit changes

We keep the trunk in a constant state of stability to allow for a release at any time and to minimize wasted time by developers due to broken code.

Overview

At Percona we use Git for source control, GitHub for code hosting, and Jira for release management.

We change our software to implement new features and/or to fix bugs. Refactoring could be classed either as a new feature or a bug depending on the scope of work.

New features and bugs are targeted to specific releases. A release is part of a series. For example, 2.4 is a series in Percona XtraBackup and 2.4.15, 2.4.16, and 2.4.17 are releases in this series.

Code is proposed for merging in the form of pull requests on GitHub.

For Percona Server for MySQL, we have several Git branches on which development occurs: 5.5, 5.6, 5.7, and 8.0. As Percona Server for MySQL is not a traditional project, instead of being a set of patches against an existing product, these branches are not related. In other words, we do not merge from one release branch to another. To have your changes in several branches, you must propose branches to each release branch.

Making a change to a project

In this case, we are going to use percona-xtrabackup as an example. The workflow is similar for Percona Server for MySQL, but the patch will need to be modified in all release branches of Percona Server for MySQL.

  • git branch https://github.com/percona/percona-xtrabackup/featureX (where ‘featureX’ is a sensible name for the task at hand)

  • (developer makes changes in featureX, testing locally)

  • The Developer pushes to https://github.com/percona/username/percona-xtrabackup/featureX

  • The developer can submit a pull request to https://github.com/percona/percona-xtrabackup,

  • Code undergoes a review

  • Once code is accepted, it can be merged

If the change also applies to a stable release (e.g. 2.4) then changes should be made on a branch of 2.4 and merged to a branch of the trunk. In this case, there should be two branches run through the param build and two merge proposals (one for the stable release and one with the changes merged to the trunk). This prevents somebody else from having to guess how to merge your changes.

Percona Server for MySQL

The same process for Percona Server for MySQL, but we have several different branches (and merge requests).

Telemetry and data collection

Percona Server for MySQL includes two telemetry systems. Each system is optional.

  • Installation-time telemetry runs once at package install or container startup.

  • Continuous telemetry uses a database (DB) component and the percona-telemetry-agent host process. The pair collects metrics and sends them on a daily schedule.

You control whether to share telemetry data. Disable either system, or both, when you do not want anonymous usage data sent to Percona.

Percona does not collect personal information. All telemetry data is anonymous. See the Percona Privacy policy .

Packages, compressed archives (tarballs), and Docker images include telemetry. Tarball deployments require the telemetry agent and a writable directory under /usr/local/percona/telemetry.

Telemetry overview

Telemetry reports anonymous deployment and usage metrics to Percona. Percona uses the data to prioritize fixes and feature work. Telemetry does not include database names, credentials, or user data.

Installation-time telemetry

Installation-time telemetry runs once. The process collects host and version metadata during package installation or container startup. The process does not run again after installation completes.

Installation-time telemetry file example

The following JSON shows a typical installation-time payload:

[{"id" : "c416c3ee-48cd-471c-9733-37c2886f8231",
"product_family" : "PRODUCT_FAMILY_PS",
"instanceId" : "6aef422e-56a7-4530-af9d-94cc02198343",
"createTime" : "2026-03-26T15:43:18Z",
"metrics":
[{"key" : "deployment","value" : "PACKAGE"},
{"key" : "pillar_version","value" : "8.4.11-11"},
{"key" : "OS","value" : "Oracle Linux Server 10"},
{"key" : "hardware_arch","value" : "x86_64 x86_64"}]}]

Disable installation-time telemetry

Installation-time telemetry is enabled by default. Set the environment variable PERCONA_TELEMETRY_DISABLE=1 before you install packages or start a container. The variable does not disable continuous telemetry. Installation-time telemetry uses PERCONA_TELEMETRY_DISABLE; continuous telemetry uses the server option percona_telemetry_disable.

sudo PERCONA_TELEMETRY_DISABLE=1 apt install percona-server-server

sudo PERCONA_TELEMETRY_DISABLE=1 dnf install percona-server-server

docker run -d -e MYSQL_ROOT_PASSWORD=test1234# -e PERCONA_TELEMETRY_DISABLE=1 --name=percona-server percona/percona-server:8.4

Continuous telemetry

Continuous telemetry combines a Percona telemetry DB component with the percona-telemetry-agent service. The component writes metrics files on disk. The agent uploads the files to the Percona telemetry service. The agent waits 24 hours after startup before the first upload attempt.

Continuous telemetry: database component, metrics file, host agent, and Percona Platform

Elements of the continuous telemetry system

The continuous telemetry system includes these host-side and platform-side pieces:

Piece Role
Percona telemetry DB component Collects metrics inside the server and writes a metrics file on disk
Metrics file JavaScript Object Notation (JSON) file on the host that stores collected metrics
Telemetry agent (percona-telemetry-agent) Host process that collects operating system metrics, reads metrics files, merges payloads, and queries the package manager for Percona packages
Telemetry service Application programming interface (API) endpoint that receives telemetry payloads
Telemetry storage Long-term storage for telemetry data on the Percona platform

Overview of the DB component

Percona Server for MySQL installs the telemetry DB component by default. The component registers as file://component_percona_telemetry.

The DB component performs these tasks:

  • Collects database metrics once per day

  • Writes a timestamped .json file under the telemetry directory

  • Retains metrics files for seven days and deletes older files before creating a new file

The DB component does not collect these data types:

  • Database names

  • User names or credentials

  • Application or user data

Locations of metrics files and telemetry history

The telemetry root path on the host is /usr/local/percona/telemetry.

Product-specific directories use the following paths under the root:

Product Path
Percona Server for MySQL ${telemetry root}/ps/
Percona Server for MongoDB (mongod) ${telemetry root}/psmdb/
Percona Server for MongoDB (mongos) ${telemetry root}/psmdbs/
Percona XtraDB Cluster ${telemetry root}/pxc/
PostgreSQL products ${telemetry root}/pg/

After a successful upload, the agent stores a copy under ${telemetry root}/history/.

Metrics file format

Metrics files use JSON. Percona may extend the schema in future releases. Production files can list many active_plugins entries. The following example shows the core fields:

{
  "db_instance_id": "e83c568c-e140-11ee-8320-7e207666b18a",
  "pillar_version": "8.4.11-11",
  "active_plugins": [
    "binlog",
    "caching_sha2_password",
    "InnoDB",
    "PERFORMANCE_SCHEMA"
  ],
  "active_components": [
    "file://component_percona_telemetry"
  ],
  "uptime": "6185",
  "databases_count": "7",
  "databases_size": "33149",
  "se_engines_in_use": [
    "InnoDB"
  ],
  "replication_info": {
    "is_semisync_source": "1",
    "is_replica": "1"
  }
}

Percona telemetry agent

The percona-telemetry-agent process runs on the database host. The agent manages JSON files under the telemetry root path.

The agent follows the following schedule:

  • Log file path: /var/log/percona/telemetry-agent.log

  • First 24 hours: no collection and no upload

  • After 24 hours: one upload attempt per day, with up to five retries on failure

  • After a successful upload: copy the file to history/ and delete the source file from the DB component directory

  • When the telemetry directory has no Percona product files: send nothing

Network access and corporate proxies

Continuous telemetry uploads use Hypertext Transfer Protocol Secure (HTTPS). The default endpoint is:

https://check.percona.com/v1/telemetry/GenericReport

Firewall teams must allow outbound HTTPS on port 443 to check.percona.com. Override the endpoint with PERCONA_TELEMETRY_URL or the --telemetry.url argument for percona-telemetry-agent.

To send traffic through a corporate proxy, add environment variables in a systemd drop-in for percona-telemetry-agent.service:

[Service]
Environment="HTTPS_PROXY=http://PROXY_HOST:PROXY_PORT"
Environment="NO_PROXY=localhost,127.0.0.1"

Apply the drop-in and restart the agent:

sudo systemctl edit percona-telemetry-agent
sudo systemctl daemon-reload
sudo systemctl restart percona-telemetry-agent

Replace PROXY_HOST and PROXY_PORT with your proxy hostname and port.

Air-gapped and isolated networks

Upload failures trigger five retries. The agent then waits until the next daily check. Unsent metrics files remain on disk until an upload succeeds.

Disk use stays bounded in isolated networks:

  • The DB component retains seven days of metrics files and prunes older files before each daily write, regardless of upload status

  • The agent prunes archived history on a seven-day interval (PERCONA_TELEMETRY_HISTORY_KEEP_INTERVAL, default 604800 seconds)

Disable continuous telemetry to stop new metrics files and outbound uploads.

Telemetry agent configuration

The agent reads these environment variables at startup. Restart the service after you change a value.

Variable Default Description
PERCONA_TELEMETRY_ROOT_PATH /usr/local/percona/telemetry Root directory for metrics and history files
PERCONA_TELEMETRY_CHECK_INTERVAL 86400 Seconds between upload checks
PERCONA_TELEMETRY_RESEND_INTERVAL 60 Seconds between retry attempts after a failed upload
PERCONA_TELEMETRY_HISTORY_KEEP_INTERVAL 604800 Seconds between history directory cleanup runs
PERCONA_TELEMETRY_URL https://check.percona.com/v1/telemetry/GenericReport Upload endpoint URL

Telemetry agent payload example

{
  "reports": [
    {
      "id": "B5BDC47B-B717-4EF5-AEDF-41A17C9C18BB",
      "createTime": "2026-03-26T15:44:54Z",
      "instanceId": "B5BDC47B-B717-4EF5-AEDF-41A17C9C18BA",
      "productFamily": "PRODUCT_FAMILY_PS",
      "metrics": [
        {
          "key": "OS",
          "value": "Ubuntu"
        },
        {
          "key": "pillar_version",
          "value": "8.4.11-11"
        }
      ]
    }
  ]
}

Telemetry agent payload fields

Each report object uses these fields:

Field Description
id Random universally unique identifier (UUID) version 4 for the request
createTime Request timestamp
instanceId Host ID from the metrics file, /usr/local/percona/telemetry_uuid, or a generated UUID version 4 when the file is absent
productFamily Product family derived from the metrics file path, such as PRODUCT_FAMILY_PS
metrics Key and value pairs from the metrics file

Operating system metrics in each upload

Each upload can include these operating system keys:

Key Description
OS Operating system name
hardware_arch CPU architecture
deployment Deployment method, such as PACKAGE or DOCKER
installed_packages Installed Percona packages with name, version, and repository when available

The agent queries the local package manager. The query matches only package names that fit Percona patterns, including percona-*, Percona-*, proxysql*, pmm, etcd*, haproxy, patroni, pg*, postgis, and wal2json. The agent does not report non-Percona packages.

Disable continuous telemetry

Continuous telemetry is enabled by default. Complete the steps in the following order.

Restart required

Steps 1 and 3 require a server or service restart. Plan a maintenance window before you change telemetry settings in production.

  1. Add the server option percona_telemetry_disable=1 under [mysqld] in my.cnf, or in an included option file. Restart the server. The server option prevents the telemetry component from loading after restart. Do not rely on UNINSTALL COMPONENT alone. A restart without percona_telemetry_disable=1 can reload telemetry.

  2. Run UNINSTALL COMPONENT "file://component_percona_telemetry"; to stop metrics file generation.

  3. Stop and disable percona-telemetry-agent as described in Disable the telemetry agent.

These steps disable continuous telemetry only. For install-time opt-out, see Disable installation-time telemetry.

Disable the telemetry agent

Stopping the agent blocks uploads. The DB component continues to write metrics files until you uninstall the component.

Run both commands for a permanent disable:

sudo systemctl stop percona-telemetry-agent
sudo systemctl disable percona-telemetry-agent

systemctl stop ends the running agent process. systemctl disable prevents the agent from starting at boot.

To pause the agent until the next host reboot, run sudo systemctl stop percona-telemetry-agent only. Do not run disable when you plan to re-enable the agent after reboot.

Agent dependencies and removal

The telemetry agent is a mandatory package dependency for Percona Server for MySQL. Review package dependencies before you remove the agent.

Package manager behavior differs:

  • DNF can remove the agent when you remove the last dependent package. Removing the agent alone can remove Percona Server for MySQL when dependencies require the agent.

  • APT may keep the agent after you remove only the server package. Use apt autoremove when appropriate. Removing the agent without checking dependencies can affect the server package.

Verify continuous telemetry is disabled

Run these checks from the MySQL client and on the host.

Confirm the telemetry component is absent:

SELECT component_urn
FROM mysql.component
WHERE component_urn LIKE '%telemetry%';
Expected output when telemetry is disabled 
Empty set (0.00 sec)

Confirm the disable option is set:

SHOW GLOBAL VARIABLES LIKE 'percona_telemetry_disable';
Expected output when telemetry is disabled 
+---------------------------+-------+
| Variable_name             | Value |
+---------------------------+-------+
| percona_telemetry_disable | ON    |
+---------------------------+-------+
1 row in set (0.00 sec)

Confirm the agent is inactive on the host:

systemctl is-enabled percona-telemetry-agent
systemctl is-active percona-telemetry-agent

After a permanent disable, is-enabled returns disabled and is-active returns inactive.

Disable the DB component

The DB telemetry component can write metrics files for seven days while the agent is stopped. Uninstall the component after you set percona_telemetry_disable=1 and restart the server. See the ordered steps in Disable continuous telemetry.

UNINSTALL COMPONENT "file://component_percona_telemetry";

Add percona_telemetry_disable=1 to my.cnf when the line is missing, then restart:

[mysqld]
percona_telemetry_disable=1

Trademark policy

This Trademark Policy is to ensure that users of Percona-branded products or services know that what they receive has really been developed, approved, tested, and maintained by Percona. Trademarks help to prevent confusion in the marketplace, by distinguishing one company’s or person’s products and services from another’s.

Percona owns a number of marks, including but not limited to Percona, XtraDB, Percona XtraDB, XtraBackup, Percona XtraBackup, Percona Server for MySQL, and Percona Live, plus the distinctive visual icons and logos associated with these marks. Both the unregistered and registered marks of Percona are protected.

Use of any Percona trademark in the name, URL, or another identifying characteristic of any product, service, website, or other use is not permitted without Percona’s written permission with the following three limited exceptions.

First, you may use the appropriate Percona mark when making a nominative fair use reference to a bona fide Percona product.

Second, when Percona has released a product under a version of the GNU General Public License (“GPL”), you may use the appropriate Percona mark when distributing a verbatim copy of that product in accordance with the terms and conditions of the GPL.

Third, you may use the appropriate Percona mark to refer to a distribution of GPL-released Percona software that has been modified with minor changes for the sole purpose of allowing the software to operate on an operating system or hardware platform for which Percona has not yet released the software, provided that those third party changes do not affect the behavior, functionality, features, design or performance of the software. Users who acquire this Percona-branded software receive substantially exact implementations of the Percona software.

Percona reserves the right to revoke this authorization at any time in its sole discretion. For example, if Percona believes that your modification is beyond the scope of the limited license granted in this Policy or that your use of the Percona mark is detrimental to Percona, Percona will revoke this authorization. Upon revocation, you must immediately cease using the applicable Percona mark. If you do not immediately cease using the Percona mark upon revocation, Percona may take action to protect its rights and interests in the Percona mark. Percona does not grant any license to use any Percona mark for any other modified versions of Percona software; such use will require our prior written permission.

Neither trademark law nor any of the exceptions set forth in this Trademark Policy permit you to truncate, modify, or otherwise use any Percona mark as part of your own brand. For example, if XYZ creates a modified version of the Percona Server for MySQL, XYZ may not brand that modification as “XYZ Percona Server” or “Percona XYZ Server”, even if that modification otherwise complies with the third exception noted above.

In all cases, you must comply with applicable law, the underlying license, and this Trademark Policy, as amended from time to time. For instance, any mention of Percona trademarks should include the full trademarked name, with proper spelling and capitalization, along with attribution of ownership to Percona Inc. For example, the full proper name for XtraBackup is Percona XtraBackup. However, it is acceptable to omit the word “Percona” for brevity on the second and subsequent uses, where such omission does not cause confusion.

In the event of doubt as to any of the conditions or exceptions outlined in this Trademark Policy, please contact [email protected] for assistance and we will do our very best to be helpful.

Index of INFORMATION_SCHEMA tables

This is a list of the INFORMATION_SCHEMA TABLES that exist in Percona Server for MySQL with XtraDB. The entry for each table points to the page in the documentation where it’s described.

Frequently asked questions

Q: Will Percona Server for MySQL with XtraDB invalidate our MySQL support?

A: We don’t know the details of your support contract. You should check with your Oracle representative. We have heard anecdotal stories from MySQL Support team members that they have customers who use Percona Server for MySQL with XtraDB, but you should not base your decision on that.

Q: Will we have to GPL our whole application if we use Percona Server for MySQL with XtraDB?

A: This is a common misconception about the GPL. We suggest reading the Free Software Foundation ‘s excellent reference material on the GPL Version 2 , which is the license that applies to MySQL and therefore to Percona Server for MySQL with XtraDB. That document contains links to many other documents which should answer your questions. Percona is unable to give legal advice about the GPL.

Q: Do I need to install Percona client libraries?

A: No, you don’t need to change anything on the clients. Percona Server for MySQL is 100% compatible with all existing client libraries and connectors.

Q: When using the Percona XtraBackup to set up a replication replica on Debian-based systems I’m getting: “ERROR 1045 (28000): Access denied for user ‘debian-sys-maint’@’localhost’ (using password: YES)”

A: In case you’re using the init script on Debian-based system to start mysqld, be sure that the password for debian-sys-maint user has been updated and it’s the same as that user’s password from the server that the backup has been taken from. The password can be seen and updated in /etc/mysql/debian.cnf. For more information on how to set up a replication replica using Percona XtraBackup see this how-to ).

Copyright and licensing information

Percona Server for MySQL documentation is (C)2009-2026 Percona LLC and/or its affiliates and is distributed under the Creative Commons Attribution 4.0 International License .

Percona Server for MySQL is built upon MySQL from Oracle. Along with making our own modifications, we merge in changes from other sources such as community contributions and changes from MariaDB.

The original SHOW USER/TABLE/INDEX statistics code came from Google.

Percona does not require copyright assignment.

See the COPYING files accompanying the software distribution.

Glossary

ACID

Set of properties that guarantee database transactions are processed reliably. Stands for Atomicity, Consistency, Isolation, Durability.

Atomicity

Atomicity means that database operations are applied following a “all or nothing” rule. A transaction is either fully applied or not at all.

Atomic DDL (Data Definition Language)

Atomic DDL (Data Definition Language) refers to the concept where DDL operations such as CREATE, ALTER, DROP, or RENAME statements are executed in a way that ensures atomicity, meaning they either fully succeed or have no effect at all.

This means that if a DDL statement fails, the database automatically rolls back any partial changes, leaving the schema in a consistent state.

Consistency

Consistency means that each transaction that modifies the database takes it from one consistent state to another.

Durability

Once a transaction is committed, it will remain so.

Foreign Key

A referential constraint between two tables. Example: A purchase order in the purchase_orders table must have been made by a customer that exists in the customers table.

General Availability (GA)

A finalized version of the product which is made available to the general public. It is the final stage in the software release cycle.

Isolation

The Isolation requirement means that no transaction can interfere with another.

InnoDB

A Storage Engine for MySQL and derivatives (Percona Server, MariaDB) originally written by Innobase Oy, since acquired by Oracle. It provides ACID compliant storage engine with foreign key support. As of MySQL version 5.5, InnoDB became the default storage engine on all platforms.

Jenkins

Jenkins is a continuous integration system that we use to help ensure the continued quality of the software we produce. It helps us achieve the aims of:

  • no failed tests in the trunk on any platform

  • aid developers in ensuring merge requests build and test on all platform

  • no known performance regressions (without a damn good explanation).

LSN

The Log Sequence Number (LSN) is an 8-byte number. Every data change adds an entry to the redo log and generates an LSN. The server increments the LSN with every change.

MariaDB

A fork of MySQL that is maintained primarily by Monty Program AB. It aims to add features, and fix bugs while maintaining 100% backward compatibility with MySQL.

my.cnf

The file name of the default MySQL configuration file.

MyISAM

A MySQL Storage Engine that was the default until MySQL 5.5.

MySQL

An open source database that has spawned several distributions and forks. MySQL AB was the primary maintainer and distributor until bought by Sun Microsystems, which was then acquired by Oracle. As Oracle owns the MySQL trademark, the term MySQL is often used for the Oracle distribution of MySQL as distinct from the drop-in replacements such as MariaDB and Percona Server for MySQL.

NUMA

Non-Uniform Memory Access (NUMA ) is a computer memory design used in multiprocessing, where the memory access time depends on the memory location relative to a processor. Under NUMA, a processor can access its own local memory faster than non-local memory, that is, memory local to another processor or memory shared between processors. The whole system may still operate as one unit, and all memory is basically accessible from everywhere but at a potentially higher latency and lower performance.

Percona Server for MySQL

The Percona branch of MySQL with performance and management improvements.

Storage Engine

A storage engine is a piece of software that implements the details of data storage and retrieval for a database system. This term is primarily used within the MySQL ecosystem due to it being the first widely used relational database to have an abstraction layer around storage. It is analogous to a Virtual File System layer in an Operating System. A VFS layer allows an operating system to read and write multiple file systems (e.g. FAT, NTFS, XFS, ext3) and a Storage Engine layer allows a database server to access tables stored in different engines (for example, MyISAM or InnoDB).

Tech Preview

A tech preview item can be a feature, a variable, or a value within a variable. The term designates that the item is not yet ready for production use and is not included in support by SLA. A tech preview item is included in a release so that users can provide feedback. The item is either updated and released as general availability(GA) or removed if not useful. The item’s functionality can change from tech preview to GA.

XtraDB

The Percona improved version of InnoDB provides performance, features, and reliability above what is shipped by Oracle in InnoDB.

How we use artificial intelligence

The technical writer oversees the integration of AI-driven tools and platforms into the documentation workflow, ensuring that AI-generated text meets the standards for clarity, coherence, and accuracy. While AI assists in tasks such as content generation, language enhancement, and formatting optimization, the technical writer is responsible for validating and refining the output to ensure its suitability for the intended audience.

Throughout the documentation process, the technical writer reviews the quality and relevance of AI-generated content in detail and with critical judgment. By leveraging their expertise in language, communication, and subject matter knowledge, the technical writer collaborates with AI systems to refine and tailor the documentation to meet the specific needs and preferences of the audience.

While AI accelerates the documentation process and enhances productivity, the technical writer verifies the information’s accuracy and maintains consistency in terminology, style, and tone. The technical writer ensures that the final document reflects the company’s commitment to excellence.

Documentation Index