Skip to content
logo
Percona Server for MySQL
XtraDB changed page tracking
Initializing search
    percona/psmysql-docs
    percona/psmysql-docs
    • Home
      • Release notes index
      • Percona Server for MySQL 8.0.34-26 (2023-09-26)
      • Percona Server for MySQL 8.0.33-25 Update (2023-08-02)
      • Percona Server for MySQL 8.0.33-25 (2023-06-15)
      • Percona Server for MySQL 8.0.32-24 (2023-03-20)
      • Percona Server for MySQL 8.0.31-23 (2023-02-09)
        • Percona Server for MySQL 8.0.30-22 Update (2022-11-21)
        • Percona Server for MySQL 8.0.30-22 (2022-11-21)
        • Percona Server for MySQL 8.0.29-21 (2022-08-08)
        • Percona Server for MySQL 8.0.28-20 (2022-06-20)
        • Percona Server for MySQL 8.0.28-19 (2022-05-12)
        • Percona Server for MySQL 8.0.27-18 (2022-03-02)
        • Percona Server for MySQL 8.0.26-17 (2022-01-26)
        • Percona Server for MySQL 8.0.26-16 (2021-10-20)
        • Percona Server for MySQL 8.0.25-15 (2021-07-13)
        • Percona Server for MySQL 8.0.23-14 (2021-05-12)
        • Percona Server for MySQL 8.0.22-13 (2020-12-14)
        • Percona Server for MySQL 8.0.21-12 (2020-10-13)
        • Percona Server for MySQL 8.0.20-11 (2020-07-21)
        • Percona Server for MySQL 8.0.19-10 (2020-03-23)
        • Percona Server for MySQL 8.0.18-9
        • Percona Server for MySQL 8.0.17-8
        • Percona Server for MySQL 8.0.16-7
        • Percona Server for MySQL 8.0.15-6
        • Percona Server for MySQL 8.0.15-5
        • Percona Server for MySQL 8.0.14
        • Percona Server for MySQL 8.0.13-4
        • Percona Server for MySQL 8.0.13-3
        • Percona Server for MySQL 8.0.12-2rc1
      • Adaptive network buffers
        • Audit Log Filter overview
        • Install the Audit Log Filter
          • Overview
          • XML (New style)
          • XML (Old style)
          • JSON
        • Audit Log Filter security
        • Audit Log Filter compression and encryption
        • Reading Audit Log Filter files
        • Manage the Audit Log Filter files
        • Filter the Audit Log Filter logs
        • Audit Log Filter restrictions
        • Audit Log Filter file naming conventions
        • Disable Audit Log Filter logging
        • Audit log filter functions, options and variables
        • Uninstall Audit Log Filter
      • Limiting the disk space used by binary log files
      • Extended mysqlbinlog
      • Extended SELECT INTO OUTFILE/DUMPFILE
      • Expanded fast index creation
      • Kill idle transactions
      • The ProcFS plugin
      • Support for PROXY protocol
      • SEQUENCE_TABLE(n) function
      • Slow query log rotation and expiration
      • Thread pool
      • Trigger updates
      • Percona Toolkit UDFs
      • Utility user
      • Quickstart guide for Percona Server for MySQL
      • Install Percona Server for MySQL from repositories
        • Percona Product Download Instructions
        • Use APT repositories
        • Files in DEB package
        • Build APT packages
        • Downloaded DEB packages
        • Apt pinning
        • Run Percona Server for MySQL
        • Uninstall
        • Use RPM repositories
        • Files in RPM package
        • Downloaded RPM packages
        • Run Percona Server for MySQL
        • Uninstall
        • Install with binary tarballs
        • Binary tarballs available
        • Install Percona Server for MySQL from a source tarball
        • Compile Percona Server for MySQL 8.0 from source
        • Install using Docker
        • Docker environment variables
      • Upgrade from 5.7 to 8.0 overview
      • Plan an upgrade
      • Upgrade strategies
        • General changes
        • InnoDB changes
        • Security & account management changes in MySQL 8.0
        • Deprecated in MySQL 8.0
        • Removed in MySQL 8.0
      • Percona Tools that can help with an upgrade
      • Percona Server for MySQL in-place upgrade guide: from 5.7 to 8.0
      • Upgrade by migrating from one environment to another
      • Upgrade using the Percona repositories
      • Upgrade from systems that use the MyRocks or TokuDB storage engine and partitioned tables
      • Upgrade using Standalone Packages
      • Downgrade Percona Server for MySQL
      • Working with AppArmor
      • Binary logs and replication improvements
      • Post-installation
      • Working with SELinux
      • Extended SHOW GRANTS
      • UNINSTALL COMPONENT
      • Backup and restore overview
      • Backup locks
      • Extended mysqldump
      • Start transaction with consistent snapshot
        • Using LDAP authentication plugins
        • LDAP authentication plugin system variables
        • Data masking overview
        • Compare the data masking component to the data masking plugin
          • Install the data masking component
          • Data masking component functions
          • Uninstall the data masking component
          • Install and remove the data masking plugin
          • Data masking plugin functions
      • FIDO authentication plugin
      • Encryption functions
      • PAM authentication plugin
      • SSL improvements
      • The secure_log_path variable
        • Data at Rest Encryption
        • Use the keyring component or keyring plugin
          • Using the Key Management Interoperability Protocol (KMIP)
          • Use the Amazon Key Management Service (AWS KMS)
          • Encrypt File-Per-Table Tablespace
          • Encrypt schema or general tablespace
          • Encrypt system tablespace
          • Encrypt temporary files
          • Encrypt Binary Log Files and Relay Log Files
          • Encrypting the Redo Log data
          • Encrypt the undo tablespace
          • Rotate the master key
          • Advanced encryption key rotation
          • Encrypt doublewrite buffers
          • Verify the encryption for tables, tablespaces, and schemas
      • Manage group replication flow control
      • Group replication system variables
      • Audit log plugin
      • Jemalloc memory allocation profiling
      • Misc. INFORMATION_SCHEMA tables
      • Process list
      • Slow query log
      • User statistics
      • Use Percona Monitoring and Management (PMM) Advisors
      • Handle corrupted tables
      • Libcoredumper
      • Too many connections warning
      • Stacktrace
      • Thread based profiling
        • Multiple page asynchronous I/O requests
        • XtraDB changed page tracking
          • User statements for handling the XtraDB changed page bitmaps
          • Additional information in SHOW ENGINE INNODB STATUS
          • INFORMATION_SCHEMA tables
            • INFORMATION_SCHEMA.INNODB_CHANGED_PAGES
          • Version specific information
          • System variables
            • innodb_max_changed_pages
            • innodb_track_changed_pages
            • innodb_max_bitmap_file_size
        • Compressed columns with dictionaries
        • Enforcing storage engine
        • Improved MEMORY storage engine
        • InnoDB page fragmentation counters
        • InnoDB full-text search improvements
        • Improved InnoDB I/O scalability
        • Extended show engine InnoDB status
        • The Percona XtraDB storage engine
        • Prefix index queries optimization
        • Limit the estimation of records in a Query
        • Show storage engines
        • XtraDB performance improvements for I/O-bound highly-concurrent workloads
        • Percona MyRocks introduction
        • Percona MyRocks installation guide
        • Updated supported features
        • MyRocks limitations
        • Differences between Percona MyRocks and Facebook MyRocks
        • MyRocks column families
        • Performance Schema MyRocks changes
        • MyRocks Information Schema tables
        • MyRocks server variables
        • MyRocks status variables
        • Gap locks detection
        • MyRocks data loading
        • Installing and configuring Percona Server for MySQL with ZenFS support
        • TokuDB introduction
        • Get started with TokuDB
        • TokuDB installation
        • Use TokuDB
        • Fast updates with TokuDB
        • TokuDB files and file types
        • TokuDB file management
        • TokuDB background ANALYZE TABLE
        • TokuDB variables
        • TokuDB status variables
        • TokuDB fractal tree indexing
        • TokuDB troubleshooting
        • TokuDB Performance Schema integration
        • TokuDB frequently asked questions
        • Migrate and remove the TokuDB storage engine
        • Percona TokuBackup
      • Topic index
      • Reserved keywords
      • List of variables introduced in Percona Server for MySQL 8.0
      • List of features available in Percona Server for MySQL releases
      • Percona Server for MySQL feature comparison
      • Understand version numbers
      • Development of Percona Server for MySQL
      • Trademark policy
      • Index of INFORMATION_SCHEMA tables
      • Frequently asked questions
      • Copyright and licensing information
      • Glossary

    • User statements for handling the XtraDB changed page bitmaps
    • Additional information in SHOW ENGINE INNODB STATUS
    • INFORMATION_SCHEMA tables
      • INFORMATION_SCHEMA.INNODB_CHANGED_PAGES
    • Version specific information
    • System variables
      • innodb_max_changed_pages
      • innodb_track_changed_pages
      • innodb_max_bitmap_file_size

    XtraDB changed page tracking¶

    Important

    Starting with Percona Server for MySQL 8.0.30, the changed page tracking feature is removed and no longer supported.

    Starting with Percona Server for MySQL 8.0.27, the page tracking feature is deprecated and may be removed in future versions.

    We recommend using the MySQL page tracking feature. For more information, see MySQL InnoDB Clone and page tracking.

    XtraDB now tracks the pages that have changes written to them according to the redo log. This information is written out in special changed page bitmap files. This information can be used to speed up incremental backups using Percona XtraBackup by removing the need to scan whole data files to find the changed pages. Changed page tracking is done by a new XtraDB worker thread that reads and parses log records between checkpoints. The tracking is controlled by a new read-only server variable innodb_track_changed_pages.

    Bitmap filename format used for changed page tracking is ib_modified_log_<seq>_<startlsn>.xdb. The first number is the sequence number of the bitmap log file and the startlsn number is the starting LSN number of data tracked in that file. Example of the bitmap log files should look like this:

    Expected output
    ib_modified_log_1_0.xdb
    ib_modified_log_2_1603391.xdb
    

    Sequence number can be used to easily check if all the required bitmap files are present. Start LSN number will be used in XtraBackup and INFORMATION_SCHEMA queries to determine which files have to be opened and read for the required LSN interval data. The bitmap file is rotated on each server restart and whenever the current file size reaches the predefined maximum. This maximum is controlled by a new innodb_max_bitmap_file_size variable.

    Old bitmap files may be safely removed after a corresponding incremental backup is taken. For that there are server User statements for handling the XtraDB changed page bitmaps. Removing the bitmap files from the filesystem directly is safe too, as long as care is taken not to delete data for not-yet-backuped LSN range.

    This feature will be used for implementing faster incremental backups that use this information to avoid full data scans in Percona XtraBackup.

    User statements for handling the XtraDB changed page bitmaps¶

    New statements have been introduced for handling the changed page bitmap tracking. All of these statements require SUPER privilege.

    • FLUSH CHANGED_PAGE_BITMAPS - this statement can be used for synchronous bitmap write for immediate catch-up with the log checkpoint. This is used by innobackupex to make sure that XtraBackup indeed has all the required data it needs.

    • RESET CHANGED_PAGE_BITMAPS - this statement will delete all the bitmap log files and restart the bitmap log file sequence.

    • PURGE CHANGED_PAGE_BITMAPS BEFORE <lsn> - this statement will delete all the change page bitmap files up to the specified log sequence number.

    Additional information in SHOW ENGINE INNODB STATUS¶

    When log tracking is enabled, the following additional fields are displayed in the LOG section of the SHOW ENGINE INNODB STATUS output:

    • “Log tracked up to:” displays the LSN up to which all the changes have been parsed and stored as a bitmap on disk by the log tracking thread

    • “Max tracked LSN age:” displays the maximum limit on how far behind the log tracking thread may be.

    Note

    Implemented in Percona Server for MySQL 8.0.13-4, a new InnoDB monitor, log_writer_on_tracker_waits, records log writer waits due to changed page tracking lag. This log writer works in parallel with other log_writer_on_[*]_ waits monitors.

    INFORMATION_SCHEMA tables¶

    This table contains a list of modified pages from the bitmap file data. As these files are generated by the log tracking thread parsing the log whenever the checkpoint is made, it is not real-time data.

    INFORMATION_SCHEMA.INNODB_CHANGED_PAGES¶

    Column Name Description
    ‘INT(11) space_id’ ‘space id of modified page’
    ‘INT(11) page_id’ ‘id of modified page’
    ‘BIGINT(21) start_lsn’ ‘start of the interval’
    ‘BIGINT(21) end_lsn’ ‘end of the interval ‘

    The start_lsn and the end_lsn columns denote between which two checkpoints this page was changed at least once. They are also equal to checkpoint LSNs.

    Number of records in this table can be limited by using the variable innodb_max_changed_pages.

    Version specific information¶

    • 8.0.12-1: The feature was ported from Percona Server for MySQL 5.7.

    System variables¶

    innodb_max_changed_pages¶

    Option Description
    Command Line: Yes
    Config file Yes
    Scope: Global
    Dynamic: Yes
    Data type Numeric
    Default value 1000000
    Range 1 - 0 (unlimited)

    This variable is used to limit the result row count for the queries from INFORMATION_SCHEMA.INNODB_CHANGED_PAGES table.

    innodb_track_changed_pages¶

    Option Description
    Command Line: Yes
    Config file Yes
    Scope: Global
    Dynamic: No
    Data type Boolean
    Default value 0 - False
    Range 0-1

    This variable is used to enable/disable XtraDB changed page tracking feature.

    innodb_max_bitmap_file_size¶

    Option Description
    Command Line: Yes
    Config file Yes
    Scope: Global
    Dynamic: Yes
    Data type Numeric
    Default value 104857600 (100 MB)
    Range 4096 (4KB) - 18446744073709551615 (16EB)

    This variable is used to control maximum bitmap size after which the file will be rotated.

    Contact us

    For free technical help, visit the Percona Community Forum.

    To report bugs or submit feature requests, open a JIRA ticket.

    For paid support and managed or consulting services , contact Percona Sales.


    Last update: 2023-09-27
    Percona LLC and/or its affiliates, © 2023
    Made with Material for MkDocs

    Cookie consent

    We use cookies to recognize your repeated visits and preferences, as well as to measure the effectiveness of our documentation and whether users find what they're searching for. With your consent, you're helping us to make our documentation better. Read more about Percona Cookie Policy.