Skip to content
logo
Percona Server for MySQL
TokuDB files and file types
Initializing search
    percona/psmysql-docs
    percona/psmysql-docs
    • Home
      • Release notes index
      • Percona Server for MySQL 8.0.34-26 (2023-09-26)
      • Percona Server for MySQL 8.0.33-25 Update (2023-08-02)
      • Percona Server for MySQL 8.0.33-25 (2023-06-15)
      • Percona Server for MySQL 8.0.32-24 (2023-03-20)
      • Percona Server for MySQL 8.0.31-23 (2023-02-09)
        • Percona Server for MySQL 8.0.30-22 Update (2022-11-21)
        • Percona Server for MySQL 8.0.30-22 (2022-11-21)
        • Percona Server for MySQL 8.0.29-21 (2022-08-08)
        • Percona Server for MySQL 8.0.28-20 (2022-06-20)
        • Percona Server for MySQL 8.0.28-19 (2022-05-12)
        • Percona Server for MySQL 8.0.27-18 (2022-03-02)
        • Percona Server for MySQL 8.0.26-17 (2022-01-26)
        • Percona Server for MySQL 8.0.26-16 (2021-10-20)
        • Percona Server for MySQL 8.0.25-15 (2021-07-13)
        • Percona Server for MySQL 8.0.23-14 (2021-05-12)
        • Percona Server for MySQL 8.0.22-13 (2020-12-14)
        • Percona Server for MySQL 8.0.21-12 (2020-10-13)
        • Percona Server for MySQL 8.0.20-11 (2020-07-21)
        • Percona Server for MySQL 8.0.19-10 (2020-03-23)
        • Percona Server for MySQL 8.0.18-9
        • Percona Server for MySQL 8.0.17-8
        • Percona Server for MySQL 8.0.16-7
        • Percona Server for MySQL 8.0.15-6
        • Percona Server for MySQL 8.0.15-5
        • Percona Server for MySQL 8.0.14
        • Percona Server for MySQL 8.0.13-4
        • Percona Server for MySQL 8.0.13-3
        • Percona Server for MySQL 8.0.12-2rc1
      • Adaptive network buffers
        • Audit Log Filter overview
        • Install the Audit Log Filter
          • Overview
          • XML (New style)
          • XML (Old style)
          • JSON
        • Audit Log Filter security
        • Audit Log Filter compression and encryption
        • Reading Audit Log Filter files
        • Manage the Audit Log Filter files
        • Filter the Audit Log Filter logs
        • Audit Log Filter restrictions
        • Audit Log Filter file naming conventions
        • Disable Audit Log Filter logging
        • Audit log filter functions, options and variables
        • Uninstall Audit Log Filter
      • Limiting the disk space used by binary log files
      • Extended mysqlbinlog
      • Extended SELECT INTO OUTFILE/DUMPFILE
      • Expanded fast index creation
      • Kill idle transactions
      • The ProcFS plugin
      • Support for PROXY protocol
      • SEQUENCE_TABLE(n) function
      • Slow query log rotation and expiration
      • Thread pool
      • Trigger updates
      • Percona Toolkit UDFs
      • Utility user
      • Quickstart guide for Percona Server for MySQL
      • Install Percona Server for MySQL from repositories
        • Percona Product Download Instructions
        • Use APT repositories
        • Files in DEB package
        • Build APT packages
        • Downloaded DEB packages
        • Apt pinning
        • Run Percona Server for MySQL
        • Uninstall
        • Use RPM repositories
        • Files in RPM package
        • Downloaded RPM packages
        • Run Percona Server for MySQL
        • Uninstall
        • Install with binary tarballs
        • Binary tarballs available
        • Install Percona Server for MySQL from a source tarball
        • Compile Percona Server for MySQL 8.0 from source
        • Install using Docker
        • Docker environment variables
      • Upgrade from 5.7 to 8.0 overview
      • Plan an upgrade
      • Upgrade strategies
        • General changes
        • InnoDB changes
        • Security & account management changes in MySQL 8.0
        • Deprecated in MySQL 8.0
        • Removed in MySQL 8.0
      • Percona Tools that can help with an upgrade
      • Percona Server for MySQL in-place upgrade guide: from 5.7 to 8.0
      • Upgrade by migrating from one environment to another
      • Upgrade using the Percona repositories
      • Upgrade from systems that use the MyRocks or TokuDB storage engine and partitioned tables
      • Upgrade using Standalone Packages
      • Downgrade Percona Server for MySQL
      • Working with AppArmor
      • Binary logs and replication improvements
      • Post-installation
      • Working with SELinux
      • Extended SHOW GRANTS
      • UNINSTALL COMPONENT
      • Backup and restore overview
      • Backup locks
      • Extended mysqldump
      • Start transaction with consistent snapshot
        • Using LDAP authentication plugins
        • LDAP authentication plugin system variables
        • Data masking overview
        • Compare the data masking component to the data masking plugin
          • Install the data masking component
          • Data masking component functions
          • Uninstall the data masking component
          • Install and remove the data masking plugin
          • Data masking plugin functions
      • FIDO authentication plugin
      • Encryption functions
      • PAM authentication plugin
      • SSL improvements
      • The secure_log_path variable
        • Data at Rest Encryption
        • Use the keyring component or keyring plugin
          • Using the Key Management Interoperability Protocol (KMIP)
          • Use the Amazon Key Management Service (AWS KMS)
          • Encrypt File-Per-Table Tablespace
          • Encrypt schema or general tablespace
          • Encrypt system tablespace
          • Encrypt temporary files
          • Encrypt Binary Log Files and Relay Log Files
          • Encrypting the Redo Log data
          • Encrypt the undo tablespace
          • Rotate the master key
          • Advanced encryption key rotation
          • Encrypt doublewrite buffers
          • Verify the encryption for tables, tablespaces, and schemas
      • Manage group replication flow control
      • Group replication system variables
      • Audit log plugin
      • Jemalloc memory allocation profiling
      • Misc. INFORMATION_SCHEMA tables
      • Process list
      • Slow query log
      • User statistics
      • Use Percona Monitoring and Management (PMM) Advisors
      • Handle corrupted tables
      • Libcoredumper
      • Too many connections warning
      • Stacktrace
      • Thread based profiling
        • Multiple page asynchronous I/O requests
        • XtraDB changed page tracking
        • Compressed columns with dictionaries
        • Enforcing storage engine
        • Improved MEMORY storage engine
        • InnoDB page fragmentation counters
        • InnoDB full-text search improvements
        • Improved InnoDB I/O scalability
        • Extended show engine InnoDB status
        • The Percona XtraDB storage engine
        • Prefix index queries optimization
        • Limit the estimation of records in a Query
        • Show storage engines
        • XtraDB performance improvements for I/O-bound highly-concurrent workloads
        • Percona MyRocks introduction
        • Percona MyRocks installation guide
        • Updated supported features
        • MyRocks limitations
        • Differences between Percona MyRocks and Facebook MyRocks
        • MyRocks column families
        • Performance Schema MyRocks changes
        • MyRocks Information Schema tables
        • MyRocks server variables
        • MyRocks status variables
        • Gap locks detection
        • MyRocks data loading
        • Installing and configuring Percona Server for MySQL with ZenFS support
        • TokuDB introduction
        • Get started with TokuDB
        • TokuDB installation
        • Use TokuDB
        • Fast updates with TokuDB
        • TokuDB files and file types
          • tokudb.environment
          • tokudb.rollback
          • tokudb.directory
          • Dictionary files
          • Recovery log files
          • Temporary files
          • Lock files
        • TokuDB file management
        • TokuDB background ANALYZE TABLE
        • TokuDB variables
        • TokuDB status variables
        • TokuDB fractal tree indexing
        • TokuDB troubleshooting
        • TokuDB Performance Schema integration
        • TokuDB frequently asked questions
        • Migrate and remove the TokuDB storage engine
        • Percona TokuBackup
      • Topic index
      • Reserved keywords
      • List of variables introduced in Percona Server for MySQL 8.0
      • List of features available in Percona Server for MySQL releases
      • Percona Server for MySQL feature comparison
      • Understand version numbers
      • Development of Percona Server for MySQL
      • Trademark policy
      • Index of INFORMATION_SCHEMA tables
      • Frequently asked questions
      • Copyright and licensing information
      • Glossary

    • tokudb.environment
    • tokudb.rollback
    • tokudb.directory
    • Dictionary files
    • Recovery log files
    • Temporary files
    • Lock files

    TokuDB files and file types¶

    Important

    Starting with Percona Server for MySQL 8.0.28-19, the TokuDB storage engine is no longer supported. We have removed the storage engine from the installation packages and disabled the storage engine in our binary builds.

    Starting with Percona Server for MySQL 8.0.26-16, the binary builds and packages include but disable the TokuDB storage engine plugins. The tokudb_enabled option and the tokudb_backup_enabled option control the state of the plugins and have a default setting of FALSE. The result of attempting to load the plugins are the plugins fail to initialize and print a deprecation message.

    We recommend Migrating the data to the MyRocks storage engine. To enable the plugins to migrate to another storage engine, set the tokudb_enabled and tokudb_backup_enabled options to TRUE in your my.cnf file and restart your server instance. Then, you can load the plugins.

    The TokuDB storage engine was declared as deprecated in Percona Server for MySQL 8.0. For more information, see the Percona blog post: Heads-Up: TokuDB Support Changes and Future Removal from Percona Server for MySQL 8.0.

    The TokuDB file set consists of many different files that all serve various purposes.

    If you have any TokuDB data your data directory should look similar to this:

    root@server:/var/lib/mysql# ls -lah
    

    The output could be:

    ...
    -rw-rw----  1 mysql mysql  76M Oct 13 18:45 ibdata1
    ...
    -rw-rw----  1 mysql mysql  16K Oct 13 15:52 tokudb.directory
    -rw-rw----  1 mysql mysql  16K Oct 13 15:52 tokudb.environment
    -rw-------  1 mysql mysql    0 Oct 13 15:52 __tokudb_lock_dont_delete_me_data
    -rw-------  1 mysql mysql    0 Oct 13 15:52 __tokudb_lock_dont_delete_me_environment
    -rw-------  1 mysql mysql    0 Oct 13 15:52 __tokudb_lock_dont_delete_me_logs
    -rw-------  1 mysql mysql    0 Oct 13 15:52 __tokudb_lock_dont_delete_me_recovery
    -rw-------  1 mysql mysql    0 Oct 13 15:52 __tokudb_lock_dont_delete_me_temp
    -rw-rw----  1 mysql mysql  16K Oct 13 15:52 tokudb.rollback
    ...
    

    This document lists the different types of TokuDB and Percona Fractal Tree files, explains their purpose, shows their location and how to move them around.

    tokudb.environment¶

    This file is the root of the Percona FT file set and contains various bits of metadata about the system, such as creation times, current file format versions, etc.

    Percona FT will create/expect this file in the directory specified by the MySQL datadir.

    tokudb.rollback¶

    Every transaction within Percona FT maintains its own transaction rollback log. These logs are stored together within a single Percona FT dictionary file and take up space within the Percona FT cachetable (just like any other Percona FT dictionary).

    The transaction rollback logs will undo any changes made by a transaction if the transaction is explicitly rolled back, or rolled back via recovery as a result of an uncommitted transaction when a crash occurs.

    Percona FT will create/expect this file in the directory specified by the MySQL datadir.

    tokudb.directory¶

    Percona FT maintains a mapping of a dictionary name (example: sbtest.sbtest1.main) to an internal file name (example: _sbtest_sbtest1_main_xx_x_xx.tokudb). This mapping is stored within this single Percona FT dictionary file and takes up space within the Percona FT cachetable just like any other Percona FT dictionary.

    Percona FT will create/expect this file in the directory specified by the MySQL datadir.

    Dictionary files¶

    TokuDB dictionary (data) files store actual user data. For each MySQL table there will be:

    • One status dictionary that contains metadata about the table.

    • One main dictionary that stores the full primary key (an imaginary key is used if one was not explicitly specified) and full row data.

    • One key dictionary for each additional key/index on the table.

    These are typically named: _<database>_<table>_<key>_<internal_txn_id>.tokudb

    Percona FT creates/expects these files in the directory specified by tokudb_data_dir if set, otherwise the MySQL datadir is used.

    Recovery log files¶

    The Percona FT recovery log records every operation that modifies a Percona FT dictionary. Periodically, the system will take a snapshot of the system called a checkpoint. This checkpoint ensures that the modifications recorded within the Percona FT recovery logs have been applied to the appropriate dictionary files up to a known point in time and synced to disk.

    These files have a rolling naming convention, but use: log<log_file_number>.tokulog<log_file_format_version>.

    Percona FT creates/expects these files in the directory specified by tokudb_log_dir if set, otherwise the MySQL datadir is used.

    Percona FT does not track what log files should or shouldn’t be present. Upon startup, it discovers the logs in the log directory, and replays them in order. If the wrong logs are present, the recovery aborts and possibly damages the dictionaries.

    Temporary files¶

    Percona FT might need to create some temporary files in order to perform some operations. When the bulk loader is active, these temporary files might grow to be quite large.

    As different operations start and finish, the files will come and go.

    There are no temporary files left behind upon a clean shutdown,

    Percona FT creates/expects these files in the directory specified by tokudb_tmp_dir if set. If not, the tokudb_data_dir is used if set, otherwise the MySQL datadir is used.

    Lock files¶

    Percona FT uses lock files to prevent multiple processes from accessing and writing to the files in the assorted Percona FT functionality areas. Each lock file will be in the same directory as the file(s) that it is protecting.

    These empty files are only used as semaphores across processes. They are safe to delete/ignore as long as no server instances are currently running and using the data set.

    __tokudb_lock_dont_delete_me_environment

    __tokudb_lock_dont_delete_me_recovery

    __tokudb_lock_dont_delete_me_logs

    __tokudb_lock_dont_delete_me_data

    __tokudb_lock_dont_delete_me_temp

    Percona FT validates its data set. If a file goes missing or not found, or seems to contain some nonsensical data, it will assert, abort or fail to start. It does this not to annoy you, but to try to protect you from doing any further damage to your data.

    Contact us

    For free technical help, visit the Percona Community Forum.

    To report bugs or submit feature requests, open a JIRA ticket.

    For paid support and managed or consulting services , contact Percona Sales.


    Last update: 2023-09-27
    Percona LLC and/or its affiliates, © 2023
    Made with Material for MkDocs

    Cookie consent

    We use cookies to recognize your repeated visits and preferences, as well as to measure the effectiveness of our documentation and whether users find what they're searching for. With your consent, you're helping us to make our documentation better. Read more about Percona Cookie Policy.