Skip to content
logo
Percona Server for MySQL
TokuDB introduction
Initializing search
    percona/psmysql-docs
    percona/psmysql-docs
    • Home
      • The Percona XtraDB storage engine
      • List of features available in Percona Server for MySQL releases
      • Percona Server for MySQL feature comparison
      • Changed in Percona Server 8.0
      • Understand version numbers
      • Install Percona Server for MySQL
        • Install using APT repository
        • Files in DEB package
        • Build APT packages
        • Install from downloaded DEB packages
        • Apt pinning
        • Run Percona Server for MySQL
        • Uninstall
        • Install using a RPM repository
        • Files in RPM package
        • Install from downloaded RPM packages
        • Run Percona Server for MySQL
        • Uninstall
        • Install with binary tarballs
        • Binary tarballs available
        • Install Percona Server for MySQL from a source tarball
        • Compile Percona Server for MySQL 8.0 from source
      • Post-installation
      • Percona Server for MySQL in-place upgrading guide: from 5.7 to 8.0
      • Upgrade using the Percona repositories
      • Upgrade from systems that use the MyRocks or TokuDB storage engine and partitioned tables
      • Upgrade using Standalone Packages
      • Downgrade Percona Server for MySQL
      • Running Percona Server for MySQL in a Docker Container
      • Docker environment variables
      • Improved InnoDB I/O scalability
      • Adaptive network buffers
      • Multiple page asynchronous I/O requests
      • Thread pool
      • XtraDB performance improvements for I/O-bound highly-concurrent workloads
      • Prefix index queries optimization
      • Limit the estimation of records in a Query
      • Jemalloc memory allocation profiling
      • The ProcFS plugin
      • Binary logs and replication improvements
      • Compressed columns with dictionaries
      • Extended SELECT INTO OUTFILE/DUMPFILE
      • Extended SET VAR optimizer hint
      • Improved MEMORY storage engine
      • Suppress warning messages
      • Limiting the disk space used by binary log files
      • Support for PROXY protocol
      • SEQUENCE_TABLE(n) function
      • Slow query log rotation and expiration
      • Trigger updates
      • Extended mysqlbinlog
      • Extended mysqldump
      • InnoDB full-text search improvements
      • Too many connections warning
      • Handle corrupted tables
      • Percona Toolkit UDFs
      • Kill idle transactions
      • XtraDB changed page tracking
      • Enforcing storage engine
      • Expanded fast index creation
      • Backup locks
      • Audit log plugin
      • Start transaction with consistent snapshot
      • Extended SHOW GRANTS
      • Utility user
      • Working with SELinux
      • Working with AppArmor
      • PAM authentication plugin
      • Server variables
      • SSL improvements
      • Data masking
        • Using LDAP authentication plugins
        • LDAP authentication plugin system variables
        • Data at Rest Encryption
        • Use the keyring component or keyring plugin
          • Using the Key Management Interoperability Protocol (KMIP)
          • Use the Amazon Key Management Service (AWS KMS)
          • FIDO authentication plugin
        • Encryption functions
        • Encrypt File-Per-Table Tablespace
        • Encrypt schema or general tablespace
        • Encrypt system tablespace
        • Encrypt temporary files
        • Encrypt Binary Log Files and Relay Log Files
        • Encrypting the Redo Log data
        • Encrypt the undo tablespace
        • Rotate the master key
        • Advanced encryption key rotation
        • Encrypt doublewrite buffers
        • Verify the encryption for tables, tablespaces, and schemas
      • User statistics
      • Slow query log
      • Extended show engine InnoDB status
      • Show storage engines
      • Process list
      • Misc. INFORMATION_SCHEMA tables
      • Thread based profiling
      • InnoDB page fragmentation counters
      • Stacktrace
      • Libcoredumper
      • Manage group replication flow control
      • Group replication system variables
      • Percona MyRocks introduction
      • Percona MyRocks installation guide
      • Updated supported features
      • MyRocks limitations
      • Differences between Percona MyRocks and Facebook MyRocks
      • MyRocks Information Schema tables
      • MyRocks server variables
      • MyRocks status variables
      • Gap locks detection
      • Data loading
      • Installing and configuring Percona Server for MySQL with ZenFS support
      • TokuDB introduction
        • Getting the Most from TokuDB
      • TokuDB installation
      • Use TokuDB
      • Fast updates with TokuDB
      • TokuDB files and file types
      • TokuDB file management
      • TokuDB background ANALYZE TABLE
      • TokuDB variables
      • TokuDB status variables
      • TokuDB fractal tree indexing
      • TokuDB troubleshooting
      • TokuDB Performance Schema integration
      • Frequently asked questions
      • Migrate and removing the TokuDB storage engine
      • Percona TokuBackup
      • Release notes index
      • Percona Server for MySQL 8.0.32-24 (2023-03-20)
      • Percona Server for MySQL 8.0.31-23 (2023-02-09)
      • Percona Server for MySQL 8.0.30-22 Update (2022-11-21)
      • Percona Server for MySQL 8.0.30-22 (2022-11-21)
      • Percona Server for MySQL 8.0.29-21 (2022-08-08)
      • Percona Server for MySQL 8.0.28-20 (2022-06-20)
      • Percona Server for MySQL 8.0.28-19 (2022-05-12)
      • Percona Server for MySQL 8.0.27-18 (2022-03-02)
      • Percona Server for MySQL 8.0.26-17 (2022-01-26)
      • Percona Server for MySQL 8.0.26-16 (2021-10-20)
      • Percona Server for MySQL 8.0.25-15 (2021-07-13)
      • Percona Server for MySQL 8.0.23-14 (2021-05-12)
      • Percona Server for MySQL 8.0.22-13 (2020-12-14)
      • Percona Server for MySQL 8.0.21-12 (2020-10-13)
      • Percona Server for MySQL 8.0.20-11 (2020-07-21)
      • Percona Server for MySQL 8.0.19-10 (2020-03-23)
      • Percona Server for MySQL 8.0.18-9
      • Percona Server for MySQL 8.0.17-8
      • Percona Server for MySQL 8.0.16-7
      • Percona Server for MySQL 8.0.15-6
      • Percona Server for MySQL 8.0.15-5
      • Percona Server for MySQL 8.0.14
      • Percona Server for MySQL 8.0.13-4
      • Percona Server for MySQL 8.0.13-3
      • Percona Server for MySQL 8.0.12-2rc1
      • List of variables introduced in Percona Server for MySQL 8.0
      • Development of Percona Server for MySQL
      • Trademark policy
      • Index of INFORMATION_SCHEMA tables
      • Frequently asked questions
      • Copyright and licensing information
      • Glossary

    • Getting the Most from TokuDB

    TokuDB introduction¶

    Important

    Starting with Percona Server for MySQL 8.0.28-19, the TokuDB storage engine is no longer supported. We have removed the storage engine from the installation packages and disabled the storage engine in our binary builds.

    Starting with Percona Server for MySQL 8.0.26-16, the binary builds and packages include but disable the TokuDB storage engine plugins. The tokudb_enabled option and the tokudb_backup_enabled option control the state of the plugins and have a default setting of FALSE. The result of attempting to load the plugins are the plugins fail to initialize and print a deprecation message.

    We recommend Migrating the data to the MyRocks storage engine. To enable the plugins to migrate to another storage engine, set the tokudb_enabled and tokudb_backup_enabled options to TRUE in your my.cnf file and restart your server instance. Then, you can load the plugins.

    The TokuDB storage engine was declared as deprecated in Percona Server for MySQL 8.0. For more information, see the Percona blog post: Heads-Up: TokuDB Support Changes and Future Removal from Percona Server for MySQL 8.0.

    TokuDB is a highly scalable, zero-maintenance downtime MySQL storage engine that delivers indexing-based query acceleration, improved replication performance, unparalleled compression, and live schema modification. The TokuDB storage engine is a scalable, ACID and MVCC compliant storage engine that provides indexing-based query improvements, offers online schema modifications, and reduces replica lag for both hard disk drives and flash memory. This storage engine is specifically designed for high performance on write-intensive workloads which is achieved with Fractal Tree indexing.

    Percona Server for MySQL is compatible with the separately available TokuDB storage engine package. The TokuDB engine must be separately downloaded and then enabled as a plug-in component. This package can be installed alongside with standard Percona Server for MySQL releases and does not require any specially adapted version of Percona Server for MySQL.

    Warning

    Only the Percona supplied TokuDB engine should be used with Percona Server for MySQL. A TokuDB engine downloadedfrom other sources is not compatible. TokuDB file formats are not the same across MySQL variants. Migrating from one variant to any other variant requires a logical data dump and reload.

    Additional features unique to TokuDB include:

    • Up to 25x Data Compression

    • Fast Inserts

    • Eliminates Replica Lag with Read Free Replication

    • Hot Schema Changes

    • Hot Index Creation - TokuDB tables support insertions, deletions and queries with no down time while indexes are being added to that table

    • Hot column addition, deletion, expansion, and rename - TokuDB tables support insertions, deletions and queries without down-time when an alter table adds, deletes, expands, or renames columns

    • On-line Backup

    Note

    The TokuDB storage engine does not support the nowait and skip locked modifiers introduced in the InnoDB storage engine with MySQL 8.0.

    For more information on installing and using TokuDB click on the following links:

    • TokuDB Installation

    • Using TokuDB

    • Getting Started with TokuDB

    • TokuDB Variables

    • Percona TokuBackup

    • TokuDB Troubleshooting

    • Frequently Asked Questions

    • Migrating and Removing the TokuDB storage engine

    Getting the Most from TokuDB¶

    Compression

    *TokuDB* compresses all data on disk, including indexes. Compression
    lowers cost by reducing the amount of storage required and frees up disk
    space for additional indexes to achieve improved query performance. Depending
    on the compressibility of the data, we have seen compression ratios up to 25x
    for high compression. Compression can also lead to improved performance since
    less data needs to be read from and written to disk.
    

    Fast Insertions and Deletions

    TokuDB’s Fractal Tree technology enables fast
    indexed insertions and deletions. Fractal Trees match B-trees in their
    indexing sweet spot (sequential data) and are up to two orders of magnitude
    faster for random data with high cardinality.
    

    Eliminates Replica Lag

    *TokuDB* replication replicas can be configured to process
    the replication stream with virtually no read IO. Uniqueness checking is
    performed on the *TokuDB* source and can be skipped on all *TokuDB*
    replica. Also, row based replication ensures that all before and after row
    images are captured in the binary logs, so the *TokuDB* replicas can harness
    the power of Fractal Tree indexes and bypass traditional read-modify-write
    behavior. This “Read Free Replication” ensures that replication replicas do not
    fall behind the source and can be used for read scaling, backups, and
    disaster recovery, without sharding, expensive hardware, or limits on what
    can be replicated.
    

    Hot Index Creation

    *TokuDB* allows the addition of indexes to an existing table
    while inserts and queries are being performed on that table. This means that
    *MySQL* can be run continuously with no blocking of queries or insertions
    while indexes are added and eliminates the down-time that index changes would
    otherwise require.
    

    Hot Column Addition, Deletion, Expansion and Rename

    *TokuDB* allows the addition
    of new columns to an existing table, the deletion of existing columns from an
    existing table, the expansion of `char`, `varchar`, `varbinary`, and
    `integer` type columns in an existing table, and the renaming of an
    existing column while inserts and queries are being performed on that table.
    

    Online (Hot) Backup

    The *TokuDB* can create backups of online database servers without downtime.
    

    Fast Indexing

    In practice, slow indexing often leads users to choose a smaller
    number of sub-optimal indexes in order to keep up with incoming data
    rates. These sub-optimal indexes result in disproportionately slower queries,
    since the difference in speed between a query with an index and the same
    query when no index is available can be many orders of magnitude. Thus, fast
    indexing means fast queries.
    

    Clustering Keys and Other Indexing Improvements

    *TokuDB* tables are clustered on
    the primary key. *TokuDB* also supports clustering secondary keys, providing
    better performance on a broader range of queries. A clustering key includes
    (or clusters) all of the columns in a table along with the key. As a result,
    one can efficiently retrieve any column when doing a range query on a
    clustering key. Also, with *TokuDB*, an auto-increment column can be used in
    any index and in any position within an index. Lastly, *TokuDB* indexes can
    include up to 32 columns.
    

    Less Aging/Fragmentation

    *TokuDB* can run much longer, likely indefinitely,
    without the need to perform the customary practice of dump/reload or
    `OPTIMIZE TABLE` to restore database performance. The key is the
    fundamental difference with which the Fractal Tree stores data on
    disk. Since, by default, the Fractal Tree will store data in 4MB chunks
    (pre-compression), as compared to InnoDB’s 16KB, *TokuDB* has the ability to
    avoid “database disorder” up to 250x better than InnoDB.
    

    Bulk Loader

    *TokuDB* uses a parallel loader to create tables and offline
    indexes. This parallel loader will use multiple cores for fast offline table
    and index creation.
    

    Full-Featured Database

    *TokuDB* supports fully ACID-compliant transactions, MVCC
    (Multi-Version Concurrency Control), serialized isolation levels, row-level
    locking, and XA. *TokuDB* scales with high number of client connections, even
    for large tables.
    

    Lock Diagnostics

    *TokuDB* provides users with the tools to diagnose locking and
    deadlock issues. For more information, see [Lock Visualization in TokuDB](tokudb_troubleshooting.md#lock-visualization-in-tokudb).
    

    Progress Tracking

    Running `SHOW PROCESSLIST` when adding indexes provides
    status on how many rows have been processed. Running `SHOW PROCESSLIST`
    also shows progress on queries, as well as insertions, deletions and
    updates. This information is helpful for estimating how long operations will
    take to complete.
    

    Fast Recovery

    *TokuDB* supports very fast recovery, typically less than a minute.
    

    Contact us

    For free technical help, visit the Percona Community Forum.

    To report bugs or submit feature requests, open a JIRA ticket.

    For paid support and managed or consulting services , contact Percona Sales.


    Last update: 2023-01-12
    Percona LLC and/or its affiliates, © 2023
    Made with Material for MkDocs

    Cookie consent

    We use cookies to recognize your repeated visits and preferences, as well as to measure the effectiveness of our documentation and whether users find what they're searching for. With your consent, you're helping us to make our documentation better.