Skip to content
logo
Percona XtraBackup
Use xbcloud Binary with Amazon S3
Initializing search
    percona/pxb-docs
    percona/pxb-docs
    • Home
      • About Percona XtraBackup
      • How Percona XtraBackup works
      • Understand version numbers
      • Install Percona XtraBackup 8.0
      • Use an APT repo to install Percona XtraBackup
      • Use a YUM repo to install Percona XtraBackup
      • Use DEB or RPM downloaded packages to install Percona XtraBackup
      • Install Percona XtraBackup from a Binary Tarball
      • Compile and install Percona XtraBackup from source code
      • What’s in the packages
      • Uninstall Percona XtraBackup
      • Run Percona XtraBackup in a Docker container
      • Implementation details
      • Connection and privileges needed
      • Configure xtrabackup
      • Server version and backup version comparison
      • xtrabackup exit codes
      • The backup cycle - full backups
      • Incremental backup
      • Compressed backup
      • Partial backups
      • Percona XtraBackup user manual
      • Throttling backups
      • Encrypted InnoDB tablespace backups
      • Encrypt backups
      • LRU dump backup
      • Point-in-time recovery
      • Restore individual tables
      • Smart memory estimation
      • Work with binary logs
      • Improved log statements
      • Work with SELinux
      • Work with AppArmor
      • Enable the server to communicate via TCP/IP
      • Install and configure an SSH server
      • Analyze table statistics
      • FLUSH TABLES WITH READ LOCK option
      • lock-ddl-per-table option improvements
      • Take an incremental backup using page tracking
      • The xbcloud binary
      • Use the xbcloud binary with Swift
      • Use xbcloud Binary with Amazon S3
        • Create a full backup with Amazon S3
        • Permissions setup
        • Environment variables
        • Restore with S3
      • Xbcloud with an IAM instance profile
      • Use the xbcloud binary with MinIO
      • Use the xbcloud with Google Cloud Storage
      • Exponential backoff
      • Use the xbcloud binary with Microsoft Azure Cloud Storage
      • How-tos and recipes
      • Release notes index
      • Percona XtraBackup 8.0.33-27 (2023-05-25)
      • Percona XtraBackup 8.0.32-26 (2023-04-04)
      • Percona XtraBackup 8.0.32-25 (2023-02-27)
      • Percona XtraBackup 8.0.31-24 (2023-02-07)
      • Percona XtraBackup 8.0.30-23 (2022-11-14)
      • Percona XtraBackup 8.0.29-22 (2022-07-19)
      • Percona XtraBackup 8.0.28-21 (2022-05-25)
      • Percona XtraBackup 8.0.28-20
      • Percona XtraBackup 8.0.27-19
      • Percona XtraBackup 8.0.26-18.0
      • Percona XtraBackup 8.0.25-17.0
      • Percona XtraBackup 8.0.23-16.0
      • Percona XtraBackup 8.0.22-15.0
      • Percona XtraBackup 8.0.14
      • Percona XtraBackup 8.0.13
      • Percona XtraBackup 8.0.12
      • Percona XtraBackup 8.0.11
      • Percona XtraBackup 8.0.10
      • Percona XtraBackup 8.0.9
      • Percona XtraBackup 8.0.8
      • Percona XtraBackup 8.0.7
      • Percona XtraBackup 8.0.6
      • Percona XtraBackup 8.0.5
      • Percona XtraBackup 8.0.4
      • Percona XtraBackup 8.0-3-rc1
      • Error Message: Found tables with row versions due to INSTANT ADD/DROP columns
      • The xtrabackup option reference
      • The xbcrypt binary
      • The xbstream binary
      • Frequently asked questions
      • Glossary
      • Index of files created by Percona XtraBackup
      • Trademark policy
      • Copyright and licensing information
      • Version checking

    • Create a full backup with Amazon S3
    • Permissions setup
    • Environment variables
    • Restore with S3

    Use xbcloud Binary with Amazon S3¶

    Create a full backup with Amazon S3¶

    $ xtrabackup --backup --stream=xbstream --extra-lsndir=/tmp --target-dir=/tmp | \
    xbcloud put --storage=s3 \
    --s3-endpoint='s3.amazonaws.com' \
    --s3-access-key='YOUR-ACCESSKEYID' \
    --s3-secret-key='YOUR-SECRETACCESSKEY' \
    --s3-bucket='mysql_backups'
    --parallel=10 \
    $(date -I)-full_backup
    

    The following options are available when using Amazon S3:

    Option Details
    –s3-access-key Use to supply the AWS access key ID
    –s3-secret-key Use to supply the AWS secret access key
    –s3-bucket Use supply the AWS bucket name
    –s3-region Use to specify the AWS region. The default value is us-east-1
    –s3-api-version = Select the signing algorithm. The default value is AUTO. In this case, xbcloud will probe.
    –s3-bucket-lookup = Specify whether to use bucket.endpoint.com or endpoint.com/bucket*style requests. The default value is AUTO. In this case, xbcloud will probe.
    –s3-storage-class= Specify the S3 storage class. The default storage class depends on the provider. The name options are the following:
    • STANDARD
    • STANDARD_IA
    • GLACIER
    NOTE If you use the GLACIER storage class, the object must be restored to S3 before restoring the backup. Also supports using custom S3 implementations such as MinIO or CephRadosGW.

    Permissions setup¶

    Following the principle of “least-privilege”, these are the minimum bucket permissions needed for xbcloud to write backups to S3: LIST/PUT/GET/DELETE.

    The following example shows the policy definition for writing to the xbcloud-testing bucket on the AWS S3 storage.

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "s3:ListBucket"
                ],
                "Resource": "arn:aws:s3:::xbcloud-testing"
            },
            {
                "Effect": "Allow",
                "Action": [
                    "s3:PutObject",
                    "s3:PutObjectAcl",
                    "s3:GetObject",
                    "s3:GetObjectAcl",
                    "s3:DeleteObject"
                ],
                "Resource": "arn:aws:s3:::xbcloud-testing/*"
            }
        ]
    }
    

    Environment variables¶

    The following environment variables are recognized. xbcloud maps them automatically to corresponding parameters applicable to the selected storage.

    • AWS_ACCESS_KEY_ID (or ACCESS_KEY_ID)

    • AWS_SECRET_ACCESS_KEY (or SECRET_ACCESS_KEY)

    • AWS_DEFAULT_REGION (or DEFAULT_REGION)

    • AWS_ENDPOINT (or ENDPOINT)

    • AWS_CA_BUNDLE

    Restore with S3¶

    $ xbcloud get s3://operator-testing/bak22 \
    --s3-endpoint=https://storage.googleapis.com/ \
    --parallel=10 2>download.log | xbstream -x -C restore --parallel=8
    

    Contact us

    For free technical help, visit the Percona Community Forum.

    To report bugs or submit feature requests, open a JIRA ticket.

    For paid support and managed or consulting services , contact Percona Sales.


    Last update: 2022-12-06
    Percona LLC and/or its affiliates, © 2023
    Made with Material for MkDocs

    Cookie consent

    We use cookies to recognize your repeated visits and preferences, as well as to measure the effectiveness of our documentation and whether users find what they're searching for. With your consent, you're helping us to make our documentation better.