Skip to content
Percona Documentation

For help, click the link below to get free database assistance or contact our experts for personalized support.

Get help from Percona

Import external database backups into Percona Everest

Important

The external backup import feature in Percona Everest is currently in Technical Preview. Early adopters are advised to use this feature only for testing purposes and not in production environments.

This new feature in Percona Everest enables you to import database backups stored in external object storage into clusters managed by Percona Everest using an extensible design.

Objectives

This feature can help you achieve the following objectives:

  • Allow you to import backup data stored in external storage like Amazon S3 into database cluster managed by Percona Everest.

  • Allow you to customize the import process using your preferred tools, such as mongodump, pg_dump, or mysqldump.

  • Provide a pluggable and extensible framework that can adapt to different import needs and workflows.

DataImporters

What are DataImporters?

Important

Percona Everest includes three pre-installed DataImporters (everest-percona-pg-operator, everest-percona-pxc-operator, and everest-percona-psmdb-operator), designed to restore external backups created using Percona operators.

A DataImporter is a cluster-scoped CRD (Custom Resource Definition). It represents a self-contained, reusable blueprint for importing data into a newly created database cluster managed by Percona Everest. It defines:

  • The container to be run for your data restore logic.
  • The database engines it supports.
  • The input configurations it requires (e.g., S3 path, credentials, and custom inputs).
  • Any constraints that your database cluster must meet, such as engine version requirements.

What is a DataImport job?

DataImportJob is a namespace-scoped CRD that represents a single execution of a DataImporter. Percona Everest automatically creates it when a user initiates a backup import using a specified DataImporter. Internally, it runs a Kubernetes Job that runs the specified import logic on a newly created DatabaseCluster, using the credentials and configuration provided by the user (e.g., S3 bucket, file path, access keys).

Why use DataImporters?

Organizations use different backup/restore tools, such as pg_dump, mysqldump, mongodump, physical snapshots, or vendor-specific tools. While Percona Everest does not natively offer support for such tools, with DataImporters, you can write custom restore logic for your preferred backup tools or formats, and Percona Everest executes them.

The benefits of using DataImporters are:

  • Reusable: You can define it once and use it across various environments.

  • Flexible: You can perform imports using any scripting or programming language and restore tool.

  • Extensible: It supports custom backup formats or workflows.

  • Decoupled: Percona Everest manages the infrastructure while you focus on the data logic.

Do you want to customize your import process? You can create a custom DataImporter tailored to your specific use case. See the documentation for detailed instructions.

Use cases

Here are some common scenarios for importing database backups into Percona Everest.

Restore a PostgreSQL cluster from an S3 backup

You have a logical backup of a PostgreSQL database, created using pg_dump, stored in an Amazon S3 bucket. You want to use this backup to initialize a new PostgreSQL cluster managed by Percona Everest.

Build your own DataImporter

You can create your own DataImporter to implement custom restore logic using any language or tool. This allows for seamless integration into the cluster provisioning workflow.

See the documentation for detailed instructions.

Limitations

There are a few limitations to be aware of when importing external database backups:

  • This feature only supports backups created using Percona Operators. Backups created by other tools directly in an S3 bucket are not supported.

    Example

    Supported

    • Backups created via Percona Operator for MySQL, PostgreSQL, or MongoDB
    • Stored in S3-compatible storage

    Unsupported

    • Backups taken by third-party tools
    • Direct uploads to S3 without using Percona Operator

  • Percona Everest 1.8.0 does not guarantee successful imports for physical backups because it lacks an encryption key configuration. Although some cases may work depending on the backup method and environment, there is currently no official support for this feature.

  • Certain import methods require database user credentials that exactly match those from the source system. Since Percona Everest does not validate these credentials, you must ensure they are accurate before starting the import.

  • Percona Everest does not verify the compatibility of imported data with the version of the target DatabaseCluster. Ensure that the backup is compatible with the database version managed by Percona Everest.

Next steps

DataImporter: Percona Operator for MySQL (XtraDB)

DataImporter: Percona MongoDB Operator

DataImporter: Percona Operator for PostgreSQL