Skip to content

Remote backup storage

On this page:

Overview

Percona Backup for MongoDB supports the following types of remote backup storage:

S3 compatible storage

Percona Backup for MongoDB should work with other S3-compatible storages, but was only tested with the following ones:

As of version 1.3.2, Percona Backup for MongoDB supports server-side encryption for S3 buckets with customer managed keys stored in AWS KMS.

See also

Protecting Data Using Server-Side Encryption with CMKs Stored in AWS Key Management Service (SSE-KMS)

New in version 1.7.0: You can enable debug logging for different types of S3 requests in Percona Backup for MongoDB. Percona Backup for MongoDB prints S3 log messages in the pbm logs output so that you can debug and diagnose S3 request issues or failures.

To enable S3 debug logging, set the storage.s3.DebugLogLevel option in Percona Backup for MongoDB configuration. The supported values are: LogDebug, Signing, HTTPBody, RequestRetries, RequestErrors, EventStreamBody.

Starting with version 1.7.0, Percona Backup for MongoDB supports Amazon S3 storage classes. Knowing your data access patterns, you can set the S3 storage class in Percona Backup for MongoDB configuration. When Percona Backup for MongoDB uploads data to S3, the data is distributed to the corresponding storage class. The support of S3 bucket storage types allows you to effectively manage S3 storage space and costs.

To set the storage class, specify the storage.s3.storageClass option in Percona Backup for MongoDB configuration file

storage:
  type: s3
  s3:
    storageClass: INTELLIGENT_TIERING

When the option is undefined, the S3 Standard storage type is used.

See also

Using Amazon S3 storage classes

As of version 1.7.0, you can set up the number of attempts for Percona Backup for MongoDB to upload data to S3 storage as well as the min and max time to wait for the next retry. Set the options storage.s3.retryer.numMaxRetries, storage.s3.retryer.minRetryDelay and storage.s3.retryer.maxRetryDelay in Percona Backup for MongoDB configuration.

retryer:
       numMaxRetries: 3
       minRetryDelay: 30
       maxRetryDelay: 5

This upload retry increases the chances of data upload completion in cases of unstable connection.

New in version 1.7.0: Percona Backup for MongoDB supports data upload to S3-like storage that supports self-issued TLS certificates. To make this happen, disable the TLS verification of the S3 storage in Percona Backup for MongoDB configuration:

$ pbm config --set storage.s3.insecureSkipTLSVerify=True

Warning

Use this option with caution as it might leave a hole for man-in-the-middle attacks.

Remote Filesystem Server Storage

This storage must be a remote file server mounted to a local directory. It is the responsibility of the server administrators to guarantee that the same remote directory is mounted at exactly the same local path on all servers in the MongoDB cluster or non-sharded replica set.

Warning

Percona Backup for MongoDB uses the directory as if it were any normal directory, and does not attempt to confirm it is mounted from a remote server. If the path is accidentally a normal local directory, errors will eventually occur, most likely during a restore attempt. This will happen because pbm-agent processes of other nodes in the same replica set can’t access backup archive files in a normal local directory on another server.

Local Filesystem Storage

This cannot be used except if you have a single-node replica set. (See the warning note above as to why). We recommend using any object store you might be already familiar with for testing. If you don’t have an object store yet, we recommend using MinIO for testing as it has simple setup. If you plan to use a remote filesytem-type backup server, please see the Remote Filesystem Server Storage above.

Microsoft Azure Blob Storage

As of v1.5.0, you can use Microsoft Azure Blob Storage as the remote backup storage for Percona Backup for MongoDB.

This gives users a vendor choice. Companies with Microsoft-based infrastructure can set up Percona Backup for MongoDB with less administrative efforts.

Note

Regardless of the remote backup storage you use, grant the List/Get/Put/Delete permissions to this storage for the user identified by the access credentials.

The following example shows the permissions configuration to the pbm-testing bucket on the AWS S3 storage.

{
    "Version": "2021-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": "arn:aws:s3:::pbm-testing"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:DeleteObject"
            ],
            "Resource": "arn:aws:s3:::pbm-testing/*"
        }
    ]
}

Please refer to the documentation of your selected storage for the data access management.

See also

Example config files

Provide the remote backup storage configuration as a YAML config file. The following are the examples of config fles for supported remote storages. For how to insert the config file, see Insert the whole Percona Backup for MongoDB config from a YAML file.

S3-compatible remote storage

Amazon Simple Storage Service

storage:
  type: s3
  s3:
    region: us-west-2
    bucket: pbm-test-bucket
    prefix: data/pbm/backup
    credentials:
      access-key-id: <your-access-key-id-here>
      secret-access-key: <your-secret-key-here>
    serverSideEncryption:
      sseAlgorithm: aws:kms
      kmsKeyID: <your-kms-key-here>

GCS

storage:
 type: s3
 s3:
     region: us-east1
     bucket: pbm-testing
     prefix: pbm/test
     endpointUrl: https://storage.googleapis.com
     credentials:
       access-key-id: <your-access-key-id-here>
       secret-access-key: <your-secret-key-here>

MinIO

storage:
  type: s3
  s3:
    endpointUrl: "http://localhost:9000"
    region: my-region
    bucket: pbm-example
    prefix: data/pbm/test
    credentials:
      access-key-id: <your-access-key-id-here>
      secret-access-key: <your-secret-key-here>

Remote filesystem server storage

storage:
  type: filesystem
  filesystem:
    path: /data/local_backups

Microsoft Azure Blob Storage

storage:
  type: azure
  azure:
    account: <your-account>
    container: <your-container>
    prefix: pbm
    credentials:
      key: <your-access-key>

For the description of configuration options, see Configuration file options.