Skip to content
logo
Percona Operator for MongoDB
Creating a private S3-compatible cloud for backups
Initializing search
    percona/k8spsmdb-docs
    percona/k8spsmdb-docs
    • Welcome
      • System requirements
      • Design and architecture
      • Comparison with other solutions
      • Install with Helm
      • Install with kubectl
      • Install on Minikube
      • Install on Google Kubernetes Engine (GKE)
      • Install on Amazon Elastic Kubernetes Service (AWS EKS)
      • Install on Microsoft Azure Kubernetes Service (AKS)
      • Generic Kubernetes installation
      • Install on OpenShift
      • Application and system users
      • Changing MongoDB options
      • Anti-affinity and tolerations
      • Labels and annotations
      • Exposing the cluster
      • Local storage support
      • Arbiter and non-voting nodes
      • MongoDB sharding
      • Transport encryption (TLS/SSL)
      • Data at rest encryption
      • Telemetry
        • About backups
        • Configure storage for backups
        • Making scheduled backups
        • Making on-demand backup
        • Storing operations logs for point-in-time recovery
        • Restore from a previously saved backup
        • Delete the unneeded backup
      • Upgrade MongoDB and the Operator
      • Horizontal and vertical scaling
      • Multi-cluster and multi-region deployment
      • Monitor with Percona Monitoring and Management (PMM)
      • Add sidecar containers
      • Restart or pause the cluster
      • Debug and troubleshoot
      • OpenLDAP integration
      • How to use private registry
      • Creating a private S3-compatible cloud for backups
      • Restore backup to a new Kubernetes-based environment
      • How to use backups to move the external database to Kubernetes
      • Install Percona Server for MongoDB in multi-namespace (cluster-wide) mode
      • Upgrading Percona Server for MongoDB manually
      • Custom Resource options
      • Percona certified images
      • Operator API
      • Frequently asked questions
      • Old releases (documentation archive)
      • Release notes index
      • Percona Operator for MongoDB 1.14.0 (2023-03-13)
      • Percona Operator for MongoDB 1.13.0 (2022-09-15)
      • Percona Operator for MongoDB 1.12.0 (2022-05-05)
      • Percona Distribution for MongoDB Operator 1.11.0 (2021-12-21)
      • Percona Distribution for MongoDB Operator 1.10.0 (2021-09-30)
      • Percona Distribution for MongoDB Operator 1.9.0 (2021-07-29)
      • Percona Kubernetes Operator for Percona Server for MongoDB 1.8.0 (2021-05-06)
      • Percona Kubernetes Operator for Percona Server for MongoDB 1.7.0 (2021-03-08)
      • Percona Kubernetes Operator for Percona Server for MongoDB 1.6.0 (2020-12-22)
      • Percona Kubernetes Operator for Percona Server for MongoDB 1.5.0 (2020-09-07)
      • Percona Kubernetes Operator for Percona Server for MongoDB 1.4.0 (2020-03-31)
      • Percona Kubernetes Operator for Percona Server for MongoDB 1.3.0 (2019-12-11)
      • Percona Kubernetes Operator for Percona Server for MongoDB 1.2.0 (2019-09-20)
      • Percona Kubernetes Operator for Percona Server for MongoDB 1.1.0 (2019-07-15)
      • Percona Kubernetes Operator for Percona Server for MongoDB 1.0.0 (2019-05-29)

    Creating a private S3-compatible cloud for backups¶

    As it is mentioned in backups, any cloud storage which implements the S3 API can be used for backups. The one way to setup and implement the S3 API storage on Kubernetes or OpenShift is Minio - the S3-compatible object storage server deployed via Docker on your own infrastructure.

    Setting up Minio to be used with Percona Operator for MongoDB backups involves the following steps:

    1. Install Minio in your Kubernetes or OpenShift environment and create the correspondent Kubernetes Service as follows:

      $ helm install \
        --name minio-service \
        --set accessKey=some-access-key \
        --set secretKey=some-secret-key \
        --set service.type=ClusterIP \
        --set configPath=/tmp/.minio/ \
        --set persistence.size=2G \
        --set environment.MINIO_REGION=us-east-1 \
        stable/minio
      

      Don’t forget to substitute default some-access-key and some-secret-key strings in this command with actual unique key values. The values can be used later for access control. The storageClass option is needed if you are using the special Kubernetes Storage Class for backups. Otherwise, this setting may be omitted. You may also notice the MINIO_REGION value which is may not be used within a private cloud. Use the same region value here and on later steps (us-east-1 is a good default choice).

    2. Create an S3 bucket for backups:

      $ kubectl run -i --rm aws-cli --image=perconalab/awscli --restart=Never -- \
        bash -c 'AWS_ACCESS_KEY_ID=some-access-key \
        AWS_SECRET_ACCESS_KEY=some-secret-key \
        AWS_DEFAULT_REGION=us-east-1 \
        /usr/bin/aws \
        --endpoint-url http://minio-service:9000 \
        s3 mb s3://operator-testing'
      

      This command creates the bucket named operator-testing with the selected access and secret keys (substitute some-access-key and some-secret-key with the values used on the previous step).

    3. Now edit the backup section of the deploy/cr.yaml file to set proper values for the bucket (the S3 bucket for backups created on the previous step), region, credentialsSecret and the endpointUrl (which should point to the previously created Minio Service).

      ...
      backup:
        enabled: true
        version: 0.3.0
        ...
        storages:
          minio:
            type: s3
            s3:
              bucket: operator-testing
              region: us-east-1
              credentialsSecret: my-cluster-name-backup-minio
              endpointUrl: http://minio-service:9000
        ...
      

      The option which should be specially mentioned is credentialsSecret which is a Kubernetes secret for backups. Sample backup-s3.yaml can be used to create this secret object. Check that the object contains the proper name value and is equal to the one specified for credentialsSecret, i.e. my-cluster-name-backup-minio in the backup to Minio example, and also contains the proper AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY keys. After you have finished editing the file, the secrets object are created or updated when you run the following command:

      $ kubectl apply -f deploy/backup-s3.yaml
      
    4. When the setup process is completed, making the backup is based on a script. Following example illustrates how to make an on-demand backup:

      $ kubectl run -it --rm pbmctl --image=percona/percona-server-mongodb-operator:0.3.0-backup-pbmctl --restart=Never -- \
         run backup \
         --server-address=<cluster-name>-backup-coordinator:10001 \
         --storage <storage> \
         --compression-algorithm=gzip \
         --description=my-backup
      

      Don’t forget to specify the name of your cluster instead of the <cluster-name> part of the Backup Coordinator URL (the cluster name is specified in the deploy/cr.yaml file). Also substitute <storage> with the actual storage name located in a subsection inside of the backups in the deploy/cr.yaml file. In the earlier example this value is minio.

    5. To restore a previously saved backup you must specify the backup name. With the proper Backup Coordinator URL and storage name, you can obtain a list of the available backups:

      $ kubectl run -it --rm pbmctl --image=percona/percona-server-mongodb-operator:0.3.0-backup-pbmctl --restart=Never -- list backups --server-address=<cluster-name>-backup-coordinator:10001
      

      Now, restore the backup, using backup name instead of the backup-name parameter:

      $ kubectl run -it --rm pbmctl --image=percona/percona-server-mongodb-operator:0.3.0-backup-pbmctl --restart=Never -- \
         run restore \
         --server-address=<cluster-name>-backup-coordinator:10001 \
         --storage <storage> \
         backup-name
      

    Contact Us

    For free technical help, visit the Percona Community Forum.

    To report bugs or submit feature requests, open a JIRA ticket.

    For paid support and managed or consulting services , contact Percona Sales.


    Last update: 2023-03-22
    Percona LLC and/or its affiliates, © 2009 - 2023
    Made with Material for MkDocs

    Cookie consent

    We use cookies to recognize your repeated visits and preferences, as well as to measure the effectiveness of our documentation and whether users find what they're searching for. With your consent, you're helping us to make our documentation better.