SingleStore DB

Using Storage Repositories with Backups

Overview

For those Toolbox commands that support backup operations, the --repository option can be used to specify the data storage location where Toolbox reads data from and writes data to. For example, the sdb-admin create-backup command can write a database backup to the specified repository, whereas the sdb-admin list-backups command can read data from the specified repository to display the list of available backups.

SingleStore supports Amazon S3, Azure Blob Storage, and Network File System (NFS) for backup storage options. Consequently, the --repository option can specify a location in an S3 bucket, Azure container, or NFS mount.

The sdb-report calibrate-blob-storage command also supports the --repository option, using Amazon S3-compatible blob storage exclusively. An unlimited storage database is created in the specified repository which is then used to test blob storage performance.

Usage

The --repository option takes the following components as input: {protocol}://{path}[?{options}].

Where:

  • protocol determines how the path to the storage location is interpreted, and which options are supported. Backup commands support s3, azure, and file protocols for Amazon S3, Azure Blob Storage, and NFS, respectively. The sdb-report calibrate-blob-storage command only supports s3.

  • path defines the path to the storage location and restricts the scope to access specific resources within the specified storage. For example, when a backup command is run against s3://bucket-name/path/inside/bucket?region=us-east-1, only the backup files located under /path/inside/bucket of the bucket-name S3 bucket will be accessed. In this example, the s3://bucket-name/path/inside repository is different from, let’s say, s3://bucket-name/path/inside/bucket, which is another repository in the same S3 bucket.

  • options specify additional configuration settings. For example, when using S3 for storage, you may specify all of the options in the BACKUP DATABASE command, including region, endpoint URL, and x-amz-server-side-encryption.

Note that the --repository option takes values in a URL format that corresponds to the storage protocol used. The following examples demonstrate the URL format of the --repository value for different storage options:

Amazon S3: s3://bucket-name/path/inside/bucket?region=us-east-1

Azure Blob Storage: azure://container/path/inside/container

NFS: file://host/some/directory