How to Restore an S3 Backup Locally and Vice Versa

1. Pull all the backup files from S3 and copy them to MA plus all the leaf nodes.

2. Rename the files to match the local backup naming convention. To convert a full backup taken directly in S3 to the on-disk format you need to append .backup after the databasename_ordinal for files excluding .manifest and BACKUP_COMPLETE.

So snapshot filenames go from testing_7 to testing_7.backup And columnar blobs go from testing_7.columns0.tar to testing_7.backup_columns0.tar.

Change directory into the backup folder, execute the following two commands to change the file names. Execute the same on all the leaf nodes backup folders. Command 1:

for file in *[0-9]; do mv "$file" "$(basename "$file").backup"; done;

Command 2:

find . ! -name '*.backup' ! -name '*.manifest' ! -name 'BACKUP_COMPLETE' ! -name '*.' -exec bash -c 'mv $0 $0.backup' {} \;

The folder name is irrelevant for local backups.

3. After renaming the files, restore the backup like a regular local backup.

For example:

mysql> RESTORE DATABASE testing FROM "/ebs/memsql/s3_test_backup/testing.backup";

Ensure permissions are given to access the files and folders.

If you want to convert a local backup to S3 you would reverse this process with an additional step of ensuring the containing folder/subobject/object prefix is databasename.backup

Last modified: May 5, 2023

Was this article helpful?

Verification instructions

Note: You must install cosign to verify the authenticity of the SingleStore file.

Use the following steps to verify the authenticity of singlestoredb-server, singlestoredb-toolbox, singlestoredb-studio, and singlestore-client SingleStore files that have been downloaded.

You may perform the following steps on any computer that can run cosign, such as the main deployment host of the cluster.

  1. (Optional) Run the following command to view the associated signature files.

    curl undefined
  2. Download the signature file from the SingleStore release server.

    • Option 1: Click the Download Signature button next to the SingleStore file.

    • Option 2: Copy and paste the following URL into the address bar of your browser and save the signature file.

    • Option 3: Run the following command to download the signature file.

      curl -O undefined
  3. After the signature file has been downloaded, run the following command to verify the authenticity of the SingleStore file.

    echo -n undefined |
    cosign verify-blob --certificate-oidc-issuer https://oidc.eks.us-east-1.amazonaws.com/id/CCDCDBA1379A5596AB5B2E46DCA385BC \
    --certificate-identity https://kubernetes.io/namespaces/freya-production/serviceaccounts/job-worker \
    --bundle undefined \
    --new-bundle-format -
    Verified OK