How to Restore an S3 Backup Locally and Vice Versa

1. Pull all the backup files from S3 and copy them to MA plus all the leaf nodes.

2. Rename the files to match the local backup naming convention. To convert a full backup taken directly in S3 to the on-disk format you need to append .backup after the databasename_ordinal for files excluding .manifest and BACKUP_COMPLETE.

So snapshot filenames go from testing_7 to testing_7.backup And columnar blobs go from testing_7.columns0.tar to testing_7.backup_columns0.tar.

Change directory into the backup folder, execute the following two commands to change the file names. Execute the same on all the leaf nodes backup folders. Command 1:

for file in *[0-9]; do mv "$file" "$(basename "$file").backup"; done;

Command 2:

find . ! -name '*.backup' ! -name '*.manifest' ! -name 'BACKUP_COMPLETE' ! -name '*.' -exec bash -c 'mv $0 $0.backup' {} \; 

The folder name is irrelevant for local backups.

3. After renaming the files, restore the backup like a regular local backup.

For example:

mysql> RESTORE DATABASE testing FROM "/ebs/memsql/s3_test_backup/testing.backup";

Ensure permissions are given to access the files and folders.

If you want to convert a local backup to S3 you would reverse this process with an additional step of ensuring the containing folder/subobject/object prefix is databasename.backup