Backups
MemSQL Operator 1.2.0 inadvertently removed two optional backup-related command-line parameters: backup-s3-endpoint
and backup-compatibility-mode
. You may remedy their absence by either:
- Upgrading to MemSQL Operator 1.2.1, which contains these parameters
- Using CR spec
backupSpec.s3Endpoint
andbackupSpec.compatibilityMode
to specify their values
As of Operator 1.0.0, the deployment.yaml
file can be configured so that the Operator will manage and perform backups. These new parameters are:
backup-bucket-name
: The AWS S3 bucket, including the path if desiredbackup-default-schedule
: Default schedule for backup databases, incron
format. See the below for more information. Ifbackup-incremental-default-schedule
is specified, the scheduled job will perform a backup usingBACKUP DATABASE ... WITH INIT ...
, otherwise, it will perform a plainBACKUP DATABASE
without using theWITH INIT
option.backup-s3-endpoint
: Provide compatible AWS S3 endpoint URL, otherwise leave blank (optional)backup-s3-region
: Name of the AWS S3 region (optional)backup-secret-name
: Name of the AWS Secret for the S3 backup
As of Operator 1.1.1, the deployment.yaml
file may also include:
backup-compatibility-mode
: Disables certain optimizations that only work with native S3, the primary use case for which is backing up to GCS. (optional) Refer to BACKUP DATABASE for more information.
As of Operator 1.2.0, the deployment.yaml
file may also include:
backup-incremental-default-schedule
: Default schedule for databases incremental backup, incron
format. If this is configured, it will perform an incremental backup usingBACKUP DATABASE ... WITH DIFFERENTIAL ...
option if there is already a backup created by usingWITH INIT
option, otherwise the first time the incremental backup is run, it will perform a backup usingWITH INIT
option. See the below for more information.backup-default-deadline-seconds
: Default deadline in seconds that the backup job has to finish before it is terminated. Defaults to3600 seconds
if not specified.backup-incremental-default-deadline-seconds
: Default deadline in seconds that the incremental backup job has to finish before it is terminated. Defaults to3600
seconds if not specified. It’s suggested to make this value the same value asbackup-default-deadline-seconds
.
As of Operator 1.2.0, you may also specify backup details via backupSpec
in the CR, which includes the following fields. These fields all have matching parts in the deployment.yaml
file. Field specified in backupSpec
in the CR takes precedence over the parameter specified in deployment.yaml
.
bucketName:
s3Region:
s3Endpoint:
secretName:
Schedule:
deadlineSeconds:
scheduleIncremental:
deadlineSecondsIncremental:
compatibilityMode:
You may define the --backup-default-schedule
and/or backup-incremental-default-schedule
using the following cron
job scheduling format:
Unit | Description | Value |
---|---|---|
MIN |
Minute field | 0 to 59 |
HOUR |
Hour field | 0 to 23 |
DOM |
Day of Month | 1 to 31 |
MON |
Month field | 1 to 12 |
DOW |
Day of Week | 0 to 6 |
For example:
+---------------- MIN
| +------------- HOUR
| | +---------- DOM
| | | +------- MON
| | | | +---- DOW
| | | | |
"--backup-default-schedule", "0 1 * * *",
The following is an example deployment.yaml
file that’s configured for backups using the parameters above:
apiVersion: apps/v1
kind: Deployment
metadata:
name: memsql-operator
namespace: memsql
spec:
replicas: 1
selector:
matchLabels:
name: memsql-operator
template:
metadata:
labels:
name: memsql-operator
spec:
serviceAccountName: memsql-operator
containers:
- name: memsql-operator
image: "memsql/operator"
imagePullPolicy: Never
args: [
# Cause the operator to merge rather than replace annotations on services
"--merge-service-annotations",
# Allow the process inside the container to have read/write access to the volume.
"--fs-group-id", "5555",
# Configure operator to run backup daily at 1 AM
"--backup-bucket-name", "<bucket-name/path>",
"--backup-default-schedule", "0 1 * * *",
"--backup-s3-endpoint", "",
"--backup-s3-region", "<AWS region for example us-west-1>",
"--backup-secret-name", "aws-credential"
]
env:
- name: WATCH_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: OPERATOR_NAME
value: "memsql-operator"
The following is an example aws-secret.yaml
file for using AWS S3 to store backups. It reflects an AWS S3 Access Secret configuration.
apiVersion: v1
kind: Secret
metadata:
name: aws-credential
namespace: memsql
type: Opaque
stringData:
AWS_ACCESS_KEY: <AWS-Access-Key>
AWS_SECRET_ACCESS_KEY: <AWS-Secret>
As of Operator 1.2.3, you may also specify the name of a Secret for the backup job’s user. The default resource pool associated with that user will be used for the backup job. The user can be specified either via userSecret
in backupSpec
, or via backup-user-secret
Operator command line parameter in deployment.yaml
. The field specified in backupSpec
in the CR takes precedence over the parameter specified in deployment.yaml
.
The following is an example backup-user-secret.yaml
file.
apiVersion: v1
kind: Secret
metadata:
name: backup-user-secret
namespace: memsql
type: Opaque
stringData:
BACKUP_USERNAME: <username>
BACKUP_PASSWORD: <password>