Backups

Backup Configurations

As of Operator 1.0.0, the sdb-operator.yaml file can be configured so that the Operator will manage and perform backups. These new parameters are:

• backup-bucket-name: The AWS S3 bucket, including the path if desired

• backup-default-schedule: Default schedule for backup databases, in cron format. See below for more information. If backup-incremental-default-schedule is specified, the scheduled job will perform a backup using BACKUP DATABASE ... WITH INIT ..., otherwise, it will perform a plain BACKUP DATABASE without using the WITH INIT option.

• backup-s3-endpoint: Provide compatible AWS S3 endpoint URL, otherwise leave blank (optional)

• backup-s3-region: Name of the AWS S3 region (optional)

• backup-secret-name: Name of the AWS Secret for the S3 backup

As of Operator 1.1.1, the sdb-operator.yaml file may also include:

• backup-compatibility-mode: Disables certain optimizations that only work with native S3, the primary use case for which is backing up to GCS (optional) Refer to BACKUP DATABASE for more information.

As of Operator 1.2.0, the sdb-operator.yaml file may also include:

• backup-incremental-default-schedule: Default schedule for databases incremental backup, in cron format. If this is configured, it will perform an incremental backup using BACKUP DATABASE ... WITH DIFFERENTIAL ... option if there is already a backup created by using WITH INIT option, otherwise the first time the incremental backup is run, it will perform a backup using WITH INIT option. See below for more information.

• backup-default-deadline-seconds: Default deadline in seconds that the backup job has to finish before it is terminated. Defaults to 3600 seconds if not specified.

• backup-incremental-default-deadline-seconds: Default deadline in seconds that the incremental backup job has to finish before it is terminated. Defaults to 3600 seconds if not specified. It’s suggested to make this value the same value as backup-default-deadline-seconds.

As of Operator 1.2.0, you may also specify backup details via backupSpec in the CR, which includes the following fields. These fields all have matching parts in the sdb-operator.yaml file. The field specified in backupSpec in the CR takes precedence over the parameter specified in sdb-operator.yaml file.

• bucketName:

• s3Region:

• s3Endpoint:

• secretName:

• schedule:

• deadlineSeconds:

• scheduleIncremental:

• deadlineSecondsIncremental:

• compatibilityMode:

You may define the --backup-default-schedule and/or backup-incremental-default-schedule using the following cron job scheduling format:

For example:

                              +---------------- MIN
                              | +------------- HOUR
                              | | +---------- DOM
                              | | | +------- MON
                              | | | | +---- DOW
                              | | | | |
"--backup-default-schedule", "0 1 * * *",

The following is an example sdb-operator.yaml file that’s configured for backups using the parameters above:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sdb-operator
  labels:
    app.kubernetes.io/component: operator
spec:
  replicas: 1
  selector:
    matchLabels:
      name: sdb-operator
  template:
    metadata:
      labels:
        name: sdb-operator
    spec:
      serviceAccountName: sdb-operator
      containers:
        - name: sdb-operator
          image: "singlestore/operator"
          imagePullPolicy: Never
          args: [
            # Cause the operator to merge rather than replace annotations on services
            "--merge-service-annotations",
            # Allow the process inside the container to have read/write access to the volume.
            "--fs-group-id", "5555",
            # Configure operator to run backup daily at 1 AM
            "--backup-bucket-name", "<bucket-name/path>",
            "--backup-default-schedule", "0 1 * * *",
            "--backup-s3-endpoint", "",
            "--backup-s3-region", "<AWS region for example us-west-1>",
            "--backup-secret-name", "aws-credential"
          ]
          env:
            - name: WATCH_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: OPERATOR_NAME
              value: "sdb-operator"

The following is an example aws-secret.yaml file for using AWS S3 to store backups. It reflects an AWS S3 Access Secret configuration.

apiVersion: v1
kind: Secret
metadata:
  name: aws-credential
  namespace: singlestore
type: Opaque
stringData:
  AWS_ACCESS_KEY: <AWS-Access-Key>
  AWS_SECRET_ACCESS_KEY: <AWS-Secret>

As of Operator 1.2.3, you may also specify the name of a Secret for the backup job’s user. The default resource pool associated with that user will be used for the backup job. The user can be specified either via userSecret in backupSpec, or via backup-user-secret Operator command line parameter in sdb-operator.yaml. The field specified in backupSpec in the CR takes precedence over the parameter specified in sdb-operator.yaml.

The following is an example backup-user-secret.yaml file.

apiVersion: v1
kind: Secret
metadata:
  name: backup-user-secret
  namespace: singlestore
type: Opaque
stringData:
  BACKUP_USERNAME: <username>
  BACKUP_PASSWORD: <password>