Continuous Backups

Note

For AWS customers, continuous backups will replace the current processes described in Backup Data in Back Up and Restore Data from April 1, 2025. For GCP and Azure customers, the current process of taking admin backup will remain unchanged.

Continuous backups asynchronously store all customer data in unlimited storage (bottomless storage) ensuring that every change (inserts, updates, and deletes) is durably persisted with 99.9999% reliability.

With a default 7-day retention period, they allow you to restore your data to a point in time within this window using the always available point-in-time recovery (PITR) option.

Benefits

  • Continuous backups do not interrupt active jobs and do not impact your workload. Since data is continuously backed up in the background, it does not consume additional CPU resources, unlike full backups.

  • You can restore your data whenever required, without the need to create Support tickets.

  • You can recover your data while your primary database is online.

Standard vs Enterprise Edition

Restoration Point

The Standard edition allows restoration to 00:00 UTC on any day within the retention window, whereas the Enterprise edition offers finer-grained restoration with one-second granularity to any point in time within the retention window. Both editions support self-service data restoration.

Data Retention Period

For both editions, the default retention period is 7 days and the minimum is 1 day. Standard edition’s retention period cannot be extended beyond 7 days. Enterprise edition has the option to extend this period up to 90 days or longer.

Increasing the retention period results in higher storage costs due to the accumulation of data over time. Even if you delete your tables or records, your data is still retained for the duration of the retention period, potentially impacting your storage costs.

Although the minimum retention period can be set to 1 day, SingleStore recommends setting the retention period to at least 3 days. Retention periods of less than 3 days significantly reduce the reaction time available to address any issues before data is permanently deleted.

Note

To retain your data for extended time periods beyond what is available with continuous backups, you can use the BACKUP DATABASE command to back up your database to your own object storage.

Restoring Data from Continuous Backups

Restoring data from continuous backups is a self-service online operation.

To restore data:

  1. Create a branch of your database by specifying the relevant timestamp.

  2. Use this new branch database to query and restore the data as needed.

For example, consider a scenario where a bad update was executed on January 2nd. You can branch the database to a specific timestamp before the update was executed and recover the data.

In the Standard edition, you can create a branch by specifying a specific date. The timestamp of the branch is automatically set to 00:00:00 and cannot be modified.

ATTACH DATABASE DB_X AS recover_DB_X AT TIME '2025-01-02';

In the Enterprise edition, the data can be restored to a granularity of 1 second.

ATTACH DATABASE DB_X AS recover_DB_X AT TIME '2025-01-02 21:57:31';

Durability Guarantee for Continuously Backed up Data

SingleStore has built-in safeguards to ensure your data is never accidentally deleted.

  • The Metadata Service (Inhouse service) : safeguards your data by continuously monitoring and managing the metadata. Regular blob quality checks and checksum validations on metadata service are conducted to identify and address potential corruption.

  • Robust GC (Garbage Collection) Process: the GC process is designed to prevent accidental data deletion, even in the event of bugs.

  • Cloud Provider’s versioning capability: this serves as a last-resort failsafe for recovering data beyond your specified retention period. Leveraging the public cloud provider's versioning feature, data is retained for an additional 3 days. It is important to note that this mechanism should not be considered an extension of your primary retention policy. To recover data beyond your standard retention period, you must submit a support ticket.

Last modified: February 14, 2025

Was this article helpful?

Verification instructions

Note: You must install cosign to verify the authenticity of the SingleStore file.

Use the following steps to verify the authenticity of singlestoredb-server, singlestoredb-toolbox, singlestoredb-studio, and singlestore-client SingleStore files that have been downloaded.

You may perform the following steps on any computer that can run cosign, such as the main deployment host of the cluster.

  1. (Optional) Run the following command to view the associated signature files.

    curl undefined
  2. Download the signature file from the SingleStore release server.

    • Option 1: Click the Download Signature button next to the SingleStore file.

    • Option 2: Copy and paste the following URL into the address bar of your browser and save the signature file.

    • Option 3: Run the following command to download the signature file.

      curl -O undefined
  3. After the signature file has been downloaded, run the following command to verify the authenticity of the SingleStore file.

    echo -n undefined |
    cosign verify-blob --certificate-oidc-issuer https://oidc.eks.us-east-1.amazonaws.com/id/CCDCDBA1379A5596AB5B2E46DCA385BC \
    --certificate-identity https://kubernetes.io/namespaces/freya-production/serviceaccounts/job-worker \
    --bundle undefined \
    --new-bundle-format -
    Verified OK