Benefits

• Continuous backups do not interrupt active jobs and do not impact your workload. Since data is continuously backed up in the background, it does not consume additional CPU resources, unlike full backups.

• You can restore your data whenever required, without the need to create Support tickets.

• You can recover your data while your primary database is online.

Standard vs Enterprise Edition

Restoration Point

The Standard edition allows restoration to 00:00 UTC on any day within the retention window, whereas the Enterprise edition offers finer-grained restoration with one-second granularity to any point in time within the retention window. Both editions support self-service data restoration.

Data Retention Period

For both editions, the default retention period is 7 days and the minimum is 1 day. Standard edition’s retention period cannot be extended beyond 7 days. Enterprise edition has the option to extend this period up to 90 days or longer.

Increasing the retention period results in higher storage costs due to the accumulation of data over time. Even if you delete your tables or records, your data is still retained for the duration of the retention period, potentially impacting your storage costs.

Although the minimum retention period can be set to 1 day, SingleStore recommends setting the retention period to at least 3 days. Retention periods of less than 3 days significantly reduce the reaction time available to address any issues before data is permanently deleted.

Restoring Data from Continuous Backups

Restoring data from continuous backups is a self-service online operation.

To restore data:

1. Create a branch of your database by specifying the relevant timestamp.

2. Use this new branch database to query and restore the data as needed.

For example, consider a scenario where a bad update was executed on January 2nd. You can branch the database to a specific timestamp before the update was executed and recover the data.

In the Standard edition, you can create a branch by specifying a specific date. The timestamp of the branch is automatically set to 00:00:00 and cannot be modified.

ATTACH DATABASE DB_X AS recover_DB_X AT TIME '2025-01-02';

In the Enterprise edition, the data can be restored to a granularity of 1 second. You must ensure the timestamp is specified in UTC.

ATTACH DATABASE DB_X AS recover_DB_X AT TIME '2025-01-02 21:57:31';

Durability Guarantee for Continuously Backed up Data

SingleStore has built-in safeguards to ensure your data is never accidentally deleted.

• The Metadata Service (Inhouse service): safeguards your data by continuously monitoring and managing the metadata. Regular blob quality checks and checksum validations on metadata service are conducted to identify and address potential corruption.

• Robust GC (Garbage Collection) Process: the GC process is designed to prevent accidental data deletion, even in the event of bugs.

• Cloud Provider’s versioning capability: this serves as a last-resort failsafe for recovering data beyond your specified retention period. Leveraging the public cloud provider's versioning feature, data is retained for an additional 3 days. It is important to note that this mechanism should not be considered an extension of your primary retention policy. To recover data beyond your standard retention period, you must submit a support ticket.

Types of Cloud Storage

SingleStore Helios cloud storage has two primary categories:

1. Active Storage: This represents the storage you are currently using to support your specific applications. It includes your primary data, logs, and snapshots that are actively in use.

2. Continuous Backups Storage: This is the space consumed by your configured backup retention settings. The storage volume in this category will vary depending on your retention period, which is 7 days by default.

How to Check Your Cloud Storage Usage

You can calculate your storage usage by querying the MV_CLOUD_REMOTE_STATS information schema.

Here is a sample query to help you calculate:

• Your total storage in megabytes.

• The storage used for continuous backups.

• Your active storage usage.

SELECT rs.STORAGE_ID, GROUP_CONCAT(DISTINCT pc.DATABASE_NAME) AS DATABASE_NAME,    
    ROUND((MAX(rs.BLOBS_BYTES)/1024/1024) + 
         (MAX(rs.LOG_CHUNKS_BYTES)/1024/1024) + 
         (MAX(rs.SNAPSHOTS_BYTES)/1024/1024), 2) AS TOTAL_STORAGE_MB,    
    ROUND(MAX(rs.RETENTION_BLOBS_BYTES)/1024/1024, 2) AS CONTINUOUS_BACKUPS_MB,    
    ROUND(((MAX(rs.BLOBS_BYTES)/1024/1024) +             
           (MAX(rs.LOG_CHUNKS_BYTES)/1024/1024) +             
           (MAX(rs.SNAPSHOTS_BYTES)/1024/ 1024)) -            
           (MAX(rs.RETENTION_BLOBS_BYTES)/1024/1024), 2) AS ACTIVE_STORAGE_MB     
 FROM information_schema.MV_CLOUD_REMOTE_STATS rs
       INNER JOIN     
     information_schema.MV_CLOUD_PER_COMPUTE_REMOTE_STATS pc    
       ON rs.STORAGE_ID = pc.STORAGE_ID    
 GROUP BY ALL;

Cloud Storage Calculations

The total storage is calculated as the sum of your blob storage, log storage, and snapshots:

Total Storage = BLOBS + LOGS + SNAPSHOTS

Continuous backups can be calculated by looking at the number of blobs retained during your retention period.

Continuous Backups = RETENTION BLOBS

To determine your active storage, subtract the continuous backups storage from the total storage.

Active Storage = Total Storage - Continuous Backups Storage