Configuring Transaction Logs and Snapshots

The following engine variables are used to configure transaction logs and snapshots.

• log_file_size_partitions: Defines the size of each transaction log file created for a partition. This variable is specified in bytes and can range from 1 MB to 1 TB. It can be configured before creating a database or before restoring a database from a backup. Default: 268435456 bytes (or 256 MB).

  When a database is created, each partition is preallocated with two log files. The default preallocated size of each log file is 256 MB of disk space, for a total of 512 MB. When the preallocated log files grow to a size of 256 MB each, a new log file will be created as long as the total transaction log size falls within the snapshot_trigger_size value.

• log_file_size_ref_dbs: Defines the size of each transaction log file created for a reference database. This variable is specified in bytes and can range from 1 MB to 1 TB. It can be configured before creating a database or before restoring a database from a backup. Default: 67108864 bytes (or 64 MB).

  Each reference database is preallocated with two log files. The default preallocated size of each log file is 64 MB of disk space, for a total of 128 MB per reference database per node. When the preallocated log files grow to a size of 64 MB each, a new log file will be created as long as the total transaction log size falls within the snapshot_trigger_size value.

• snapshot_trigger_size: Defines the size of transaction logs in bytes, which, when reached, triggers a snapshot that is written to disk. Following the creation of a snapshot, subsequent database modifications are written to new transaction logs until the snapshot_trigger_size is reached again. Default: 2147483648 bytes (or 2048 MB).

  Old transaction logs that are committed to snapshots are periodically deleted based on the value defined by the snapshots_to_keep variable.

• snapshots_to_keep: Defines the number of snapshots to keep for backup and replication. In addition to the predefined number of snapshots, transaction logs that were used to create those snapshots will also be preserved. Consequently, older (or "unused") transaction logs are periodically deleted, with the exception of one to four most recent unused logs, depending on your workload. Default: 2.

• datadir: Specifies the path to the data directory on each node, where transaction logs reside (among other data files). By default, the data directory is located under /var/lib/memsql/<node_ID>/data. However, this path may change depending on how your cluster was deployed.

• minimal_disk_space: Defines the minimal disk space in MB, which, when reached on the disk where the data directories (such as logs, snapshots, blobs, plancache, auditlogs, and tracelogs) are mounted, SingleStore will halt new write queries. As transaction logs can potentially take up significant space on a disk, it is important to monitor this configuration.

In addition to the above engine variables, you can also configure snapshot_wait_for_blob_gc_seconds and bottomless_snapshot_trigger_log_chunks. For information about how and when to configure these variables, see List of Engine Variables.