Cluster Architecture

Unique Design

SingleStore is a real-time hybrid transaction/analytics processing (HTAP) database designed with a distributed SQL architecture. This means that compute can be scaled out using a clustered architecture, rather than only scaling up using larger machines.

Clusters further enhance this distributed architecture by freeing databases from the confines of a single workload, delivering true separation of compute and storage.

Implementation

Each cluster is a part of a Group, and Databases can be attached or detached from any cluster within a group. Each group can contain up to five clusters. Groups are deployed in a cloud provider and region. An organization can have many groups deployed across multiple cloud providers and regions.

Databases can be attached to multiple clusters simultaneously. Each database can have one R/W attachment and many R/O attachments. This allows data to be shared across workloads, providing scalability and isolation.

The R/W attachment writes logs/blobs to shared storage and sends logs to the R/O attached clusters asynchronously. The R/O attached clusters replay these logs, creating local blobs.

For example, a customer-facing SaaS application is running on cluster 1 and writing data into DB01. An internal telemetry application running on cluster 2 is reading data from DB01. In this scenario, DB01 has an R/W attachment to cluster 1 and a R/O attachment to cluster 2.

Attachments are created on each database, and it is possible for a cluster to have a combination of R/W and R/O attachments.

Writes made to a database from a cluster with an R/W attachment will appear on clusters with R/O attachments near instantaneously. The transaction log tail is sent to the replica and applied there asynchronously in real time. Thus, R/W-R/O sync is eventually made consistent.

All new writes to a database are typically replicated immediately, allowing clusters with R/O attachments to serve real-time workloads without waiting for data to load or going through complex ETL processes.

There are two situations in which replay latency can happen:

• Load replay needs the CPU to go through the log, so if this part is slowed down, replay will fall behind.

• SingleStore processes and saves transactions in a certain order, but it does not always commit them in that order. For example, if you have a big load followed by a small load, the system might finish the small task first. However, when it needs to replay or sync this on a replica, the replica has to wait for the big load to finish before replaying the small one.

When attaching a database to a cluster as R/O, all new data written will be replicated to the new cluster.

When sizing the cluster ensure that the selected size can accommodate the size of the working data set. A very small cluster attaching a database with a large working set will not have sufficient resources to process the incoming replication stream or reads against the database.

For example, if a rowstore table is replicated to a small R/O cluster, replay on the R/O cluster will stop once the memory limit is reached, and the database can become unrecoverable. If replay stops, then eventually the write-side may also stop. An atomic write for two databases within a single cluster is not supported.