Sign inTry Free

Load Data with SingleStore Flow

On this page

Overview

SingleStore Flow (or, "Flow") is SingleStore's data migration and change data capture (CDC) solution which is comprised of two primary components:

  • SingleStore Ingest, also referred to as "Ingest," transfers the schema for all tables, moves table data for tables up to 10GB in size, and handles ongoing CDC for all tables into SingleStore.

  • SingleStore XL Ingest, also referred to as "XL Ingest," handles the initial transfer of large tables by breaking them into smaller logical partitions. It then copies multiple partitions from the source to the target in parallel. This helps transfer large tables in a reasonable amount of time.

Flow supports the following source databases for transfer to SingleStore:

  • Oracle

  • Microsoft SQL Server

  • MySQL

  • PostgreSQL

  • Any Database (Snowflake)

Prerequisites

While the database prerequisites will vary based on the source, the following are the minimum requirements for using both Ingest and XL Ingest.

Component

Requirements

Virtual Machine (VM)

The sizing recommendation is based on the size of databases/tables being transferred.

For smaller databases sized 100GB or less, a minimum of 2 vCPUs, 8 GB RAM, and 100 GB of free disk space is recommended.

For large transfers, more disk space may be required. The required free space for Ingest is twice the size of the largest table to be transferred.

For very large tables, where XL Ingest must be used, the amount of local disk needed is 2 x (logical partition size) x (number of concurrent partition transfers).

More CPU cores and RAM is required for sources and destinations that contain a large amount of data and have an aggressive timeline in which to transfer it.

The VM must be close to the source database to minimize latency.

Refer to the Recommended Hardware Configuration section in the Ingest documentation for more information.

Browser

A Chromium-based browser, such as Google Chrome or Microsoft Edge, installed on the VM

Java

Amazon Corretto Java 21, a no-cost distribution of the Open Java Development Kit (OpenJDK), installed on the VM

In this section

Last modified: March 20, 2025

Was this article helpful?

On this page

Was this article helpful?

Verification instructions

Note: You must install cosign to verify the authenticity of the SingleStore file.

Use the following steps to verify the authenticity of singlestoredb-server, singlestoredb-toolbox, singlestoredb-studio, and singlestore-client SingleStore files that have been downloaded.

You may perform the following steps on any computer that can run cosign, such as the main deployment host of the cluster.

  1. (Optional) Run the following command to view the associated signature files.

    curl undefined

  2. Download the signature file from the SingleStore release server.

    • Option 1: Click the Download Signature button next to the SingleStore file.

    • Option 2: Copy and paste the following URL into the address bar of your browser and save the signature file.

    • Option 3: Run the following command to download the signature file.

      curl -O undefined

  3. After the signature file has been downloaded, run the following command to verify the authenticity of the SingleStore file.

    echo -n undefined |
        cosign verify-blob --certificate-oidc-issuer https://oidc.eks.us-east-1.amazonaws.com/id/CCDCDBA1379A5596AB5B2E46DCA385BC \
          --certificate-identity https://kubernetes.io/namespaces/freya-production/serviceaccounts/job-worker \
          --bundle undefined \
          --new-bundle-format -
    Verified OK