Sign inTry Free

Prerequisites

On this page

The following are the prerequisites for launching Ingest on Amazon EC2:

  • Selection of the Ingest volume.

  • Selection of the EC2 instance type.

  • Ensure connectivity between the server/EC2 hosting the Ingest software and the source. Additionally, ensure connectivity to DynamoDB if the high availability option is required.

To create the necessary AWS services, refer to Environment Setup.

The following are the steps to take before launching Ingest in AWS via custom installation on an EC2:

  1. Create a policy with a relevant name for EC2, such as FlowEC2Policy. Refer to the Define custom IAM permissions with customer managed policies for creating policies.

  2. Refer to AWS Identity and Access Management (IAM) for SingleStore Flow for JSON policy.

  3. Create an IAM role called FlowEC2Role. Refer to Create a role to delegate permissions to an IAM user for creating roles.

  4. Attach the FlowEC2Policy to the role.

  5. Create a Lambda policy for disk checks and attach the Lambda policy JSON. Refer to AWS Recovery for Lambda Policy JSON.

The following are the recommended EC2 options for replicating source data volumes.

Total Data Volume

EC2 Recommended

< 100 GB

t2.small

100GB – 300GB

t2.medium

300GB – 1TB

t2.large

> 1TB

Contact SingleStore Support

These recommendations serve as a starting point. If you have any questions, please contact SingleStore Support or your technical account team representative.

The following are the system requirements when not using the Amazon EC2:

  • Port 8081 must be open on the server hosting the Ingest software.

  • Google Chrome is required as the internet browser on the server hosting Ingest software.

  • Java version 21 or higher is required.

  • If using Microsoft SQL Server as a source, download and install the BCP utility.

  • Ensure connectivity between the server hosting the Ingest software and the source, and DynamoDB (if the high availability option is required).

Recommended Hardware Configuration

The following describes the hardware configuration for a Windows server, assuming that there are a few sources and target combinations (3 medium, ideally). It also depends on how intensively the data is being replicated from these sources, so this is a guide, but will need extra resources depending on the amount of data being replicated. The amount of disk space will also be dependent on the amount of data being replicated.

The following describes the hardware configuration for a Windows server; similar configuration is recommended for a Linux or Ubuntu based server. The configuration also depends on the intensity of data replication from these sources. Additional resources may be required based on the volume of data being replicated. The disk space required also depends on the amount of data being replicated.

Component

Specification

Processor

4 cores

Memory

16 GB

Disk requirements

Varies based on the data being extracted, with a minimum of 300 GB

Network performance

High

Prerequisites for Software on Server

The following software must be installed on the server:

Required Skills

Flow is a suite of robust applications that makes seamless data replication to the cloud. It handles large data volumes with ease, and the process is fully automated. The setup takes only three simple steps. The application does not require highly technical resources, but basic knowledge of the following is recommended for deployment:

  • AWS Cloud Fundamentals

  • Basic database skills, including writing and executing database queries (for RDBMS endpoints)

  • Familiarity with using Microsoft Windows or Linux-based systems

Installation

For details on how to install Ingest and other Flow components, refer to Install SingleStore Flow.

Last modified: September 1, 2025

Was this article helpful?

On this page

Was this article helpful?

Verification instructions

Note: You must install cosign to verify the authenticity of the SingleStore file.

Use the following steps to verify the authenticity of singlestoredb-server, singlestoredb-toolbox, singlestoredb-studio, and singlestore-client SingleStore files that have been downloaded.

You may perform the following steps on any computer that can run cosign, such as the main deployment host of the cluster.

  1. (Optional) Run the following command to view the associated signature files.

    curl undefined

  2. Download the signature file from the SingleStore release server.

    • Option 1: Click the Download Signature button next to the SingleStore file.

    • Option 2: Copy and paste the following URL into the address bar of your browser and save the signature file.

    • Option 3: Run the following command to download the signature file.

      curl -O undefined

  3. After the signature file has been downloaded, run the following command to verify the authenticity of the SingleStore file.

    echo -n undefined |
        cosign verify-blob --certificate-oidc-issuer https://oidc.eks.us-east-1.amazonaws.com/id/CCDCDBA1379A5596AB5B2E46DCA385BC \
          --certificate-identity https://kubernetes.io/namespaces/freya-production/serviceaccounts/job-worker \
          --bundle undefined \
          --new-bundle-format -
    Verified OK