Important

Self-managed SingleStore will soon transition from version 9.1 RC to version 10. This new semantic versioning scheme will provide SingleStore with finer control over engine and feature releases that were not possible with the current versioning scheme.

In the interim, SingleStore 9.1 RC can be used to preview, evaluate, and provide feedback on the new and upcoming features in SingleStore 10 prior to its general availability. Ahead of this transition, SingleStore 9.0 is recommended for production workloads, which can later be upgraded to SingleStore 10.

Load Data with Pipelines

This part of the tutorial shows how to ingest stock trades data from a public AWS S3 bucket into the SingleStore database using pipelines.

Note

The SQL Editor only runs the queries that you select, so ensure you have them all selected before selecting Run.

Run the following SQL commands to create the pipelines:

SQL

USE trades;

CREATE OR REPLACE PIPELINE company
AS LOAD DATA S3 's3://singlestore-docs-example-datasets/trades/company.csv'
CONFIG '{"region":"us-east-1"}'
SKIP DUPLICATE KEY ERRORS
INTO TABLE company;

CREATE OR REPLACE PIPELINE trade
AS LOAD DATA S3 's3://singlestore-docs-example-datasets/trades/trade.csv'
CONFIG '{"region":"us-east-1"}'
SKIP DUPLICATE KEY ERRORS
INTO TABLE trade;

Run the following SQL commands to start the pipelines:
SQL
```
USE trades;
START ALL PIPELINES;
```

Once the Success message is returned for all the created pipelines, SingleStore starts ingesting the data from the S3 bucket.

Load Data with Pipelines

Was this article helpful?

Was this article helpful?