Aggregator Pipelines

Aggregator pipelines load data through the aggregator instead of loading data directly to the leaves. Using aggregator pipelines can improve efficiency for low parallelism pipelines, such as single file S3 loads or single partition Kafka topics, and it is required for pipelines that load data into reference tables and tables with auto increment columns.

Aggregator pipelines only support stored procedures for MongoDB and MySQL sources, i.e., CREATE AGGREGATOR PIPELINE ... AS LOAD DATA {MONGODB | MYSQL} statements.

To create an aggregator pipeline, use the CREATE AGGREGATOR PIPELINE command.

Last modified: February 4, 2025

Was this article helpful?

Verification instructions

Note: You must install cosign to verify the authenticity of the SingleStore file.

Use the following steps to verify the authenticity of singlestoredb-server, singlestoredb-toolbox, singlestoredb-studio, and singlestore-client SingleStore files that have been downloaded.

You may perform the following steps on any computer that can run cosign, such as the main deployment host of the cluster.

(Optional) Run the following command to view the associated signature files.
Shell
```
curl undefined
```
Download the signature file from the SingleStore release server.
- Option 1: Click the Download Signature button next to the SingleStore file.
- Option 2: Copy and paste the following URL into the address bar of your browser and save the signature file.
- Option 3: Run the following command to download the signature file.
  Shell
```
curl -O undefined
```

After the signature file has been downloaded, run the following command to verify the authenticity of the SingleStore file.

Shell

echo -n undefined |
    cosign verify-blob --certificate-oidc-issuer https://oidc.eks.us-east-1.amazonaws.com/id/CCDCDBA1379A5596AB5B2E46DCA385BC \
      --certificate-identity https://kubernetes.io/namespaces/freya-production/serviceaccounts/job-worker \
      --bundle undefined \
      --new-bundle-format -

Verified OK