Sign inTry Free

Connect with Dataloop

On this page

Dataloop is an end-to-end AI development platform that enables you to integrate data models and applications for building and deploying AI systems (applications). The SingleStore integration in Dataloop streamlines data processing, table updates, and data uploads between SingleStore Helios and Dataloop datasets.

This integration allows you to:

  • Directly run queries from Dataloop in SingleStore using the integrated service.

  • Automatically create and update tables based on Dataloop dataset information.

  • Upload SingleStore query results directly to Dataloop datasets.

The SingleStore integration in Dataloop uses the SingleStore Python Client.

Prerequisites

  • An active SingleStore Helios deployment.

  • Access to the Dataloop Platform (console).

  • A table in the SingleStore database with at least the following columns:

    • id: Auto-generated field.

    • prompt: Contains the prompt to create in Dataloop.

    • response: Stores the model responses. This field is automatically populated from the Reinforcement Learning Human Feedback (RLHF) pipeline.

Pipeline Nodes

The SingleStore integration in Dataloop provides the following nodes:

  • Import SingleStore

  • Export SingleStore

Import SingleStore

The Import SingleStore node retrieves the prompts from a SingleStore table, adds them to a specified dataset in Dataloop, and creates the prompt items accordingly.

Export SingleStore

The Export SingleStore node updates the best marked response, model name, and id from Dataloop in the corresponding row in the SingleStore table.

Connect to SingleStore from Dataloop

To connect to SingleStore:

  1. Install the SingleStore Integration from Dataloop Marketplace.

  2. (Optional) Create a Secret and store the password of the SingleStore database user. The Secret (password) can also be created/specified while configuring the Dataloop pipelines.

  3. Create a Dataloop pipeline to SingleStore and specify the SingleStore connection configuration in the Import/Export node configuration.

Install the SingleStore Integration

To install the SingleStore integration in Dataloop,

  1. Log in to the Dataloop Platform (console).

  2. On the left navigation pane, select Marketplace.

  3. Under Dataloop Hub, select Applications.

  4. From the list of applications, select SingleStore Integration.

  5. Select Install to install the SingleStore Integration.

  6. On the Install Application dialog, select Proceed.

  7. From the SingleStore Password list, select Add New Secret. You can also select an existing Secret. To create or specify a Secret later, select Set Up Later.

  8. On the Create Secret dialog, enter the following:

    1. Secret Name: A name for the Secret.

    2. Secret: The password for the SingleStore database user used to connect to your SingleStore deployment.

  9. Select Create Secret > Install Application. The SingleStore Integration is now installed.

    Note

    The SingleStore Integration only supports username/password based authentication. Specify the SingleStore connection configuration in the Import/Export node configuration.

Create a Pipeline to SingleStore

To create a pipeline in Dataloop,

  1. On the Dataloop Platform, on the left navigation pane, select Pipelines > Create Pipeline > Start from Scratch.

  2. On the Set Pipeline Name dialog, enter a name for the pipeline and select Create Pipeline.

  3. From the Node Library pane, drag the Import SingleStore or Export SingleStore node to the canvas.

  4. Select the node added to the canvas, and set the following parameters for each node:

    • host: IP address or hostname of the SingleStore Helios deployment.

    • user: Name of the SingleStore database user with which to connect.

    • database: Name of the SingleStore database.

    • table_name: Name of the table in the SingleStore database.

  5. To specify a different access credential, select Actions > Edit Access Credentials, and select (or create) a Secret.

  6. Configure the pipelines as required, and start the pipelines to start data processing.

Last modified: April 10, 2025

Was this article helpful?

On this page

Was this article helpful?

Verification instructions

Note: You must install cosign to verify the authenticity of the SingleStore file.

Use the following steps to verify the authenticity of singlestoredb-server, singlestoredb-toolbox, singlestoredb-studio, and singlestore-client SingleStore files that have been downloaded.

You may perform the following steps on any computer that can run cosign, such as the main deployment host of the cluster.

  1. (Optional) Run the following command to view the associated signature files.

    curl undefined

  2. Download the signature file from the SingleStore release server.

    • Option 1: Click the Download Signature button next to the SingleStore file.

    • Option 2: Copy and paste the following URL into the address bar of your browser and save the signature file.

    • Option 3: Run the following command to download the signature file.

      curl -O undefined

  3. After the signature file has been downloaded, run the following command to verify the authenticity of the SingleStore file.

    echo -n undefined |
        cosign verify-blob --certificate-oidc-issuer https://oidc.eks.us-east-1.amazonaws.com/id/CCDCDBA1379A5596AB5B2E46DCA385BC \
          --certificate-identity https://kubernetes.io/namespaces/freya-production/serviceaccounts/job-worker \
          --bundle undefined \
          --new-bundle-format -
    Verified OK