Sign inTry Free

PIPELINES_FILES

Warning

SingleStore 9.0 gives you the opportunity to preview, evaluate, and provide feedback on new and upcoming features prior to their general availability. In the interim, SingleStore 8.9 is recommended for production workloads, which can later be upgraded to SingleStore 9.0.

On this page

This view stores information about files that have been extracted from a file system-like data source, such as Amazon S3. Each row represents a single file.

information_schema.PIPELINES_FILES Schema

Column Name

Description

DATABASE_NAME

The name of the database associated with the file.

PIPELINE_NAME

The name of the pipeline associated with the file.

SOURCE_TYPE

The name of the pipeline’s data source, such as S3.

FILE_NAME

The name of the file.

FILE_SIZE

The size of the file in bytes.

FILE_STATE

Shows the state of the file, which is one of the following values:

  • Unloaded: The pipeline has not yet loaded the data in the file into any tables.

  • Loaded: The pipeline has successfully processed the file. The data in the file has been loaded into one or more tables.

  • Skipped: The pipeline encountered an error while processing the file; the file is skipped and no data in the file has been loaded into any tables.

Note

Monitor the information_schema.pipelines_files table. It stores data for the duration of the pipeline’s lifetime, resulting in a large number of files referenced in the table. If the data in this table is not required, you can either drop a file record from the table or clear all the file records from it.

Last modified: August 1, 2024

Was this article helpful?

On this page

Was this article helpful?

Verification instructions

Note: You must install cosign to verify the authenticity of the SingleStore file.

Use the following steps to verify the authenticity of singlestoredb-server, singlestoredb-toolbox, singlestoredb-studio, and singlestore-client SingleStore files that have been downloaded.

You may perform the following steps on any computer that can run cosign, such as the main deployment host of the cluster.

  1. (Optional) Run the following command to view the associated signature files.

    curl undefined

  2. Download the signature file from the SingleStore release server.

    • Option 1: Click the Download Signature button next to the SingleStore file.

    • Option 2: Copy and paste the following URL into the address bar of your browser and save the signature file.

    • Option 3: Run the following command to download the signature file.

      curl -O undefined

  3. After the signature file has been downloaded, run the following command to verify the authenticity of the SingleStore file.

    echo -n undefined |
        cosign verify-blob --certificate-oidc-issuer https://oidc.eks.us-east-1.amazonaws.com/id/CCDCDBA1379A5596AB5B2E46DCA385BC \
          --certificate-identity https://kubernetes.io/namespaces/freya-production/serviceaccounts/job-worker \
          --bundle undefined \
          --new-bundle-format -
    Verified OK