Sign inTry Free

PIPELINES_CURSORS

Warning

SingleStore 9.0 gives you the opportunity to preview, evaluate, and provide feedback on new and upcoming features prior to their general availability. In the interim, SingleStore 8.9 is recommended for production workloads, which can later be upgraded to SingleStore 9.0.

On this page

This view contains information about a pipeline’s offset ranges. Each row represents a single data source partition.

information_schema.PIPELINES_CURSORS Schema

Column Name

Description

DATABASE_NAME

The name of the database associated with the offsets.

PIPELINE_NAME

The name of the pipeline associated with the offsets.

SOURCE_TYPE

The name of the pipeline’s data source, such as KAFKA or S3.

SOURCE_PARTITION_ID

The identifier for a data source’s partition from which offsets are extracted. For an S3 pipeline, this value represents the name of a bucket’s object.

EARLIEST_OFFSET

The earliest offset for an offset range. This value indicates the starting offset and LATEST_OFFSET indicates the ending offset in an offset range. This value may be NULL if data extraction is still in progress or failed.

LATEST_OFFSET

The bounding offset for an offset range, exclusive. This value indicates the ending offset and EARLIEST_OFFSET indicates the starting offset in an offset range. This value may be NULL if data extraction is still in progress or failed.

CURSOR_OFFSET

The next offset at which the pipeline will start loading in this partition. This number represents the end of the range which has already been loaded by this pipeline, exclusive; all future considered offsets will be beyond this.

SUCCESSFUL_CURSOR_OFFSET

The offset cursor position in this source partition that has been successfully loaded. If this is lower than CURSOR_OFFSET, it implies the intermediate offsets were skipped. Note: offsets will only be skipped if pipelines_stop_on_error is false.

UPDATED_UNIX_TIMESTAMP

The last updated time for an offset range in Unix timestamp format.

EXTRA_FIELDS

Some extractors output additional information when extracting data from the source. For S3 pipelines, you may encounter a value of {"size":<bytes>}, where <bytes> represents the size of an object in the source bucket.

Last modified: January 10, 2025

Was this article helpful?

On this page

Was this article helpful?

Verification instructions

Note: You must install cosign to verify the authenticity of the SingleStore file.

Use the following steps to verify the authenticity of singlestoredb-server, singlestoredb-toolbox, singlestoredb-studio, and singlestore-client SingleStore files that have been downloaded.

You may perform the following steps on any computer that can run cosign, such as the main deployment host of the cluster.

  1. (Optional) Run the following command to view the associated signature files.

    curl undefined

  2. Download the signature file from the SingleStore release server.

    • Option 1: Click the Download Signature button next to the SingleStore file.

    • Option 2: Copy and paste the following URL into the address bar of your browser and save the signature file.

    • Option 3: Run the following command to download the signature file.

      curl -O undefined

  3. After the signature file has been downloaded, run the following command to verify the authenticity of the SingleStore file.

    echo -n undefined |
        cosign verify-blob --certificate-oidc-issuer https://oidc.eks.us-east-1.amazonaws.com/id/CCDCDBA1379A5596AB5B2E46DCA385BC \
          --certificate-identity https://kubernetes.io/namespaces/freya-production/serviceaccounts/job-worker \
          --bundle undefined \
          --new-bundle-format -
    Verified OK