PIPELINES_OFFSETS

On this page

This view contains information about a pipeline's offset ranges. Each row represents a single data source partition and tracks various offset positions throughout the pipeline's extraction and loading process.

Note

This view contains integer-based offsets for traditional pipelines (Kafka, S3, Azure, GCS, HDFS, FS). The offset values in this view represent numeric positions or ranges within the data source partitions.

information_schema.PIPELINES_OFFSETS Schema

Column Name	Description
`DATABASE_NAME`	The name of the database that contains the pipeline.
`PIPELINE_NAME`	The name of the pipeline associated with the offsets.
`SOURCE_TYPE`	The pipeline data source type, such as KAFKA, S3, AZURE, GCS, HDFS, or FS(filesystem). This column identifies the external system from which data is extracted.
`SOURCE_PARTITION_ID`	The identifier of the source partition from which offsets are extracted. For S3 pipelines, this value represents an object name in a bucket. For Kafka pipelines, it represents the partition number. For file-based sources, it represents a file path.
`EARLIEST_OFFSET`	The starting offset of an offset range. Together with `LATEST_OFFSET`, this column defines the range of data that has been or will be processed. This value may be `NULL` if extraction has not started or is still in progress.
`LATEST_OFFSET`	The ending offset of an offset range (exclusive). Together with `EARLIEST_OFFSET`, this column defines the available data range in the source partition. This value may be `NULL` if extraction is in progress or has failed.
`LATEST_LOADED_OFFSET`	The most recent offset that the pipeline has extracted. This data may not yet be committed to the destination table. This value advances as the pipeline reads from the source.
`LATEST_SUCCESSFUL_OFFSET`	The most recent offset that has been successfully loaded and committed. If this value is lower than `LATEST_LOADED_OFFSET`, extracted data is still being processed or has encountered errors. This column represents the safe restart position for the pipeline.
`LATEST_EXPECTED_OFFSET`	The next offset at which the pipeline expects to resume loading. Under normal operation, this value closely follows `LATEST_SUCCESSFUL_OFFSET`. Large differences may indicate processing delays or partition rebalancing.
`UPDATED_UNIX_TIMESTAMP`	The time when the offset metadata was last updated, expressed as a Unix timestamp. Use this column to detect stalled pipelines or inactive partitions.
`EXTRA_FIELDS`	Additional extractor-specific metadata in JSON format. Some extractors include extra information, such as `{"size": <bytes>}` for S3 objects. For other sources, this column may contain connector-specific metadata, such as file timestamps, compression details, or partition information. This value may be `NULL` if no additional metadata is available.

Last modified: February 5, 2026

Was this article helpful?

Verification instructions

Note: You must install cosign to verify the authenticity of the SingleStore file.

Use the following steps to verify the authenticity of singlestoredb-server, singlestoredb-toolbox, singlestoredb-studio, and singlestore-client SingleStore files that have been downloaded.

You may perform the following steps on any computer that can run cosign, such as the main deployment host of the cluster.

(Optional) Run the following command to view the associated signature files.
Shell
```
curl undefined
```
Download the signature file from the SingleStore release server.
- Option 1: Click the Download Signature button next to the SingleStore file.
- Option 2: Copy and paste the following URL into the address bar of your browser and save the signature file.
- Option 3: Run the following command to download the signature file.
  Shell
```
curl -O undefined
```

After the signature file has been downloaded, run the following command to verify the authenticity of the SingleStore file.

Shell

echo -n undefined |
    cosign verify-blob --certificate-oidc-issuer https://oidc.eks.us-east-1.amazonaws.com/id/CCDCDBA1379A5596AB5B2E46DCA385BC \
      --certificate-identity https://kubernetes.io/namespaces/freya-production/serviceaccounts/job-worker \
      --bundle undefined \
      --new-bundle-format -

Verified OK