SingleStore Managed Service

information_schema.PIPELINES_BATCHES_SUMMARY Table

The PIPELINES_BATCHES_SUMMARY table contains high-level information about individual batches as they are loaded into the database. The columns in this table are described below. It contains one row per batch.

information_schema.PIPELINES_BATCHES_SUMMARY Table Schema

Name

Description

DATABASE_NAME

The name of the database associated with the batch.

PIPELINE_NAME

The name of the pipeline associated with the batch.

BATCH_ID

The internal unique identifier for the batch. Values of BATCH_ID are unique in this table.

BATCH_STATE

Specifies whether the batch transaction was successful. There are five possible values: Succeeded, In Progress, Failed, Queued or No Data. Succeeded: The batch was successfully loaded into the destination table. In Progress: The batch is currently being loaded into the destination table. Failed: The batch failed to be written. Batch failures are written as errors in the information_schema.PIPELINES_ERRORS table and you can use the BATCH_ID to correlate a specific batch failure with more detailed error information. Queued: The batch is waiting for enough resources in the cluster to free up so it can run. No Data: The batch contained no data. Batches that don’t contain data will be deleted as new batches are written.

START_TIME

The time that the batch transaction started in DATETIME format.

BATCH_TIME

The number of seconds the batch took to run, or if the batch is In Progress, the number of seconds it has been running for.

ROWS_PER_SEC

Equal to ROWS_STREAMED / BATCH_TIME.

ROWS_STREAMED

The number of rows extracted extracted into SingleStore DB. It can be higher or lower than the number of rows written, if, for instance, rows are filtered by a WHERE clause or SKIP clause, or if rows are modified by an upsert clause.

NUM_PARTITIONS

The total number of batch partitions for this batch.

NUM_PARTITIONS_FINISHED

The total number of finished batch partitions for this batch. A batch cannot finish until all batch partitions are finished.

MB_STREAMED

The number of megabytes extracted into SingleStore DB.

MB_PER_SEC

Equal to MB_STREAMED / BATCH_TIME.