PIPELINES_BATCHES_SUMMARY

DATABASE_NAME

The name of the database associated with the batch.

PIPELINE_NAME

The name of the pipeline associated with the batch.

BATCH_ID

The internal unique identifier for the batch. Values of BATCH_ID are unique in this table.

BATCH_STATE

Specifies whether the batch transaction was successful. There are five possible values: Succeeded, In Progress, Failed, Queued, or Cancelled.

Succeeded: The batch was successfully loaded into the destination table.

In Progress: The batch is currently being loaded into the destination table.

Failed: The batch failed to be written. Batch failures are written as errors in the information_schema.PIPELINES_ERRORS table and you can use the BATCH_ID to correlate a specific batch failure with more detailed error information.

Queued: The batch is waiting for enough resources in the cluster to free up so it can run.

Cancelled: A pipeline batch's internal query is killed. That can be triggered internally, if you run STOP PIPELINE while a batch is running, or externally, by CTRL-C-ing START PIPELINE FOREGROUND.

START_TIME

The time that the batch transaction started in DATETIME format.

EXTRACTOR_WAIT_TIME

The amount of time, in seconds, that the pipeline spent waiting on the extractor to complete its operation.

TRANSFORM_WAIT_TIME

The amount of time, in seconds, that the pipeline spent waiting on the transform to complete its operation.

BATCH_TIME

The number of seconds the batch took to run, or if the batch is In Progress, the number of seconds it has been running for.

ROWS_PER_SEC

Equal to ROWS_STREAMED / BATCH_TIME.

ROWS_STREAMED

The number of rows extracted into SingleStore. It can be higher or lower than the number of rows written, if, for instance, rows are filtered by a WHERE clause or SKIP clause, or if rows are modified by an upsert clause.

NUM_PARTITIONS

The total number of batch partitions for this batch.

NUM_PARTITIONS_FINISHED

The total number of finished batch partitions for this batch. A batch cannot finish until all batch partitions are finished.

MB_STREAMED

The number of megabytes extracted into SingleStore.

MB_PER_SEC

Equal to MB_STREAMED / BATCH_TIME.