PIPELINES_ BATCHES
Warning
SingleStore 9.0 gives you the opportunity to preview, evaluate, and provide feedback on new and upcoming features prior to their general availability. In the interim, SingleStore 8.9 is recommended for production workloads, which can later be upgraded to SingleStore 9.0.
On this page
This view contains detailed, low-level information about individual batches as they’re loaded into the database.
It contains one row per batch partition and is useful for understanding partition-level details of a pipeline’s behavior.
The data in this view is not persistent across node restarts.
The following pipeline engine variable is used to control an aspect of pipeline behavior.
-
pipelines_
- controls the number of batch metadata entries to keep before they are overwritten by incoming batches.batches_ metadata_ to_ keep
information_ schema. PIPELINES_ BATCHES Schema
Name |
Description |
---|---|
|
The name of the database associated with the batch. |
|
The name of the pipeline associated with the batch. |
|
The internal unique identifier for the batch. |
|
Specifies whether the batch transaction was successful. |
|
The number of rows that were affected in the destination table as a result of the batch transaction. |
|
The elapsed time in seconds from the start of the batch transaction to its success or failure. |
|
The time that the batch transaction started in Unix timestamp format. |
|
Specifies whether the batch transaction was successful for a given partition. |
|
The number of rows that were parsed during the batch transaction for a given partition. |
|
The identifier for a data source’s partition from which batch partition data is extracted. |
|
Specifies the earliest offset for the batch partition. |
|
Specifies the latest offset for the batch. |
|
The elapsed time in seconds from the start of the batch transaction to its success or failure. |
|
The number of bytes that was extracted from the data source for the batch. |
|
The number of bytes that were transformed for the batch. |
|
The amount of time, in seconds, that the pipeline spent waiting on the extractor to complete its operation. |
|
The amount of time, in seconds, that the pipeline spent waiting on the transform to complete its operation. |
|
The hostname or host IP address for the leaf node that processed the batch. |
|
The port number for the leaf node that processed the batch. |
|
Specifies the partition ID on a leaf node that processed the batch. |
Last modified: March 4, 2024