Pipeline Overview

A pipeline is a mechanism for continuously loading data into a SingleStoreDB database from external sources. With pipelines, users can extract, shape, and load data without the need for additional ETL (Extract, Transform, Load) tools.

Users can monitor the status and performance of pipelines in SingleStoreDB via the dashboard feature. The Pipeline Summary portion of the dashboard provides a high-level overview of the state of all pipelines, including the number and percentage of pipelines in various states such as running, stopped, and errored. The Pipeline Performance section provides more insights into the operation of pipelines. For example, monitor metrics such as execution count, average CPU time per execution, average elapsed time per execution, and others, which can aid identifying and optimizing aspects of pipeline performance.