SingleStore DB

Verify Pipeline Success

First, navigate to Pipelines within Studio. This is where you can see the status of each pipeline job. This is helpful for verifying whether there have been any errors during the loading, and other status related information.

The entire loading process takes around four minutes, but you do not need to wait for Pipelines to finish before querying. You can query the data as soon as you have started the loading process; this is part of the powerful functionality of Pipelines.

If you want to know whether the pipeline has finished loading, you can check the progress of a pipeline via the pipelines_files Information Schema table. For example, to see whether the "lineitem" pipeline has finished:

SELECT * FROM information_schema.pipelines_files WHERE pipeline_name = "tpch_100_lineitem";

You can also run your pipeline in the foreground, which makes it easy to verify that all the data has been loaded:

START PIPELINE <pipeline_name> FOREGROUND;

In the next page, we will show you how to get business-critical metrics.