PROFILE PIPELINE
On this page
Runs a single batch in the foreground to provide detailed resource metrics for each operation in your pipeline.
Syntax
PROFILE PIPELINE pipeline_name
Remarks
-
This command causes implicit commits.
Refer to COMMIT for more information. -
pipeline_is the configured pipeline.name -
Refer to the Permission Matrix for the required permission.
-
To use
PROFILE PIPELINE, you must gather data source offsets throughTEST PIPELINE.Verify that TEST PIPELINEdoes not return an empty set before callingPROFILE PIPELINE.See TEST PIPELINE for more details.
Note
You cannot run PROFILE PIPELINE when the pipeline is in a Running or Error state.
Example
The output of PROFILE PIPELINE is similar to the SHOW PROFILE output described in PROFILE.
Example
TEST PIPELINE kafka LIMIT 10;+---------+| id |+---------+| rlkj || slkjrl || sdlkj || sk || slkjrls || lkjrljs || slkjr |+---------+
PROFILE PIPELINE kafka;
+------------------------------------------------------------------------------------------------------------------------------------------------------+
| PROFILE |
+------------------------------------------------------------------------------------------------------------------------------------------------------+
| Gather partitions:all est_rows:264 alias:remote_0 actual_rows: 7 exec_time: 0ms |
| InsertInto quickstart_kafka.messages [messages_src.id] local:yes est_rows:264 actual_rows: 7 exec_time: 0ms start_time: [00:00:00.259, 00:00:00.280] |
| ExternalIngest pipeline:kafka |
| |---CSVParse actual_rows: 7 exec_time: 0ms start_time: [00:00:00.005, 00:00:00.030] end_time: [00:00:00.259, 00:00:00.280] |
| |---Transform streamed_bytes: 0.043000 KB exec_time: 4ms start_time: [00:00:00.005, 00:00:00.030] end_time: [00:00:00.259, 00:00:00.280] |
| +---Extract streamed_bytes: 0.092000 KB exec_time: 269ms start_time: [00:00:00.005, 00:00:00.030] end_time: [00:00:00.256, 00:00:00.278] |
+------------------------------------------------------------------------------------------------------------------------------------------------------+Last modified: January 27, 2025