PROFILE PIPELINE
On this page
Runs a single batch in the foreground to provide detailed resource metrics for each operation in your pipeline.
Syntax
PROFILE PIPELINE pipeline_name
Remarks
-
This command causes implicit commits.
Refer to COMMIT for more information. -
pipe_is the configured pipeline.line -
Refer to the Permission Matrix for the required permission.
To use PROFILE PIPELINE, you must gather data source offsets through TEST PIPELINE.TEST PIPELINE does not return an empty set before calling PROFILE PIPELINE.
See TEST PIPELINE for more details.
Note
You cannot run PROFILE PIPELINE when the pipeline is in a Running or Error state.
Output
The output of PROFILE PIPELINE is similar to the SHOW PROFILE output described in PROFILE.
Example
TEST PIPELINE kafka LIMIT 10;+---------+| id |+---------+| rlkj || slkjrl || sdlkj || sk || slkjrls || lkjrljs || slkjr |+---------+7 rows in set (0.82 sec)PROFILE PIPELINE kafka;+------------------------------------------------------------------------------------------------------------------------------------------------------+| PROFILE |+------------------------------------------------------------------------------------------------------------------------------------------------------+| Gather partitions:all est_rows:264 alias:remote_0 actual_rows: 7 exec_time: 0ms || InsertInto quickstart_kafka.messages [messages_src.id] local:yes est_rows:264 actual_rows: 7 exec_time: 0ms start_time: [00:00:00.259, 00:00:00.280] || ExternalIngest pipeline:kafka || |---CSVParse actual_rows: 7 exec_time: 0ms start_time: [00:00:00.005, 00:00:00.030] end_time: [00:00:00.259, 00:00:00.280] || |---Transform streamed_bytes: 0.043000 KB exec_time: 4ms start_time: [00:00:00.005, 00:00:00.030] end_time: [00:00:00.259, 00:00:00.280] || +---Extract streamed_bytes: 0.092000 KB exec_time: 269ms start_time: [00:00:00.005, 00:00:00.030] end_time: [00:00:00.256, 00:00:00.278] |+------------------------------------------------------------------------------------------------------------------------------------------------------+6 rows in set (0.98 sec)
Last modified: April 4, 2023