PROFILE PIPELINE
Runs a single batch in the foreground to provide detailed resource metrics for each operation in your pipeline. The batch itself is not persisted.
Syntax
PROFILE PIPELINE pipeline_name
Remarks
This command causes implicit commits. See COMMIT for more information.
pipe_line
is the configured pipeline.See the Permission Matrix for the required permission.
To use PROFILE PIPELINE
, you must gather data source offsets through TEST PIPELINE
. Verify TEST PIPELINE
does not return an empty set before calling PROFILE PIPELINE
.
See TEST PIPELINE for more details.
Note
You cannot run PROFILE PIPELINE
when the pipeline is in a Running
or Error
state.
Output
The output of PROFILE PIPELINE
is similar to the SHOW PROFILE
output described in PROFILE.
Example
TEST PIPELINE kafka LIMIT 10; +---------+ | id | +---------+ | rlkj | | slkjrl | | sdlkj | | sk | | slkjrls | | lkjrljs | | slkjr | +---------+ 7 rows in set (0.82 sec) PROFILE PIPELINE kafka; +------------------------------------------------------------------------------------------------------------------------------------------------------+ | PROFILE | +------------------------------------------------------------------------------------------------------------------------------------------------------+ | Gather partitions:all est_rows:264 alias:remote_0 actual_rows: 7 exec_time: 0ms | | InsertInto quickstart_kafka.messages [messages_src.id] local:yes est_rows:264 actual_rows: 7 exec_time: 0ms start_time: [00:00:00.259, 00:00:00.280] | | ExternalIngest pipeline:kafka | | |---CSVParse actual_rows: 7 exec_time: 0ms start_time: [00:00:00.005, 00:00:00.030] end_time: [00:00:00.259, 00:00:00.280] | | |---Transform streamed_bytes: 0.043000 KB exec_time: 4ms start_time: [00:00:00.005, 00:00:00.030] end_time: [00:00:00.259, 00:00:00.280] | | +---Extract streamed_bytes: 0.092000 KB exec_time: 269ms start_time: [00:00:00.005, 00:00:00.030] end_time: [00:00:00.256, 00:00:00.278] | +------------------------------------------------------------------------------------------------------------------------------------------------------+ 6 rows in set (0.98 sec)