Data Loading for HDFS Pipelines

When the master aggregator reads an HDFS output directory’s contents, it schedules each file on a single SingleStoreDB Cloud partition. After each leaf partition across the workspace has finished extracting, transforming, and loading its file, a batch operation has been completed.