Sign inTry Free
  • Getting Started with SingleStoreDB Cloud
    • Connect to Your Workspace
      • Create a Database
        • Integrate with SingleStoreDB Cloud
        • Load Data
          • Load Data from a Data Source
            • Load a Type of Data
              • Load Data with Pipelines
                • Pipeline Data Sources and Formats
                  • The Lifecycle of a Pipeline
                  • Parallelized Data Extraction with Pipelines
                    • Data Loading for Azure Blob Pipelines
                    • Data Loading for HDFS Pipelines
                    • Data Loading for Kafka Pipelines
                    • Data Loading for S3 Pipelines
                    • Pipelines Scheduling
                  • Data Shaping with Pipelines
                    • Avro Schema Evolution with Pipelines
                      • Pipeline Procedures
                      • Load Data - Related Topics
                    • Query Data
                      • Manage Data
                        • Developer Resources
                          • User and Workspace Administration
                            • Security
                              • Reference
                                • Release Notes
                                  • Support
                                    • Glossary
                                    1. Cloud
                                    2. /
                                    3. ...
                                    4. /
                                    5. Data Loading for HDFS Pipelines

                                    Data Loading for HDFS Pipelines

                                    When the master aggregator reads an HDFS output directory’s contents, it schedules each file on a single SingleStoreDB Cloud partition. After each leaf partition across the workspace has finished extracting, transforming, and loading its file, a batch operation has been completed.

                                    Last modified: June 22, 2022

                                    Was this article helpful?

                                    Data Loading for Azure Blob Pipelines
                                    Data Loading for Kafka Pipelines

                                    Product

                                    • Product Overview
                                    • Solutions
                                    • Pricing
                                    • Support

                                    Community

                                    • Forums
                                    • Developer Hub
                                    • Events
                                    • Training

                                    Company

                                    • Contact Us
                                    • Blog
                                    • Careers
                                    • SingleStore Connect

                                    This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

                                    • © SingleStore
                                    • |

                                    • Privacy Policy
                                    • |

                                    • Terms of Service
                                    • |

                                    • Legal Terms and Conditions

                                    Was this article helpful?