SingleStore XL Ingest ================== ## Overview SingleStore XL Ingest ("XL Ingest"), a component of SingleStore Flow, is companion software to [SingleStore Ingest](https://docs.singlestore.com/db/v9.1/load-data/load-data-with-singlestore-flow/singlestore-ingest.md) ("Ingest"). When large tables (greater than 10GB) are included in Ingest, a full extract may not be feasible due to long processing times. This increases the likelihood of encountering issues, and if a problem occurs, the entire initial extract must be rerun. Therefore, XL Ingest is essential for working with large tables. XL Ingest handles the initial transfer of large tables by dividing them into smaller logical partitions. It then transfers multiple partitions from the source to the target in parallel. This ensures the transfer happens within a reasonable amount of time. Use XL Ingest to transfer the identified logical partitions of large tables in parallel to SingleStore for storage and processing. To transfer large tables from source database to SingleStore, perform the steps outlined in [Select Tables](https://docs.singlestore.com/#section-idm234737541647291.md). Complete these steps before triggering an XL Ingest job to prevent data loss during the transition. **Note**: Pausing updates on the source database is not required during this process. Both Ingest and XL Ingest can operate independently and concurrently without disrupting the source data. ## Select Tables To ingest large data table(s) from source database to SingleStore, perform the following steps in Ingest before triggering a job in XL Ingest. 1. Navigate to **Dashboard** > **Tables** and select the gear icon. 2. Define a primary key (**Pkey**) and any necessary partitions for the table. 3. Enable **Skip Initial Extract** to bypass the initial extract and directly proceed with the delta load. 4. Select **Apply** to save the changes. 5. Navigate to the **Dashboard** tab. Under **Schedule**, disable the Ingest scheduler to ensure that all tables are moved to the destination at the same time. 6. Navigate to the **Operations** tab and initiate **Full Extract** to trigger the initial bulk load for all the selected tables, except for those tables marked as **Skip Initial Extract**. 7. Initiate **Sync New Tables** to trigger the initial bulk load for tables marked as **Redo Initial Extract** and newly-added tables in an ongoing replication. This captures the watermark for CDC and creates the table in the destination database (SingleStore). 8. Enable the ingest scheduler in Ingest after transferring tables using XL Ingest. **Note**: After marking tables with **Skip Initial Extract**, the next scheduled delta run automatically captures CDC for all tables, including those loaded with XL Ingest. XL Ingest prevents duplication during the CDC load by using a watermark to track changes. ## Split Table into Slices Large tables must be divided into notional slices based on the value of a single slice column, for example, primary key. For automatic slice determination, XL Ingest uses parameters like the number of slices needed and how many characters from the start of the slice column value must be used. **Note**: Date and time columns (for example `DATE`, `DATETIME`, `TIMESTAMP`) are not supported as slice columns in XL Ingest. Refer to [Configure Table Slicing](https://docs.singlestore.com/db/v9.1/load-data/load-data-with-singlestore-flow/singlestore-xl-ingest/configure-table-slicing.md) for more information on slicing. ## For Large Tables XL Ingest automatically determines the slices based on the specified parameters. For example, a slice column like `names` can be divided by the first 3 characters. Alternatively, you can manually enter the slice values instead of using auto-slice. ## For Smaller Tables Slicing may not be necessary. The entire table can be processed as a single slice. ## For PostgreSQL Partitioned Tables To use XL Ingest with PostgreSQL partitioned tables: 1. In Ingest, select only the parent PostgreSQL partitioned table on the **Tables** page. Child partitions do not appear separately. 2. Configure the primary key and other table options on the parent table. 3. Set up slicing on the partition key column using Auto-Slice or by manually entering slice boundary values. 4. Sync the table in XL Ingest to load all slices in parallel. 5. After XL Ingest completes, enable the Ingest scheduler so CDC continues from the same parent table. ## In this section * [Start and Stop XL Ingest](https://docs.singlestore.com/db/v9.1/load-data/load-data-with-singlestore-flow/singlestore-xl-ingest/start-and-stop-xl-ingest.md) * [Configure XL Ingest](https://docs.singlestore.com/db/v9.1/load-data/load-data-with-singlestore-flow/singlestore-xl-ingest/configure-xl-ingest.md) * [Configure Table Slicing](https://docs.singlestore.com/db/v9.1/load-data/load-data-with-singlestore-flow/singlestore-xl-ingest/configure-table-slicing.md) *** Modified at: May 28, 2026 Source: [/db/v9.1/load-data/load-data-with-singlestore-flow/singlestore-xl-ingest/](https://docs.singlestore.com/db/v9.1/load-data/load-data-with-singlestore-flow/singlestore-xl-ingest/) (An index of the documentation is available at /llms.txt)