# SingleStore Ingest Operational FAQ

This document describes some of the most frequent operational user queries.

## Stop Ingest

Perform the following steps to stop Ingest.

1. Navigate to the **Dashboard** tab. Under **Schedule**, disable the scheduler and wait for the load to complete.

2. Open **Windows Services** and stop the Ingest Windows service.

3. If the Ingest Windows service stops successfully but the Ingest page is still loading, you may need to terminate the non-responding processes.

4. To terminate the non-responding Ingest process, follow these steps.

   1. Open the command prompt as **Administrator**.

   2. Run the following command:
      ```shell
      netstat -ano | find "8081"
      ```
      Replace `8081` with the port number on which Ingest is running.

   3. Note the process ID (PID) of the Ingest service from the above command output.

   4. Open **Windows Task Manager**, navigate to the **Details** tab, and find the process with corresponding PID.

   5. Right-click on the process and select **End Task**. This terminates the runaway Ingest process.

## Restart Ingest

Perform the following steps to restart Ingest.

1. Navigate to the **Dashboard** tab. Under **Schedule**, disable the scheduler and wait for the load to complete. Then, stop the Ingest Windows service.

2. Restart the Ingest Windows service and wait for the Ingest dashboard to appear, refreshing the page as needed.

3. Navigate to the **Dashboard** tab, under **Schedule**, and enable the scheduler.

4. Ingest picks up the correct processing sequence. Confirm that the last sequence number loaded correctly and that the new sequence number started correctly. If needed, [perform a rollback](https://docs.singlestore.com/#section-idm234737465163327.md) to the correct sequence number for processing.

## Perform a Full Extract

Use the following steps to perform a full extract for an initial sync of all tables in a pipeline.

1. Navigate to the **Dashboard** tab. Under **Schedule**, disable the scheduler and wait for the load to complete.

2. Navigate to **Dashboard** > **Tables**.

   1. Select the tables (one or more) for the full extract.

   2. Ensure that the primary key (**PKey**) column(s) are applied.

   3. Enable **Delta Extract** (**With History**).

   4. Select **Apply**.

3. Navigate to the **Operations** tab and select **Full Extract**.

4. Navigate to the **Logs** tab. Check the logs to confirm that the selected tables have started the full extract.

## Skip an Initial Extract

Use the following steps to perform deltas without an initial sync, or when using XL Ingest.

1. Navigate to the **Dashboard** tab. Under **Schedule**, disable the scheduler and wait for the load to complete.

2. Navigate to **Dashboard** > **Tables**.

   1. Select the table(s) to perform deltas without an initial sync of the data.

   2. Ensure that the primary key (**PKey**) column(s) are applied.

   3. Enable **Delta Extract** (**With History**).

   4. Enable **Skip Initial Extract**.

   5. Select **Apply**.

3. Navigate to the **Operations** tab and select **Full Extract**.

4. Navigate to the **Logs** tab. Check the logs to confirm that the selected table(s) skipped the full extract.

## Redo an Initial Extract

Use the following steps to redo an initial extract for existing tables.

1. Navigate to the **Dashboard** tab. Under **Schedule**, disable the scheduler and wait for the load to complete.

2. Navigate to **Dashboard** > **Tables**.

   1. Select the table(s) for which to redo the initial extract.

   2. Ensure that the primary key (**PKey**) column(s) are applied.

   3. Enable **Delta Extract** (**With History**).

   4. Enable **Redo Initial Extract**.

   5. Select **Apply**.

3. Navigate to the **Operations** tab and select **Sync New Tables**.

4. Navigate to the **Logs** tab. Check the logs to confirm that the selected table(s) have started the full extract.

## Perform a Rollback

Use the following steps to perform a rollback in Ingest.

1. Navigate to the **Dashboard** tab. Under **Schedule**, disable the scheduler and wait for the load to complete.

2. Navigate to the **Operations** tab, select the gear icon, and then select **Rollback**.

3. In the pop-up window, select the sequence number from which you want to replay the log files.

   *Confirm that the log files are available on the source server.*

4. Select **Select**.

5. Navigate to the **Dashboard** tab. Under **Schedule**, enable the scheduler.

## Add New Tables to an Existing Pipeline

Use the following steps to add only new tables to existing pipelines.

1. Navigate to the **Dashboard** tab. Under **Schedule**, disable the scheduler and wait for the load to complete.

2. Navigate to **Dashboard** > **Tables**:

   1. Select one or more new tables as required.

   2. Ensure the primary key (**PKey**) column(s) are applied.

   3. Enable **Delta Extract** (**With History**).

   4. Select **Apply**.

3. Navigate to the **Operations** tab and select **Sync New Tables**.

4. Navigate to the **Logs** tab. Confirm that the existing tables were ignored and the new table(s) started to sync.

## Create a New Pipeline

Use the following steps to create an additional Ingest pipeline.

1. Unzip the Ingest ZIP file into a folder, such as `C:\singlestore-flow\pipeline\`.

2. Edit the `ingest.xml` file located in the `C:\singlestore-flow\pipeline\` folder.

3. Change the `<id>Bryteflow-Ingest</id>` and `<name>BryteFlow Ingest</name>` tags to reference the new pipeline names.

4. Check that port `8081` is free and not being used by other processes.

5. Open a command prompt as Administrator, navigate to the new pipeline folder, run `Install-Service.bat`, and then close the command prompt.

6. Open the Windows Services console and start the newly-created service.

7. Launch the pipeline at `localhost:8081`.

8. To avoid using port `8081` for future pipelines, SingleStore recommends changing the port to another preferred Web port by navigating to **Settings** > **Instance Details** > **Web Port**.

9. Restart the Windows service and launch the Ingest application on the new Web port, such as `localhost:8082`.

10. Ensure you have a valid license key to get started.

## Enable Windows Authentication for SQL Server

Use the following steps to enable Windows authentication in Ingest.

1. Stop the Ingest service.

2. Download and copy the `sqljdbc_auth.dll` file to the `bin` directory of your Java installation. You can download the DLL from [here](https://bryteflow.com/release/files/sqljdbc_auth.zip) or from [Microsoft](https://learn.microsoft.com/en-us/sql/connect/jdbc/download-microsoft-jdbc-driver-for-sql-server).

3. Start the Ingest service and add the following JDBC option in the **Source Database** connection settings.
   ```
   integratedSecurity=true
   ```

## Latency Calculation

Latency is calculated based on the time from when the record is committed to the source until it is available at the destination.

## Prerequisites for Oracle Source

The Flow installation guide provides details on how to enable supplemental logging for source table(s) for delta replication. Refer to [Install SingleStore Flow](https://docs.singlestore.com/db/v9.1/load-data/load-data-with-singlestore-flow/install-singlestore-flow.md) for more information.

**Note**:

* If database supplemental logging is enabled, only inserts and deletes are captured, and updates are missed.
* If table-level supplemental logging is enabled, only updates are captured, and inserts and deletes are missed.

Therefore, both database and table-level supplemental logging are required.

## Synchronize Schema Changes Using Sync Struct

Use the following steps to perform a Sync Struct for existing tables when the source structure changes.

1. Navigate to the **Dashboard** tab. Under **Schedule**, disable the scheduler and wait for the load to complete.

2. Navigate to the **Operations** tab and select **Sync Struct**.

3. Navigate to the **Logs** tab to check the logs. Ingest compares the structures and synchronizes any differences.

## Use of BCP by Flow is not supported on Linux for SQL Server

Use the following steps to disable BCP when enabling change data capture or change tracking on SQL Server running on Linux:

**Note**: It is recommended to test this in a non-production environment before deploying to production.

1. Check if Ingest is currently running any extract or loads. If no extracts are running, stop the Ingest service.

2. Navigate to the Ingest installation directory. Locate the `config.xml` file in: `/ingest/config/config.xml.`

3. Open the file in a text editor. In the `<Setting>` section, and set the "`use_bcp`" parameter to `"N"`:
   ```xml
   <Param name="use_bcp" value="N"/>
   ```
   If the parameter does not exist, add the line manually.

4. Save the file.

5. Restart the Ingest service.

## Loading Data into Reference Tables

Loading data into reference tables using Flow requires using the DDL endpoint for destination connection. Using the DML endpoint may result in an error like:

```
LOAD DATA into reference table on a child aggregator is not permitted on child aggregators. Try the command again on the master aggregator.Loading data into reference tables using Flow requires using the DML endpoint for destination connection. Using the DDL endpoint may result in an error like:
```

To resolve this:

1. Identify the workspace group ID of your SingleStore workspace.

2. In the Flow instance, update the hostname for the SingleStore destination connection by replacing `svc-<id>-dml` with `svc-<workspace-group-id>-ddl` so that the `id` is replaced with the workspace group ID and the `dml` tag is changed to `ddl`. Keep the rest of the hostname unchanged.

   Example:
   ```
   svc-78636867-cf31-4b41-4765-24a3997fd429-ddl.aws-ireland-2.svc.singlestore.com
   ```

3. Test the connection again. It should now allow ingesting data into the reference table.

## Load Data from Views

Loading data from views is not supported through the Ingest UI. For self-managed deployments, you can use a `config.xml` workaround to perform a **one-time full extract** from a view. CDC (change data capture) from views is not supported.

* Use a separate Flow instance dedicated to views, configured only for the one-time full extract.
* As views do not define primary keys, configure the destination table with either a unique indexed column as the primary key, or multiple columns that together form a natural key.

**Note**: This workaround is not recommended for production use without prior testing.

To load data from a view in a full extract pipeline, add the view as a table entry in `config.xml` as follows:

1. Stop the Ingest service and ensure that no extracts are running.

2. Open the following configuration file:
   ```shell
   <flow_install_dir>/ingest/config/config.xml
   ```

3. In the `<TableList>` section, add a `<Table>` entry for the view using the following template:
   ```
   <Table>
     <Param name="id" value="<source_db>:<source_schema>.<view_name>"/>
     <Param name="pkey" value="<primary_key_column>"/>
     <Param name="select" value="Y"/>
     <Param name="xfer" value="F"/>
   </Table>
   ```

4. Save `config.xml`.

5. Restart the Ingest service.

6. From the Ingest UI, run a **Full Extract** operation. Flow treats the view as if it were a table and loads its result set into the destination table.

***

Modified at: May 28, 2026

Source: [/db/v9.1/load-data/load-data-with-singlestore-flow/singlestore-ingest-operational-faq/](https://docs.singlestore.com/db/v9.1/load-data/load-data-with-singlestore-flow/singlestore-ingest-operational-faq/)

(An index of the documentation is available at /llms.txt)