# Settings

## Instance Details

Navigate to **Settings > Instance Details**, and enter the following configuration details:

1. **Ingest Instance Name**: Enter a name for the current instance of Ingest.

2. **Web Port**: Enter the port on which the Ingest server runs.

3. **Icon Text**: Enter the text for your instance icon. The text takes only 2 characters.

4. **Icon Color**: Select the color for your instance icon.

Select **Apply** to save the settings.

## License

To obtain a valid license, contact [SingleStore Sales](https://www.singlestore.com/contact/) or your SingleStore technical account team representative with the **Product ID**. Navigate to **Settings >** **Licence**. In **Licence Key**, enter the license key, and then select **Apply** to save the settings.

## History

Navigate to **Settings > History**, and enter the following configuration details:

1. Enable **Keep history by default**, if you want to keep history always.

2. In **History Start Date Column**, enter the source date for type-2 SCD records.

3. In **History End Date Column**, enter the history end date for type-2 SCD records.

4. In **Open End Date Column (YYYY-MM-DD)**, enter the end date used for history records.

5. In **Max. Updates in batch**, enter the value that determines when combined updates exceed this threshold in a batch.

Select **Apply** to save the settings.

## About Ingest

To view the **Version**, **Build**, **Host Name**, and **Ingest Location** of your installed Ingest Instance, navigate to **Settings > About Ingest**.

## User-Defined Settings

To add custom fields, navigate to **Settings > User Defined Settings**. Enter the **Name** and **Value** of your custom field, and select the **Save** icon. You can also **Delete** and **Undo** the name and value of your custom field.

**Note**: You can add only one custom field.

## Email Notification

To configure email updates, navigate to **Settings > Email Notification**, and then perform the following steps:

1. To receive email notifications, enable **Enable Email Notifications**.

2. In **Host Name**, enter the address of your SMTP server.

3. In **Port**, enter the port number that the SMTP server is listening on.

4. In **Sender**, enter the email address from which the notifications are sent. This must be a valid email address on the server.

5. In **Recipient**, enter the email address to which the notifications are sent.

6. In **User ID**, enter the full email address to authenticate with the SMTP server.

7. Enter the **Password** for the email, then confirm it in the **Confirm Password** field.

   **Note**: Passwords are encrypted within Ingest.

Select **Apply** to save the settings.

## AWS Proxy Settings

Navigate to **Settings** > **AWS Proxy Settings**, enter the following configuration details.

1. **Proxy Host**: Enter the S3 proxy host name.

2. **Proxy Port**: Enter the S3 proxy port.

3. **Proxy User Id**: Enter the S3 proxy user ID.

4. **Proxy Password**: Enter the S3 proxy password.

Select **Apply** to save the settings.

## AWS Credentials

Ingest can access AWS services using either IAM roles or access keys when deployed on-premises. You must configure the access method and credentials in Ingest.

To access AWS services from an on-premises SingleStore Flow server, navigate to **Settings** > **AWS Credentials**, and perform the following steps.

1. Enable **Use EC2 IAM Role**, if you are using.

2. Select the **AWS Region** from the list.

3. Enter the AWS access key ID in **AWS Key** and **AWS Secret Key** to access the S3 service (if installed on-premises). For installations using IAM roles, these keys are not required. **Note**: Keys are encrypted within Ingest.

4. If you are using KMS, enter the KMS Key in **AWS Key**. **Note**: Keys are encrypted within Ingest.

Select **Apply** to confirm and save the details.

## AWS Recovery

Ingest supports high availability, which means it automatically saves the current configuration and execution state to S3. This enables recovery of the Ingest instance (including its current state) if it is catastrophically lost.

SingleStore Flow keeps a backup of every successful job execution in S3 that makes the latest version available for recovery. To configure it, navigate to **Settings** > **AWS Recovery**, and perform the following steps.

1. In **Ingest Instance Name**, enter a name for the current instance of Ingest.

2. Check **Enable Recovery**.

3. In **S3 Recovery Location**, enter the destination for recovery data in S3, for example: `s3://your_bucket_name/your_folder_name/Ingestyour__name`.

4. In **S3 KMS Id**, enter the KMS ID to encrypt logs in S3. This is optional but recommended. When provided, logs are encrypted using KMS encryption.

Select **Apply** to save the settings.

## Recovery Utilization

To recover an instance of Ingest, source a new instance of Ingest on a new server or Amazon EC2 server:

1. Download and install Ingest in the newly launched EC2. Refer to the [Install SingleStore Flow](https://docs.singlestore.com/db/v9.1/load-data/load-data-with-singlestore-flow/install-singlestore-flow.md).

2. Once the EC2 instance is running, the endpoint is an EC2 instance hosting Ingest as a Windows service on port 8081. The EC2 instance's assigned role must include the required policies. Refer to the [AWS Identity and Access Management (IAM) for SingleStore Flow](https://docs.singlestore.com/db/v9.1/load-data/load-data-with-singlestore-flow/singlestore-ingest/aws-environment-setup/#aws-identity-and-access-management-iam-for-singlestore-flow.md).

   1. The list of instances is stored in DynamoDB. Ensure that the EC2 role has the necessary permissions to access DynamoDB and the underlying table containing saved instances.

3. Once the EC2 instance is launched, open the Ingest web console by entering localhost:8081 in a Chrome browser.

4. Select **Existing Instance**.

5. Select the instance you want to restore from the list (e.g., `Your_Ingest_Name`).

6. Select **Select**.

7. Ingest retrieves the configuration and saved execution state of the selected instance and restores it.

8. After restoration, SingleStore recommends to stop the faulty EC2 instance (the previous installation):

   1. Go to the AWS EC2 console.

   2. Search for the Ingest tag (or the tag used to launch the application).

   3. Select the older EC2 instance.

   4. Navigate to **Actions > Instance State**, and then select **Terminate**.

**Note**: Recovery can also be used as a method for partial migration between environments (e.g., from DEV to PROD stacks). Since the restore clones the source environment and state, further configuration may be needed (e.g., updating configuration options for the PROD stack S3 location, etc.). This method can reduce the workload when migrating hundreds of tables to a new EC2 instance.

## Recovery from Faults

SingleStore Flow supports high availability and auto-recovery mechanisms in case of faults or failures. These include:

* **EC2 Instance Failures**: If the EC2 instance is terminated or affected, Flow saves the last successful loads as a savepoint. It resumes from the savepoint when restarted on another EC2 instance or AZ.
* **SingleStore Connection Issues**: Flow retries until successful.
* **Source DB Connection Issues**: Flow retries until successful.
* **AWS S3 Connection Issues**: Flow retries until successful.

Customers seeking high availability must configure their Ingest instance for high availability and recovery. Refer to [AWS Recovery](https://docs.singlestore.com/db/v9.1/load-data/load-data-with-singlestore-flow/singlestore-ingest/settings/#aws-recovery-2.md) for setup details.

In the event of application faults or failures:

* Enable CloudWatch Logs and metrics to get notified. AWS CloudWatch events can be used for alerting by writing custom Lambda functions.
* Enable SNS notifications in setup details Ingest and subscribe to the SNS topic via the AWS console.

To monitor EC2 instances:

* Enable CloudWatch monitoring for EC2 or set up health checks with proper alerts.
* For disk space, setup details Ingest sends status updates to CloudWatch Logs, including free disk space in GB. Users can create Lambda functions to raise alarms when space is low.

Refer to [Remote Monitoring](https://docs.singlestore.com/db/v9.1/load-data/load-data-with-singlestore-flow/singlestore-ingest/settings/#aws-cloudwatch-logs-2.md) for more information.

## Disk Space Monitoring via Lambda

**Prerequisites**

1. Create an IAM role with the following policy to allow Lambda functions to call AWS services.
   ```
   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Sid": "Stmt1",
         "Action": [
           "logs:FilterLogEvents",
           "logs:GetLogEvents",
           "logs:PutLogEvents"
         ],
         "Effect": "Allow",
         "Resource": "arn:aws:logs:<region>:<account_ID>:log-group:<log-group-name>:log-stream:<log-stream-name>"
       },
       {
         "Sid": "Stmt2",
         "Action": [
           "sns:Publish",
           "sns:TagResource"
         ],
         "Effect": "Allow",
         "Resource": "arn:aws:sns:<region>:<account_ID>:<topic_name>"
       }
     ]
   }
   ```

2. Create an SNS topic (refer to [What is Amazon CloudWatch?](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/WhatIsCloudWatch.html) for more information) and use the ARN in your Lambda function code.

**Lambda Function Steps**

1. Log in to the AWS console and go to the AWS Lambda dashboard.

2. Select **Create Function** and enter a function name.

3. Select Python 3.8 as the runtime.

4. Select **Create Function**.

5. Add a trigger for **CloudWatch Log** and select your log group.

6. Add the Lambda code for disk check and select the **Lambda Execution Role**.

7. Select **Save**.

Here is sample Lambda code to check disk space:

```
import json
import boto3

def lambda_handler(event, context):
    freeGb = 100
    cloudwatch = boto3.client('logs')
    response = cloudwatch.get_log_events(
        logGroupName='Oracle_LogGroup',
        logStreamName='Oracle_LogStream',
        startFromHead=False,
        limit=100
    )
    for i in response['events']:
        msg = json.loads(i['message'])
        if msg['type'] == "Status" and msg['diskFreeGB'] < freeGb:
            sns = boto3.client('sns')
            sns.publish(
                TopicArn='arn:aws:sns:us-west-2:689564010160:LambdaTrigger',
                Message='Your free disk size is getting low, please contact the concerned team!'
            )
```

**Note**: Customize the email body as needed.

## Time to Recover

## Recovery Point Objective (RPO)

Ingest automatically recovers from most failure scenarios that leverages durable services like S3 for unlimited data retention. Data recovery depends on the customer's source database retention settings.

**Note**: No full reload is required, unlike other solutions.

## Recovery Testing

After recovery, perform the following tasks before resuming replication:

1. Start the Ingest service.

2. Open the web console in Chrome.

3. Navigate to **Dashboard** > **Source** and test the connection for the source database.

4. Navigate to **Dashboard** > **Destination** and test the connection for the destination database.

5. If there are any connection issues, troubleshoot and resolve them.

6. Once successful, navigate to **Dashboard**, under **Schedule**, enable the Scheduler, and resume replication.

## AWS CloudWatch Logs

Ingest comes pre-configured with remote monitoring capabilities that leverage existing AWS technologies, such as CloudWatch Logs and Events. CloudWatch, along with other AWS services, can monitor the execution of Ingest and trigger appropriate alarms in case of errors or failures. The events from Ingest application flows to CloudWatch Logs. These events provide detailed application statistics that can be used for custom monitoring.

In addition to integration with CloudWatch, Ingest writes internal logs directly to S3, including execution and error logs from the Ingest console. Navigate to **Settings** > **AWS CloudWatch Logs**, and perform the following steps.

1. Check **Enable CloudWatch Logs** and metrics to get notified. AWS CloudWatch events can be used for alerting by writing custom Lambda functions.

2. Enter the name of the **CloudWatch Log Group.** This needs to be created first in the AWS console.To create a CloudWatch Log Group, refer to [CreateLogGroup](https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_CreateLogGroup.html) for more information.

3. Enter the name of the **CloudWatch Log Stream**. This also needs to be created first in the AWS console. To create a CloudWatch Log Stream, refer to [CreateLogStream](https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_CreateLogStream.html) for more information.

4. Select **Apply** to save the settings.

Refer to [Appendix](https://docs.singlestore.com/db/v9.1/load-data/load-data-with-singlestore-flow/singlestore-ingest/appendix.md) for more information.

## AWS CloudWatch Metrics

Navigate to **Settings** **AWS CloudWatch Metrics**, and perform the following steps.

1. Check **Enable CloudWatch Metrics** if required.

2. In **CloudWatch Metrics Namespace,** enter the name of CloudWatch metric. Refer to [View available metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/viewing_metrics_with_cloudwatch.html) for more information.

3. Select **Apply** to save the settings.

The events that Ingest pushes to AWS CloudWatch Logs and the metrics console are as follows. Refer to [Appendix](https://docs.singlestore.com/db/v9.1/load-data/load-data-with-singlestore-flow/singlestore-ingest/appendix.md) and [Appendix](https://docs.singlestore.com/db/v9.1/load-data/load-data-with-singlestore-flow/singlestore-ingest/appendix.md) for more information.

## AWS SNS Notifications

Navigate to **Settings** > **AWS SNS Notifications**, and perform the following steps.

1. Check **Enable SNS notifications** and subscribe to the SNS topic via the AWS console.

2. In **SNS Topic**, enter the Topic ARN or SNS Topic Name. To create an SNS Topic, refer to [Creating an Amazon SNS Topic](https://docs.aws.amazon.com/sns/latest/dg/sns-create-topic.html) for more information.

3. Select **Apply** to save the settings.

Refer to [Appendix](https://docs.singlestore.com/db/v9.1/load-data/load-data-with-singlestore-flow/singlestore-ingest/appendix.md) for more information.

## AWS S3 Logging

Navigate to **Settings** > **AWS S3 Logging**, and perform the following steps.

1. Check **Enable S3 Logging** if you want to record data to S3 (console/execution logs).

2. In **S3 Log Location**, enter the destination of the logging data in S3, for example: `s3://your_bucket_name/your_folder_name.`

3. In **S3 KMS Id**, enter the KMS ID to encrypt logs in S3. This is optional but recommended. When provided, logs are encrypted using KMS encryption.

4. Select **Apply** to save the settings.

## Metadata Settings

Navigate to **Settings** > **Metadata Settings**, and perform the following steps.

1. Check **Enable Operational Metadata**, if required.

2. Select metadata database from the **Database Type** list: **AWS Aurora**.

3. In **Host Name**, enter the IP address or hostname of the database server.

4. In **Port**, enter the port number on which the database server is listening.

5. In **Database Name**, enter the name of your database.

6. In **Database Schema**, enter the schema name of your database.

7. In **User Id**, enter a valid database user ID to be used with Ingest.

8. Enter the **Password** and confirm it by re-entering it in the **Confirm Password** field.

9. **JDBC Options** are optional and can be used to extend the JDBC URL for accessing the database.

10. Select **Apply** to save the settings.

***

Modified at: April 29, 2026

Source: [/db/v9.1/load-data/load-data-with-singlestore-flow/singlestore-ingest/settings/](https://docs.singlestore.com/db/v9.1/load-data/load-data-with-singlestore-flow/singlestore-ingest/settings/)

(An index of the documentation is available at /llms.txt)