Replicate MongoDB® Collections via AWS PrivateLink

SingleStore supports loading data from MongoDB® using CDC pipelines via AWS PrivateLink.

To replicate your MongoDB® databases using private links:

1. Create an outbound connection from SingleStore. Refer to Connect to SingleStore Helios using AWS PrivateLink for more information.

   While configuring the outbound connection, on the Create Connection dialog, enter or select the following information:

   1. Endpoint: Select SingleStore Endpoint.

   2. Service Name: Enter your MongoDB® Atlas Endpoint Service Name.

   Even though data is ingested into SingleStore, an outbound connection is used because the ingestion process is initiated by connecting from SingleStore Helios to the AWS instance/resource.

2. Request the VPC ID and Subnet ID for the AWS instance that hosts your SingleStore Helios workspace.

   Contact SingleStore Support and provide the following information:

   • Workspace ID and name.

   • In the support ticket, specify that the request is for VPC ID and Subnet ID for the AWS instance that hosts this workspace.

3. Set up a private endpoint to the MongoDB® Atlas cluster using AWS PrivateLink. Specify the VPC ID and Subnet ID received from SingleStore while configuring the private endpoint in MongoDB®. Refer to Configure Private Endpoints for more information.

4. Ensure that the Prerequisites are met.

5. Copy the Service Name of your outbound connection from the Cloud Portal.

   1. Select <your_workspace_group> > Firewall > Private Links.

   2. Under the Actions column of your private connection, select the three dots, and select View Connection.

   3. From the View Connection dialog, copy the VPC Endpoint Service Name.

6. Create a link to the VPC Endpoint Service. Specify the Service Name copied earlier in the mongodb.hosts parameter. For example:

   SQL

   Copy

   CREATE LINK <link_name> AS MONGODB
     CONFIG '{
       "mongodb.hosts": "<vpc_endpoint_service_name>"
       [, "mongodb.ssl.enabled":"true" ]
       [, "mongodb.authsource":"admin" ]}' 
     CREDENTIALS 
       '{"mongodb.password": "<password>", "mongodb.user": "<user>"}';

   Refer to CREATE LINK for more information.

7. Replicate the tables.

   SQL

   Copy

   CREATE TABLES AS INFER PIPELINE AS LOAD DATA 
     LINK <link_name> "*" FORMAT AVRO;
   
   START ALL PIPELINES;

   Refer to Replicate MongoDB® Collections using SQL for more information.