Connect to SingleStore Helios using AWS PrivateLink

Configure both outbound and inbound connections to connect your workspace to AWS PrivateLink. For information on managing private connections, refer to SingleStore Private Connections.

To connect via AWS PrivateLink using Flow, specify the VPC endpoint service name of your private link in the destination database configuration. Refer to Configure Flow.

Contact SingleStore Support for assistance with setting up or configuring private connections.

Note

This tutorial builds cross-account connectivity to Amazon MSK clusters with AWS PrivateLink by fronting all brokers in the cluster with a single NLB that has cross-zone load balancing enabled. Refer to Pattern 2: Front all MSK brokers with a single shared interface endpoint in the tutorial for more information.

Configure Inbound Connections

To successfully set up an inbound connection to SingleStore Helios using AWS PrivateLink, perform the following tasks:

  1. Create an Inbound Connection on the Cloud Portal

  2. Create a Private Endpoint on the Amazon VPC Console

Create an Inbound Connection on the Cloud Portal

On the Cloud Portal,

  1. Select Workspaces.

  2. Select the three dots under Actions for your workspace and select Access & Security from the list.

  3. Under Private Links, select Create Connection.

  4. On the Create Connection dialog, enter or select the following information:

    1. Endpoint: Select SingleStore Endpoint.

    2. Connection Type: Select the Inbound connection type from the list.

    3. AWS Account ID (Inbound connections only): Enter the AWS Account ID associated with your VPC/private endpoint.

  5. Select Create Connection.

Once the connection is ready to use, which may take a few minutes, its status changes to ACTIVE. If an error occurs while creating the private connection, the connection is deleted automatically. Hover over the DELETED status indicator to view the error message.

Copy the VPC Endpoint Service Name of your connection, and enter it in the Service name field while creating a private endpoint on the Amazon VPC Console. Refer to Manage Private Connections for information on how to view the private connection details.

Create a Private Endpoint on the Amazon VPC Console

Note

Your workspace and endpoint must be in the same region.

Create a private endpoint using the Service name copied earlier:

  1. On the Amazon VPC console, select Endpoints > Create endpoint.

  2. Under Service Category, select Other endpoint services.

  3. Enter the Service name copied from the Cloud Portal in the Service name box.

  4. Select Verify service to verify the Service name.

  5. Under VPC, select the VPC from which you'll connect with the AWS service.

  6. Under Subnets, select one subnet per Availability Zone from which you'll connect to the AWS service.

  7. Select Create endpoint.

You can use the endpoint after it enters the Available state. Refer to Endpoint states for more information. Create a security group to control access to the endpoint, and then attach the security group to the endpoint. Refer to Control traffic to resources using security groups for more information.

Note

SingleStore Helios does not support Certificate Authority (CA) verification for inbound connections. For information on connecting to SingleStore Helios using SSL, refer to Connect to SingleStore Helios using TLS/SSL.

Configure Outbound Connections

To successfully set up an outbound connection to SingleStore Helios using AWS PrivateLink, perform the following tasks:

  1. Copy the AWS account ID from the Cloud Portal

  2. Create an Endpoint Service on the AWS Console

  3. Create an Outbound Connection on the Cloud Portal

  4. Accept the Connection Request in your AWS Console

If you are using Kafka brokers with AWS MSK, you must specify the IP address of the broker endpoints while creating the target groups of the load balancer. Run the nslookup command with the DNS names of the MSK brokers to get their IP addresses. Note that the IP address of the endpoint does not change since it is attached to the VPC ENI (elastic network interfaces). Hence, resolve the broker endpoint IP address before initiating the connection. When using Kafka brokers, use the broker name with the port instead of the endpoint name in the CREATE PIPELINE command.

Copy the AWS Account ID from the Cloud Portal

On the Cloud Portal,

  1. Select Workspaces.

  2. Select the three dots under Actions for your workspace and select Access & Security from the list.

  3. Under Private Links, select Create Connection.

  4. On the Create Connection dialog, from the Connection Type list, select Outbound. Copy the AWS account ID displayed.

You'll need to whitelist this ID while creating your endpoint service.

Create an Endpoint Service on the AWS Console

On the AWS Console,

  1. Create a target group for each of the AWS services that you want to access using AWS PrivateLink, select EC2 > Target groups > Create Target group. Refer to Target Groups for more information.

  2. Create a network load balancer, select EC2 > Load Balancers > Create Load Balancer.

  3. Under Network Load Balancer, select Create. Your workspace and the load balancer must be in the same region. Ensure that Cross-zone load balancing is enabled. Refer to Create a Network Load Balancer for related information.

  4. In the AWS Console, select VPC > Endpoint Services > Create Endpoint Service.

    Note

    Your workspace and endpoint service must be in the same region.

    • Associate the endpoint service with the Network Load Balancer created in the previous step.

    • Enable Require acceptance for endpoint for additional security.

  5. For this service, under Allow principals, add the AWS account ID copied from the Cloud Portal in the "arn:aws:iam::<account id>:root" format. This enables SingleStore to find and access the private endpoint service.

  6. Verify that the security group rules in your VPC allow inbound traffic from the endpoint service. Refer to Control traffic to resources using security groups for more information.

  7. Copy the Service Name of this AWS endpoint service.

Create an Outbound Connection on the Cloud Portal

On the Cloud Portal,

  1. Select Workspaces.

  2. Select the three dots under Actions for your workspace and select Access & Security from the list.

  3. Under Private Links, select Create Connection.

  4. On the Create Connection dialog, enter or select the following information:

    1. Endpoint: Select SingleStore Endpoint.

    2. Connection Type: Select the Outbound connection type from the list.

    3. Service name (Outbound connections only): Enter the Service Name associated with your AWS endpoint service.

  5. Select Create Connection.

  6. (Optional) Accept the connection request in your AWS Console.

The connection is ready to use once the endpoint status changes to ACTIVE. If an error occurs while creating the private connection, the connection is deleted automatically. Hover over the DELETED status indicator to view the error message.

Accept the Connection Request in your AWS Console

If Require acceptance for endpoint is enabled while creating the endpoint service, you must accept the connection request from SingleStore in your AWS account. On the AWS Console,

  1. Select VPC > Endpoint Services, and then select your endpoint service.

  2. On the Endpoints Connections tab, find the request from the SingleStore AWS account with the Pending Acceptance status.

  3. From the Actions menu, select Accept Endpoint Connection Request.

The connection status changes to Available, indicating that the connection is successfully established and is ready to use.

Configure Flow

To configure Flow to connect using AWS PrivateLink:

  1. Log in to the Cloud Portal.

  2. Copy the VPC Endpoint Service Name of your outbound private link.

    1. Select Workspaces.

    2. Select the three dots under Actions for your workspace and select Access & Security from the list.

    3. Under Private Links, select the three dots under Actions for your private link, and then select View Connection.

    4. Copy the VPC Endpoint Service Name for your private link.

  3. Select Ingestion > Load Data, and then select a source supported by Flow.

  4. Configure the destination database, connection name, Flow instance size, and then select Create Flow Instance.

  5. Select Open Flow under the Actions column of the Flow instance created in the previous step.

  6. On the Setup tab, configure the source database and then select Next.

  7. Under Destination Database, enter the VPC Endpoint Service Name copied earlier in the Host Name field.

  8. Enter the username and password of the SingleStore database user with which to connect.

  9. Select Test to test the connection.

Once the connection is verified, configure the Flow instance as required and proceed with data ingestion.

Refer to Load Data with SingleStore Flow on Helios for more information.

References

In this section

Last modified:

Was this article helpful?

Verification instructions

Note: You must install cosign to verify the authenticity of the SingleStore file.

Use the following steps to verify the authenticity of singlestoredb-server, singlestoredb-toolbox, singlestoredb-studio, and singlestore-client SingleStore files that have been downloaded.

You may perform the following steps on any computer that can run cosign, such as the main deployment host of the cluster.

  1. (Optional) Run the following command to view the associated signature files.

    curl undefined
  2. Download the signature file from the SingleStore release server.

    • Option 1: Click the Download Signature button next to the SingleStore file.

    • Option 2: Copy and paste the following URL into the address bar of your browser and save the signature file.

    • Option 3: Run the following command to download the signature file.

      curl -O undefined
  3. After the signature file has been downloaded, run the following command to verify the authenticity of the SingleStore file.

    echo -n undefined |
    cosign verify-blob --certificate-oidc-issuer https://oidc.eks.us-east-1.amazonaws.com/id/CCDCDBA1379A5596AB5B2E46DCA385BC \
    --certificate-identity https://kubernetes.io/namespaces/freya-production/serviceaccounts/job-worker \
    --bundle undefined \
    --new-bundle-format -
    Verified OK

Try Out This Notebook to See What’s Possible in SingleStore

Get access to other groundbreaking datasets and engage with our community for expert advice.