Load Data from Azure Event Hubs

Prerequisites

You need an Azure account with access to Event Hubs. You will also need to create an Event Hubs namespace, which gives you a unique FQDN.

Get Connection String from Azure Portal

After creating the namespace, you’ll need a connection string to communicate with Event Hubs.

1. On the Azure portal, in the left navigation menu, select All services.

2. In the Analytics section, select Event Hubs.

3. From the list of event hubs, select the SingleStore event hub.

4. On the Event Hubs Namespace page, in the left navigation menu, select Shared Access Policies.

5. From the list of policies, select the relevant shared access policy.

6. In the selected policy dialog, copy the Connection string-primary key.

Here is a sample pipeline:

CREATE PIPELINE `data_ingest`
AS LOAD DATA KAFKA 'front2back2eventhub.servicebus.windows.net:9093/table-update'
CONFIG '{"security.protocol": "sasl_ssl","sasl.mechanism": "PLAIN","sasl.username": "$ConnectionString"}'
CREDENTIALS '{"sasl.password": "<password>"}'
BATCH_INTERVAL 2500
ENABLE OUT_OF_ORDER OPTIMIZATION
INTO TABLE `test`;