Sign inTry Free

Mutual TLS for Kafka Pipelines

On this page

SingleStore Pipelines support mutual TLS (mTLS) when loading data from Kafka. With mTLS, both the Kafka broker and the SingleStore pipeline authenticate each other using certificates.

Certificate Requirements

Before configuring Kafka and SingleStore, generate the required certificates and keys in .pem format.

The following are the required files:

  • CA certificate configured on the Kafka broker for client certificate verification

  • Client certificate, signed by the CA in step 1

  • Client private key

  • CA certificate used by the client to verify the broker certificate

  • Kafka broker certificate, signed by the CA in step 4

  • Kafka broker private key

Note

The CA certificates used for signing and verification can be either self-generated or issued by well-known third-party CA certificates.

Generate Kafka Keystore and Truststore

Kafka uses Java KeyStore (JKS) files for SSL configuration. The required format depends on the Java version used by the Kafka broker.

JKS Keystore Format (Java 8 and Earlier)

In Java 8 and earlier, the default keystore and truststore format is JKS (Java KeyStore). JKS supports different passwords for the keystore and the private key.

Use the following commands to generate the keystore and truststore from the .pem files.

  • Use the following command to create a PKCS12 keystore file:

    openssl pkcs12 -export \
 -in /path/to/server-cert.pem \
 -inkey /path/to/server-key.pem \
 -certfile /path/to/ca-cert.pem \
 -out kafka-server.p12 \
 -name kafka-server \
 -passout pass:<PKCS12_PASSWORD>

  • Use the following command to convert the PKCS12 keystore to JKS file:

    keytool -importkeystore \
 -destkeystore kafka.server.keystore.jks \
 -srckeystore kafka-server.p12 \
 -srcstoretype PKCS12 \
 -alias kafka-server \
 -deststorepass <JKS_KEYSTORE_PASSWORD> \
 -destkeypass <JKS_KEY_PASSWORD> \
 -srcstorepass <PKCS12_PASSWORD>

  • Use the following command to create the truststore file:

    keytool -import \
 -trustcacerts \
 -alias CARoot \
 -file /path/to/ca-cert.pem \
 -keystore kafka.server.truststore.jks \
 -storepass <JKS_TRUSTSTORE_PASSWORD> \
 -noprompt

Use these JKS files in the Kafka broker configuration.

PKCS12 Keystore Format (Java 9 and Later)

In Java 9 and later, the default keystore and truststore format is PKCS12 (.p12), unless explicitly configured to use JKS. PKCS12 does not support separate passwords for the keystore and the private key.

Use the following commands to generate the keystore and truststore from .pem files.

  • Use the following command to create a PKCS12 keystore file:

    openssl pkcs12 -export \
 -in path/to/server-cert.pem \
 -inkey path/to/server-key.pem \
 -certfile path/to/ca-cert.pem \
 -out kafka.server.keystore.p12 \
 -name kafka-server \
 -passout pass:<SERVER_KEYSTORE_PASSWORD>

  • Use the following command to create the PKCS12 truststore file:

    keytool -import \
 -trustcacerts \
 -alias CARoot \
 -file certs/ca-cert.pem \
 -keystore kafka-stores-pkcs12/kafka.server.truststore.p12 \
 -storetype PKCS12 \
 -storepass <SERVER_TRUSTSTORE_PASSWORD> \
 -noprompt

Use these .p12 files in the Kafka broker configuration.

Configure Kafka Brokers

Configure the Kafka broker with an SSL listener and enable client authentication by adding the following properties to server.properties file. The following configurations use KRaft mode.

JKS Keystore Configuration (Java 8 and Earlier)

listeners=PLAINTEXT://:9092,SSL://:9093,CONTROLLER://:9096
advertised.listeners=PLAINTEXT://host.example.com:9092,SSL://host.example.com:9093
listener.security.protocol.map=PLAINTEXT:PLAINTEXT,SSL:SSL,CONTROLLER:PLAINTEXT
inter.broker.listener.name=SSL
ssl.keystore.location=/path/to/kafka.server.keystore.jks
ssl.keystore.password=<JKS_KEYSTORE_PASSWORD>
ssl.key.password=<JKS_KEY_PASSWORD>
ssl.truststore.location=/path/to/kafka.server.truststore.jks
ssl.truststore.password=<JKS_TRUSTSTORE_PASSWORD>
ssl.client.auth=required

Replace host.example.com with the Kafka broker endpoint.

Enabling ssl.client.auth=required requires clients to present a valid certificate that enables mutual TLS.

PKCS12 Keystore Configuration (Java 9 and Later)

listeners=PLAINTEXT://:9092,SSL://:9093,CONTROLLER://:9096
advertised.listeners=PLAINTEXT://host.example.com:9092,SSL://host.example.com:9093
listener.security.protocol.map=PLAINTEXT:PLAINTEXT,SSL:SSL,CONTROLLER:PLAINTEXT
inter.broker.listener.name=SSL
ssl.keystore.location=/path/to/kafka.server.keystore.p12
ssl.keystore.password=<SERVER_KEYSTORE_PASSWORD>
ssl.key.password=<SERVER_KEYSTORE_PASSWORD>
ssl.truststore.location=/path/to/kafka.server.truststore.p12
ssl.truststore.password=<SERVER_TRUSTSTORE_PASSWORD>
ssl.client.auth=required

Replace host.example.com with the Kafka broker endpoint.

Enabling ssl.client.auth=required requires clients to present a valid certificate that enables mutual TLS.

Configure a Pipeline to Use mTLS

When creating a pipeline, configure the Kafka connection to use SSL and specify the client certificate, private key, and CA certificate.

Example: CREATE PIPELINE with mTLS

The following example demonstrates how to create a Kafka pipeline that authenticates to the Kafka broker using mTLS.

CREATE PIPELINE p AS
LOAD DATA KAFKA 'host.example.com:9093/test_topic'
CONFIG '{
  "security.protocol": "ssl",
  "ssl.ca.location": "/path/to/ca-cert.pem",
  "ssl.certificate.location": "/path/to/client-cert.pem",
  "ssl.key.location": "/path/to/client-key.pem"
}'
CREDENTIALS '{
  "ssl.key.password": "<CLIENT_PRIVATE_KEY_PASSWORD>"
}'
INTO TABLE t;

The host and port mentioned must be the same as specified in advertised.listeners in the Kafka server configuration. This example uses port 9093 as the SSL port and must be used for the mTLS connection.

Last modified: December 22, 2025

Was this article helpful?

On this page

Was this article helpful?

Verification instructions

Note: You must install cosign to verify the authenticity of the SingleStore file.

Use the following steps to verify the authenticity of singlestoredb-server, singlestoredb-toolbox, singlestoredb-studio, and singlestore-client SingleStore files that have been downloaded.

You may perform the following steps on any computer that can run cosign, such as the main deployment host of the cluster.

  1. (Optional) Run the following command to view the associated signature files.

    curl undefined

  2. Download the signature file from the SingleStore release server.

    • Option 1: Click the Download Signature button next to the SingleStore file.

    • Option 2: Copy and paste the following URL into the address bar of your browser and save the signature file.

    • Option 3: Run the following command to download the signature file.

      curl -O undefined

  3. After the signature file has been downloaded, run the following command to verify the authenticity of the SingleStore file.

    echo -n undefined |
        cosign verify-blob --certificate-oidc-issuer https://oidc.eks.us-east-1.amazonaws.com/id/CCDCDBA1379A5596AB5B2E46DCA385BC \
          --certificate-identity https://kubernetes.io/namespaces/freya-production/serviceaccounts/job-worker \
          --bundle undefined \
          --new-bundle-format -
    Verified OK

Try Out This Notebook to See What’s Possible in SingleStore

Get access to other groundbreaking datasets and engage with our community for expert advice.

Sign UpLog In