AWS Elastic Kubernetes Service  (EKS)  IAM Roles for Service Accounts (IRSA) authentication can be used to allow SingleStore cluster pods to communicate with S3 buckets (data, backup and pipelines).

This applies to Operator deployments only and is supported for SingleStore Self-Managed Engine version >= 8.5.

The following steps enable IRSA:

1. Set the enable_eks_irsa global variable to ON.

   Copy

   SET GLOBAL enable_eks_irsa = ON

2. Create an IAM role with proper policies attached to choose which services the role can access and the scope of access for those services.

3. Create a Kubernetes service account with annotation using the IAM role arn.

   Copy

   apiVersion: v1
   kind: ServiceAccount
   metadata:  
     annotations:    
       eks.amazonaws.com/role-arn: arn:aws:iam::<aws_account_id>:role/<role_name>  
      name: <service_account_name> # Must be set to the name of the cluster

4. Update the Memsql CR spec with the service account name from the step above.

   Copy

   apiVersion: v1alpha1
   type: MemsqlCluster
   metadata:  
     name: <name> # This is the name of the cluster
   spec:  
     ...  
     serviceAccountName: <service_account_name>  
     backupSpec:    
       credsMode: "eks_irsa"    
       CloudServiceProvider: "AWS"    
       bucketName: "<bucket_name>"    
       s3Region: "<bucket_region; e.g. us-east-1>"

5. Operator will restart the nodes and mount the JWT token to each node.

6. Update the SQL statements to use the hint creds_mode: eks_irsa

   • Data:

     Copy

     CREATE LINK <db_name> AS S3 CREDENTIALS '{}'
     CONFIG '{"region":"us-east-1", "creds_mode":"eks_irsa"}'

   • Pipelines:

     Copy

     CREATE PIPELINE <pipeline_name> AS LOAD DATA S3 'path'
     CONFIG '{"region": "us-east-1", "creds_mode": "eks_irsa"}

   • Backup:

     Copy

     BACKUP DATABASE <db_name> TO S3 "<bucket_name/path>"
     CONFIG '{"region":"us-east-1", “creds_mode”: “eks_irsa”}' CREDENTIALS '{}';

   • Restore:

     Copy

     RESTORE DATABASE <db_name> FROM S3 'bucket_name/path'
     CONFIG '{"region":"us-east-1", "creds_mode":"eks_irsa"}' CREDENTIALS '{}';

CREATE PIPELINE s3_pipeline AS
  LOAD DATA S3 's3://<bucket-name>/<path>/'
    CONFIG '{  
             "region": "us-east-1",  
             "creds_mode": "eks_irsa"
            }'
    CREDENTIALS '{  
                  "role_arn":
                  "arn:aws:iam::123456789012:role/singlestore-s3-pipeline"
                 }'
  INTO TABLE <table_name>
    FIELDS TERMINATED BY ',';

START PIPELINE s3_pipeline;