SingleStore Kubernetes Operator Release Notes

Maintenance Release Changelog

2024-09-05 Version 3.292.2

• Re-introduced the exporter container. To enable or disable the exporter, update the DisableExporter field in the CR.

2024-08-02 Version 3.292.0

• Changes the default security context to use non-root containers

• Reduces rebalances when upgrading Kubernetes

• The Operator container no longer uses the root user

• Adds a security context to support Kubernetes restricted mode

• Support for SingleStore versions 7.6 and earlier has been deprecated. To use SingleStore versions 7.8 and later, customers can upgrade from SingleStore versions 7.6 and earlier using Operator versions 3.258.0 and earlier.

• Adds security fixes to the Operator image

• Default settings have been optimized

2024-04-04 Version 3.258.0

• Removes the need for Operator RBAC access to leases

• To mitigate partition movement during scaling, falls back to scaling full availability groups at a time with a single rebalance when the SingleStore engine version is earlier than 8.1.28

2024-03-13 Version 3.246.0

• Adds a feature flag to disable all volume recreation. To set, add the following to the spec:

  Copy

     featureFlags:
      - DisablePVRecreation

• Fixes an issue with Operator rebalances causing partition movement

2024-01-22 Version 3.230.0

• Adds the --log-json argument to the Operator binary to output logs as JSON

• Adds the SkipMetadataUpload: true argument to the backup spec to skip uploading the metadata about the databases that were backed up

• Adds the ServiceAccountName argument to the backup spec to override the default service account name used by the backup CronJob

• Fixes node image file permissions when upgrading from SingleStore engine images earlier than v7.6.16

2023-12-05 Version 3.194.3

• Upgraded the Go version to 1.21.4

• Fixed the HTTP Web proxy certificate rotation to use new certificates

• Added parallelized backups across databases; repeated backups are skipped due to job failures

• Added support for requiring SSL for the root user

  • This feature can be enabled by adding the following fields to the cluster spec:

    YAML

    Copy

    featureFlags:
    	- EnableRequireRootSSL

2023-10-13 Version 3.162.3

• Updated the RBAC (the sdb-rbac.yaml file) that must be reapplied to avoid potential issues

• Added support for cert-manager SSL secrets by using the key ca.crt for the certificate authority (if present, and in lieu of tls.ca). Refer to Configure TLS/SSL/WebSocket for more information.

• Exposed tolerations on the backup pod on field spec.backupspec.tolerations. Refer to Taints and Tolerations for more information.

• Backup logic will now perform backups in parallel and not repeat backups for the same databases in the case of a failure

• Backup pod will now use SSL if enabled on the cluster

• Reduced database memory overhead by setting the maximum_memory variable to a maximum of either 90% of pod memory or pod memory - 10 GB, whichever is greater

• Updated the leaf_failover_fanout global variable to always use load_balanced mode and removed support for setting it to paired mode

• Various bug fixes and scaling improvements

2023-07-13 Version 3.40.5

• Updated the Go version to 1.20.5

2023-05-15 Version 3.40.3

• Removed the exporter sidecar from non-MA pods

• Introduced online password rotation via RootPasswordSecret

• Changed the pod management policy from OrderedReady to Parallel, allowing aggregators to spin up in parallel rather than in series

• Added support for online SSL certificate rotation. Changed the underlying secret of SecureConnectionSpec.SSLSecretName which will reload the certificates in the background without downtime. Refer to Secrets for more information.

• Exposed backup pod affinities via MemsqlClusterBackupSpec.Affinity. Refer to the Kubernetes API Reference Docs for more information.

• Exposed backup pod labels and annotations via MemsqlClusterBackupSpec.ObjectMetaOverrides. Refer to ObjectMeta for more information.

• Added support for upgrades between SingleStore engine versions 8.0 and later

2023-01-17 Version 3.0.98

• Fixed exporter command line options to reduce memory usage

• A snapshot is now created before and after SingleStore upgrades

• Introduced a child aggregator startup probe to prevent sending traffic to nodes that are not ready for queries

• Reduced the number of rebalances during cluster updates

• Introduced an online root password setup and rotation using RootPasswordSecret in the spec

• Kubernetes probe logs now output to container logs

• Reduced snapshots during StatefulSet changes

• Removed the liveness probe

• Added support for Kubernetes 1.25

• Incremental backups of unlimited storage databases are now skipped

• Added ansi_quotes SQL mode compatibility

• Disabled service account token for server pods

2022-10-19 Version 3.0.60

• Incremental backup fixed to correctly pick up INIT backup

• Adjusted backup cron job settings to keep the logs of failed Pods

• Improvements to reduce the impact of database node reboots

• Operator Docker image moved to a scratch base image

• Allows the specification of more than one SecurityContext for a Pod

2022-08-26 Version 3.0.29

• Reduced the timeout for checking whether a SingleStore node is down

• Added exporting of Prometheus metrics

• Set the default TLS version to 1.2 for SingleStore clusters

• Modifying an SSL certificate secret no longer triggers a cluster restart

• Added ExposeAggs CR flag to expose aggregator information in the status

• Added support for configuring network policies via networkPolicySpec

• Added support for configuring imagePullSecrets for backup jobs via imagePullSecrets in the backup spec

• Added support for setting the number of cores and memory directly through the cluster CR

• Updated various dependencies to remediate known vulnerabilities

• Changes on the ingress controller will not impact the LoadBalancer Service in the Kubernetes cluster

• Added support for Azure backups

• Fixed various minor bugs

2022-06-19 Version 2.0.23

• This version of the Operator requires that the startupProbe flag is enabled in Kubernetes 1.17 - 1.20. For Kubernetes 1.20 and later, startupProbe is enabled by default.

• To promote safe restarts, the Master Aggregator, child aggregator, and leaf nodes now wait for terminationGracePeriodSeconds, allowing all in-flight queries to complete.

• Supports SingleStore node Docker images without Python installed.

• Various minor bug fixes.

2022-05-25 Version 2.0.8

• Leaf nodes now automatically detach on container termination and wait for queries to drain before terminating

• Leaf nodes now automatically attach themselves on container start if they were detached previously

• ms-pusher resources are now configurable via the memsql CR

• imagePullSecrets was added to memsql CR

• memsql CRD version upgraded from v1beta1 to v1

2022-04-29 Version 1.2.8

• Updated the Operator's container base image to use AlmaLinux

• Changed parked PVC deletion to take cluster metadata into account

• The Operator now waits for the first aggregator Pod before creating a leaf StatefulSet

• The Operator now waits for the reconciliation loop to finish before declaring a cluster active

• Added server-side filtering to the Operator client cache to prevent out of memory (OOM) errors

• The Operator will not restart a cluster when updating labels

• Made the collocated CA affinity rule preferred, but not required

• Improvements with liveness/readiness probes

• Added the Operator version to the cluster status

2022-03-11 Version 1.2.7

• Fixes a bug where configuring the SSL and/or the Disaster Recovery spec would cause the SingleStore deployment to fail.

• Required --cluster-id argument has been added to the Operator deployment.yaml file.

• Updates the RBAC permissions to include two new API groups, networking.k8s.io and coordination.k8s.io, which have been added to the rbac.yaml file.

2021-05-04 Version 1.2.5

• Changes the image pull policy from Always to IfNotPresent.

• Changes PodDisruptionBudget to use maxUnavailable instead of minAvailable to avoid an unnecessary PodDisruptionBudget update when horizontally scaling a cluster up or down.

• Allows the admin password to be updated either via MySQL command or the Operator. Previously, if the admin password was updated via MySQL command, it would be reverted by the Operator.

• For engine version 7.3.3 and later, the value of the global variable failover_on_low_disk is OFF by default, which prevents a cluster from failing over when out of disk space.

• Allows the admin user to configure and use connection links on engine versions 7.3.2 and later.

• Child aggregators will now be added before leaf nodes when a new cluster is deployed, which allows child aggregators to have smaller node IDs.

• Fixes a bug where the Operator failed to close a connection to the Master Aggregator.

• Improves the rebalance logic to avoid unnecessary addition/removal of leaf nodes during scale up/down if rebalancing fails.

2021-01-15 Version 1.2.4

• Allows annotations to be passed to aggregators and/or leaf nodes which will then be merged into the corresponding StatefulSet’s annotations.

• Allows the Liveness and/or Readiness probe time-related parameters to be customized.

• Adds an option to disable DDL/DML service creation.

• Adds WebSocket connection support to the cluster.

• Adds the option to enforce a secure connection for the admin user (the database user created by the Operator) when connecting to the cluster.

• Exposes DDL, DML, and WebSocket ports (if WebSocket is enabled) in MemsqlCluster’s status.

• Allows global variables to be set separately for aggregator pods and leaf pods, or for all pods in the clusters (pertains to 2019-10-14 Version de65d489 and later). If a global variable is specified at both the cluster-level and in an aggregator or leaf spec, the latter has priority.

• Supports upgrading SingleStore from v7.1.x to v7.3.x.

2020-10-27 Version 1.2.3

• If a backup cron job was scheduled, was triggered, and is later removed, the associated batch job and pods will be deleted.

• Allows the user that runs the backup job to be specified over the default root user. The default resource pool associated with that user will be used in the backup job.

• When scaling down the redundancy level from level 2 to level 1, the associated ConfigMap for the second availability group (AG) will now be deleted.

• Fixes a performance degradation issue introduced in Operator release 1.2.2 where the primary partitions would become imbalanced across both availability groups during an engine upgrade and/or other condition.

2020-10-01 Version 1.2.2

• Added support for using ClusterIP addresses as DDL and DML endpoint when service type is configured as ClusterIP.

• Added rootServiceUser flag in the CR to control whether to grant SERVICE_USER to root.

• Improved Operator and engine performance by avoiding unnecessary database rebalances.

• Fixed a stuck reconciliation loop when PersistentVolumeClaim allocated more storage size than requested.

• Fixed the service update logic to avoid unnecessary service updates.

• Added support to allow the Operator to pass optional startup parameters to the master exporter.

2020-08-18 Version 1.2.1

• Adds support for setting local_file_system_access_restricted global variable.

• Adds support for setting priorityClassName via an Operator command-line parameter priority-class-name. The priorityClassName will be passed to all StatefulSet’s PodSpec.

• Re-adds the Operator command-line parameters backup-s3-endpoint and backup-compatibility-mode that were inadvertently removed from the Operator 1.2.0 release. Refer to the Backups reference for more information.

2020-08-04 Version 1.2.0

• Inadvertently removed the backup-s3-endpoint and backup-compatibility-mode command-line parameters. Refer to the Backups reference for more information.

• Adds support for setting almost all engine variables via globalVariables section in the CR. Refer to List of Engine Variables for more information. Adding, removing, or updating engine variables will cause pods to restart. The following variables are explicitly prohibited:

  • redundancy_level

  • sync_permissions

  • local_file_system_access_restricted

• Adds a phase field to CR status.

  • Running indicates that the Operator is happy with the state of the cluster and that there are no more changes to be made.

  • Pending indicates that the Operator is still working towards a desired state of the cluster.

• Adds support for scaling up volume storage size when larger numbers are specified in storageGB field in aggregatorSpec and/or leafSpec in CR.

• Adds support for incremental backups which allows an admin to specify both full and incremental backup schedules.

• Adds memsqlPusherSpec inside a monitoringSpec in the CR to allow SingleStore to be configured to push metrics to Kafka.

2020-07-06 Version 1.1.1

• As of SingleStore 7.1.4, license checks are now cgroup-aware and respect container resource boundaries for containerized deployments.

  • While this does not change how license checks are performed, nor does it change how capacity is allocated, it does change how the resources allocated to the container are checked.

• Changed the label on the backup pod from app.kubernetes.io/name=memsql-cluster to app.kubernetes.io/name=backup

• Triggers pod restarts when the content of dependent secrets/configmaps are updated

  • When data in secrets/configmaps are used as a container’s environment variable, changes in those values will trigger a pod restart

• Adds support for running backups with compatibility mode

• S3 Region is no longer required when running backups

2020-06-18 Version 1.1.0

• Adds support for Disaster Recovery (DR).

  • DR requires either of the two following requirements to be met by the underlying infrastructure:

    • Kubernetes hosts in primary and secondary clusters can reach each other via host IPs across clusters

    • Kubernetes pods in primary and secondary clusters can reach each other via pod IPs across clusters

  • In addition, the following requirements must be met:

    • SingleStore7.1.3 or newer must be deployed on both the primary and secondary clusters

    • The primary and secondary clusters’ DDL endpoints are stable

• Adds support for both client-server and intra-cluster secure connections.

  • For client-server secure connections:

    • Once configured, the server permits, but does not require, a secure connection

    • Supports both initial deployments and upgrades from existing deployments that are not already configured for client-server secure connections

    • Downgrades are not supported

  • For intra-cluster secure connections:

    • Once configured, intra-cluster secure connections are required between all nodes. Secure connections are also used between the primary cluster and secondary cluster if DR is configured.

    • Supports initial deployments but does not support upgrades from existing deployments that are not already configured with intra-cluster secure connections

    • Downgrades are not supported

• Improves readiness probe by checking each SingleStore node’s online status.

• Improves leaf nodes StatefulSet’s update performance by using OnDelete update strategy.

• Fixes the inability to set the number of arenas via glibc tunable by introducing an envVariables section in the CR and allowing users to set MALLOC_ARENA_MAX. Default: (node height) * (CPUs per unit)

• Adds support for auditlog_level global variable. The following variables are currently supported:

  • default_partitions_per_leaf

  • columnstore_segment_rows

  • columnstore_flush_bytes

  • columnstore_window_size

  • transaction_buffer

  • snapshot_trigger_size

  • minimal_disk_space

  • pipelines_max_concurrent

  • auditlog_level

2020-05-15 Version 1.0.0

• Adds support for custom scheduling in MemsqlCluster spec

  • Supported scheduling constraints include: nodeSelector, affinity, anti-affinity, toleration, schedulerName, and nodeName in PodSpec

• Adds support for automated backup scheduling

• Adds support for setting whitelisted sync and non-sync variables

  • The following variables are supported: default_partitions_per_leaf, columnstore_segment_rows, columnstore_flush_bytes, columnstore_window_size, transaction_buffer, snapshot_trigger_size, minimal_disk_space, pipelines_max_concurrent

• Deploys Prometheus exporter as a sidecar to all SingleStore nodes

• Adds support for a custom container image repository

  • If the environment variable RELATED_IMAGE_NODE is specified, the Operator will pull and use this image to launch all SingleStore node pods as well as the sidecar container.

• Updates liveness probe to use the ProcessState property returned from memsqlctl

• Adds labels to all cluster components and services

• Adds support for custom glibc tunables in CR

  • Default: glibc.malloc.arena_max = 8 * (node height) * (CPUs per unit)

• Updates leaf-node restart mechanism to facilitate faster recovery

• Improves Operator performance by filtering controller events

• Adds support for vertical scaling of child aggregators

• Updates the ConfigMaps mechanism

  • As a consequence, do not create an unused configuration when redundancy level is set to 1.

• Updates readiness probe to include node role type and database status

• Updates Operator to check leaf-node status before detaching/attaching a pod

2019-10-14 Version de65d489

• Added support for the --fs-group-id flag, which allows you to inject an additional group id into the container running SingleStore. This is used to ensure that the process inside the container has the correct permissions to read/write to the /var/lib/memsql volume.

• Added support for additional control over the services used to handle DDL and DML queries from client applications. These controls are specified in a new key in the MemsqlCluster spec called serviceSpec.

• The top level attributes loadBalancerSourceRanges and serviceObjectMetaOverrides used in the MemSQLCluster spec are now deprecated. Use serviceSpec moving forward.

• Some global variables can now be specified through the new attribute globalVariables. The supported variables are: license_visibility, default_partitions_per_leaf, columnstore_segment_rows, columnstore_flush_bytes, columnstore_window_size, transaction_buffer, and snapshot_trigger_size.

• The attributes license and adminHashedPassword can now be specified through secrets. To do this, use the alternative attribute licenseSecret or adminHashedPasswordSecret.