7.9 Maintenance Release Changelog (SingleStore Helios Only)

Fixed an issue where the root password failed to initialize during startup when the @@password_min_uppercase_chars variable is set to a value greater than zero.

Fixed an issue where unlimited storage databases stored on Amazon S3 could run out of connections under intense loads.

Fixed an issue where an incorrect result is produced when the engine variable query_parallelism is set with flexible_parallelism. query_parallelism is deprecated and query_parallelism_per_leaf_core should be used instead.

Removed an unexpected warning related to the engine variable query_parallelism (which is deprecated) when a query contains an ORDER BY clause with an IN list filter that matches the table's shard key.

Fixed a bug where an aggregator result table hits a lockdown when sql_select_limit is set.

Fixed a memory leak issue in the memsql_exporter process.

Fixed an issue where VECTOR_MUL was returning incorrect results on shard tables.

The SHOW COLUMNAR MERGE STATUS command now outputs status for tables with empty sort keys.

The following commands now respect non-default columnstore_table_lock_threshold values: INSERT ... SELECT ... ON DUPLICATE UPDATE, REPLACE ... SELECT, and REPLACE ... LOAD.

Fixed an issue where LIMIT and OFFSET after a window function with no PARTITION BY clause were returning incorrect information.

Improved performance of multi-part GCS backups.

Increased mv_plancache.activity_name from varchar(64) to varchar(512).

Introduced a new global variable, dr_min_connection_timeout_ms, which allows users to adjust the minimum timeout period in Disaster Recovery (DR) replication.

Fixed a memory leak when using backreferences inside regexp built-ins (e.g. regexp_substr()).

Fixed a potential crash in NFS backup when encountering an IO error.

Added support for Control Group v2 (cgroup v2).

Fixed an issue with the incremental restore of an unlimited database that would fail if the restore was larger than the disk.

Fixed a performance bug regarding the prefetch queue's size when flexible parallelism is enabled.

Fixed a potential issue where changing a user password would not change the connection pool password in the Data API.

Fixed a JSON formatting issue for the PARTITION_UNRECOVERABLE event in MV_EVENTS details column.

Fixed a bug where a HashJoin on a materialized CTE gets executed as a NestedLoopJoin in some query shapes causing slow join performance.

Fixed the SPLIT string function that was not detecting the correct argument type when used inside a table-valued function (TVF) or a user-defined scalar-valued function (UDF).

Fixed an issue on sample tables that can be created with different collations on different partitions.

Introduced an optimizer feature where a single sharded table may be gathered to the optimizer in order to reduce duplicate work across partitions. Two new engine variables were added to control this behavior: optimizer_min_reference_tables_for_gathered_join and optimizer_min_reference_rows_for_gathered_join.

Improved the performance of backups to GCS (Google Cloud Storage).

Fixed a rare case where the query optimizer re-wrote a query into a non-equivalent query which produced more result rows.

Reduced memory usage in situations where tables are repeatedly created and dropped.

Fixed unlimited storage S3 multipart uploads to properly retry on 200 responses which contain an embedded "SlowDown" error, instead of failing immediately.

Fixed an issue where memory accumulates over time if reshuffle or repartition operators are executed for many unique query shapes.

Introduced a new global engine variablebackup_multipart_upload_concurrency that can be used to improve blobstore backup performance. The default value (3) can improve performance out of the box.

Improved the performance of a left join of a reference table with a sharded table by pushing the reference table clauses to the leaves. The engine variable disable_reference_table_leftjoin_where_pushdown must be set to "OFF" to enable this operation.

Fixed the query_text output in profiles generated while the profiled query is executing.

Fixed a bug where the DEFAULT NOW() data type and the UUID/SYS_GUID functions get ignored in some cases when specified on a column in the primary key.

Fixed an issue where errors could occur during backup when large files are copied.

Fixed an issue with DATETIME columns with ON UPDATE CURRENT_TIMESTAMP defaults that could cause incorrect results.

Fixed an issue where a NULL could be pushed down to a union which could cause the type-cast/collation to be lost, resulting in an error: "Operation 'Scanning a hash join build' is not allowed".

The sql_mode session variable now shows up in SHOW VARIABLES and the mv_global_variables view.

Fixed an issue that could cause an incorrect error message to be generated on an out of memory error.

Fixed an issue where S3 pipelines could stop loading data from the bucket if files are deleted from the bucket while the pipeline is active.

Improvements to memsql_exporter: Improved error handling and reduced memory usage.

Fixed an issue where, in rare cases, an UPDATE statement with a left join in the filter clauses could fail to execute.

Fixed an issue where replica blobs could be loaded into the blob cache with an incorrect Least Recently Used (LRU) timestamp.

Fixed an uncommon issue where a Point-in-Time-Restore may fail because the log is perceived as corrupt.

Fixed an issue where CLEAR ORPHAN DATABASE could cause data loss if run when the master aggregator has a database in an unrecoverable state.

Fixed an issue with DDL endpoint queries using a lower-than-specified query parallelism setting when workspaces are enabled.

Fixed a potential crash issue in UPDATE queries that involve joins and that have scalar subselects in SET clauses.

Fixed an issue with missing row counts during optimization when sampling is disabled.

Fixed an issue where running DROP PLAN FROM PLANCACHE ... FOR QUERY on a query with invalid syntax could cause a crash.

Updated timezone metadata to include Mexico's latest timezone change.

Fixed an issue where incorrect results could be returned when segment-level flexible parallelism is used inside a subquery containing both a shard key filter and either an aggregate, a window function, or the limit clause.

Added a new information schema view internal_table_statistics which shows memory use of SingleStore internal metadata tables. The columns displayed are the same as those shown for table_statistics.

The  read_advanced_counters, snapshot_trigger_size, and  snapshot_to_keep  engine variables can now be set on SingleStore Helios.

Added the option to use HTTPS with memsql_exporter. To use HTTPS, enable the engine variables exporter_use_https, exporter_ssl_cert, and exporter_ssl_key.

Fixed an issue where the memory used by an external function was not being freed quickly enough.

Fixed an error that could occur when attaching databases with a snapshot file of greater than 5 GB.

Improved the query parsing performance of queries with many tables.

Fixed an error caused by setting the collation_server global variable to a non-default value when performing a REBALANCE PARTITIONS resource availability check.

Fixed a bug where too many rows are sampled from columnstore tables with more than 1 billion rows.

Fixed an issue with histogram estimation in columns with a negative value TIME type.

Added ability to cache histogram results to reduce the work histograms perform during optimization.

Fixed "table partition count mismatch errors" that occur due to the following conditions: the system variable enable_workspace_ha is set and there is an upgraded workspace with an attached read replica.

Improved error handling when restoring from an incomplete incremental backup.

Improved the performance of S3 pipelines when Garbage Collection (GC) is enabled.

Fixed an issue with the schema registry so only superusers can see redacted pipeline metadata such as credentials and URLs.

Fixed an issue with promote lock timeout errors that may occur during a rebalance due to a heavy ingest workload, which causes the merger to be slow to pause.

Introduced a new global variable subprocess_max_retries, which is used for retrying on retry-able connection failures during select into/backup queries for S3 and GCS.

Fixed an issue that may cause a "Table doesn't exist" error when a multi-insert contains expressions and the target table has a computed column as its shard key.

Fixed a crash when a computed column definition refers to a table name.

Background snapshots are now allowed to run during BACKUP DATABASE commands. This prevents increased disk usage by logs during a long-running backup.

The CREATE_TIME, CREATE_USER, ALTER_TIME, and ALTER_USER columns in the information_schema.TABLES view are now properly set for views and TVFs (table-valued functions).

Fixed a parsing issue for queries containing multi-line comments near GROUP BY or ORDER BY clauses.

Added support for incremental backups to blob stores where a single partition contains more than 5 TB of data.

The num_background_merger_threads engine variable is now settable on Cloud.

Fixed the case where the REGEXP_REPLACE and REGEXP_SUBSTR expressions can produce non-utf8 strings by introducing the new engine variableregexp_output_validation_mode. Regular expression built-ins can produce non-utf8 strings because they don't have full support for multi-byte characters. The engine variable controls this behavior if regular expression built-ins return strings that are invalid under its collation settings.

Added the DATETIME_PRECISION column to both PARAMETER and ROUTINES information_schema views. Also, the DATETIME_PRECISION column will include TIME and TIMESTAMP data types in the COLUMNSinformation_schema view.

Sampling will no longer be used for table size estimation when statistics are present

Fixed an issue causing incorrect trace messages in master logs where clocks were incorrectly advancing from "0".

Added a new information_schema view named LMV_LOCAL_DATABASES. This view shows the state of local databases like SHOW DATABASES EXTENDED, but it can be queried against unlike show commands.

Fixed an issue where extra CPU was used when a read-only database is attached to a workspace without any writable mount for the read-only database.

The spilling_minimal_disk_space engine variable can now be set at startup.

Allow modifying the json_extract_string_collation engine variable as long as a computed column that modifies the shard key does not contain a call to JSON_EXTRACT_STRING.

Added a new option, SERVER_V2, to the json_extract_string_collation engine variable. This new, recommended option is the default for new clusters, and allows comparison of utf8mb4 strings extracted from JSON to utf8 string constants. Existing clusters will retain their original setting upon upgrade.

Improved the performance of various commands (SHOW commands, DDL, etc.) when there are very many views or tables in the database (100s of thousands).

Fixed a performance regression when running SET <system variable> = <expression> commands. These commands are often run by client drivers on new connections.

Fixed a MySQL protocol incompatibility where SingleStore was not properly sending back the SERVER_STATUS_IN_TRANS flag. This was impacting some MySQL php drivers.

Fixed an upgrade issue where a database that was originally created on SingleStore 6.8 or earlier and was then upgraded to 7.3 is then REPLICATED to SingleStore version 7.8 could result in broken shard keys.

Added a new OPTIMIZE TABLE <table_name> INDEX; command for columnstore tables. This command runs the optimization routine for columnstore secondary indexes manually.

Added a new global variable, num_background_merger_threads, which controls the number of background merger threads to start for each node. The default value is 2.

Fixed an issue introduced in version 8.0.4 where SHOW TABLE STATUS was incorrectly displaying views.

The SHOW STATUS EXTENDED command contains a new "Gv_clock" key whose value is the current logical clock of the server.

The SHOW DATABASE STATUS command contains a new "gv_clock" key whose value is the current logical clock of the server.

information_schema.table_statistics now correctly shows information about global temporary tables.

Fixed some error handling issues with unlimited storage download and upload processes.

The SHOW TABLE STATUS command now displays the memory usage by GLOBAL TEMPORARY tables.

Fixed an issue that may cause the old master aggregator to lose users/grants metdata when it rejoins the cluster after a new master aggregator was promoted using the AGGREGATOR SET AS MASTER command.

Fixed a crash when parsing certain Parquet data into a pipeline.

Added support for using a connection link for inserting data with the FORMAT PARQUET option.

Fixed an issue with aggregate functions using incorrect enum/set types that may result in inaccurate output in the operator tree.

Fixed an issue with a transaction leak on the master aggregator when running CREATE TABLE AS SELECT on a child aggregator using autocommit=0 on the connection.

Fixed a bug that may cause a query to hang when comparing an utf8 column with an utf8mb4 constant. This issue occurs when collation_server is set to one of the utf8mb4 collations.

Improved the accuracy of network time reporting in query profiles regarding the time spent sending the results back to the user connection.

Fixed an edge case issue causing a potential memory leak when running an UPSERT statement against a columnstore table.

Fixed an issue that could cause the engine to crash from information_schema query submissions.

Improved the performance of bushy join rewrites.

Fixed an edge case issue where the engine could crash when performing multi-inserts.

The avro schema registry URL portion of the CREATE PIPELINE syntax is now redacted in processlist.

Fixed an issue where the engine could crash during recursive set operations.

The information_schema.statistics "collation" column now correctly indicates whether an index is ascending ("A") or descending ("D").

Improved performance of comparing utf8mb4 strings.

Fixed an edge case issue which could cause the engine to hang during shutdown.

Added the skip_segelim_with_inlist_threshold engine variable, which will skip segment elimination with the IN list if its size is larger than threshold (default is 1000 elements).

Fixed a performance issue when creating materialized aggregator result tables.

Fixed an issue when auto-creation is turned off (auto.create.topics.enable=false); SELECT INTO KAFKA returns a success message even if the insert fails.

Fixed an issue with tables created on child aggregators not using the correct create_user in the information_schema.tables.

Improved the ALTER TABLE error message received when run on a view on a child aggregator.

Added the session variable histogram_column_correlation, which controls the assumed correlation between columns when using histograms for selectivity estimation.

Added ability to use ALTER and TRUNCATE table commands when BACKUP is running.

Fixed an issue where wrong results may occur when a shard key column is set to the value of a non-deterministic function (RAND(), UUID(), etc.).

SingleStore Helios no longer schedules pipelines to run internally if there is no writable database attached.

Improved join estimations when using a unique key.

Fixed an incorrect error message when GRANTing table-level permission if the default_user_require_ssl is set.

Improved incremental backup performance.

Fixed a potential issue that could cause a NFS backup error to trigger a crash.

Fixed an issue where having too many rows on the same columnstore hash key could return "Read failed with error 14 (Bad address)".

Added the ability to handle 429 (too many requests) errors the same way as 503 (slow down) errors.

Enabled histograms to estimate predicates that use Stored Procedure parameters or implicit/explicit typecasted literals. This change is controlled via the enable_histogram_to_unwrap_literals sync variable. This is Off by default. It can be enabled by setting the variable to ‘On’ or 1.

Added the ability to limit the application of join predicate transitivity to predicates that may be estimated. To use this feature, the session variable verify_fields_in_transitivity needs to be set to ON.

Fixed an issue that may cause a restarted master aggregator to crash when the STOP PIPELINE command runs on a detached pipeline.

Fixed a potential issue that could cause a lost second prepare statement when using a binary protocol over SSL.

Fixed an issue where an incorrect result could be returned when a query contains multiple joins that all match a multi-column hash index.

Fixed a performance issue with count distinct aggregation and streaming subqueries.

Reduced the memory usage of querying information_schema.pipeline_errors when there are many past pipeline errors.

Decreased the cost of working with large project lists during optimization.

ATTACH DATABASE...READ ONLY no longer blocks the DDL or code-generation against the primary writable workspace when the read-only workspace is being ATTACHED. This also applies to DETACH of a read-only site.

Introduced new syntax for DROP FROM PLANCACHE to drop a plan based on query text instead of plan_id. The syntax also supports dropping plans across all aggregators instead of on a particular node_id.

Introduced a new engine variable: clamp_histogram_date_estimates. When enabled, SingleStore will use a heuristic to more accurately estimate date predicates that are larger than the histograms range. This variable is set to AUTO (OFF) by default.

Added the ability to allow code-generation of queries against the information schema when the cluster is very low on disk/memory.

Fixed a potential issue when a database is ATTACH ... READ ONLY is re-attached to the cluster for the first time. Auto-increment columns metadata was not resyncing properly, causing false, duplicate key errors.

Fixed a potential issue that may cause an infinite loop when some information_schema tables are filtered using the utf8m4 string constant.

Fixed a potential issue using multi-statement batches inside of prepared statements without parameters that may result in a crash.

Fixed a potential issue that may cause unlimited databases to rebuild unnecessary auto-stats.

Fixed an issue which caused unlimited database S3 uploads to fail due to an ExpiredToken error. This issue impacted the refresh of EC2 instance meta credentials caused by the public open-source AWS C++ SDK after encountering a transient network issue.

Allow REVOKE from ROLE permissions for tables that have been dropped.

Fixed a specific query shape which failed CodeGen for columnstore unique keys.

Added engine variable: exporter_ssl_key_passphrase. It is used for the decryption of the exporter_ssl_key file by the monitoring process if the file is encrypted (password protected).

Fixed an issue where memsql_exporter could start with an incorrect user.

Fixed a potential issue regarding diagnostic messages appearing to start successfully but were silently failing.

Improved the hash function used for strings.

Updated behavior of the DROP FROM PLANCACHE command; it now supports ON NODE syntax to drop a query plan from a specific node.

Fixed an inconsistent error message on CREATE USER when the target user already exists.

Added ability to assign a custom name to a Wasm UDF in CREATE FUNCTION by specifying the USING EXPORT clause.

Fixed an incorrect error message for EXPLAIN REBALANCE ALL DATABASES when a read-only database was attached to a workspace.

Added ability for information_schema.mv_plancache to be workspace aware. Only displays plans executed on the connected user's workspaces.

Added support for SELECT INTO ... FORMAT PARQUET.

Added ability to show CTE (common table expressions) in SHOW CREATE VIEW.

Fixed a memory leak issue due to alloc_warnings when using reference table pipelines.

Fixed an issue where commands forwarded to the master aggregator during an online upgrade could cause a crash.

Fixed an invalid op tree error when CCM matching is enabled on a query with CTEs.

Fixed a performance issue for sampling queries with large IN lists executed by the query optimizer.

Fixed an issue with the aggregate result tables not being parameterized correctly, which resulted in unnecessary plans being generated.

Increased the default value of the following engine variables: http_api_pool_max_idle to 1024 and http_api_pool_capacity to 2048.

Added http_api_max_lifetime_seconds engine variable which controls how long a pooled connection can remain open.

Added http_api_max_idle_seconds engine variable which controls how long a pooled connection can remain idle (or unused) before it will be closed.

Improved snapshot trigger for databases with a slow rate of writes. A snapshot occurs when at least 128MB of logs have been written in the last hour.

The CREATE DATABASE command now performs a disk space availability check to ensure enough space exists to store some initial logs and blobs for the database. If the disk space is too low, the create command will be rejected.

Added ability for automatically triggered rebalance operations to restart any long-running write queries which would prevent the rebalance from making progress. The user executed rebalance commands now have a KILL option to trigger this behavior from a manually executed REBALANCE.

Added support for running INSERT SELECTS using an aggregator result table on a child aggregator.

Fixed an issue where canceling a RESTORE would not completely drop the partially restored database.

Fixed a lockdown feature error that may occur for queries with multiple in-subselects.

Fixed an issue where SHOW GRANTS output was incorrect for users with REQUIRE SSL WITH GRANT OPTIONS.

Improved reliability of full and incremental cloud backups that experience network connectivity issues.

Improved the accuracy of the FIRST_LOADED and LAST_EXECUTED timestamp columns in information_schema.plancache.

Fixed an issue where S3 can silently fail to upload a file for an unlimited database.

Fixed an issue that may cause error 200 to occur when running CREATE DATABASE or RESTORE DATABASE commands on an unlimited database.

Fixed an edge case issue where the blob cache eviction algorithm could cause an infinite loop.

Added the ability to query MV_BACKUP_STATUS for users with SHOW METADATA permissions.

Fixed a crash when creating a WASM user-defined function from an S3 bucket with an invalid prefix in the path name.

Fixed an incorrect error message when a computed column refers to a two part table name (i.e., table.column).

Fixed an issue that may cause insert into mv_queries to become invalid when the cluster is in ONLY_FULL_GROUP_BY sql_mode.

Fixed an issue that returned 'zero rows' when sampling fails or there is no sampling data. The corrected behavior will use heuristic or histogram estimate depending on what is available. To enable this behavior, set estimate_zero_rows_when_sampling_data_is_missing to ‘false’.

Fixed parsing of microseconds in STR_TO_DATE to give fewer than six digits a higher priority.

Fixed an issue where queries that contain CTE (common table expressions) with the same name may not traverse correctly.

Fixed an issue where S3 can silently fail to upload a file for an unlimited database.

Added the ability to configure SingleStore to estimate the equality predicate for uncorrelated scalar subselects as 1/cardinality; and use heuristics for all other scalar subselect predicates. To enable the new estimation logic, set exclude_scalar_subselects_from_filters to 'false'.

Improved error messages generated when a PITR cannot find a valid time range for restoring.

Added support for Negative Lookahead, Positive Lookahead, Non-greedy quantifiers, and Non-capturing group regular expressions.

Fixed an issue where large amounts of consecutive aborted messages were not being handled correctly in a Kafka pipeline.

Improved the Subselect To Join rewrite to recognize more cases with subselects in filter conditions for left join.

Fixed an issue with window functions which refer to the window definition at the end of a SELECT statement.

Removed ability to use utf8_general_mysql500_ci collation. This collation is not supported by SingleStore Helios.

Added support for a new AES encryption mode called CBC (cipher block chaining) for 128-bit and 256-bit keys. See AES_ENCRYPT for more information.

Fixed an issue with unnecessary data sorting when single partition queries contain an ORDER BY clause. The sort is redundant because the data is sorted on the leaves for these types of queries.

Fixed a crash caused by UPDATE statements with a joined table where the optimizer pushes the JOIN into a derived table whose name is automatically generated and different from the original table being updated.

Added support for s3_force_path_style to S3 backup. s3_force_path_style (optional boolean, defaults to true) specifies whether to use path style (region.amazonaws.com/bucket) or virtual address style (bucket.region.amazonaws.com) syntax when specifying the location of the bucket. For S3 compatible services that only support virtual address paths, specify "s3_force_path_style":"false" in the configuration options. Refer to the BACKUP DATABASE section for more information.

Fixed an error regarding failover in progress that occurs when running EXPLAIN REBALANCE PARTITIONS FULL.

Fixed an issue where auto-attach of leaf nodes was not being retried on repeatable errors due to excessive errors when checking on the state of the leaf partitions.

Fixed an issue where DETACH DATABASE of a read-only replica was not correctly removing the partitions from offline leaves.

Fixed an issue where CLEAR ORPHAN commands were not correctly removing the multiple replica partitions for databases attached as read replicas.

Fixed an issue where ATTACH DATABASE would attach a replica partition for an existing online read replica partition.

Fixed an issue with REBALANCE PARTITION operations that do not move master partitions. Adding or removing a read replica will not impact temporary tables on the primary writable copy of the database.

Added ability to SHOW METADATA permissions so users can query information_schema.mv_events.

Added requirement that ALTER USER statements in privilege_transfer_mode now need GRANT permissions.

Added ability to enable/disable AWS EKS IRSA auth for S3 pipelines. The default state for the variable enable_eks_irsa is OFF.

Improved GLOBAL temporary table errors for queries from a read replica.

Fixed an error that was possible while querying information_schema.TABLE_STATISTICS.

Fixed an issue causing some queries with table value functions to report the error, "Partition's table metadata has an unexpected table type," when flexible parallelism is enabled.

Improved CREATE RESULT TABLE error when run from a read replica. This feature is not supported for read replicas.

Fixed an issue where replication between primaries and read replicas may not pause properly during an online upgrade.

Fixed a typo in the system variable aes_default_encryption_mode.

Fixed incorrect behavior when dropping and adding a multicolumn columnstore hash index on the same set of columns in a single alter.

Improved the error information when creating a database without specifying a workspace name in the syntax on a master aggregator on a cluster with multiple workspaces.

Improved the way bad memory allocations are handled during Wasm UDF execution.

Fixed an error when using certain DATE_FORMAT options via Tableau.

Improved the performance of downloading blob files for unlimited databases when SSL is enabled between SingleStore and the blobstore.

Fixed an issue with backing up an unlimited database. Backup could fail if the blob gets evicted from the blob cache during backup.

Fixed an issue that may cause the DATA API to become unresponsive under a highly concurrent workload.

Fixed an issue where the incorrect backup size showed in mv_backup_history for incremental backups.

Added support for backticks ( ` ) in column names.

Improved the performance of hash joins on expressions.

Fixed a potential issue with transaction rolled back-mid query errors that may occur when a database name has a number suffix.

Fixed an issue when specifying all columns in the SELECT command resulted in columns being out of order rather than matching the table definition.

Fixed a crash due to using the LINES TERMINATED BY clause when using INFER PIPELINE.

Fixed an issue that may cause information_schema.processlist to show “Sanitizing . . .“ instead of the actual query text.

Reduce the memory used to track cached blob files and run scans over columnstore tables.

Fixed an issue that may cause a deadlock when using the query SELECT INTO <objectstore> when flexible parallelism is enabled.

Added support for forwarding SHOW RESULT TABLES from child aggregators to the master aggregator when DDL forwarding is enabled.

Fixed crashes that may occur when downloading blobs from Azure bottomless databases.

Fixed an issue where a syntax error would occur when running queries against reference tables with comments following the query text.

Added support for NOT NULL in persisted computed columns in CREATE TABLE statements.

Added support for 256-bit keys for AES_ENCRYPT and AES_DECRYPT functions.

Fixed status so Pipelines running START PIPELINE FOREGROUND, TEST PIPELINE, and PROFILE PIPELINE will now show "Running Foreground", "Testing", and "Profiling" in information_schema.pipelines.state, rather than "Stopped".

DROP DATABASE is now only allowed from a workspace with the database attached as writable.

Disabled ability to ATTACH DATABASE as READ ONLY.