Maintenance Release Changelog

Removed ability to use utf8_general_mysql500_ci collation. This collation is not supported by SingleStore Helios .

Improved the Subselect To Join rewrite to recognize more cases with subselects in filter conditions for left join.

Fixed an issue that may cause insert into mv_queries to become invalid when the cluster is in ONLY_FULL_GROUP_BY sql_mode.

Allow REVOKE from ROLE permissions for tables that have been dropped.

Fixed an issue where the incorrect backup size showed in mv_backup_history for incremental backups.

Fixed a potential issue with transaction rolled back-mid query errors that may occur when a database name has a number suffix.

Fixed issues with the information_schema.mv_activities and information_schema.mv_activities_extended tables reporting incorrect activities on database names ending with an integer.

Fixed a server crash caused by serializing big table or view definitions in snapshots.

Fixed an issue when running DROP TABLE on a function instead of DROP FUNCTION.

Fixed a server crash caused by a rewrite trying to inline nested expressions.

Fixed an issue with out of memory errors being generated while compiling queries that use multiple UNION clauses.

Adjusted the maximum memory allocation function to proactively run even when the total_server_memory usage is well below the maximum_memory allocation.

Fixed an issue with error generation for aggregator result tables used in a parallel read. DROP RESULT TABLE now returns the correct error.

Improved heuristic estimates for queries with complex join predicates.

Reduced the amount of stack space required to perform code generation for large in-lists.

Fixed an issue where table-valued functions crashed when a parameter was used more than once while compiling a query.

Improved the performance of sampling queries executed by the query optimizer.

Fixed an issue with query forwarding incorrectly triggering aggregator sync causing the aggregators to run out of disk space.

Fixed an issue with incorrect nullCount and rowCount statistics for larger tables with autostats enabled.

Fixed an issue with memsql_exporter ceasing to produce monitoring results.

Fixed a workspace database log corruption that could occur if the root user was dropped when the sync_permissions global variable was enabled.

Fixed the CLEAR ORPHAN DATABASES command to ensure all databases are dropped when the leaf_failover_fanout global variable is set to load_balanced and there are partition databases with more than replica partition.

Fixed a bug that incorrectly pushes a join before a window function when attempting to perform a distributed join in an aggregator.

Improved the predicate transitivity rewrite during query execution to cover more cases in an EXISTS/NOT EXISTS clause.

Reduce probability of failed connections to workspaces due to WebSockets proxy issues.

Fixed the calculation for total_runtime_ms for queries profiled from within stored procedures. Prior to this release, total_runtime_ms was not calculated accurately.

Added support for BACKUP DATABASE into S3 buckets using the sts_regional_endpoints configuration option.

Fixed issue with crash occurring when attempting to compile queries for UPDATE statements with multiple columns due to insufficient memory resources. The amount of memory necessary to compile these query shapes was reduced.

Fixed sync timeout errors occurring when REBALANCE PARTITIONS of large databases is being processed. The REBALANCE PARTITIONS operation now waits for the replication of large snapshot files. 

Added password_expiration column to information_schema.USERS table. If the password_expiration_seconds engine variable is not enabled, the password_expiration column will be NULL. If the password_expiration_seconds engine variable is enabled, the password_expiration column will display the number of seconds remaining for the password to expire.

Updated time zone data for some countries that recently changed their time zone rules.

Added Linux dependencies for SASL to SingleStore installation packages so they do not need to be installed separately.

Fixed an issue with possible connection crashes on disaster recovery workspaces when DDL queries are forwarded to the master aggregator and a database has not been selected.

Fixed an issue with table-valued functions crashing when the parameters used contain full-text match functions and window functions.

Added support for running PROFILE on queries that execute inside of transactions the include multiple statements.

Fixed a crash caused by a check on the maximum number of partitions allowed on a leaf node. An error message is displayed when this threshold is exceeded.

Fixed an issue where the memsql_exporter would sometimes produce invalid JSON errors. The memsql_exporter is a satellite process that collects data about a running workspace.

Added support for SKIP CONSTRAINT ERRORS options for Avro and JSON pipelines.

Increased the timeout used when connecting to AWS to determine credentials if none are specified in a BACKUP/RESTORE command.  Also added a new engine variable subprocess_ec2_metadata_timeout_ms which allows users to set that timeout.

Fixed auto histograms to determine when to build new histograms based on both master and child aggregators.

Masked blob store CREDENTIALS text in BACKUP, RESTORE, and LINK commands in SHOW PROCESSLIST, information_schema.PROCESSLIST, and all error messages.

Added support for incremental backups to utilize LINK syntax to specify CONFIG and CREDENTIALS clauses.

Added support for optimizing cases of IN joins in subselects in the project list.

Replaced the symlink from /opt/memsql to /etc/systemdb/system/memsql.service with copying the memsql.service file to avoid issues with some filesystems not being mounted.

Removed the 20 second limit when checking for expired query plan at the disk_plan_expiration_minutes intervals.

Fixed an issue that could cause pipelines to hang when encountering errors.

Added session variable named subquery_merge_with_outer_joins that fine-tunes the behavior of the rewrite SubQueryMerge. If the variable is set to ON: always merge the sub-select with outer join(s). If the variable is set to HEURISTIC: check to see if merging the sub-select with outer join(s) will cause negative performance for the overall query. If the variable is set to OFF: don’t merge sub-selects with outer join(s). If the variable is set to AUTO (this is the default): For SingleStore 7.5 and previous versions, AUTO will behave like ON. For SingleStore versions after 7.5, AUTO will behave like HEURISTIC.

Added support for complex joins (joins with non-linear/nested join tree) involving correlated nested SELECT queries.

Fixed an issue with query failures occurring if leaf_pushdown=1 hint is set. The query will be run without the hint and a plan warning will be emitted.

Fixed an aggregator crash that was caused when a malformed INIT_DB command is sent by a client.

Added anonymous code blocks which are unnamed blocks of SingleStore Procedural SQL (MSQL) code that is executed immediately. They allow you to create a code block without defining and calling a stored procedure. They are useful in situations such as testing and debugging.

Fixed a crash that could occur when ALTER TABLE COMPRESSION=SPARSE is run in the same ALTER statement with an operation that doesn’t require data movement (ie., AUTOSTATS_ENABLED = ON).

Fixed the cardinality estimates for columns with an equality predicate.

Fixed a crash that can occur when a named window clause is used in multiple window functions in a query.

Added a new system variable called master_promote_kill_timeout_seconds which specifies how long an online failover will wait to drain open write transactions before issuing a kill to those transactions. It is limited to 120 seconds. After the write transactions are killed, an application-level retry is required.

Fixed an issue that can cause defunct processes for pipelines or code generation in a rare race condition.

Updated memsqlctl start-node to check the validity of NUMA configurations when nodes are restarted. If the system doesn’t have NUMA configurations, the nodes will be restarted. If any of the nodes have invalid NUMA configurations, the NUMA bindings will be recalculated for all the nodes on the host.

Fixed an issue that could cause backups to GCS to fail with the error "The difference between the request time and the server's time is too large".

Added a new system variable called parametrizer_query_max_params which allows users to set the maximum number of scalar constants a single query can contain. Any query with more than parametrizer_query_max_params will be failed with an error. The default value is 1048576.

Updated memsqlctl to establish an SSL connection to the node if a CA certificate is set explicitly. SingleStore now falls back to an SSL connection via the host's default certificates if the target user requires SSL.

Fixed an issue where the exporter process was not being cleaned up when memsqld is shut down.

Fixed an issue where modifying the case of a column via ALTER TABLE could cause a pipeline using that table to crash.

Fixed an issue where DDL forwarding could crash when DROP USER is run.

Properly set keep alive and timeouts on the persistent leaf-to-master aggregator connection that is used to communicate replication link status. This fixes an issue with REBALANCE commands that failed with synchronization timeouts.

Fixed a synchronization bottleneck, improving the ability of the pipeline scheduler to run many pipelines in parallel.

Improved the performance of pipeline and database DDL actions when a database has many pipelines running.

Fixed a blocking issue that could occur when several pipelines are being background scheduled simultaneously.

Fixed an issue where incorrect results could be returned when using INSERT_ALL to insert into a temporary table.

Fixed an issue that disabled a compilation optimization for selected queries. Disabling this optimization made certain queries slower.

Fixed an issue that could cause the internal statistics tracked by workload management to become incorrect after a "too many queued" query error has been hit.

Fixed an issue with CTE predicate pushdown where predicates were incorrectly split across HAVING and WHERE commands.

Fixed a rare race condition that could cause sync replication to get stuck.

Added a new system variable optimizer_disable_subselect_to_join. Setting this to ON disables the optimizer rewrite that converts subselect to joins automatically.

Disabling this rewrite may cause queries to fail if they require the rewrite to run.

Fixed an issue where a crash could occur when data_conversion_compatibility_level is set to a value other than 6.0 and a computed column is set to an invalid value via an UPDATE statement.

Fixed an issue where a crash could occur when leaf_failover_fanout is set to load_balanced mode and REMOVE LEAF is run on larger workspaces.

Fixed an issue where a crash could occur when running UPDATE on a query with a scalar subselect in a SET clause after ALTER TABLE.

Fixed an issue where in some rare cases PROFILE queries could cause a crash.

Fixed an issue where ROWSTORE_PROGRESS could exceed 100% in mv_backup_status.

Added a new system variable auditlog_retention_period which controls the retention period (in days) of audit log files on disk. The default is 0 which means files remain on disk indefinitely.

ADMIN_ONLY audit log level now audits DDL statements only. Previously it would audit other non-DDL statements, which could cause bloat in the audit logs in ADMIN_ONLY mode.

Fixed an issue where a crash could occur during concurrent columnstore DELETEs while escalating from row level locking to table level locking.

Fixed an issue in universal storage unique keys where hash collisions with null values could produce incorrect results.

Fixed an issue where incorrect results could be returned when using UNION ALL and SELECT ... INTO S3.

Optimized partition placements when rebalancing partitions in load_balanced mode.

Fixed a permissions error where the stated required permission was incorrect.

Fixed reader quota errors that could be encountered by DELETE queries with limits that run distributed joins on derived tables.

Fixed an issue where CREATE GLOBAL TEMPORARY TABLE LIKE ... would create a normal user table instead of a global temporary table.

Fixed an issue with DATETIME equality testing when updating columnstore tables.

Added support for temporary stored procedures. For more information, see CREATE TEMPORARY PROCEDURE.

Added support for bloom filters on right joins using columnstore indexes and segment elimination. Previously, bloom filters were only supported on inner joins.

Added a new engine variable, backup_max_threads, which controls the number of concurrent threads used to run backups on leaves. By default, each backup uses one thread per partition.

Fixed an issue where queries filtering on the the transaction_state column of information_schema.PROCESSLIST or information_schema.MV_PROCESSLIST could return the wrong results.

Fixed a wrong results issue that could occur during online upgrade to version 7.3.

Fixed a crash that could occur when using a table reference like `table_name.*` inside TO_JSON

Fixed a bug on the websocket proxy where a connection could hang on the client after being killed on the server.

Fixed a crash that could occur when a query included user defined variables.

Running DDL inside of stored procedures from child aggregators is now properly supported.

Fixed a crash that previously occurred in cases where SHOW CLUSTER STATUS was run while a leaf node was unresponsive.

Fixed an issue that caused incorrect tracing to the error log when SHOW WORKLOAD MANAGEMENT STATUS was executed.

Fixed an issue with the system variable redundancy_level being incorrectly set on leaf nodes, which previously caused errors when CHECK BLOB CHECKSUM WITH REPAIR was run.

Monitoring workspaces are now supported via SSL.

Fixed an issue that previously caused an incorrect error when running LOAD DATA from inside a stored procedure.

REBALANCE ALL DATABASES and REBALANCE PARTITIONS FULL now fully rebalance redundancy 1 workspaces.

Added support for connecting to Avro Confluent Schema Registry, by specifying SSL configuration settings in CREATE PIPELINE.

Fixed an issue with automatic histograms on workspaces that were upgraded with the variable cardinality_estimation_level set to 6.0.

Fixed an issue that occurred when running ALTER or TRUNCATE against a table that included ON DUPLICATE KEY UPDATE in its CREATE PIPELINE statement.

Fixed an issue that caused read actions on columnstore JSON objects to result in an error (Error reading from parquet: Column not found in schema) in cases where trailing whitespaces were present in the JSON key names stored for a given table.

Improved the optimizer’s cost estimates for hash joins when one of the tables in a given join has an index matching the join condition.

When inserting into a columnstore table, columns not needed to deal with a duplicate key are no longer decompressed.

Fixed a crash that occurred when an aggregate function was used in an unsupported context inside a stored procedure.

Added a new engine variable optimizer_empty_tables_limit, which sets a threshold for the number of empty tables that must be present in a query before the optimizer falls back to being rule based, instead of cost based.

When running memsql_exporter, the password for the monitoring user is no longer exposed in the ps output.

Allowed memsqlctl to establish a secure connection when a node is configured for SSL using either the engine variable ssl_ca, or the host system’s trusted certificates for verification.

Increased TasksMax to 128000 for memsql.service in order to prevent errors and/or unexpected node behavior. Previously, this setting was often exceeded, causing systemd to limit the ability of memsqld to fork processes.

Fixed an issue that caused auto-rebalance operations occuring when leaf_failover_fanout is set to load_balanced not to fully rebalance the workspace.

Fixed a timeout error that occurred when running REBALANCE PARTITIONS on databases with large columnstore tables.

Improved the speed of server shutdown and of recovery for databases with large numbers of tables. This improvement is evident at 1,000 tables or more, and increases significantly after 10,000 tables.

Fixed an issue that caused RESTORE commands to fail in cases where a database contained a large number of reference tables.

Fixed an issue that caused DROP TEMPORARY TABLE IF NOT EXISTS to incorrectly return an error in cases where the table didn’t exist.

Fixed a crash caused by a check on the maximum number of partitions allowed on a leaf node. Now, an error message is surfaced when this is exceeded.

Fixed issue that caused an error (Transaction rolled back mid-query) to occur when running cross-database queries in multi-statement transactions.

Reduced blocks on UPDATE and DELETE commands via background merger.

Added a new variable, default_columnstore_table_lock_threshold, which sets a threshold for the number of rows that are locked when updating rows in a columnstore table before a table lock is acquired.

Initial GA release of SingleStore Helios 7.3.2