Maintenance Release Changelog

Fixed a server crash caused by table-valued functions parameters that assume the parameter name is the same as the alias of the table-valued function operator.

Fixed an issue with out of memory errors being generated while compiling queries that use multiple UNION clauses.

Adjusted the maximum memory allocation function to proactively run even when the total_server_memory usage is well below the maximum_memory allocation.

Fixed an issue where an incorrect error was generated for an aggregator result table when it was used by a parallel read. DROP RESULT TABLE now returns the correct error.

Improved heuristic estimates for queries with complex join predicates.

Fixed an issue with incremental autostats where duplicate rows were included in the autostats resulting in invalid statistics in a disaster recovery cluster when executing an ALTER statement in the primary cluster.

Fixed an issue with calling nested stored procedures that have different definers.

Allowed the execution of UPDATE/UPSERT command on sort key columns via pipelines. Prior to this update, an error was generated.

Fixed an issue where table-valued functions crashed when a parameter is used more than once while compiling the query.

Improved the performance of sampling queries executed by the query optimizer.

Fixed the CLEAR ORPHAN DATABASES command to ensure all databases are dropped when the leaf_failover_fanout global variable is set to load_balanced and there are partition databases with more than replica partition.

Fixed an issue with query forwarding incorrectly triggering aggregator sync causing the aggregators to run out of disk space.

Fixed an issue with incorrect nullCount and rowCount statistics for larger tables with autostats enabled.

Fixed an issue with memsql_exporter ceasing to produce monitoring results.

Fixed an issue with C# MySql client drivers getting "no database errors" on connection.

Improved rowstore garbage collection by eliminating the delay for unlimited storage uploads to occur after a snapshot.

Fixed an issue with partitions in unlimited storage databases going into an unrecoverable state while snapshots download during server restarts.

Fixed the CREATED column in the information_schema.ROUTINES table by now recording the creation date for table-valued functions.

Fixed a bug that incorrectly pushes a join before a window function when attempting to perform a distributed join in an aggregator.

Fixed a bug that a variety of conversion operators failed to check errors during the construction of the operator tree.

Fixed timeout errors when performing large BACKUP/RESTORE operations against an Azure blob store.

Fixed an issue where utf8mb4 characters were mistakenly extracted to the resulting utf8 string for the JSON_EXTRACT_STRING command.

Fixed a cluster database log corruption that could occur if the root user was dropped when sync_permissions was enabled.

Improved the predicate transitivity rewrite during query execution to cover more cases in an EXISTS/NOT EXISTS clause.

Stopped row-level locking when an UPDATE or DELETE join condition uses a full-text index.

Fixed an issue with garbage collection in unlimited storage databases that could cause a point in time restore to fail.

Fixed an issue with inconsistent snapshots caused by running SNAPSHOT DATABASE concurrently with an ALTER TABLE command against a columnstore table.

Clarified error messaging when an incorrect configuration is given while creating an unlimited storage database.

Reduced the probability of failed connections to clusters due to WebSockets proxy issues.

Added support for multipart_chunk_size_mb configuration if compatibility_mode=true. See SELECT … INTO S3 for more information on the multipart_chunk_size_mb configuration.

Fixed the calculation for total_runtime_ms for queries profiled from within stored procedures. Prior to this release, total_runtime_ms was not calculated accurately.

Improved how the engine starts the prometheus exporter.

Fixed the default value for max_async_compilation_concurrency so that it is now set appropriately when control groups are in use limiting CPU usage.

Added control group awareness for information_schema.mv_nodes which now displays the correct number of CPUs a process has access to.

Fixed an issue with queries not assigned to resource group pools with hard CPU limits from using the CPU assigned to the pool.

Fixed an issue with a crash occurring when an incremental backup was performed on an empty database.

Clarified error messaging when invalid data is inserted into a utf8 charset datatype column.

Fixed an issue with an aggregator results table checking INSERT privileges.

Fixed an issue where results data is incorrectly blank in particular columns when querying information_schema.mv_plancache.

Fixed a race condition involving concurrent UPDATE/UPSERT queries that update sort keys against columnstore tables.

Fixed an issue with a crash occurring when DDL is run inside a stored procedure that is executed within a resource pool on a child aggregator.

Block the DROP TEMPORARY TABLE command from running in a transaction that writes to the temporary table being dropped. The DROP TEMPORARY TABLE command result will display an error.

Removed the disk space check that occurred when ALTER TABLE ADD INDEX is run. The check occurred prior to altering a columnstore table in an unlimited storage database.

Fixed an intermittent "Failed to GetObject" error when restoring a non-unlimited storage database into a cluster where unlimited storage is configured.

Locked down SET clauses that attempt to update sparse columns in correlated subselects.

Added support for MAX_ERRORS clause use when using LOAD DATA into columnstore tables with unique keys.

Removed restrictions on nested SELECT queries that utilize the PARTITION_ID() function in their intermediate results.

Improved the performance of columnstore UPDATE queries that do not modify unique key columns.

Improved the performance of UPDATE queries that do not modify unique key columns against columnstore tables.

Added support for BACKUP DATABASE into S3 buckets using the sts_regional_endpoints configuration option.

Introduced mv_bottomless_summaryinformation_schema table that shows various aggregated statistics about unlimited storage databases. The mv_bottomless_summary table is an at-a-glance rollup table that shows what is happening right now with the unlimited storage cluster.

Fixed an issue with possible connection crashes on disaster recovery clusters when DDL queries are forwarded to the master aggregator and a database has not been selected.

Fixed sync timeout errors occurring when REBALANCE PARTITIONS of large databases is being processed. The REBALANCE PARTITIONS operation now waits for the replication of large snapshot files.

Added password_expiration column to information_schema.USERS table. If the password_expiration_seconds engine variable is not enabled, the password_expiration column will be NULL. If the password_expiration_seconds engine variable is enabled, the password_expiration column will display the number of seconds remaining for the password to expire.

Updated time zone data for some countries that recently changed their time zone rules.

Fixed a crash caused by a check on the maximum number of partitions allowed on a leaf node. An error message is displayed when this threshold is exceeded.

Added support for allowing spilling to disk of hash join output row.

Added support for running PROFILE on queries that execute inside of transactions the include multiple statements.

Fixed an issue where a crash could occur while attempting to compile queries for UPDATE statements with multiple columns due to insufficient memory resources. The amount of memory necessary to compile these query shapes was reduced.

Fixed a race condition involving concurrent UPDATE/UPSERT queries that update sort keys against columnstore tables.

Fixed an issue in the join optimizer that incorrectly eliminated candidate plans from the search.

Fixed a write amplification issue associated with row versioning. The behavior is disabled if the database is not unlimited storage.

Added support for MySQLDump 8.0.

Fixed a server startup failure when LD_PRELOAD is set. The behavior now reverts to a previous version which would overwrite any existing LD_PRELOAD setting.

Fixed an issue where the memsql_exporter would sometimes produce invalid JSON errors. The memsql_exporter is a satellite process that collects data about a running cluster.

Reduced the memory used to download blob files from remote storage.

Fixed a regression in recovery performance for databases using remote storage.

Fixed a crash that could occur when using LINK syntax with Avro pipelines.

Added Linux dependencies for SASL to SingleStore installation packages so they do not need to be installed separately.

Masked blob store CREDENTIALS text in CREATE DATABASE command for unlimited storage in SHOW PROCESSLIST, information_schema.PROCESSLIST, and all error messages.

Removed the 16384 size limit on the Avro array and map sizes in pipelines and LOAD DATA when pipelines aren't utilized.

Masked blob store CREDENTIALS text in BACKUP, RESTORE, and LINK commands in SHOW PROCESSLIST, information_schema.PROCESSLIST, and all error messages.

Increased incremental blob file streaming to disk to reduce memory overhead for unlimited storage.

Added support for incremental backups to utilize LINK syntax to specify CONFIG and CREDENTIALS clauses.

Added support for READ ONLY or READ WRITE syntax to the SET TRANSACTION statement for MySQL compatibility.

Added support for optimizing cases of IN joins in subselects in the project list.

Added session variable named subquery_merge_with_outer_joins that fine-tunes the behavior of the rewrite SubQueryMerge. If the variable is set to ON: always merge the sub-select with outer join(s). If the variable is set to HEURISTIC: check to see if merging the sub-select with outer join(s) will cause negative performance for the overall query. If the variable is set to OFF: don’t merge sub-selects with outer join(s). If the variable is set to AUTO (this is the default): For SingleStore 7.5 and previous versions, AUTO will behave like ON. For SingleStore versions after 7.5, AUTO will behave like HEURISTIC.

Fixed auto histograms to determine when to build new histograms based on both master and child aggregators.

Fixed an aggregator crash that was caused when a malformed INIT_DB command is sent by a client.

Added support for complex joins (joins with non-linear/nested join tree) involving correlated nested SELECT queries.

Added an new engine variable named rewrite_verbose to output tracings of the rewrite phase on the current query in the tracelogs.

Fixed warning indicating "partition metadata is out of sync on table", which could appear in the tracelog after a node restart.

Added support for SKIP CONSTRAINT ERRORS options for Avro and JSON pipelines.

Increased the timeout used when connecting to AWS to determine credentials if none are specified in a BACKUP/RESTORE command.  Also added a new engine variable subprocess_ec2_metadata_timeout_ms which allows users to set that timeout.

New Feature: Added support for alias references in WHERE clauses and as column expressions. See SELECT for more information.

Fixed an issue where Disaster Recovery of a non-unlimited storage database was not blocked if a cluster was configured to use remote storage.

The AGGREGATOR SYNC AUTO_INCREMENT command now can be performed by anyone with the ability to update the target database(s) or table in any way.

Fixed an issue that could cause defunct processes for pipelines or code generation in a rare race condition.

Increased batch deletions from unlimited storage to improve the performance of GCing files from remote storage.

Improved JSON built-in functions to better handle out of memory errors.

Improved the cardinality estimates for columns with an equality predicate.

Fixed per-predicate PROFILE statistics on columnstore filter operations.

Fixed a crash that could occur when running CREATE PIPELINE or REPLACE PIPELINE on a pipeline that already exists.

Removed truncation of the query text in query.log when general_log=on.

Fixed a crash that could occur when ALTER TABLE COMPRESSION=SPARSE is run in the same ALTER statement with an operation that doesn’t require data movement (ie., AUTOSTATS_ENABLED = ON).

New Feature: Added anonymous code blocks which are unnamed blocks of SingleStore (MemSQL) Procedural SQL (MPSQL) code that are executed immediately. They allow you to create a code block without defining and calling a stored procedure and are useful in situations such as testing or debugging.

Added a new system variable called parametrizer_query_max_params which allows users to set the maximum number of scalar constants a single query can contain. Any query with more than parametrizer_query_max_params will be failed with an error. The default value is 1048576.

Fixed an issue where the exporter process was not being cleaned up when memsqld is shut down.

Fixed an issue where modifying the case of a column via ALTER TABLE could cause a pipeline using that table to crash.

Improved usability of various JSON-builtins to support utf8mb4 charset arguments.

Increased frequency of merging during load for unique keys on columnstores.

Added information_schema.mv_bt which shows backtrace information for all nodes in the cluster.

Added information_schema.mv_plancache which shows plancache information for all nodes in the cluster (cluster-wide version of information_schema.plancache).

Added support for the transaction_read_only and tx_read_only session variables for MySQL compatible client drivers that query them.

Fixed a performance regression establishing SSL connections introduced in 7.5.

Fixed a deadlock which can occur if a failover happens during an ALTER TABLE.

Changed the SET PASSWORD statement to query forward from the child aggregator to the master aggregator.

Initial GA release of version 7.5.5

Stability improvements to version 7.5.4

Fixed an "invalid LSN" error when restoring a local storage database into a remote storage database.

Fixed an issue that could cause DETACH DATABASE to hang while a pipeline that is attempting to load batches is failing.

Fixed an issue that could generate incorrect out of memory errors when using a remote storage database.

Beta release of SingleStore DB 7.5