Maintenance Release Changelog

Fixed unlimited storage S3 multipart uploads to properly retry on 200 responses which contain an embedded "SlowDown" error, instead of failing immediately.

Reduced memory usage in situations where tables are repeatedly created and dropped.

Fixed an issue where memory accumulates over time if reshuffle or repartition operators are executed for many unique query shapes.

Introduced a new global engine variablebackup_multipart_upload_concurrency that can be used to improve blobstore backup performance. The default value (3) can improve performance out of the box.

Improved the performance of a left join of a reference table with a sharded table by pushing the reference table clauses to the leaves. The engine variable disable_reference_table_leftjoin_where_pushdown must be set to "OFF" to enable this operation.

Fixed the query_text output in profiles generated while the profiled query is executing.

Fixed an issue where the query optimizer could choose incorrectly between using the HashGroupBy or the StreamingGroupBy operators.

Fixed an issue where JSON computed column matching incorrectly triggers a JSON_EXTRACT_<type> function.

Fixed a STR_TO_DATE function bug when used with '%T' or '%r' in the format string.

Fixed a rare log file corruption issue with sync durability in unlimited storage databases.

Fixed an issue with handling a query that contains a common table expression (CTE) in a CREATE VIEW statement that has multiple UNION operators.

Added cross-database support for Recursive CTEs.

Fixed a bug where DEFAULT NOW/UUID/SYS_GUID is sometimes ignored when specified on a column in the primary key.

Fixed an issue where JSON_INCLUDE_MASK built-in returns a document with all fields, instead of zero fields for an empty mask '{}' case.

Added ability to JSON_MATCH_ANY() to receive the columnstore optimization even if the entire JSON column is projected in the query.

Fixed a crash caused by reusing plancache for a deleted link.

Fixed an issue with DATETIME columns with ON UPDATE CURRENT_TIMESTAMP defaults that could cause incorrect results.

The sql_mode session variable now shows up in SHOW VARIABLES and the mv_global_variables view.

Fixed an issue that could cause an incorrect error message to be generated on an out of memory error.

Fixed an issue where a NULL could be pushed down to a union which could cause the type-cast/collation to be lost, resulting in an error: "Operation 'Scanning a hash join build' is not allowed".

Fixed an issue where errors could occur during backup when large files are copied.

Fixed an issue where S3 pipelines could stop loading data from the bucket if files are deleted from the bucket while the pipeline is active.

Fixed an issue where, in rare cases, an UPDATE statement with a left join in the filter clauses could fail to execute.

Fixed an issue where replica blobs could be loaded into the blob cache with an incorrect LRU timestamp.

Improvements to memsql_exporter: Improved error handling and reduced memory usage.

Fixed an issue with ALTER USER, and now the following syntax works as expected:

ALTER USER user IDENTIFIED BY 'password';

Added  JSON_INCLUDE/EXCLUDE_MASK function when applied to a JSON document; it will return a subset of the document based on the mask.

JSON_EXCLUDE_MASK(<json>,<mask>);

JSON_INCLUDE_MASK(<json>,<mask>);

Added support for LOAD DATA from S3 for Avro and Parquet data.

Fixed a potential crash issue in UPDATE queries that involve joins and that have scalar subselects in SET clauses.

Fixed an issue where running DROP PLAN FROM PLANCACHE ... FOR QUERY on a query with invalid syntax could cause a crash.

Updated timezone metadata to include Mexico's latest timezone change.

Fixed an issue where DETACH DATABASE was not killing long running queries that were blocking the database from getting detached.

Fixed an issue with missing row counts during optimization when sampling is disabled.

Fixed an issue where background mergers were not stopping quickly enough, resulting in delayed DDL operations or REBALANCE commands.

Fixed an issue with setting maximum_blob_cache_size_percent on a single site that could cause the blob cache to shrink incorrectly.

Fixed an issue where CLEAR ORPHAN DATABASE could cause data loss if run when the master aggregator has a database in an unrecoverable state.

Fixed an issue in information schema view JSON_COLUMN_SCHEMA where incorrect details were being shown for the leaf columns.

Fixed an issue where the memory used by an external function was not being freed quickly enough.

Fixed a column name resolution issue for recursive CTEs when the column type is the same across the base branch and the recursive branch.

The default_partitions_per_leaf global variable will no longer be user-settable in SingleStoreDB Cloud, ensuring that it's always defined for the most optimal performance, according to the current resource size.

Fixed an issue where filters using the range_stats column in the information_schema.optimizer_statistics table were not allowed.

Fixed an issue where incorrect results could be returned when segment-level flexible parallelism is used inside a subquery containing both a shard key filter and either an aggregate, a window function, or the limit clause.

Fixed an issue where an internal result table was not created for a recursive CTE involving data across databases.

Fixed an error that could occur when attaching databases with a snapshot file of greater than 5 GB.

Fixed an issue where too many rows are sampled from columnstore tables with more than 1 billion rows.

Fixed an issue with histogram estimation in columns with a negative value TIME type.

Fixed an error caused by setting the collation_server global variable to a non-default value when performing a REBALANCE PARTITIONS resource availability check.

Fixed "table partition count mismatch errors" that occur due to the following conditions: the system variable enable_workspace_ha is set and there is an upgraded workspace with an attached read replica.

Fixed an issue with DDL endpoint queries using a lower-than-specified query parallelism setting when workspaces are enabled.

Improved column type resolution for base and recursive branches in recursive common table expressions (CTEs).

Fixed an issue that prevents GROUP BY Push-Down optimization if the join filter contains a mismatched column type.

Fixed a data conversion compatibility level 8.0 error that may occur when sampling columnstore tables.

Fixed a possible deadlock that may occur between the blob cache and the Rowstore Garbage Collection (GC) when the blob cache encounters an out-of-memory error.

Add an information schema table (JSON_COLUMN_SCHEMA) which shows the schema inferred for JSON columns in columnstore tables.

Improved the parsing performance of queries that contain several tables.

Improved error handling when restoring from an incomplete incremental backup.

Improved the performance of S3 pipelines when Garbage Collection (GC) is enabled.

The  read_advanced_counters, snapshot_trigger_size, and  snapshot_to_keep  engine variables can now be set on SingleStoreDB Cloud.

Added new session variable disable_remove_redundant_gby_rewrite to prevent the GROUP BY columns from being removed when used in an ORDER BY clause.

Added syntax to allow multiple leaf nodes to be detached using the DETACH LEAF command.

Introduced disk and memory availability checks that run before a database is allowed to be attached to a workspace.

Added the ability to cache histogram results during optimization to reduce the work performed by the histograms.

Added the ability to backup a database to an HTTPS S3 target with an unverified SSL certificate when using the option: verify_ssl: false.

Fixed an issue with column type checking on base and recursive case of a recursive common table expression.

Fixed an issue that may cause a "Table doesn't exist" error when a multi-insert contains expressions and the target table has a computed column as its shard key.

Expanded existing Unicode characters to support Private Use Area (PUA) code points. Including one in the Basic Multilingual Plane (U+E000–U+F8FF) and one in each plane 15 and 16 (U+F0000–U+FFFFD, U+100000–U+10FFFD).

Fixed a crash that could occur when a computed column definition refers to a table name.

Increased default blob cache size on hosts with less than 200GB of total disk space.

Added the ability to set the maximum_blob_cache_size_percent global variable for workspaces.

Fixed an issue with promote lock timeout errors that may occur during a rebalance due to a heavy ingest workload, which causes the merger to be slow to pause.

Introduced a new global variable subprocess_max_retries, which is used for retrying on retry-able connection failures during select into/backup queries for S3 and GCS.

New feature: ORDER BY SELF JOIN, creates a self join on ORDER BY LIMIT queries to take advantage of differences in bandwidth.

Fixed an issue with the schema registry so only superusers can see redacted pipeline metadata such as credentials and URLs.

Background snapshots are now allowed to run during BACKUP DATABASE commands. This prevents increased disk usage by logs during a long-running backup.

The CREATE_TIME, CREATE_USER, ALTER_TIME, and ALTER_USER columns in the information_schema.TABLESview are now properly set for views and TVFs (table-valued functions).

Fixed an issue that occurred when the MATCH_PARAM_<type> argument of the JSON_MATCH_ANYfunction was not in a predicate expression.

Improved the performance of the JSON_MATCH_ANY_EXISTSfunction over columnstore tables.

Fixed a profiling issue specific to non-collocated hash joins where the memory usage and disk spilling are missing under the join operators.

Fixed an issue where the database engine locks up on certain out-of-memory errors.

Fixed an issue where the audit logs now show the original user of a query forwarded to the master aggregator instead of the internal distributed user.

Fixed a parsing issue for queries containing multi-line comments near GROUP BY or ORDER BY clauses.

The num_background_merger_threadsengine variable is now settable on Cloud.

The ORDER BY ALL [DESC|ASC] (or ORDER BY *) syntax is now supported. 

The GROUP BY ALL [DESC|ASC] (or GROUP BY *) syntax is now supported. 

Improved the query execution performance of JSON columns under a higher level of parallelism.

Expanded support for encoded GROUP BY query shapes containing expressions in aggregates.

Added support for incremental backups to blob stores where a single partition contains more than 5 TB of data.

Added the REVERSE() built-in string function that reverses the target string.

Added the DATETIME_PRECISION column to both PARAMETER and ROUTINESinformation_schema views. Also, the DATETIME_PRECISION column will include TIME and TIMESTAMP data types in the COLUMNSinformation_schema view.

Fixed an issue causing incorrect trace messages in master logs where clocks were incorrectly advancing from "0".

Added a new information_schema view named LMV_LOCAL_DATABASES. This view shows the state of local databases like SHOW DATABASES EXTENDED, but it can be queried against unlike show commands.

Improved the code generation performance of tables with a large number of indexes.

Added the /api/v2/jwks_setup endpoint to Data API to allow users to enable JWT Auth in Data API on Cloud. See jwks_setup for more information.

Sampling will no longer be used for table size estimation when statistics are present.

Fixed an issue where extra CPU was used when a read-only database is attached to a workspace without any writable mount for the read-only database.

The spilling_minimal_disk_spaceengine variable can now be set at startup.

Fixed an issue with the master aggregator leaking memory while performing a background task.

Fixed the case where the REGEXP_REPLACE and REGEXP_SUBSTR expressions can produce non-utf8 strings by introducing the new engine variableregexp_output_validation_mode. Regular expression built-ins can produce non-utf8 strings because they don't have full support for multi-byte characters. The engine variable controls this behavior if regular expression built-ins return strings that are invalid under its collation settings.

Improved the performance on columnstore scans that perform multiple JSON extraction operations on the same JSON column.

Added a new OPTIMIZE TABLE <table_name> INDEX; command for columnstore tables. This command runs the optimization routine for columnstore secondary indexes manually.

Fixed an issue where recursive CTEs could fail to execute on upgrade to 8.0 if materialized CTEs were disabled.

Added the ignore_foreign_keys system variable, which allows foreign key syntax in CREATE TABLE commands, but completely ignores the key (it will not show up in metadata).

Allow modifying the json_extract_string_collation engine variable as long as a computed column that modifies the shard key does not contain a call to JSON_EXTRACT_STRING.

Added a new option, SERVER_V2, to the json_extract_string_collationengine variable. This new, recommended option is the default for new clusters, and allows comparison of utf8mb4 strings extracted from JSON to utf8 string constants. Existing clusters will retain their original setting upon upgrade.

Fixed an issue with databases becoming unrecoverable when upgrading to version 8.0 if any tables contained shard keys using computed columns that call JSON_EXTRACT_STRING.

Added the option to authenticate Data API requests using JWT.

Added the skip_segelim_with_inlist_threshold engine variable, which controls when segment elimination will not use an IN list that is too large (default 1000 elements).

Fixed a performance regression when running SET <system variable> = <expression> commands. These commands are often run by client drivers on new connections.

Improved the performance of various commands (SHOW commands, DDL, etc.) when there are very many views or tables in the database (100s of thousands).

Added CREATE_TIME and ALTER_TIME columns to information_schema.pipelines. CREATE_TIME shows the date/time a pipeline was created or recreated. ALTER_TIME shows the date/time a pipeline was altered via an ALTER PIPELINE statement.

Fixed an issue introduced in version 8.0.4 where SHOW TABLE STATUS was incorrectly displaying views.

Added a new global variable, num_background_merger_threads, which controls the number of background merger threads to start for each node.

information_schema.table_statistics now correctly shows information about global temporary tables.

Added support to PROFILE for hash join spilling.

Fixed an upgrade issue where a database that was originally created on SingleStoreDB 6.8 or earlier and was then upgraded to 7.3 is then REPLICATED to SingleStoreDB version 7.8 could result in broken shard keys.

Added support for Recursive CTEs in cases where a Recursive CTE has dependency on a streaming result table outside of its dependency cycle.

Fixed an issue with Recursive CTEs where one Recursive CTE refers to another Recursive CTE and that second CTE would only read the rows from the first CTE’s final iteration.

The SHOW STATUS EXTENDED command contains a new "Gv_clock" key whose value is the current logical clock of the server.

The SHOW DATABASE STATUS command contains a new "gv_clock" key whose value is the current logical clock of the server.

Fixed an issue with aggregate functions using incorrect enum/set types that may result in inaccurate output in the operator tree.

Fixed an edge case issue which could cause the engine to hang during shutdown.

Improved performance of comparing utf8mb4 strings.

Added a new information schema view internal_table_statistics which shows memory use of SingleStoreDB internal metadata tables. The columns displayed are the same as those shown for table_statistics.

Fixed an edge case issue that could cause ADD AGGREGATOR to fail.

Reduced the memory used by ANALYZE queries by splitting approx count queries to only query a single JSON column at a time.

Improved the accuracy of network_time in PROFILE output for some query shapes.

The blob cache is now allowed to use more disk space by default on larger local disks. It will now use up to 90% of local disk space on disks over 2 TB in size instead of 75%.

Added a new global variable, maximum_blob_cache_size_percent, which can set the blob cache size as a value from 0 to 1 that is percentage of local disk the blob cache is allowed to use. The default value is 0.

Added a new global variable, num_background_merger_threads, which controls the number of background merger threads to start for each node. The default value is 2.

Fixed a MySQL protocol incompatibility where SingleStore was not properly sending back the SERVER_STATUS_IN_TRANS flag. This was impacting some MySQL php drivers.