Maintenance Release Changelog

2024-03-14 Version 8.1.38

  • Bugfix: Fixed a regression with sparse JSON when use_seekable_json is disabled.

2024-03-05 Version 8.1.37

  • Enhancement: Added numInferredPaths and schemaSize as new columns to JSON_COLUMN_SCHEMA table. numInferredPaths is the number of key paths inferred for the segment. schemaSize is the size of schema_json in bytes.

  • Bugfix: Added safety checks around dropping databases without exclusive access to storage.

  • Enhancement: Added spilling metrics for TopSort (ORDER BY with LIMIT).

  • Bugfix: The existing websocket connection will be closed when variables are updated in global scope.

  • Bugfix: Removed the hardcoded 'collation_server' from constant.StrictModeParams.

  • Enhancement: Added new metrics to memsql_exporter based on information_schema.mv_sysinfo_disk columns.

  • Bugfix: Fixed a potential data leak in out-of-memory reports.

  • Bugfix: Fixed an issue in backup restore when the backup has a corrupted GV timestamp in the about snapshot record.

  • Bugfix: Fixed an issue with exporter not working with Public-Key Cryptography Standards #8(PKCS#8) pem format.

  • Bugfix: Fixed an issue with hex() not handling decimal types correctly.

  • Bugfix: Fixed a crash that occurred when trying to grant access to the table with no context database in per-privilege mode.

  • Enhancement: Added logging to heartbeat thread in order to better diagnose process stalls. This logging is enabled/disabled via the new verbose_time_heartbeat_thread engine variable. The default value is off, setting this to on will enable the logging.

  • Bugfix: Fixed an issue in backup restore when the backup has a corrupted GV timestamp in the about snapshot record.

  • Bugfix: Addressed a family of issues relating to optimal execution of JSON extracts on schema'd data.

2024-02-14 Version 8.1.36

  • Enhancement: The new engine variable parametrize_json_keys allows JSON keys to be parametrized and plans can be reused.

  • Bugfix: Fixed the change to aggregator activity queries in memsql_exporter not being applied properly.

  • Enhancement: Reduced amount of memory used by unlimited storage file listing, including point-in-time recovery (PITR).

  • Enhancement: Added an engine variable, innodb_lock_wait_timeout. This variable exists for backwards compatibility with MySQL and is non-operational in SingleStore Helios.

  • Enhancement: Backups now throttle after the snapshot.

  • BugFix: Fixed an issue where a backup would carry metadata about the retention period of the backed up unlimited storage database.

2024-02-06 Version 8.1.35

  • Enhancement: Added ability to memsql_exporter to always return all sub-activities of an aggregator activity.

  • Enhancement: Improved ability to configure librdkafka through credentials.

  • Enhancement: Added the column name to the error message when an invalid date/time is inserted into a column.

  • Enhancement: The new engine variable max_table_memory_room_mb sets the maximum amount of memory required when creating or attaching a database. Configuring this engine variable allows more control over whether a detached database can be reattached.

  • Enhancement: Increased the maximum value to 15 for the backup_multipart_upload_concurrency engine variable.

  • Bugfix: Fixed a segmentation fault (crash) from appearing after terminating a query on the remote node.

  • Bugfix: Fixed the out of memory issue caused by operators generated for wide user tables during the optimization phase.

  • Bugfix: Fixed an issue where the incorrect number of rows would be shown when performing a select into object storage.

  • Bugfix: Fixed an issue with FROM_BASE64 and TO_BASE64 built-ins when processing large string inputs, preventing potential errors in reading communication packets.

  • Bugfix: Fixed an issue with START PIPELINE FOREGROUND skipping files in some circumstances.

2024-01-24 Version 8.1.34

  • Enhancement: Improved error messages when downloading backups.

  • Enhancement: Added the blob_cache_miss_b and blob_cache_wait_time_ms columns to the following information schema views: information_schema.plancache, information_schema.MV_ACTIVITIES, information_schema.MV_ACTIVITIES_CUMULATIVE, information_schema.MV_TASKS, information_schema.MV_FINISHED_TASKS, and information_schema.MV_QUERY_ACTIVITIES_EXTENDED_CUMULATIVE.

  • Bugfix: Fixed an issue where INSERT...SELECT queries with a partition_id() filter generating an error.

  • Bugfix: Fixed an issue with memory crashing when using REGEXP_MATCH, JSON_INCLUDE_MASK, or JSON_EXCLUDE_MASK built-ins.

  • Enhancement: Improved performance for JSON_EXTRACT_<type> built-ins in ORDER BY clauses.

  • Enhancement: Added the ability for users to kill queries on a remote node without the user having PROCESS privileges.

  • Bugfix: Fixed an allocation issue that caused poor performance on high load insertion queries.

  • Bugfix: Fixed an issue with memsql_exporter working improperly in ANSI_QUOTES sql_mode.

  • Enhancement: ALTER PIPELINE setting max_partitions_per_batch to use a default 0 value is now allowed.

  • Bugfix: JSON_KEY escape characters were not working as expected.

  • Enhancement: Added the bottomless_experimental_blobstore_mode engine variable, which when enabled, completes additional verification of persisted files immediately after upload. This mode is experimental and may reduce upload speed. Please use with caution.

  • Enhancement: Introduced blob cache profiling metrics for columnstore tables on unlimited storage databases. Refer to the PROFILE page for more information on what blob cache metrics are available.

  • Bugfix: Fixed a bug which corrupts databases whose log size is not a power of two pages.

2024-01-03 Version 8.1.32

  • Enhancement: Updated the information_schema.USERS view to reflect the account status for locked users.

  • Bugfix: Fixed a crash that occurred when user-defined variables are used in the GROUP BY and ORDER BY clauses.

  • Bugfix: Fixed an issue where filtering with a JSON_EXTRACT_<type> function performs inconsistently.

  • Bugfix: Fixed a network communication error that occurred after query rewrites.

  • Bugfix: Fixed an issue where JSON computed column matching incorrectly triggers a JSON_EXTRACT_<type> function.

  • Bugfix: Modified computed column matching to accurately evaluate JSON expressions containing equality and non-safe equality comparisons.

  • Bugfix: Fixed an error that occurred when a user attempts to access a view that is based on a table that has no privileges granted on it.

  • Enhancement: Improved the ability to kill queries containing JSON built-in functions.

  • Enhancement: The query optimizer now considers more LEFT JOINelimination cases.

  • Enhancement: Introduced a new session engine variable optimizer_disable_transitive_predicates which disables predicate transitivity on query rewrites.

  • Enhancement: Added more log messages for backup sub-process errors.

2023-12-5 Version 8.1.31

  • Added support for creating a DEEP COPY of tables with computed columns. (8.1.31)

  • Added additional config and credential option validation while creating pipelines. (8.1.31)

  • Fixed an issue where valid LINK config and credential parameters were not supported for both reading from and writing to a datasource.

2023-11-28 Version 8.1.30

  • Fixed an issue where the number of rows affected with SELECT INTO object storage was erroneously reported as 0 rows.

  • Fixed an issue where small tables use an inappropriately small sample size by setting a lower bound. This mostly affects reference tables.

2023-11-21 Version 8.1.29

  • Fixed a potential crash that could occur when LOAD DATA is used without a database context.

  • Pipeline state is now shown consistently across primary and child nodes.

  • Fixed an issue with the SET clause of the CREATE PIPELINE statement: now it is possible to use type options like NULL/NOT NULL/COLLATE... with :>/!:> typecast operators.

  • Added a new engine variable pipelines_cdc_java_heap_size to specify the JVM heap size limit for CDC-in pipelines.

2023-11-13 Version 8.1.28

  • Fixed an issue with JSON_SET_JSON used in UDFs.

  • Fixed a potential issue related to hash join spilling.

  • Reduced blocking in query compilation after evicting many (10K+) plans in the plancache.

2023-11-07 Version 8.1.27

  • Added three new fields to MV_SYSINFO_DISK:

    • READ_OPERATIONS_CUMULATIVE_PER_DEVICE: Number of read operations performed by the device since start up.

    • WRITE_OPERATIONS_CUMULATIVE_PER_DEVICE: Number of write operations performed by the device since start up.

    • DEVICE_NAME: Name of the device to which the values in read_operations_cumulative_per_device and write_operations_cumulative_per_device are associated.

  • Disabled order_by_self_join rewrite when an ORDER BY contains an indeterminate function.

  • Disabled group-by on encoded data when there is an aggregate on full-text match.

  • Fixed a potential security issue with users created without a password.

  • Fixed some cases where APPROX_COUNT_DISTINCT_ACCUMULATE erroneously included nulls in the count.

  • Fixed an issue preventing correct ingestion from MongoDB® collections with dot symbols.

  • Fixed an issue in CDC pipelines that could lead to Java Out Of Memory errors.

  • Improved performance during snapshotting for CDC-in pipelines.

  • Changed the SingleStore Helios workspace default and range for the pipelines_cdc_row_emit_delay_us engine variable. Throttling default value is set to 1. The supported range is from 0 to 1000000.

  • Addressed some performance issues that occur when performing large sets of security operations (creating a lot of groups/users/roles, etc.) via the new privilege_caches_update_mode engine variable.

    This variable can be used to temporarily disable updating privilege caches while performing security operations such as: granting/revoking permissions to/from users, granting/revoking permissions to/from roles, adding/deleting roles to/from groups, adding/deleting users to/from groups, role deletion, group deletion. This can result in a performance gain.

    IT IS IMPORTANT that the privilege cache updates be reenabled (value set back to always) once the administration work is finished.

    To use this variable:

    1. Set the value to never.

    2. Perform the operations.

    3. Set the value back to always.

  • Fixed a crash caused when AS WASM/WITH WIT is used in a CREATE AGGREGATE statement that (1) doesn't have HANDLE as the state type, and (2) doesn't supply the SERIALIZE and DESERIALIZE callbacks.

  • Fixed an issue where error messages were not generated when corrupt blob was encountered during json encoding read/write.

  • Added additional node metrics to the /cluster-metrics endpoint of the memsql_exporter.

2023-10-17 Version 8.1.26

  • Reverted the ability to use SSL keys with a password in the HTTP API.

2023-10-11 Version 8.1.25

  • Fixed an upgrade issue where some databases could temporarily become unrecoverable if snapshots were skipped during a pre-upgrade on a recently attached unlimited database.

  • Fixed an issue preventing nodes from attaching during upgrade.

  • Added ability to use SSL keys with a password in the HTTP API.

  • Added support for simple multi-column update with sub-query.

2023-10-03 Version 8.1.24

Warning

Versions 8.1.24 and newer of SingleStore are not forward compatible with any of the previous 8.1 maintenance releases, meaning that a backup from 8.1.24 or newer cannot be restored on any 8.1 release prior to 8.1.24.

  • Added resource pool metrics to the /cluster-metrics endpoint of the memsql_exporter.

  • Added ability to infer CSV data with text boolean values.

  • Fixed background pipelines not respecting max concurrency limit in the resource pool. Please note that this change will affect previously created pipelines using a constrained resource pool, including the error state if MAX_QUEUE_DEPTH is exceeded.

  • Fixed an issue with two-phase commit that could lead to loss of atomicity on databases restored from 8.1 backups.

  • Fixed an issue with the transaction log not being downgradable to previous 8.1 patch releases.

  • Fixed an issue where two-phase commit would not work correctly on disaster recovery failovers.

  • Improved the performance of mv_replication_status for clusters with large DR databases.

2023-09-25 Version 8.1.23

  • Improved blob cache sync performance when there is no running query.

  • Added support for flipping join order for FULL OUTER JOIN operations.

2023-09-19 Version 8.1.22

  • Fixed an issue where unlimited storage databases stored on Amazon S3 could run out of connections under intense loads.

2023-09-15 Version 8.1.21

  • Fixed a bug where leaf failover and ATTACH LEAF would copy extra partitions.

  • Fixed an issue where the rebalance done by ATTACH LEAF was unstable with respect to REBALANCE ALL DATABASES.

  • Fixed an issue where an incorrect result is produced when the engine variable query_parallelism is set with flexible_parallelism. query_parallelism is deprecated and query_parallelism_per_leaf_core should be used instead.

  • Removed an unexpected warning related to the engine variable query_parallelism (which is deprecated) when a query contains an ORDER BY clause with an IN list filter that matches the table's shard key.

  • The START TRANSACTION command is no longer allowed from Data API unless it is inside a stored procedure.

  • Now redirecting the AWS-GO-SDK log to a file instead of writing to stdout.

  • Fixed a bug where an aggregator result table hits a lockdown when sql_select_limit is set.

  • Added two engine variables: max_expression_query_limit which sets a limit on the number of expressions within an entire query and max_expression_item_limit which sets a limit on the number of expressions within a query item. Both can be set to a range between 100 and the maximum unsigned INT value. Setting these engine variables to the maximum unsigned INT value disables both features.

  • Fixed an issue where a crash occurs where the engine improperly rewrites queries with a UNION in an EXCEPT clause.

2023-09-05 Version 8.1.20

  • Fixed several issues causing slow compilation for queries over wide tables.

  • Added two engine variables, jwks_username_field and jwks_require_audience to add flexibility and improve security.

  • Added support for materializing CTEs without recomputing them when the query contains UNION, UNION ALL, and other SET operations. To enable the feature, set the engine variable allow_materialize_cte_with_union to TRUE. The default value for this engine variable is FALSE for version 8.1.19 and earlier.

  • Fixed a memory leak issue in the memsql_exporter process.

  • Enhanced the performance of DDL statements for role manipulation.

2023-08-28 Version 8.1.19

  • The SHOW COLUMNAR MERGE STATUS command now outputs status for tables with empty sort keys.

  • The following commands now respect non-default columnstore_table_lock_threshold values: INSERT ... SELECT ... ON DUPLICATE UPDATE, REPLACE ... SELECT, and REPLACE ... LOAD.

  • Fixed an issue where VECTOR_MUL was returning incorrect results on shard tables.

  • Fixed an issue where memsql_exporter was generating too much "pipeline_error" data.

  • Fixed an issue with the Kerberos cache for HDFS pipelines.

  • Improved memory consumption in json decoding.

  • Fixed an issue where LIMIT and OFFSET after a window function with no PARTITION BY clause were returning incorrect information.

  • The WARM BLOB CACHE FOR INDEX command can now accept index names that conflict with reserved identifiers (such as PRIMARY, etc.).

2023-08-23 Version 8.1.18

  • Added support for the REGEXP_MATCH() function. This function returns a JSON array of matching substring(s) within the first match of a regular expression pattern to a string.

  • Improved performance of multi-part GCS backups.

  • Introduced a new global engine variable json_document_max_leaves which limits the number of JSON key paths inferred within a segment. The default value is 10000.

2023-08-09 Version 8.1.17

  • Fixed an issue where BACKUP or RESTORE of an existing backup from GCS fails on SingleStore versions newer than 8.1.6.

  • Fixed an issue where BACKUP DATABASE WITH INIT would fail under out-of-memory conditions.

  • Fixed a potential crash in NFS backup when encountering an IO error.

2023-08-08 Version 8.1.16

  • Added support for multiple uncorrelated IN-subselects in more query shapes.

  • SKIP PARSER ERRORS is now supported for Kafka. Additionally, a new related engine variable, pipelines_parse_errors_threshold, has been added.

  • Fixed an issue with the incremental restore of an unlimited database that would fail if the restore was larger than the disk.

  • SingleStore automatically rewrites A=B OR (A IS NULL AND B IS NULL) to null safe equal (A<=>B) to enable hash joins.

  • Added support for hash joins on null-accepting expressions in the ON condition of outer joins.

  • Fixed an issue with significant memory reduction from spilling for hash join queries with variable length strings (varchar, text, etc.) involved.

2023-08-02 Version 8.1.15

  • Introduced a new global variable, dr_min_connection_timeout_ms, which allows users to adjust the minimum timeout period in Disaster Recovery (DR) replication.

2023-07-27 Version 8.1.14

  • Fixed a bug that could cause Amazon S3 pipelines to stop loading data.

2023-07-26 Version 8.1.13

  • Added support for Control Group v2 (cgroup v2).

  • Fixed an issue where a large spike in the query memory consumption on a any SingleStore node could cause replicas on the same node to become unrecoverable.

  • Fixed a pipeline wrong state issue caused by an error in a table with a computed column.

  • Fixed a potential issue where REBALANCE PARTITION would not stabilize to a partition placement on a read replica's workspace.

  • Fixed a potential crash in some edge cases when using Parquet pipelines.

2023-07-18 Version 8.1.12

Warning

Version 8.1.12 of SingleStore is not forward compatible with any of the 8.1 maintenance releases, meaning that a backup from 8.1.12 cannot be restored on any previous 8.1 release.

  • Fixed an issue with upgrade steps not running in some edge cases. A logging change was required, making the log format incompatible with earlier patch releases.

  • Fixed a crash that may occur when using SELECT … FOR UPDATE … LIMIT <n> in a multi-statement transaction when two_phase_commit is ON.

  • Fixed a bug where the permissions do not clear for Data Manipulation Language (DML) queries and then leak to subsequent Data Definition Language (DDL) queries.

  • Fixed a bug where the mv_disk_usage table would show incorrect results for the plancache directory.

  • Default values for BLOB/TEXT and JSON columns are allowed, as well as NULL and empty strings. Fixed a bug where utf8mb4 characters were not allowed inside default values for CHAR/VARCHAR columns.

    CREATE TABLE table_1(col_1 JSON DEFAULT {'amt': 10});
    CREATE TABLE table_2(col_1 BLOB DEFAULT ('xyz'));
    CREATE TABLE table_2(col_1 longtext DEFAULT '{...}');
  • Fixed a potential issue where changing a user password would not change the connection pool password in the Data API.

  • Reverted the performance improvements for backups to GCS (Google Cloud Storage).

2023-07-10 Version 8.1.11

  • Bugfix: Fixed a crash that may occur when a function calls more than one external function.

    In the following example get_embedding() is an external function:

    SELECT DOT_PRODUCT(JSON_ARRAY_PACK(get_embedding('house')),
    JSON_ARRAY_PACK(get_embedding('car'))
  • Added an opt-in optimization to Kerberos for HDFS pipelines to reduce the amount of requests.

  • Added a LimitNICE value into the service file. This change affects installations where SingleStore Helios is running as a service.

  • Fixed a JSON formatting issue for the PARTITION_UNRECOVERABLE event in MV_EVENTS details column.

  • Fixed the dependency on PIPES_AS_CONCAT sql_mode inside user defined functions. The sql_mode state is now stored when a user defined function is created and used whenever the function is executed.

2023-07-07 Version 8.1.10

  • Fixed a bug in memsql_exporter that may cause it to stop accepting HTTP requests.

2023-07-06 Version 8.1.9

  • Added support for opaque OAUTHBEARER tokens in Kafka Pipelines.

  • Added support for ROW_COUNT information function within views.

  • SingleStore does not support using the environment variables to initialize OpenSSL 3.

  • Added two engine variables (workload_management_enable_large_memory_classification and workload_management_available_leaf_memory_ratio) to help tune workload management to better support queuing with high memory queries.

2023-06-26 Version 8.1.8

  • Fixed a bug where a pipeline would fail with error code 2789. The error occurred when the pipeline's "cursor offset" was below its "earliest offset".

  • Fixed a bug where a HashJoin on a materialized CTE gets executed as a NestedLoopJoin in some query shapes causing slow join performance.

  • Fixed the SPLIT string function that was not detecting the correct argument type when used inside a table-valued function (TVF) or a user-defined scalar-valued function (UDF).

  • Improved the accuracy of the blob_fetch_network_time statistic in query profiles.

  • Added metrics to memsql_exporter that show the breakdown of disk usage by SingleStore Helios. Also, added metrics to memsql_exporter that show unlimited storage utilization statistics.

  • Fixed a bug with the blob_fetch_network_time statistic in query profiles.

  • Fixed a performance bug about queue size when flexible parallelism is enabled. Overall query performance is increased.

  • Fixed an issue where, when parsing a table with many secondary keys, an incorrect name was used to refer to table's the primary key.

  • Enhanced the on-disk plancache functionality by shrinking the plancache when disk space gets low and the on-disk plancache is using at least 25% of available disk space.

  • Fixed a bug where unlimited storage garbage collection could occur before a partition could recover from that version.

2023-06-21 Version 8.1.7

  • Fixed a vulnerability in memsql_exporter by updating Go Prometheus client to 1.15.1.

  • Added a metric to the memsql_exporter that shows whether there are unavailable partitions in any database.

  • PAM (Pluggable Authentication Module) users are now allowed to start and configure the memsql_exporter process.

  • Added support for disk spilling for LEFT OUTER JOIN query shapes.

  • Added the max_execution_time engine variable. It is unused and setting this variable has no effect. It exists so certain MySQL-compatible apps that have this variable set will not fail when connected to SingleStore Helios.

  • Added mv_disk_usage and mv_data_disk_usage information schema tables that report the breakdown of disk utilization by SingleStore Helios.

  • Fixed an issue on sample tables that can be created with different collations on different partitions.

  • Fixed an issue where JSON computed column matching incorrectly triggers a JSON function.

  • Introduced the 8.1 data_conversion_compatibility_level which provides additional out-of-range checks for the TIMESTAMP data type.

  • Fixed a rare case where the query optimizer re-wrote a query into a non-equivalent query which produced more result rows.

2023-06-12 Version 8.1.6

  • Improved the performance of backups to GCS (Google Cloud Storage). GCS backups are not using multi-part uploads.

  • Fixed unlimited storage S3 multipart uploads to properly retry on 200 responses which contain an embedded "SlowDown" error, instead of failing immediately.

  • Fixed an issue where memory accumulates over time if reshuffle or repartition operators are executed for many unique query shapes.

  • Fixed an issue with crashes that occur when SSL is enabled on universal storage databases.

  • The FLUSH EXTRACTOR POOLS command now runs on all nodes in a cluster when run on an aggregator.

  • Improved the memsqlctl management tool's describe-node property variables command to display global variables instead of session variables.

2023-05-30 Version 8.1.5

  • Disabled semi-join operator reduction optimization.

2023-05-28 Version 8.1.4

  • Fixed an issue where the query optimizer could choose incorrectly between using the HashGroupBy or the StreamingGroupBy operators.

  • Added more information about blob files that need to be downloaded from unlimited storage to the blob cache when running PROFILE.

  • Fixed rare issue with sync durability and unlimited storage that could cause log file corruption in the blob storage.

  • Fixed an issue with optimizing queries containing no-operation predicates such as 1=1.

  • Improved the performance of a left join of a reference table with a sharded table by pushing the reference table clauses to the leaves. The engine variable disable_reference_table_leftjoin_where_pushdown must be set to "OFF" to enable this operation.

  • Now allowing child aggregator root users to be dropped even if the sync_permissionengine variable is enabled.

  • Removed the restrictions on the names of JWT users. Customers may now create users with names that look like UUIDs and emails.

2023-05-24 Version 8.1.3

  • Fixed a bug with error handling that produced spam messages.

  • Backups taken from the 8.1.3 release (and subsequent 8.1 releases) are no longer restorable into the previous 8.1 releases (8.1.1 and 8.1.2).

  • Removed the EXECUTE permission requirements for Kai internal UDFs.

  • Changed the system information built-ins to use the utf8_general_ci collation for the collation_server value.

Last modified: March 14, 2024

Was this article helpful?