Maintenance Release Changelog v8.5

2024-12-18 Version 8.5.38

  • New feature: Added support for the file_compression parameter in the CONFIG clause of CREATE PIPELINE AS ... LOAD DATA S3 statement. Decompresses files with the specified extensions. Refer to S3 Configurations for more information. (8.5.38)

  • Enhancement: Added the TABLE_NAME column to the LOAD_DATA_ERRORS information schema view. TABLE_NAME is the name of the table associated with the error. (8.5.38)

  • Bugfix: Fixed bottomless upload throttling criteria.

  • Bugfix: Blocked creation of temporary table as a shallow copy.

  • Bugfix: Fixed display of default BSON and string values with null-terminators in the information_schema.

  • Bugfix: Fixed IN-list index matching for columnstore tables when a query has multiple IN-list predicates.

2024-11-26 Version 8.5.37

  • Bugfix: Fixed a bug that led to a master aggregator crash when using dynamic resource pools. (8.5.37)

  • Bugfix: Fixed a crash that occurred during spilling when executing a query with a large number of GROUP BY columns. (8.5.37)

  • Bugfix: Allowed cached table memory to be freed for empty tables in replica databases. (8.5.37)

  • Bugfix: Fixed a deadlock in a corner case involving index eviction and replaying two-phase commit transactions. (8.5.37)

  • Bugfix: Fixed the result of ORDER BY ... LIMIT queries on tables with a sort key column defined in DESC order. (8.5.37)

  • Bugfix: Fixed an issue caused when multi-part names were used in common table expressions (CTEs) within UNION. (8.5.37)

2024-11-05 Version 8.5.36

  • Enhancement: Improved out of memory (OOM) error message to make the message more accurate.

  • Bugfix: Fixed a crash that could occur when columns were dropped with ALTER TABLE.

  • Bugfix: Fixed an issue where left join sometimes returned an incorrect result when spilling occurred.

2024-10-21 Version 8.5.35

  • Enhancement: Improved the performance of regular expression (regex) operations.

2024-10-15 Version 8.5.34

  • Bugfix: Fixed an issue where the garbage collection of a partition database's 2PC transaction table could unnecessarily delay the reprovisioning of the reference database.

  • Bugfix: Fixed deadlock in database recovery due to evicting table module holding an open LOCK.

2024-10-09 Version 8.5.33

  • Enhancement: Added a new max_autostats_update_workers global variable that sets the maximum number of merger threads that the autostats updater can run on.

  • Enhancement: Fixed the KILLED_QUERIES counter in MV_RESOURCE_POOL_STATUS.

  • Enhancement: Added a new filter derivation rewrite controlled by the optimizer_not_null_filter_derivation engine variable.

  • Enhancement: When printing the table digest SHA value, the table DDL schema is also printed to memsql_METADATA.log.

2024-10-02 Version 8.5.32

  • Enhancement: Added metrics to track the availability of unlimited storage and the bottle service.

  • Bugfix: Fixed a certificate refresh issue for connections to remote storage in long-running clusters.

  • Enhancement: Added bottle service reliability metrics to MV_BOTTOMLESS_STATUS_EXTENDED.

  • Bugfix: Fixed a rare issue where a membership-related clustering operation could delay the re-provision or deletion of a cluster database.

  • Bugfix: Fixed an issue where JSON_MATCH_ANY() queries with MATCH_ELEMENTS evaluated incorrectly on non-array values.

  • Bugfix: Fixed sporadic reading failures from MV_SYSINFO_CPU.

2024-09-05 Version 8.5.31

  • Bugfix: Clarified the error message produced in some cases where a snapshot gets aborted.

  • Enhancement: Updated the distributed OpenSSL license file.

  • Enhancement: Update to certificate loading behavior. Prior to this change, only the first certificate was loaded from a server's .pem file specified via the server configuration. Now all certificates are loaded, so if a .pem file contains a certificates chain, then intermediate CA certificates will be returned by the server.

2024-08-14 Version 8.5.30

  • Bugfix: Fixed an issue where concurrent DROP TABLEs could block a snapshot from running indefinitely.

  • Enhancement: Added X-Content-Type-Options: nosniff, Strict-Transport-Security: max-age=31536000, Cache-control: no-store headers to Data API responses.

  • Enhancement: Improved performance of VECTOR-type User-Defined Variables(UDVs). No longer require using extra typecast to BLOB datatype.

  • Bugfix: Fixed a rare race condition where PITR would incorrectly operate on an unrecoverable database.

2024-07-30 Version 8.5.29

  • Bugfix: Fixed a password validation error on ADD AGGREGATOR.

  • Bugfix: Fixed an issue that could occur when replacing JSON expressions with computed columns in GROUP BY lists.

  • Enhancement: Added a new engine variable, sampling_estimate_approach, used to configure sampling in columnstore tables.

    See Statistics and Sampling Concepts for more information.

  • Enhancement: Added websocket_proxy.log to the files managed via Trace Log Rotation.

  • Enhancement: Added a new engine variable: assume_udfs_deterministic. This variable controls behavior where SingleStore will do extra work to avoid issues caused by UDFs that return different values when repeatedly invoked (e.g., are non-deterministic). This variable defaults to false, preserving the existing behavior (since SingleStore version 8.0), where SingleStore will ensure these computations take place once on an aggregator node. When set to true, SingleStore will avoid this extra layer of coordination (and bottleneck) allowing leaf nodes to independently coordinate.

  • Bugfix: Fixed an issue where in a VM environment SHOW LEAVES could show corrupted latency values due to incorrect values from the system clock if the VM is stopped or migrated.

  • Enhancement: Added a method to throttle upload ingest when blob cache has low evictability and running out of disk space is imminent. This is controlled via two new engine variables: bottomless_upload_throttle_hard_limit_cache_usability and bottomless_upload_throttle_soft_limit_cache_usability.

    bottomless_upload_throttle_hard_limit_cache_usability: The usability (free space + evictable space) of blob cache below which all columnstore ingest is throttled.

    bottomless_upload_throttle_soft_limit_cache_usability: The usability (free space + evictable space) of blob cache below which some columnstore ingest is throttled.

2024-07-16 Version 8.5.28

  • Enhancement: Added a new engine variable, enable_extended_types_metadata. Used to enable the handling of extended data types that are not defined in MySQL enum_field_types. Default is disabled. When enabled, SingleStore may send extended metadata to the client. It provides the ability to distinguish between different data types, and handle them appropriately in client applications.

  • Bugfix: Fixed display of microseconds for default TIMESTAMP(6) column values in system information commands (such as DESCRIBE TABLE or INFORMATION_SCHEMA.COLUMNS.COLUMN_DEFAULT).

  • Bugfix: Fixed a potential false positive output for OBSERVE queries during the first delete of large columnstore segments.

  • Bugfix: Fixed an issue where user defined variables could read collation-related variables incorrectly during a SET @var = ... command.

  • Enhancement: Allow leading 0x prefix for OBSERVE offsets.

  • Bugfix: Fixed an issue where the output of the OBSERVE query for the first delete on a columnstore segment was the inverse of expected.

  • Bugfix: Fixed issue where, in a stored procedure query-type-variable definition, a column was incorrectly assumed to belong to a table in an outer select when it was possible that the column instead belonged to a yet to be defined table.

  • Enhancement: SingleStore throws an error when inserting incorrect data into vector column for any data_conversion_compatibility_level.

2024-07-02 Version 8.5.27

  • Bugfix: Identified an upgrade scenario that causes a blob leak. As a result, temporarily disallowing shallow table copy for tables created on version 8.0 or before.

  • Enhancement: Added support for ON NODE clause for SHOW PROFILE PROCESS|PLAN, which forwards the command to another aggregator.

  • Enhancement: Improved performance of the CREATE PROJECTION command.

  • Enhancement: Added support to execute KILL QUERY on leaf node IDs.

  • Bugfix: Unneeded sort fields will no longer be projected to the aggregator.

  • Enhancement: Enhance table pushdown to union rewrite to enable derived table pushdown based on row count estimation.

  • Bugfix: Fixed an issue where deleting a local user on a leaf made aggregates fail to run on that leaf.

2024-06-28 Version 8.5.26

  • Bugfix: Fixed a rare issue where Unlimited Storage GC could use up all of the MA's available connections.

  • Bugfix: Identified an upgrade scenario that causes a blob leak and added a command to delete the leaked blobs.

  • Enhancement: Updated the MV_BOTTOMLESS_STATUS_EXTENDED information schema table:

    • Removed the LATEST_API_FAILURE_RATE and AVERAGE_API_LATENCY_US columns.

    • Added the following columns:

      SUCCESS_API_PER_SECOND - Number of successful remote storage api calls per second over the last 30 seconds.

      FAILED_API_PER_SECOND - Number of failed remote storage api calls per second over the last 30 seconds.

      AVERAGE_API_EVENT_LATENCY_US - Average latency to complete a remote storage api event over the past 30 seconds. Includes time taken on retries or waiting for rate limiting.

      AVERAGE_API_RESPONSE_LATENCY_US - Average latency of receiving a response for a remote storage api request. Excludes waiting for rate limiting or retries. NOTE: This is relevant only for S3. otherwise this field is identical to AVERAGE_API_EVENT_LATENCY_US.

2024-06-20 Version 8.5.25

  • Bugfix: Fixed a crash that occurred when dropping a database with tables containing very high and very low column IDs.

2024-06-18 Version 8.5.24

  • Bugfix: Fixed issues with the TO_CHAR() function where results were truncated or reading communication packet errors were generated.

  • Bugfix: Fixed an integer overflow issue in unique integer columns by changing the storage type to a 32-bit unsigned integer.

  • Bugfix: Fixed incorrect results in an uncommon scenario where hash join operators with an optimization path need to spill.

  • Enhancement: The memsql_exporter now collects additional fields from the mv_activities_extended_cumulative information schema view.

  • Enhancement: Updated the default client.id for the Kafka configuration to the SingleStore partner ID.

2024-06-12 Version 8.5.23

  • Enhancement: Added the following monitoring metrics to the information_schema.mv_bottomless_status_extended view: NUM_SUCCESS_API_REQUESTS, NUM_FAILED_API_REQUESTS, LATEST_API_FAILURE_RATE, AVERAGE_API_LATENCY_US, IS_GC_RUNNING, NUM_REMOTE_FILES_DELETED, and NUM_SLOW_DOWNS.

  • Bugfix: Fixed a crash that occurred when spilling a hash join operation with at least 3 tables.

  • Bugfix: Fixed a crash that occurred when running the SHOW CREATE PROJECTIONS command when using the database name in the syntax (SHOW CREATE PROJECTIONS database_name.projection_name...).

  • Bugfix: Fixed an issue with incorrect results in information_schema.PROJECTIONS when filtering on the PROJECTION_NAME column.

  • Enhancement: Improved performance related to PIPELINES_BATCHES information schema table.

  • Enhancement: An engine variable enable_use_of_stale_incremental_stats has been added to allow the query optimizer to use stale statistics instead of querying for statistics during optimization.

  • Bugfix: Fixed an issue where histograms are built from up to 10x more rows than desired.

2024-05-28 Version 8.5.22

  • Enhancement: Added STORAGE_ID and COMPUTE_ID to INFORMATION_SCHEMA.TABLE_STATISTICS, COLUMNAR_SEGMENTS, and MV_COLUMNSTORE_FILES.

  • Bugfix: Fixed a crash issue with JSON_TO_ARRAY. Disabled JSON_TO_ARRAY TVF join optimization if query does not use the TVF output column.

    The crash specifically occurs when JSON_TO_ARRAY join optimization is triggered and when there is no usage of table_col output of the TABLE(JSON_TO_ARRAY()) statement in the evaluation of the query.

    For example, prior to this fix:

    The following query will crash (because the table_col output of the TABLE(JSON_TO_ARRAY(t.json)) statement is unused):

    SELECT t.a FROM t JOIN TABLE(JSON_TO_ARRAY(t.json)) GROUP BY 1;

    The following query will NOT crash:

    SELECT table_col::a::b, t.a FROM t JOIN TABLE(JSON_TO_ARRAY(t.json)) GROUP BY 1, 2;
  • Enhancement: Added more information to filesystem error messages.

2024-05-21 Version 8.5.21

  • Bugfix: Fixed an issue in ALTER TABLE when adding a computed column containing JSON_BUILD_OBJECT.

  • Bugfix: Addressed a crash issue with REGEX_REPLACE.

2024-05-16 Version 8.5.20

  • Bugfix: Reduced stack usage during compilation for some CASE expressions with many WHEN-THEN statements.

  • Bugfix: Fixed a potential deadlock issue between background garbage collector threads and foreground query execution threads.

  • Bugfix: Fixed a collation issue in the JSON_COLUMN_SCHEMA information schema table.

  • Enhancement: SingleStore now supports module eviction of idle tables with an ALTER history. This means that, prior to the 8.5.20 release, tables that had been altered could not be evicted. Altered here includes CREATE INDEX, ALTER TABLE ADD COLUMN, and some others.

2024-05-07 Version 8.5.19

  • Bugfix: Fixed a bug with Wasm UDFs where REPLACE FUNCTION did not correctly evict from the plancache, causing continued usage of the old UDF after replacement.

2024-04-30 Version 8.5.18

  • Bugfix: Fixed an upgrade issue present in version 8.5.17.

  • Bugfix: Fixed a BSON issue that could occur when large documents are read and mistakenly reported as corrupted.

  • Enhancement: Upgraded OpenSSL 1.0.2u to 1.0.2zj

  • Bugfix: Fixed a crash issue that could occur under a rare race condition in some distributed join queries.

  • Bugfix: Fixed statement atomicity in transactions writing to tables that have projections on them.

  • Enhancement: Allow Kerberos users to skip password validation for users who are authenticated with plugins.

2024-04-23 Version 8.5.17

Warning

Version 8.5.17 includes an issue that can lead to problems upgrading to future patches. Do not upgrade to version 8.5.17. An updated version (8.5.18) is available. The issue only affects installations using VECTOR data type in table columns. Contact SingleStore Customer Support for more information if you have cluster that uses the VECTOR data type on 8.5.17.

  • Enhancement: Added support for loading data from compressed Avro datafiles.

  • Bugfix: Fixed an issue that occurred when primary key is NULLABLE when copying a table using "Create table as select" statement.

  • Enhancement: Added a new engine variable, enable_extended_types_metadata. Used to enable the handling of extended data types that are not defined in MySQL enum_field_types. Default is disabled. When enabled, SingleStore may send extended metadata to the client. It provides the ability to distinguish between different data types, and handle them appropriately in client applications.

  • Bugfix: Addressed a crash issue occurring while executing queries with vector builtins on clusters running via Docker containers on Mac machines equipped with Apple silicon processors (M1).

  • Bugfix: No longer show information_schema pipeline metadata of databases that are not attached to the workspace.

  • Enhancement: Use SeekableString encoding by default for storing VECTOR datatype columns in columnstore tables.

  • Bugfix: Fixed an issue that caused rare "Arrangement already exists in namespace" errors in recursive CTE queries.

  • Bugfix: Improved the hashGroupBy spilling condition to trigger (spill) when appropriate.

2024-04-18 Version 8.5.16

  • Enhancement: Added a new engine variable costing_max_djo_tables which sets the maximum amount of tables we allow in full DQO estimate search when filling table row estimates for distributed join hints.

2024-04-08 Version 8.5.15

  • Enhancement: Made many more engine variables configurable on SingleStore Helios:

    multi_insert_tuple_count, columnstore_row_value_table_lock_threshold, internal_columnstore_max_uncompressed_blob_size, internal_columnstore_max_uncompressed_rowindex_blob_size, columnstore_validate_blob_before_merge, default_max_histogram_buckets, enable_varbuffer_dictionary_compression, enable_alias_space_trim, skip_segelim_with_inlist_threshold, default_autostats_histogram_mode, default_autostats_columnstore_cardinality_mode, default_autostats_rowstore_cardinality_mode, default_autostats_columnstore_sampling, experimental_disable_multipartition_parallel_read, internal_enable_parallel_query_throttling, enable_spilling, spilling_node_memory_threshold_ratio, spilling_query_operator_memory_threshold, regexp_compile_mem_mb, selectivity_estimation_min_sample_size, repartition_minimum_buffer_size, use_seekable_json, json_document_max_children, json_document_max_leaves, json_document_sparse_children_check_threshold, json_document_sparse_children_check_ratio, json_document_absolute_sparse_key_check_ratio, json_document_page_data_soft_limit, ignore_insert_into_computed_column, max_subselect_aggregator_rowcount, leaf_pushdown_enable_rowcount, reshuffle_group_by_base_cost, max_broadcast_tree_rowcount, enable_broadcast_left_join, display_full_estimation_stats, force_bushy_joins, force_heuristic_rewrites, force_table_pushdown, force_bloom_filters, explain_joinplan_costs, statistics_warnings, optimizer_warnings, optimize_constants, optimize_huge_expressions, optimize_expressions_larger_than, optimize_stmt_threshold, quadratic_rewrite_size_limit, batch_external_functions, batch_external_functions_size, optimizer_enable_json_text_matching, optimizer_enable_orderby_limit_self_join, distributed_optimizer_broadcast_mult, distributed_optimizer_old_selectivity_table_threshold, distributed_optimizer_selectivity_fallback_threshold, distributed_optimizer_verbose, old_local_join_optimizer, optimizer_hash_join_cost, optimizer_merge_join_cost, optimizer_nested_join_cost, optimizer_disable_right_join, interpreter_mode_sampling_threshold, hash_groupby_segment_distinct_values_threshold, sampling_estimates_for_complex_filters, enable_histogram_to_unwrap_literals, estimate_zero_rows_when_sampling_data_is_missing, exclude_scalar_subselects_from_filters, leaf_pushdown_default, distributed_optimizer_nodes, optimizer_num_partitions, enable_binary_protocol, enable_multipartition_queries, enable_local_shuffle_group_by, enable_skiplist_sampling_for_selectivity, columnstore_sample_per_partition_limit, disable_sampling_estimation, disable_histogram_estimation, inlist_precision_limit, allow_materialize_cte_with_union, optimizer_cross_join_cost, distributed_optimizer_run_legacy_heuristic, distributed_optimizer_min_join_size_run_initial_heuristics, distributed_optimizer_unrestricted_search_threshold, singlebox_optimizer_cost_based_threshold, distributed_optimizer_estimated_restricted_search_cost_bound, disable_reference_table_leftjoin_where_pushdown, disable_sampling_estimation_with_histograms, disable_subquery_merge_with_straight_joins, default_columnstore_table_lock_threshold, default_spill_dependent_outputters, query_rewrite_loop_iterations, verify_fields_in_transitivity, optimizer_min_reference_tables_for_gathered_join, optimizer_min_reference_rows_for_gathered_join, max_expression_query_limit, max_expression_item_limit, optimizer_empty_tables_limit, optimizer_beam_width, optimizer_disable_subselect_to_join, disable_remove_redundant_gby_rewrite, subquery_merge_with_outer_joins, optimizer_disable_operator_tree_nullability_check, clamp_histogram_date_estimates, varchar_column_string_optimization_length, histogram_column_correlation, use_column_correlation, consider_secondary_projection, optimizer_disable_semi_join_reduction, optimizer_gb_placement_table_limit, optimizer_disable_transitive_predicates

  • Enhancement: Added more config information about pipelines to memsql_exporter.

  • Enhancement: Improved error handling for VECTOR type.

  • Enhancement: Added support for aggregate functions (FIRST, LAST) with VECTOR datatype.

  • Enhancement: Added support for Kafka key (producer) in the SELECT ... INTO KAFKA command.

  • Enhancement: Improved Kafka error messages to make them more specific.

  • Bugfix: Fixed a case where a killed query could incorrectly return a JSON error rather than query interrupted message.

  • Bugfix: MV_DISK_USAGE queries can no longer block garbage collection.

  • Bugfix: Fixed a crash-on-startup issue on RHEL-based operating systems/kernels (CentOS 9, RHEL, etc.).

  • Bugfix: Fixed potential deadlock between table module eviction and GC manager.

  • Bugfix: Fixed a crash that could occur when a scalar subselect is passed as an argument to an external TVF.

  • Bugfix: Fixed an issue with restoring from public backups.

  • Bugfix: Backups now throttle after snapshot.

2024-04-02 Version 8.5.14

  • New Feature: Added ability to create a Projection which is a copy of some or all columns in a table, and may be sharded and sorted differently than the primary table. Projections can be used to speed up several types of queries including range filters, ORDER BY/LIMIT, GROUP BY, count(DISTINCT...), DISTINCT, joins, and row lookups on secondary keys. Projections depend on the table and are updated in real time when you update the table. Related information schema table: PROJECTIONS.

  • Bugfix: Removed an unnecessary metadata copy to improve JSON performance.

  • Enhancement: Multi-insert queries now respect lower columnstore_segment_rows settings.

  • Enhancement: Subprocess and S3 API tracing are now also enabled via the enable_subprocess_tracing engine variable.

  • Enhancement: Added the optional IF NOT EXISTS clause to the CREATE TABLE ... AS INFER PIPELINE statement. Refer to Replicate Data from MongoDB® for more information.

2024-03-27 Version 8.5.13

  • Enhancement: Dedicated Admin Connections via a TCP port. Added two new engine variables, enable_dac_ip_port and dac_ip_port - these can be used to open an extra mysql port on startup which will accept dedicated admin connections.

  • Enhancement: The Tracelog rotation threshold now applies to rotated logs. Added two new engine variables to control the total size of retained log files: auditlog_retention_size and tracelog_retention_size.

  • Bugfix: LAST_INSERT_ID will now correctly retrieve the last inserted ID for a forwarded INSERT query.

  • Bugfix: The rule based optimization warning message will only be shown when cost based optimization is meaningful.

  • Bugfix: Fixed a minor performance issue where a table module could be evicted prematurely during database recovery.

  • Bugfix: Fixed a very rare deadlock between ALTER of a table and a DROP of a database.

  • Bugfix: Fixed a JSON insertion failure to old utf8 sample tables that contain utf8mb4 characters by encoding it with base64 format beforehand for backward compatibility.

  • Enhancement: Disk Spilling now takes QUERY_MEMORY_PERCENTAGE resource pool setting into consideration.

2024-03-19 Version 8.5.12

  • Bugfix: Fixed an issue with a blob file leak in a rare crash scenario.

  • Bugfix: Fixed a potential deadlock in a scenario involving DDL, a 2PC transaction, and a concurrent failover.

  • Bugfix: Fixed a deadlock involving BACKUP and ALTER in a rare case.

  • Enhancement: Added support for utf8mb4 symbols in column comments, table comments, and user comments.

  • Enhancement: Enabled additional query hint for non-distributed queries so that first run of those queries can be queued.

  • Enhancement: Updated timezone data, now using IANA's tzdata version 2024a; new timezones Kyiv, Ciudad_Juarez, and Kanton supported.

2024-03-13 Version 8.5.11

  • Bugfix: Fixed an issue in background merger that allowed concurrent deletes of empty segments.

  • Bugfix: Fixed an issue to ensure preserving a field of a table correctly.

  • Enhancement: Idle Table Eviction improvement. In the 8.5 GA version, an idle table's objects cannot be evicted from memory if the table has one or more UNIQUE key columns (both PRIMARY KEYs and any other keys with a UNIQUE constraint). In the 8.5.11 patch, idle tables' code objects can be evicted from memory if the table has one or more UNIQUE key columns.

2024-03-06 Version 8.5.10

  • Enhancement: Added numInferredPaths and schemaSize as new columns to JSON_COLUMN_SCHEMA table. numInferredPaths is the number of key paths inferred for the segment. schemaSize is the size of schema_json in bytes.

  • Bugfix: Added safety checks around dropping databases without exclusive access to storage.

  • Enhancement: Added logging to heartbeat thread in order to better diagnose process stalls. This logging is enabled/disabled via the new verbose_time_heartbeat_thread engine variable. The default value is off, setting this to on will enable the logging.

  • Enhancement: Added the ability to collect and send SingleStore Self-Managed usage telemetry data to SingleStore via the EXPORT TELEMETRY sql command (for 1-time or ad hoc use) or the send_usage_telemetry engine variable. This defaults to OFF. Added an associated information schema table: usage_telemetry_status.

  • Bugfix: Fixed an issue where explicitly defining a JSON type in the RETURNS of a Wasm TVF can cause an error when it is run.

  • Bugfix: Fixed a rare crash scenario.

  • Bugfix: Fixed an issue that could lead to the risk of undefined behavior when running DROP EXTENSION IF EXISTS.

  • Enhancement: IN-lists will now use hashmap optimization for matching parameters in *_MATCH_ANY statements.

  • Enhancement: Made default_distributed_ddl_timeout a sync variable on all nodes.

  • Enhancement: Modified the conversion logic for converting from VECTOR to BSON types. Now, when casting VECTOR(F32), it will generate a BSON array of doubles, rather than a combination of numeric types.

  • Bugfix: Addressed a family of issues relating to optimal execution of JSON extracts on schema'd data.

2024-02-28 Version 8.5.9

  • Bugfix: Fixed an issue with regression in replay performance for databases with many tables.

2024-02-27 Version 8.5.8

  • Enhancement: Added spilling metrics for TopSort (ORDER BY with LIMIT).

  • Bugfix: Fixed an issue where the existing websocket connection would close when variables are updated in global scope.

  • Bugfix: Removed the hard-coded 'collation_server' from constant.StrictModeParams.

  • Enhancement: Added new metrics to memsql_exporter based on the information_schema.mv_sysinfo_disk columns.

  • Enhancement: Added support for creating numeric histograms based on JSON values. (Feature flag gated, contact your SingleStore representative)

  • Enhancement: Added ability for information_schema.optimizer_statistics to display JSON keys.

  • Enhancement: Improved JSON histograms to support numeric histograms, analyze command, info_schema.optimizer_statistics, and displays JSON histograms.

  • Enhancement: Added support for warming blob cache with table's column data. Syntax is: OPTIMIZE TABLE <table_name> WARM BLOB CACHE FOR COLUMN <column_names>:

    OPTIMIZE TABLE t WARM BLOB CACHE FOR COLUMN c1, c2;
    OPTIMIZE TABLE t WARM BLOB CACHE FOR COLUMN *;
  • Enhancement: Added ability to skip eviction when log replaying hits the blob cache space limit, providing a greater chance to succeed.

  • Enhancement: Disabled the default semi join reduction rewrite.

  • Bugfix: Fixed an issue with a sensitive information leak inside of out-of-memory reports.

  • Bugfix: Fixed the result collation from VECTOR built-ins to be utf8mb4_bin instead of binary.

  • Enhancement: Creation of computed columns with VECTOR data type is now allowed.

  • Enhancement: Added support to the VECTOR built-ins for all VECTOR elements types (e.g., F32, F64, I8, I16, I32, and I64).

  • Enhancement: Added support for usage of vector built-ins with string/JSON arguments without requiring an explicit typecast to VECTOR (e.g., 'SELECT @vec<*> '[1,2,3]').

  • Enhancement: Now recognize more EXTRACT/MATCH functions as candidates for pushdown/computed column matching.

  • Enhancement: Replaced expression pushdown (EPD) approach with more complete version that does not over project rows.

2024-02-20 Version 8.5.7

  • Bugfix: Fixed an issue with nested extracts in the JSON_MATCH_ANY() statement.

  • Bugfix: Fixed an issue with backup restore when the backup has a corrupted GV timestamp in the about snapshot record.

  • Enhancement: The new engine variable parametrize_json_keys allows JSON keys to be parametrized and plans can be reused.

  • Bugfix: Fixed an issue that causes CREATE TABLES AS INFER PIPELINE to fail with an Out Of Memory error.

  • Bugfix: Fixed an issue with exporter not working with Public-Key Cryptography Standards #8(PKCS#8) pem format.

  • Bugfix: Fixed an issue with potential crashes when specifying a vector index in index hint.

  • Bugfix: Fixed a crash that occurred when trying to grant access to the table with no context database in per-privilege mode.

  • Bugfix: Fixed a memory issue, when calling a Table-Valued Function (TVF) that has arguments with default values.

  • Bugfix: Fixed an issue with the RG pool selector function return type check. The maximum value for n in VARCHAR(n) is now 256.

  • Enhancement: information_schema.mv_connection_attributes now shows tls_version and tls_cipher for connections where SSL is enabled.

2024-02-13 Version 8.5.6

  • Enhancement: Added ability for predicate pushdown with NOW() and user defined functions (UDF).

  • Enhancement.: Reduced amount of memory used by unlimited storage file listing, including point-in-time recovery (PITR).

  • Enhancement. Added an engine variable, innodb_lock_wait_timeout. This variable exists for backwards compatibility with MySQL and is non-operational in SingleStore.

  • Enhancement. Optimized performance when using VECTOR built-ins with the VECTOR data type.

  • Bugfix: Fixed an issue where a backup would inadvertently carry metadata about the retention period of the backed-up unlimited storage database.

  • Bugfix: Fixed the change to aggregator activity queries in memsql_exporter not being applied properly.

2024-02-07 Version 8.5.5

  • Enhancement: Added ability to memsql_exporter to always return all sub-activities of an aggregator activity.

  • Enhancement: Improved ability to configure Kafka extract/egress by allowing additional options.

  • Enhancement: Improved performance for certain query shapes used by Kai.

  • Bugfix: Fixed the query-events endpoint link on the memsql_exporter landing page.

  • Bugfix: Fixed an issue where a segmentation fault (crash) appeared after terminating a query on the remote node.

  • Bugfix: Fixed an issue with crashes occurring when using certain JSON_EXTRACTs in the predicate expression of JSON_MATCH_ANY().

  • Bugfix: Fixed an issue where the number of rows affected with SELECT INTO object storage was erroneously reported as 0 rows.

  • Bugfix: Fixed an issue with START PIPELINE FOREGROUND skipping files in some circumstances.

2024-01-29 Version 8.5.4

  • Enhancement: The new engine variable max_table_memory_room_mb sets the maximum amount of memory required when creating or attaching a database. Configuring this engine variable allows more control over whether a detached database can be reattached.

  • Bugfix: Fixed an optimization out-of-memory issue cause by operators generated for wide user tables.

  • Bugfix: Fixed a crash that occurs in rare scenarios involving ALTER TABLE and failovers.

  • Bugfix: Fixed ineffective search options that change in subsequent vector search queries.

  • Bugfix: Resolved an issue related to FROM_BASE64 and TO_BASE64 builtins when processing large string inputs thereby preventing potential errors in reading communication packets.

  • Enhancement: The engine variable backup_multipart_upload_concurrency maximum value has been increased to 15.

  • Bugfix: Fixed the code involved in backups to improve download error messaging.

  • Bugfix: Fixed an bug that could cause scans using non-unique indexes which could return incorrect results or cause crashes.

2024-01-23 Version 8.5.3

  • Bugfix: Fixed a bug that could result in unrecoverable databases if the database had 1024 or more tables.

2024-01-18 Version 8.5.2

  • Enhancement: Improved performance for JSON_EXTRACT_<type> built-ins in ORDER BY clauses.

  • Enhancement: Added the blob_cache_miss_b and blob_cache_wait_time_ms columns to the following information schema views: information_schema.plancache, information_schema.MV_ACTIVITIES, information_schema.MV_ACTIVITIES_CUMULATIVE, information_schema.MV_TASKS, information_schema.MV_FINISHED_TASKS, and information_schema.MV_QUERY_ACTIVITIES_EXTENDED_CUMULATIVE.

  • Bugfix: Fixed an issue where INSERT...SELECT queries with a partition_id() filter generating an error.

  • Bugfix: Fixed an issue with memory crashing when using REGEXP_MATCH, JSON_INCLUDE_MASK, or JSON_EXCLUDE_MASK built-ins.

  • Enhancement: INFER PIPELINE for MongoDB® CDC-in now generates tables with BSON column types.

  • Enhancement: Improved performance by optimizing joins on TABLE(JSON_TO_ARRAY()) queries.

  • Enhancement: Now suppressing a harmless traceSuspiciousClockUpdate trace message during recovery.

  • Enhancement: Added the ability for users to kill queries on a remote node without the user having PROCESS privileges.

  • Bugfix: Fixed an allocation issue that caused poor performance on high load insertion queries.

  • Enhancement: Enabled support for external UDFs used in INSERT statements with multiple VALUE clauses.

  • Enhancement: Added BSON fundamentals and column type support for SingleStore Kai.

Last modified: December 18, 2024

Was this article helpful?