SingleStore DB

Maintenance Release Changelog
2021-11-30 Version 7.5.14
  • Fixed the default value for max_async_compilation_concurrency so that it is now set appropriately when control groups are in use limiting CPU usage.

  • Added control group awareness for information_schema.mv_nodes which now displays the correct number of CPUs a process has access to.

  • Fixed an issue with queries not assigned to resource group pools with hard CPU limits from using the CPU assigned to the pool.

  • Fixed an issue with a crash occurring when an incremental backup was performed on an empty database.

  • Clarified error messaging when invalid data is inserted into a utf8 charset datatype column.

  • Fixed an issue with an aggregator results table checking INSERT privileges.

  • Fixed an issue where results data is incorrectly blank in particular columns when querying information_schema.mv_plancache.

  • Fixed a race condition involving concurrent UPDATE/UPSERT queries that update sort keys against columnstore tables.

  • Fixed an issue with a crash occurring when DDL is run inside a stored procedure that is executed within a resource pool on a child aggregator.

  • Block the DROP TEMPORARY TABLE command from running in a transaction that writes to the temporary table being dropped. The DROP TEMPORARY TABLE command result will display an error.

  • Removed the disk space check that occurred when ALTER TABLE ADD INDEX is run. The check occurred prior to altering a columnstore table in an unlimited storage database.

  • Fixed an intermittent "Failed to GetObject" error when restoring a non-unlimited storage database into a cluster where unlimited storage is configured.

  • Locked down SET clauses that attempt to update sparse columns in correlated subselects.

  • Added support for MAX_ERRORS clause use when using LOAD DATA into columnstore tables with unique keys.

  • Removed restrictions on nested SELECT queries that utilize the PARTITION_ID() function in their intermediate results.

  • Improved the performance of columnstore UPDATE queries that do not modify unique key columns.

2021-11-15 Version 7.5.13
  • Improved the performance of UPDATE queries that do not modify unique key columns against columnstore tables.

  • Added support for BACKUP DATABASE into S3 buckets using the sts_regional_endpoints configuration option.

  • Introduced mv_bottomless_summaryinformation_schema table that shows various aggregated statistics about unlimited storage databases. The mv_bottomless_summary table is an at-a-glance rollup table that shows what is happening right now with the unlimited storage cluster.

    Column Name

    Description

    UPLOAD_QUEUE_WAIT_SECS

    Number of seconds the uploads to remote storage are delayed in the entire cluster

    UPLOAD_MB_PER_SEC

    The rate at which files are being uploaded to remote storage in the entire cluster

    LAST_UPLOAD_RESULT

    If all partitions in the cluster report SUCCESS for upload, then this reports SUCCESS

    BLOB_CACHE_DOWNLOAD_MB_PER_SEC

    Rate at which the blob cache is downloading files from remote storage in the entire cluster

    BLOB_CACHE_EVICTION_MB_PER_SEC

    Rate at which the blob cache is evicting files in the entire cluster

    OLDEST_DATA_AGE_DAYS

    Age of the oldest piece of data in the entire cluster

    API_REQUESTS_PER_SEC

    Rate of API requests made to remote storage in the entire cluster

    IS_BEING_RATE_LIMITED

    Indicates whether any node in the entire cluster is being rate limited on API requests to remote storage. On S3, rate limiting starts when we receive 503 SLOWDOWN errors.

  • Fixed an issue with possible connection crashes on disaster recovery clusters when DDL queries are forwarded to the master aggregator and a database has not been selected.

  • Fixed sync timeout errors occurring when REBALANCE PARTITIONS of large databases is being processed. The REBALANCE PARTITIONS operation now waits for the replication of large snapshot files.

  • Added password_expiration column to information_schema.USERS table. If the password_expiration_seconds engine variable is not enabled, the password_expiration column will be NULL. If the password_expiration_seconds engine variable is enabled, the password_expiration column will display the number of seconds remaining for the password to expire.

  • Updated time zone data for some countries that recently changed their time zone rules.

  • Fixed a crash caused by a check on the maximum number of partitions allowed on a leaf node. An error message is displayed when this threshold is exceeded.

  • Added support for allowing spilling to disk of hash join output row.

  • Added support for running PROFILE on queries that execute inside of transactions the include multiple statements.

  • Fixed an issue where a crash could occur while attempting to compile queries for UPDATE statements with multiple columns due to insufficient memory resources. The amount of memory necessary to compile these query shapes was reduced.

  • Fixed a race condition involving concurrent UPDATE/UPSERT queries that update sort keys against columnstore tables.

2021-10-12 Version 7.5.12
  • Fixed an issue in the join optimizer that incorrectly eliminated candidate plans from the search.

  • Fixed a write amplification issue associated with row versioning. The behavior is disabled if the database is not unlimited storage.

  • Added support for MySQLDump 8.0.

  • Fixed a server startup failure when LD_PRELOAD is set. The behavior now reverts to a previous version which would overwrite any existing LD_PRELOAD setting.

  • Fixed an issue where the memsql_exporter would sometimes produce invalid JSON errors. The memsql_exporter is a satellite process that collects data about a running cluster.

  • Reduced the memory used to download blob files from remote storage.

  • Fixed a regression in recovery performance for databases using remote storage.

  • Fixed a crash that could occur when using LINK syntax with Avro pipelines.

  • Added Linux dependencies for SASL to SingleStore installation packages so they do not need to be installed separately.

2021-10-01 Version 7.5.11
  • Masked blob store CREDENTIALS text in CREATE DATABASE command for unlimited storage in SHOW PROCESSLIST, information_schema.PROCESSLIST, and all error messages.

2021-09-21 Version 7.5.10
  • Removed the 16384 size limit on the Avro array and map sizes in pipelines and LOAD DATA when pipelines aren't utilized.

  • Masked blob store CREDENTIALS text in BACKUP, RESTORE, and LINK commands in SHOW PROCESSLIST, information_schema.PROCESSLIST, and all error messages.

  • Increased incremental blob file streaming to disk to reduce memory overhead for unlimited storage.

  • Added support for incremental backups to utilize LINK syntax to specify CONFIG and CREDENTIALS clauses.

  • Added support for READ ONLY or READ WRITE syntax to the SET TRANSACTION statement for MySQL compatibility.

  • Added support for optimizing cases of IN joins in subselects in the project list.

  • Added session variable named subquery_merge_with_outer_joins that fine-tunes the behavior of the rewrite SubQueryMerge. If the variable is set to ON: always merge the sub-select with outer join(s). If the variable is set to HEURISTIC: check to see if merging the sub-select with outer join(s) will cause negative performance for the overall query. If the variable is set to OFF: don’t merge sub-selects with outer join(s). If the variable is set to AUTO (this is the default): For SingleStore 7.5 and previous versions, AUTO will behave like ON. For SingleStore versions after 7.5, AUTO will behave like HEURISTIC.

  • Fixed auto histograms to determine when to build new histograms based on both master and child aggregators.

  • Fixed an aggregator crash that was caused when a malformed INIT_DB command is sent by a client.

  • Added support for complex joins (joins with non-linear/nested join tree) involving correlated nested SELECT queries.

  • Added an new engine variable named rewrite_verbose to output tracings of the rewrite phase on the current query in the tracelogs.

  • Fixed warning indicating "partition metadata is out of sync on table", which could appear in the tracelog after a node restart.

  • Added support for SKIP CONSTRAINT ERRORS options for Avro and JSON pipelines.

  • Increased the timeout used when connecting to AWS to determine credentials if none are specified in a BACKUP/RESTORE command.  Also added a new engine variable subprocess_ec2_metadata_timeout_ms which allows users to set that timeout.

2021-08-30 Version 7.5.9
  • New Feature: Added support for alias references in WHERE clauses and as column expressions. See SELECT for more information.

  • Fixed an issue where Disaster Recovery of a non-unlimited storage database was not blocked if a cluster was configured to use remote storage.

  • The AGGREGATOR SYNC AUTO_INCREMENT command now can be performed by anyone with the ability to update the target database(s) or table in any way.

  • Fixed an issue that could cause defunct processes for pipelines or code generation in a rare race condition.

  • Increased batch deletions from unlimited storage to improve the performance of GCing files from remote storage.

  • Improved JSON built-in functions to better handle out of memory errors.

  • Improved the cardinality estimates for columns with an equality predicate.

2021-08-16 Version 7.5.8
  • Fixed per-predicate PROFILE statistics on columnstore filter operations.

  • Fixed a crash that could occur when running CREATE PIPELINE or REPLACE PIPELINE on a pipeline that already exists.

  • Removed truncation of the query text in query.log when general_log=on.

  • Fixed a crash that could occur when ALTER TABLE COMPRESSION=SPARSE is run in the same ALTER statement with an operation that doesn’t require data movement (ie., AUTOSTATS_ENABLED = ON).

2021-08-09 Version 7.5.7
  • New Feature: Added anonymous code blocks which are unnamed blocks of SingleStore (MemSQL) Procedural SQL (MPSQL) code that are executed immediately. They allow you to create a code block without defining and calling a stored procedure and are useful in situations such as testing or debugging.

  • Added a new system variable called parametrizer_query_max_params which allows users to set the maximum number of scalar constants a single query can contain. Any query with more than parametrizer_query_max_params will be failed with an error. The default value is 1048576.

  • Fixed an issue where the exporter process was not being cleaned up when memsqld is shut down.

  • Fixed an issue where modifying the case of a column via ALTER TABLE could cause a pipeline using that table to crash.

  • Improved usability of various JSON-builtins to support utf8mb4 charset arguments.

  • Increased frequency of merging during load for unique keys on columnstores.

  • Added information_schema.mv_bt which shows backtrace information for all nodes in the cluster.

  • Added information_schema.mv_plancache which shows plancache information for all nodes in the cluster (cluster-wide version of information_schema.plancache).

  • Added support for the transaction_read_only and tx_read_only session variables for MySQL compatible client drivers that query them.

  • Fixed a performance regression establishing SSL connections introduced in 7.5.

  • Fixed a deadlock which can occur if a failover happens during an ALTER TABLE.

  • Changed the SET PASSWORD statement to query forward from the child aggregator to the master aggregator.

2021-07-20 Version 7.5.6
  • Initial GA release of version 7.5.5

2021-07-07 Version 7.5.5
  • Stability improvements to version 7.5.4

2021-06-28 Version 7.5.4
  • Fixed an "invalid LSN" error when restoring a local storage database into a remote storage database.

2021-06-24 Version 7.5.3
  • Fixed an issue that could cause DETACH DATABASE to hang while a pipeline that is attempting to load batches is failing.

  • Fixed an issue that could generate incorrect out of memory errors when using a remote storage database.

2021-06-22 Version 7.5.2
  • Beta release of SingleStore DB 7.5