Maintenance Release Changelog

Fixed an issue with automatic histograms on clusters that were upgraded with the variable cardinality_estimation_level set to 6.0.

Fixed an issue where an UPDATE query on a columnstore table with unique key could wrongly report a duplicate key error in some cases.

Added metrics about the availability of database partitions to memsql_exporter.

Allowed memsqlctl to establish a secure connection when a node is configured for SSL using either the engine variable ssl_ca, or the host system’s trusted certificates for verification.

Added the --insecure-ssl flag, which allows memsqlctl to support SSL connections without verification.

Fixed an issue that occurred when running ALTER or TRUNCATE against a table that included ON DUPLICATE KEY UPDATE in its CREATE PIPELINE statement.

Fixed an issue that caused read actions on columnstore JSON objects to result in an error (Error reading from parquet: Column not found in schema) in cases where trailing whitespaces were present in the JSON key names stored for a given table.

Allowed memsqlctl to be run as the root user without additional configuration.

The memsql_exporter process is now properly shut down when the exporter port is changed or the cluster is uninstalled. Previously, it left a defunct process.

When running memsql_exporter, the password for the monitoring user is no longer exposed in the ps output.

When inserting into a columnstore table, columns not needed to deal with a duplicate key are no longer decompressed.

Improved the optimizer’s cost estimates for hash joins when one of the tables in a given join has an index matching the join condition.

Fixed a crash that occurred when an aggregate function was used in an unsupported context inside a stored procedure.

Added a new engine variable optimizer_empty_tables_limit, which sets a threshold for the number of empty tables that must be present in a query before the optimizer falls back to being rule based, instead of cost based. For more information, see List of Engine Variables.

Improved the speed of server shutdown and of recovery for databases with large numbers of tables. This improvement is evident at 1,000 tables or more, and increases significantly after 10,000 tables.

Fixed an issue that caused RESTORE commands to fail in cases where a database contained a large number of reference tables.

The system’s root CA is now used where no ssl.ca.location is specified for a Kafka pipeline connection using SSL.

Fixed an issue that caused DROP TEMPORARY TABLE IF NOT EXISTS to incorrectly return an error in cases where the table didn’t exist.

Fixed a crash caused by a check on the maximum number of partitions allowed on a leaf node. Now, an error message is surfaced when this is exceeded.

Fixed issue that caused an error (Transaction rolled back mid-query) to occur when running cross-database queries in multi-statement transactions.

Enabled use of gzip encoded files in GCS pipelines.

Reduced blocks on UPDATE and DELETE commands via background merger.

Added a new variable, default_columnstore_table_lock_threshold, which sets a threshold for the number of rows that are locked when updating rows in a columnstore table before a table lock is acquired. For more information, see List of Engine Variables.

Fixed an issue that occurred when a TO_QUERY statement referred to a table that did not exist.

When using binary protocol, YEAR type results are now returned as MYSQL_TYPE_SHORT. Previously, these results were incorrectly returned as a length-encoded string.

Fixed a crash caused by timestamp values during daylight savings time changes in columnstore tables using a timestamp as a sort key or unique key.

Fixed a crash that occurred when querying information_schema.COLUMNAR_SEGMENTS, information_schema.MV_COLUMSTORE_FILES, or information_schema.MV_COLUMNAR_SEGMENT_INDEX while ALTER TABLE … DROP INDEX ... is being run concurrently in order to drop a secondary columnstore index.

APPROX_COUNT_DISTINCT now supports aggregates with a grouping set. For example, SELECT APPROX_COUNT_DISTINCT(column_name) FROM table_name GROUP BY ROLLUP(column_name).

In the output of PROFILE, indicated cases where a corrupted secondary key is encountered while running a given query.

Added Avro schema evolution capabilities to Pipelines.

Fixed an issue that caused crashes when using GROUPING() on expression inputs.

information_schema.load_data_errors now shows errors on tables a user has access to via roles and groups. Previously, only direct grants allowed access to the view.

Fixed an issue that caused crashes when using reference tables with unique keys on columnstores.

Fixed an issue that caused index corruption with columnstore secondary keys in cases where a single partition has more than 2 billion unique values.

Added support for audit logging levels other than ALL-RESULTS mode when clients connect using server side prepared statements. Previously, enabling audit logging disabled use of prepared statements.

Fixed a crash that occurred when ANY_VALUE(NULL) is included in a query.

Fixed a crash that occurred where a query-typed variable accesses a table via a 3-part name (for example, dbname.tablename.columnname) and the database referred to does not exist.

When a leaf node moves from the attaching state to the online state and an error is encountered attaching its partitions, we now more aggressively retry to ensure all partitions are attached before moving the leaf into the online state.

Snapshots of the memsql system database are now taken more frequently. Previously, the memsql database was slower to recover on higher latency disk than user databases; more frequent snapshots speed up restart times.

Blocked the creation of pipelines which refer to computed columns or columns set via a default value in the WHERE clause.

Allowed memsqlctl to register multiple nodes with the same auditlogsdir.

Fixed a wrong result issue that caused only the first of multiple values with the same hash in a segment to be output when using an IN list (for example, IN (a, b, c)) with a columnstore secondary hash index.

Added a new engine variable disable_subquery_merge_with_straight_joins that disables merging sub-queries if they contain STRAIGHT_JOIN in order to avoid unwanted query plans. Here, merging sub-queries refers to removing unnecessary nested layers from joins including sub-queries. For example, the query select * from (select * from table_a straight_join table_b) table_c straight_join table_d would be rewritten as select * from table_a straight_join table_b straight_join table_d if the engine variable disable_subquery_merge_with_straight_joins is set to OFF. If the variable is set to ON, the query would not be rewritten. The default setting is AUTO, which is equivalent to OFF in 7.1.

Added the command memsqlctl unregister-node to unregister nodes in memsqlctl.

Fixed an issue where restoring a backup from a local or network drive caused the partitions to be unevenly distributed across the leaves. The issue occurred when leaf_failover_fanout was set to load_balanced.

Fixed an issue where some memsqlctl commands that were run with the --timeout flag would not exit after the specified timeout.

Fixed an issue that could prevent MemSQL from cleaning up stale data after a node goes offline due to an out of disk condition.

Fixed a crash caused by stored procedures which include an INTO clause (that modifies variables), followed by an IN clause.

BACKUP and RESTORE commands can now be governed by resource pools.

Fixed an issue causing RESTORE to fail after upgrading to MemSQL 7.1 and executing BACKUP … SPLIT PARTITIONS.

Query hints are now supported inside query type variables in user defined functions and stored procedures.

The CHECK BLOB CHECKSUM command now supports a WITH REPAIR option that can repair file system corruptions as long as one copy of a corrupt columnstore blob file exists on the cluster.

The query hint OPTION (MATERIALIZE_CTES = AUTO) was added to enable materialized common table expressions per query, as an alternative to the existing session variable MATERIALIZE CTES.

Fixed a wrong results bug that occurred when joining a query type value (QTV) with a user table, and accessing a column that doesn’t exist in the QTV definition, but does exist in the user table schema. Previously, even if the column was specified as coming from the QTV, it was being read from the table. Now, specifying the QTV works correctly.

BACKUP … TO AZURE now waits four minutes before timing out slow operations against the Blob Store.

Fixed an issue that occurred when using server side prepared statements via the newer versions of the MariaDB client. The MySQL client is unaffected.

Fixed an issue cleaning up global temp tables on leaves during rebalancing operations.

Fixed an issue where two window functions with identical ORDER BY and PARTITION BY clauses, but differing ROWS BETWEEN clauses, incorrectly output the same values.

Fixed a performance issue with JSON_TO_ARRAY().

Fixed an issue where incremental autostats may enter an invalid state and be unable to recover automatically due to out of memory on a leaf node.

Improved error message in cases where MemSQL has issues accessing the memsql.cnf file.

Fixed an issue where REPLICATE DATABASE would fail in cases where no replicated database existed yet and the node containing the master aggregator was previously replaced with a new node.

Added a multipart_chunk_size_mb setting to SELECT .. INTO S3 that controls the size of the file uploaded to S3 to support uploading larger result sets to S3.

Fixed an issue where auto-attach would fail to attach a partition but succeed in bringing the leaf online, resulting in offline partitions.

Ensured that any re-used existing replica partition is still forced in sync as part of the copy operation run as a step of a REBALANCE operation. Previously, this was a cause of failure in cases where REBALANCE failed once prior.

Fixed a failure when running a ROLLUP query over a constant expression.

Audit logging now supports auditing logins and login attempts.

Audit logging file headers now include information about what level the file is logging at as well as if the level was changed.

Added a tracelog (memsql.log) message to indicate if audit logging has been disabled from previously having been enabled.

Added support for a SELECT containing a semicolon in the CREATE TABLE ... AS SELECT statement.

Modified the infrastructure of the native monitoring solution by removing the Kafka and memsql-pusher components and automating the monitoring setup with SingleStoreDB Toolbox commands.

Fixed a compatibility issue that occurred when running newer versions of SAP Business Object Data Services (BODS) against MemSQL. The issue was that the command SET sql_mode = <expression> was not supported by MemSQL.

Fixed an issue that could occur when using columnstore unique keys in an update query with a join.

Avoid taking unneeded locks for queries writing into columnstore tables with unique key columns. Taking these locks could cause transactions containing multiple statements to deadlock.

Improved the error message returned when REMOVE AGGREGATOR is run on the master aggregator.

The behavior of the max_compilation_time_s engine variable no longer applies to DDL queries. Eliminating compilation timeouts for these queries, especially for ALTER TABLE, will prevent the queries from potentially running forever.

Reduced the CPU usage required for many threads to allocate memory for variable length strings.

Added a new engine variable convert_nonunique_hash_to_skiplist which, when set to ON, creates a skiplist index in place of any non-unique hash index. By default, the variable is set to OFF.

Pipelines no longer stop executing if free disk space drops lower than 5 * <the value of the minimal_disk_space engine variable>. They now stop executing if free disk space drops lower than 400 mb + <the value of the minimal_disk_space engine variable>.

Now, SHOW PROCESSLIST returns the IP address of the client (in the Host column) if the host name cannot be resolved.

Fixed an issue that occurred when the engine variable interpreter_mode was set to interpret_first, where the first run of the query remained in interpreted mode even after compilation had finished.

Added support for ALTER PIPELINE ... SET RESOURCE POOL.

Improved the performance of rebalance partition operations that copy partitions with many small columnstore BLOB files.

Improved predicate transitivity rewrites to avoid adding redundant predicates. This fixes compile timeouts for some queries with large numbers of predicates.

Fixed an issue where the engine variable explicit_defaults_for_timestamp could be set to a different value on new nodes added to the cluster, as compared to the existing value already set on existing nodes. This would cause the error “partition’s table metadata are out of sync” when queries were run.

Improved heartbeat failure detection when leaves were unresponsive to queries.

Fixed an issue where setting the engine variable sql_mode to ONLY_FULL_GROUP_BY would , for queries using INTERSECT and EXCEPT, incorrectly generate an error indicating a field is unaggregated and not in the GROUP BY clause.

Fixed an issue where setting the engine variable materialized_ctes to ALL would prevent certain queries using INTERSECT and EXCEPT from running.

Re-enabled support for certain nested subselects that had been added previously and then removed erroneously.

Improved the performance of batch deletes on columnstores. The increase in performance depends on the encoding of the columns.

Now, license checks are Linux control group aware. For containerized deployments, CPU and memory limits are checked against each container.

ANALYZE TABLE now requires the SELECT and (INSERT or ALTER) permissions to execute.

Added the TO_JSON function, which converts a table column, an entire table, a scalar value, or a single row to a JSON object.

Fixed a syntax error that occurred when using the LINES STARTING BY or LINES TERMINATED BY options with SELECT ... INTO S3 or SELECT ... INTO KAFKA.

Improved the accuracy of the PROFILE commands that have ColumnStoreScan components.

When preserve_original_colstore_json is set to ON, null values and empty arrays are preserved in JSON columns in columnstores.

SSL ciphers that use elliptic curves are now supported.

Added an optimizer setting that allows you to adjust the cost of performing a cross join.

Fixed an issue where user-defined tables having the same name as system tables, such as USERS, were treated as case-sensitive when table_name_case_sensitivity was set to OFF.

Correctly generate an error when running ALTER VIEW on a schema-bound view v1, where a schema-bound view v2 refers to v1.

Fixed an auto-attach failure in specific failure conditions where a node quickly got marked offline and then online by the aggregator, but the node itself did not restart.

Fixed an issue where setting one of the three engine variables collation_server, collation_database, or collation_connection would not persist the value for all three variables on node restart. (Setting any of these variables will change the other two variables to the same value).

EXPLAIN <query>, PROFILE <query>, and SHOW PROFILE commands are now allowed inside of stored procedures.

Improved predicate transitivity rewrites to avoid adding redundant predicates. This fixes compile timeouts for some queries with large numbers of predicates.

The STATE column of the information_schema.PIPELINES view now indicates if a Pipeline has failed due to insufficient disk space.

Initial GA release of SingleStore 7.1