Upgrade to SingleStore 8.9

Backups

MemSQL 6.8 and earlier backups cannot be restored in SingleStore 7.8 or later.

If you need to keep older backups for an extended period of time, consider implementing one of the following recommendations:

1. Maintain a 7.5 or earlier installation in a test or pre-production environment to be used in case you ever need to restore 6.8 and earlier backups.

2. Upgrade the backups by restoring them into a 7.0 to 7.5 test or pre-production environment, then backing up the data again. Validate the new backups by restoring them. After that discard the older release backups.

Plancache

Plans in the plancache are dependent upon the specific SingleStore patch version, so when you upgrade to a new SingleStore version, all previously compiled plans will be invalidated. This means that any queries run against the upgraded cluster will force a one-time plan compilation, which results in slower query times the first time those queries are run. After the plans have been recompiled, they will be stored again in the plancache and query latency will return to nominal values.

Non-Sync Variables

By default, convert_nonunique_hash_to_skiplist is set to TRUE as of SingleStore 7.3. This means that any non-unique hash index will be recovered as a skiplist index, any newly created table will also have its non-unique hash indexes created as skiplists. For more information about this engine variable see the Non-Sync Variables List.

Secure Connections

As of SingleStore 8.1, OpenSSL 3.0 is now used to establish secure connections to SingleStore. As a consequence, a client certificate that uses SHA or MD5 hash functions in its signature must be replaced with a certificate that uses SHA256 at a minimum, or a secure connection to SingleStore cannot be established. While SingleStore supports TLS v1, TLS v1.1, and TLS v1.2, using TLS v1.2 is recommended. When FIPS is enabled, only TLS v1.2 is supported. Refer to Troubleshoot OpenSSL 3.0 Connections for more information.

Upgrade Versions and Methods

The tables below depict which versions of SingleStore can be upgraded to SingleStore 8.9 and the method by which the cluster can be upgraded.

• Offline upgrade: Your SingleStore cluster will be shut down and restarted over the course of the upgrade

• Online upgrade: Your SingleStore cluster will not be shut down over the course of the upgrade

Step 1: Upgrade SingleStore Toolbox

To upgrade to SingleStore 7.3, you must have Toolbox 1.5.3 or later installed prior to the SingleStore upgrade process. SingleStore recommends that you use the latest version of Toolbox when upgrading your cluster.

sudo yum install singlestoredb-toolbox -y

sudo yum install /path/to/singlestoredb-toolbox.rpm -y

sudo apt install /path/to/singlestoredb-toolbox.deb -y

Step 2: Upgrade SingleStore

There are two available options for upgrading a cluster:

• Offline Upgrade

  The simplest and preferred upgrade option is an offline upgrade. It is the least error-prone and easiest to execute; however, it requires downtime as all of the nodes in the cluster are upgraded at the same time. Your cluster will be shut down and restarted over the course of the upgrade.

  If the cluster is running with high availability (HA), you also have the option to perform an incremental online upgrade, which maintains cluster availability throughout the upgrade process.

• Online Upgrade

  For high availability (HA) clusters only. With this option, the cluster will not be shut down over the course of the upgrade. Nodes will be restarted in a specific sequence to ensure that DML-based workloads will still function.

  An online upgrade may fail if either a long-running workload that writes to the database or a workload that manipulates SingleStore files (such as an automated backup or maintenance script) is running on the target cluster. SingleStore recommends performing an online upgrade only after these workloads have completed.

Toolbox versions 1.11.7 and later provide the option to retry a failed online upgrade. Should an online upgrade fail, an offline upgrade will be attempted.

When upgrading your cluster:

• If you do not specify a version, your cluster will be upgraded to the latest version and patch release of SingleStore.

• If you specify a major version, your cluster will be upgraded to the latest patch release of that version.

• To upgrade to a specific version and patch release, use the --version option

Refer to SingleStore release notes for available patch versions and sdb-deploy upgrade for more information.

Prior to upgrading your cluster, SingleStore recommends that you take a backup as a standard precautionary measure. Refer to Back Up and Restore Data for more information. The sdb-deploy upgrade command will perform a snapshot of all databases prior to upgrade.

memsql_exporter.go:1001 failed reading ini file: open /etc/memsql/memsql_exporter.cnf: permission denied

sdb-deploy upgrade --precheck-only

Toolbox will perform the following actions:
  · Download singlestoredb-server x.yy.zx

Would you like to continue? [y/N]: y
✓ Downloaded singlestoredb-server production:latest
Toolbox is about to perform following checks:
  · Cluster health checks:
    - Check that all nodes are online and healthy
    - Check that all partitions are healthy
  · Check that there are no pending rebalance operations
  · Take snapshots of all databases

Would you like to continue? [y/N]: y
Checking cluster status
✓ Nodes are online
✓ Partitions are healthy

✓ Snapshots completed
✓ All checks passed successfully
Operation completed successfully

sdb-deploy upgrade --version 8.9

sdb-deploy upgrade --online --version 8.9

sdb-deploy upgrade --file-path /path/to/singlestoredb-server.rpm

sdb-deploy upgrade --file-path /path/to/singlestoredb-server.deb

sdb-deploy upgrade --online --file-path /path/to/singlestoredb-server.rpm

sdb-deploy upgrade --online --file-path /path/to/singlestoredb-server.deb

Collect Event Traces

Existing cluster monitoring instances can be configured to collect event traces after upgrading a cluster to SingleStore v8.5 or later. Refer to Query History for more information on how to fully enable this feature.

Run the following command to restart monitoring and collect event traces.

HTTP Connections

sdb-admin start-monitoring \
  --database-name metrics \
  --collect-event-traces \
  --exporter-host <exporter-hostname-or-IP-address> \
  --user root \
  --password <secure-password> \
  --retention-period 10

HTTPS Connections

sdb-admin start-monitoring \
  --database-name metrics \
  --collect-event-traces \
  --exporter-host=<exporter-hostname-or-IP-address> \
  --user root \
  --password=<secure-password> \
  --retention-period 10 \
  --ssl-ca=/path/to/ca-cert.pem      --or-- 
  --ssl-capath=/ca-directory/including/path

System Behavior

When upgrading to SingleStore 8.9, you should be aware of the following changes to system behavior or default configuration settings. The behavior of a cluster upgraded from an earlier version to SingleStore 8.9 may differ compared to a newly installed cluster on SingleStore 8.9 as described below. Most of the changes fall into two categories:

• In some versions, the default value for a configuration variable was changed compared to previous versions, but clusters upgraded from earlier versions retain their previous setting, both if it was set to a specific value or if it was not explicitly set and hence using the previous default. In some of these cases, SingleStore recommends to update your configuration to the new default if you were previously using the old default, after appropriate testing.

• Some new features are automatically enabled by default on newly installed SingleStore 8.9 clusters but not automatically enabled on clusters upgraded from an earlier version to 8.9. In some of these cases, SingleStore recommends to enable the new features, after appropriate testing.

Upgrades to 8.9

• To reduce your total cost of ownership (TCO), you may be able store data in Universal Storage instead of rowstores. This is because rowstores store their data in RAM, which can be costly. Universal Storage now supports upserts, which were previously only supported in rowstores.

• You may want to run the command REBALANCE ALL DATABASES. This command rebalances each database in the cluster, in alphabetical order of the database name. When a rebalance runs on a database d, it first considers the placement of the partitions of the other databases in the cluster before rebalancing the partitions of d.

• You may want to set the cardinality_estimation_level engine variable to '7.3'. This setting uses sampling and histograms together (when both are available) to improve selectivity estimation. The default setting is '7.1'.

• Changing the value of the data_conversion_compatibility_level engine variable can change the behavior of expressions in computed columns. Refer to the Data Type Conversion section of Data Types for more information.

• sp_query_dynamic_param should be turned off if an application breaks post-upgrade due to a change in type conversion behavior. See the Example: Changes in Type Conversion Behavior for more information.

• Upgrading the cluster, with json_extract_string_collation set to auto (default setting), changes the collation settings for JSON_EXTRACT_STRING from json to server. Refer to In-Depth Variable Definitions for information on json_extract_string_collation settings.