Skip to main content

Upgrade to SingleStoreDB 8.0

Notice

Please note the following before upgrading your cluster:

  • SingleStoreDB betas and release candidates cannot be upgraded unless explicitly stated in the release notes.

  • As of SingleStoreDB 7.8, a new reserved prefix for column names has been introduced: _$!$.

    • This prefix is reserved for internal use only. Before upgrading SingleStoreDB, SingleStore recommends renaming existing columns prefixed with " _$!$ " to avoid potential issues in the future.

    • While this is not mandatory, and DML commands will still allow access to existing objects using this prefix, DDL commands will no longer allow column names to be created with this prefix.

  • If any host in the cluster is near or at disk capacity, please increase available storage before upgrading to SingleStoreDB 8.0.

Important notes about upgrading

This topic describes how to upgrade SingleStoreDB. Please read the following information thoroughly before upgrading..

After you have finished upgrading, please see the Post-Upgrade Considerations section for additional information on behavioral changes that you should be aware of.

Upgrade duration and behavior

Anticipate a longer upgrade time for each node. At the end of the upgrade process, anticipate a multi-minute delay while final checks are run that ensure the upgrade has completed successfully and the cluster is ready for operation. If you connect to the cluster while these post-upgrade checks are running, you will receive an error that the upgrade is still in progress.

Plancache

Plans in the plancache are dependent upon the specific SingleStoreDB patch version, so when you upgrade to a new SingleStoreDB version, all previously compiled plans will be invalidated. This means that any queries run against the upgraded cluster will force a one-time plan compilation, which results in slower query times the first time those queries are run. After the plans have been recompiled, they will be stored again in the plancache and query latency will return to nominal values.

Non-Sync Variables

As of SingleStoreDB 7.3, convert_nonunique_hash_to_skiplist is set to TRUE by default. This means that any non-unique hash index will be recovered as a skiplist index, any newly created table will also have its non-unique hash indexes created as skiplists. For more information about this engine variable see the Non-Sync Variables List.

Verify your cluster is ready for upgrade

Warning

Only clusters that are running SingleStoreDB 7.3 and later can upgrade directly to SingleStoreDB 8.0.

If upgrading from SingleStoreDB 7.0 and later with DR clusters created via Replication, SingleStore recommends that you upgrade your DR secondary cluster(s) one at a time, and then upgrade your primary cluster last so that replication will continue to work after each upgrade.

To upgrade from MemSQL 6.7 or 6.8, or from SingleStoreDB 7.0, a two-step upgrade process is recommended.

  1. Upgrade to SingleStoreDB 7.3.

  2. Use this guide to upgrade to SingleStoreDB 8.0.

Prior to upgrading your cluster, it is recommended that you take a backup as a standard precautionary measure. See Back Up and Restore Data.

In addition, run the following SQL commands from the Master Aggregator to confirm that the following are true.

  • All leaves are online.

    SHOW LEAVES;
  • All aggregators are online.

    SHOW AGGREGATORS;
  • There are no partitions with an Orphan role.

    SHOW CLUSTER STATUS;
  • No rebalance or restore redundancy is necessary.

    EXPLAIN REBALANCE PARTITIONS;
    EXPLAIN RESTORE REDUNDANCY;

After you have backed up your data and verified your cluster is ready, you are ready to upgrade your cluster to the latest version of SingleStoreDB using the SingleStoreDB management tools.

Upgrade your cluster

Upgrade versions and methods

The tables below depict which versions of SingleStoreDB can be upgraded to SingleStoreDB 8.0 and the method by which the cluster can be upgraded.

  • Offline upgrade: Your SingleStoreDB cluster will be shut down and restarted over the course of the upgrade

  • Online upgrade: Your SingleStoreDB cluster will not be shut down over the course of the upgrade

Upgrade via SingleStoreDB Toolbox

Upgrade from

Offline upgrade

Online upgrade

7.3 and later

From 7.3 and later

Step 1: Upgrade SingleStoreDB Toolbox

To upgrade to SingleStoreDB 8.0, you must have Toolbox 1.14.0 or later installed prior to the SingleStoreDB upgrade process. It is recommended that you use the latest version of Toolbox when upgrading your cluster.

Run the install command to either install Toolbox if it is not already installed, or upgrade an existing version of Toolbox to the latest version of the package.

RHEL/CentOS

sudo yum install singlestoredb-toolbox -y

Debian

sudo apt install singlestoredb-toolbox -y

Upgrade without Internet access

If your cluster does not have internet access, use one of the following buttons to download either the latest RPM or Debian singlestoredb-toolbox package to a location accessible by your cluster:

singlestoredb-toolbox (.rpm)

singlestoredb-toolbox (.deb)

Then, run the install command for the installed package manager and specify the path to the package.

RHEL/CentOS

sudo yum install /path/to/singlestoredb-toolbox.rpm -y

Debian

sudo apt install /path/to/singlestoredb-toolbox.deb -y

Step 2: Upgrade SingleStoreDB

The simplest and preferred upgrade option is an offline cluster upgrade. It is the least error-prone and easiest to perform; however, it requires downtime as all of the nodes in the cluster will be upgraded at the same time, shutting down the entire cluster for the duration of the upgrade.

If the cluster is running with High Availability, you also have the option to perform an incremental online cluster upgrade, which maintains cluster availability throughout the upgrade process. See the instructions in the Online upgrade section for more details.

Select an option below to upgrade your cluster.

Option 1: Offline upgrade

Note: Using this method to upgrade SingleStoreDB is referred to as an offline upgrade as your SingleStoreDB cluster will be shut down and restarted over the course of the upgrade. Do not shut down your cluster prior to starting the upgrade. If the cluster or individual nodes are offline when the upgrade is started, the upgrade will fail.

Run the following command to start an offline upgrade.

  1. Confirm that the cluster can be upgraded. The cluster will not be upgraded when running this command.

    sdb-deploy upgrade --precheck-only
    
  2. Upgrade your cluster.

    Note: sdb-deploy upgrade will perform a snapshot of all databases prior to upgrade.

    sdb-deploy upgrade --version 8.0

If you do not specify a patch version, your cluster will be upgraded to the latest patch version of SingleStoreDB 8.0.

During the upgrade process, you will be prompted to enable synchronous replication on your existing databases, or to leave those databases using the previous asynchronous replication behavior.

Synchronous replication is enabled by default on all new 8.0 databases and provides an extra layer of resiliency in clusters with high availability enabled.

Note: You cannot downgrade from your current version.

If your cluster does not have internet access, use one of the following buttons to download either the latest RPM or Debian singlestoredb-server package to a location accessible by your cluster:

singlestoredb-server (.rpm)

singlestoredb-server (.deb)

The singlestoredb-server package contains both the SingleStoreDB binary and the low-level management tool, memsqlctl.

Run the sdb-deploy upgrade command and reference the appropriate package in the --file-path option. Running upgrade (as opposed to simply upgrading the package via the package manager) will perform an offline restart of all the nodes to make sure the cluster is using the new version.

RHEL/CentOS

sdb-deploy upgrade --file-path /path/to/singlestoredb-server.rpm

Debian

sdb-deploy upgrade --file-path /path/to/singlestoredb-server.deb

Refer to sdb-deploy upgrade for more information.

Once you have finished upgrading all of your nodes, see Post-upgrade considerations for any changes between versions that may impact your cluster.

Option 2: Online upgrade

This upgrade method is referred to as an online upgrade as your SingleStoreDB cluster will not be shut down over the course of the upgrade. Nodes will be restarted in a specific sequence to ensure that DML-based workloads will still function. Do not shut down your cluster prior to starting the upgrade. If the cluster or individual nodes are offline when the upgrade is started, the upgrade will fail.

Notice

An online upgrade may fail if a long-running workload that writes to the database, or a workload that manipulates SingleStoreDB files (such as an automated backup or maintenance script), is running on the target cluster. SingleStore recommends performing an online upgrade only after these workloads have completed.

Toolbox 1.11.7 and later provide the option to retry a failed online upgrade. Should the online upgrade ultimately fail, an offline upgrade will be attempted. Note that the cluster will be shut down and restarted over the course of an offline upgrade.

Run the following command to start an online upgrade.

  1. Confirm that the cluster can be upgraded. The cluster will not be upgraded when running this command.

    sdb-deploy upgrade --precheck-only
    
  2. Upgrade your cluster.

    Note: sdb-deploy upgrade will perform a snapshot of all databases prior to upgrade.

    sdb-deploy upgrade --online --version 8.0
    

By specifying --version, your cluster will be upgraded to the latest patch version of SingleStoreDB 8.0.

During the upgrade process, you will be prompted to enable synchronous replication on your existing databases. While asynchronous replication can still be chosen, the data replication mechanism as of SingleStoreDB 7.0 has been enhanced to the point where synchronous replication is now the recommended option.

Note: You cannot downgrade from your current version.

If your cluster does not have internet access, use one of the following buttons to download either the latest RPM or Debian singlestoredb-server package to a location accessible by your cluster:

singlestoredb-server (.rpm)

singlestoredb-server (.deb)

The singlestoredb-server package contains both the SingleStoreDB binary and the low-level management tool, memsqlctl.

Run the sdb-deploy upgrade --online command and reference the appropriate package in the --file-path option. Running upgrade (as opposed to simply upgrading the package via the package manager) will perform an offline restart of all the nodes to make sure the cluster is using the new version.

RHEL/CentOS

sdb-deploy upgrade --online --file-path /path/to/singlestoredb-server.rpm

Debian

sdb-deploy upgrade --online --file-path /path/to/singlestoredb-server.deb

Refer to sdb-deploy upgrade for more information.

Once you have finished upgrading all of your nodes, see Post-upgrade considerations for any changes between versions that may impact your cluster.

Post-upgrade considerations

When upgrading to SingleStoreDB 8.0, you should be aware of the following changes to system behavior or default configuration settings. The behavior of a cluster upgraded from an earlier version to SingleStoreDB 8.0 may differ compared to a newly installed cluster on SingleStoreDB 8.0 as described below. Most of the changes fall into two categories:

  • In some versions, the default value for a configuration variable was changed compared to previous versions, but clusters upgraded from earlier versions retain their previous setting, both if it was set to a specific value or if it was not explicitly set and hence using the previous default. In some of these cases, it is recommended to update your configuration to the new default if you were previously using the old default, after appropriate testing.

  • Some new features are automatically enabled by default on newly installed SingleStoreDB 8.0 clusters but not automatically enabled on clusters upgraded from an earlier version to 8.0. In some of these cases, it is recommended to enable the new features, after appropriate testing.

Upgrades to 8.0

  • To reduce your total cost of ownership (TCO), you may be able store data in Universal Storage instead of rowstores. This is because rowstores store their data in RAM, which can be costly. Universal Storage now supports upserts, which were previously only supported in rowstores.

  • You may want to run the command REBALANCE ALL DATABASES. This command rebalances each database in the cluster, in alphabetical order of the database name. When a rebalance runs on a database d, it first considers the placement of the partitions of the other databases in the cluster before rebalancing the partitions of d.

  • You may want to set the cardinality_estimation_level engine variable to '7.3'. This setting uses sampling and histograms together (when both are available) to improve selectivity estimation. The default setting is '7.1'.

  • Changing the value of the data_conversion_compatibility_level engine variable can change the behavior of expressions in computed columns. Refer to the Data Type Conversion section of Data Types for more information.

  • sp_query_dynamic_param should be turned off if an application breaks post-upgrade due to a change in type conversion behavior. See the Example: Changes in Type Conversion Behavior for more information.

  • Upgrading the cluster, with json_extract_string_collation set to auto (default setting), changes the collation settings for JSON_EXTRACT_STRING from json to server. Refer to In-Depth Variable Definitions for information on json_extract_string_collation settings.