SingleStoreDB Toolbox and memsqlctl Differences

SingleStoreDB Toolbox

SingleStoreDB Toolbox (or simply “Toolbox”) is a collection of command-line tools that can be used to deploy and manage clusters. Toolbox’s advantages are most apparent on a multi-host cluster, where it can be used to simplify high-level administrative tasks. Toolbox is packaged as Debian, RPM, or tar files, which allows it to be incorporated into common, automated deployment workflows.

Toolbox consists of four primary tools:

• sdb-deploy: Used to deploy and upgrade the SingleStoreDB server across hosts in a cluster.

• sdb-admin: Used to perform cluster administrative tasks, such as adding nodes, enabling high availability, setting up cluster monitoring, performing backup operations, optimizing performance settings, and more.

• sdb-report: Used to collect diagnostic information about a cluster.

• sdb-toolbox-config: Primarily used to register and unregister hosts in a cluster.

Toolbox can be installed on a cluster host or any Linux computer with network access to all of the associated cluster hosts. Toolbox must be able to connect via SSH to each cluster host as one of the following:

• The user that runs memsqld

• The Linux root user

• A Linux user with sudo privileges

Toolbox requires the following information to operate:

• A list of cluster hosts

• The credentials for each host, such as Linux username and SSH key

Toolbox stores the list of registered cluster hosts and their credentials in a state file. The path to the state file can be obtained by calling the env sub-command of a Toolbox command, such as sdb-admin env or sdb-report env.

Contrary to SingleStore’s legacy management tool, MemSQL Ops, Toolbox does not actively manage a cluster. When a Toolbox command is invoked, Toolbox connects to each host in the cluster via SSH and executes a memsqlctl command which, in turn, issues the command to all nodes in the cluster. Once the command execution finishes, Toolbox shuts down and does not actively engage with the cluster. In addition, you may run more than one Toolbox instance to manage a single cluster, and you can manually reconfigure Toolbox to manage a different cluster.

memsqlctl

The memsqlctl low-level management tool that can run commands against nodes on the same host, where it uses the following configuration files when manipulating the cluster:

• memsql.cnf: Stores engine variables, the values of which are loaded when a node starts.

• memsqlctl.hcl: Overrides the default settings for the entire database system. It also contains the location of the nodes.hcl file.

• nodes.hcl: Contains information about SingleStoreDB nodes that exist on that host. Included are the location of the memsql.cnf file used by that node and an encrypted password to use to connect to the node.

The memsqlctl tool is included in the singlestoredb-server / memsql-server package/tarball file (along with the SingleStoreDB engine) and does not require a separate installation. It controls the startup process of memsqld, which reads the memsql.cnf configuration file when starting up.

Several administrative tasks supported by memsqlctl can be accomplished via a SQL interface. However, memsqlctl helps avoid the manual configurations involved when performing certain tasks, thereby minimizing human error.

For example, to create a node manually, one must:

• Craft a new memsql.cnf file from scratch

• Start memsqld manually, specifying all of the required memsqld arguments on the command line

As starting memsqld this way can be error-prone, memsqlctl can (and should) be used instead. With memsqlctl, these actions are performed by reading information about the nodes on a host from the memsqlctl.hcl file. Refer to memsqlctl for more information about the actions supported by this tool.

The following directory structures show the location of key cluster processes and the associated configuration files.

For each node:

• /opt/singlestoredb-server-<version>/memsqld -- or --

  /opt/memsql-server-<version>/memsqld

• /usr/bin/memsqlctl

• /etc/memsql/memsqlctl.hcl

• /var/lib/memsql/nodes.hcl

• /var/lib/memsql/<node-ID>/memsql.cnf