setup-cluster
On this page
Description
Deploy a SingleStoreDB cluster across multiple hosts
This command conveniently runs various sdb-toolbox-config
, sdb-deploy
, and sdb-admin
commands as a single command to produce a functional SingleStoreDB cluster that can be connected to and queried.
The --password
flag is required and specifies the SingleStoreDB root
password on each node.MEMSQL_
environment variable is a safer alternative option for setting the password.
If you would instead like to set up a local SingleStoreDB "cluster" for testing purposes, see sdb-deploy cluster-in-a-box.
Examples
sdb-deploy setup-cluster \--master-host user@host1:33 \--aggregator-hosts host2 \--leaf-hosts host3,host4 \--identity-file ~/my_key.pem \--license AAAAAAa/Aaa/AA== \--password testpass
This will install the latest released version of singlestoredb-server
on each host and deploy a Master Aggregator on host1
; a child aggregator on host2
; and leaf nodes on host3
and host4
.~/my_
.MEMSQL_
environment variable, you can omit the --license
flag.
"High Availability" mode is enabled by default.--high-availability
to false:
sdb-deploy setup-cluster --master-host host1 ... --high-availability=false
The command will deploy SingleStoreDB on port 3306
on each node by default.--memsql-port
:
sdb-deploy setup-cluster --master-host host1 ... --memsql-port 3307
If you want to install a version other than the latest released version, you can specify a different released version, or the path to a singlestoredb-server
package:
sdb-deploy setup-cluster --master-host host1 ... --version 6.7.0
sdb-deploy setup-cluster --master-host host1 ... --file-path path/to/singlestoredb-server.deb
sdb-deploy setup-cluster --master-host host1 ... --file-path path/to/singlestoredb-server.rpm
If you need to specify a host's SSH user or SSH port, you can add the information to the host descriptor:
sdb-deploy setup-cluster --master-host admin@host1:10022 --leaf-hosts admin@host2:10022,admin@host3:10022 ...
If you want to use a YAML file to define your cluster you can do so by using the --cluster-file
flag which takes in the path to a YAML file as input.--cluster-file
.
Usage of the command with --cluster-file
flag:
sdb-deploy setup-cluster --cluster-file path/to/cluster-file.yml
YAML File Configuration
As of SingleStoreDB Toolbox 1.sdb-deploy setup-cluster
command now accepts YAML configuration files, and validates the configuration file format before attempting to set up the specified cluster.
The command is designed to be consistent, where re-running the sdb-deploy setup-cluster
command with the same cluster file will always produce the same cluster.sdb-deploy setup-cluster
re-run, in order to generate the desired cluster.
Steps for using a Configuration File
-
Install SingleStoreDB Toolbox onto the main deployment host.
-
Using the template below, create a YAML file with your desired cluster configuration.
license: <license-from-portal.singlestore.com>memsql_server_version: <version>package_type: <type> ← rpm, deb, or tarhosts:- hostname: <ip-address>localhost: truenodes:- register: falserole: Masterconfig:password: <secure-password>port: 3306- register: falserole: Leafconfig:password: <secure-password>port: 3307 -
Run the following with the path to the YAML file as input.
sdb-deploy setup-cluster --cluster-file <path-to-cluster-file>
Note: By default, the sdb-deploy setup-cluster
command configures the host's operating system per SingleStore's recommendations.--skip-auto-config=true
option to the hosts
field in the YAML configuration file.
Complete Cluster File Template
license: <LICENSE | /path/to/LICENSE-file> [Required to bootstrap Master Aggregator]high_availability: <true | false>memsql_server_version: <the version of memsql you want to install (6.7+)>memsql_server_file_path: <path to the downloaded memsql server file>memsql_server_preinstalled_path: <equivalent to using the '--preinstalled-path' option;the path to the unpacked singlestoredb-server filewhere the unpacked folder name must be of the form'singlestoredb-server-<version>*' or'memsql-server-<version>*'>skip_install: <true | false> [ADVANCED]skip_validate_env: <true | false> [ADVANCED]allow_duplicate_host_fingerprints: <true | false> [ADVANCED]assert_clean_state: <true | false> [ADVANCED]package_type: <deb | rpm | tar> [Required if multiple package present]root_password: <default password to be used for all nodes>optimize: <true | false>optimize_config:memory_percentage: <percentage of memory you want memsql to use>no_numa: <true | false>sync_variables: [ADVANCED]<variable's name>: <variable's value>hosts:- hostname: <host-name> [Required]localhost: <true | false>skip_auto_config: <true | false>memsqlctl_path: <path to memsqlctl> [ADVANCED]memsqlctl_config_path: <path to memsqlctl config> [ADVANCED]tar_install_dir: <path to tar install dir> [ADVANCED]tar_install_state: <path to tar install state> [ADVANCED]ssh: [Required for remote Hosts]host: <ssh host name>port: <ssh port>user: <ssh user>private_key: <path to your identity key>nodes:- register: <true | false>force_registration: <true | false> [ADVANCED]role: <Unknown | Master | Leaf | Aggregator> (case sensitive) [Required]availability_group: <availability group>no_start: <true | false>config:auditlogsdir: <path to auditlogs directory> [ADVANCED]baseinstalldir: <path to base install directory> [ADVANCED]configpath: <path to configuration path> [ADVANCED] [Required if register is true]datadir: <path to data directory> [ADVANCED]disable_auto_restart: <true | false>password: <password>plancachedir: <path to plancache directory> [ADVANCED]port: <port number> [Required for node creation]tracelogsdir: <path to tracelogs directory> [ADVANCED]bind_address: <bind address> [ADVANCED]ssl_fips_mode: <true | false > [ADVANCED]variables:<variable's name>: <variable's value>
Example Cluster Files
The following cluster file template examples can be used to set up a cluster.
Copy an example into an example.
file and pass this file to the sdb-deploy setup-cluster
command.
sdb-deploy setup-cluster --cluster-file example.yaml
Note: Depending on how you installed SingleStoreDB Toolbox (for example, via tarball-based install, you may need to include the full path to the cluster file.
./sdb-deploy setup-cluster --cluster-file /full-path/example.yaml
For Example 2, Example 3, and Example 4, you will need multiple hosts and the ability to ssh
into each host from the main deployment host.
Example 1
license: <license-from-portal.singlestore.com>memsql_server_version: 7.3.10package_type: rpmhosts:- hostname: 127.0.0.1localhost: truenodes:- register: falserole: Masterconfig:password: <secure-password>port: 3306- register: falserole: Leafconfig:password: <secure-password>port: 3307
Using this cluster file, sdb-deploy setup-cluster
:
-
Registers a local host to the cluster.
-
Installs
singlestoredb-server
v7.3. 10 on this local host. -
Creates a Master Aggregator node on port
3306
. -
Creates a leaf node on port
3307
.
Example 2
license: <license-from-portal.singlestore.com>memsql_server_version: 7.3.10package_type: rpmhosts:- hostname: <hostname1>localhost: true- hostname: <hostname2>ssh:host: <cluster-addressable-hostname>private_key: <path/to/private-key>
Using this cluster file, sdb-deploy setup-cluster
:
-
Registers a local host to the cluster.
-
Registers a remote host to the cluster using the provided
ssh
credentials. -
Installs
singlestoredb-server
v7.3. 10 on both hosts.
Example 3
license: <license-from-portal.singlestore.com>memsql_server_version: 7.3.10package_type: rpmhosts:- hostname: <hostname1>localhost: truenodes:- register: falserole: Masterconfig:password: <secure-password>port: 3306- register: falserole: Leafconfig:password: <secure-password>port: 3307- hostname: <hostname2>ssh:host: <cluster-addressable-hostname>private_key: <path/to/private-key>nodes:- register: falserole: Aggregatorconfig:password: <secure-password>port: 3306- register: falserole: Leafconfig:password: <secure-password>port: 3307
Using this cluster file, sdb-deploy setup-cluster
:
-
Registers a local host to the cluster.
-
Register a remote host to the cluster using the provided
ssh
credentials. -
Installs
singlestoredb-server
v7.3. 10 on both the hosts. -
On Host 1, creates a node on port
3306
, bootstraps it as the Master Aggregator using the specified license, and sets the password to the value provided in the cluster file. -
On Host 1, creates a node on port
3307,
assign it as a leaf node, and sets the password to the value provided in the cluster file. -
On Host 2, creates a node on port
3306
, assign it as a Child Aggregator, and sets the password to the value provided in the cluster file. -
On Host 2, creates a node on port
3307
, assign it as a leaf node, and sets the password to the value provided in the cluster file.
Example 4
license: <license-from-portal.singlestore.com>high_availability: truepackage_type: rpmmemsql_server_file_path: /home/ec2-user/singlestoredb-server-7.3.10-5aa7127018.x86_64.rpmhosts:- hostname: <hostname1>localhost: truenodes:- role: Masterconfig:password: <secure-password>port: 3306- hostname: <hostname2>ssh:host: <cluster-addressable-hostname>private_key: <path/to/private-key>nodes:- role: Aggregatorconfig:password: <secure-password>- hostname: <hostname3>ssh:host: <cluster-addressable-hostname>private_key: <path/to/private-key>nodes:- role: Aggregatorconfig:password: <secure-password>- hostname: <hostname4>ssh:host: <cluster-addressable-hostname>private_key: <path/to/private-key>nodes:- role: Leafavailability_group: 1config:password: <secure-password>- hostname: <hostname5>ssh:host: <cluster-addressable-hostname>private_key: <path/to/private-key>nodes:- role: Leafavailability_group: 1config:password: <secure-password>- hostname: <hostname6>ssh:host: <cluster-addressable-hostname>private_key: <path/to/private-key>nodes:- role: Leafavailability_group: 2config:password: <secure-password>- hostname: <hostname7>ssh:host: <cluster-addressable-hostname>private_key: <path/to/private-key>nodes:- role: Leafavailability_group: 2config:password: <secure-password>
Using this cluster file, sdb-deploy setup-cluster
:
-
Enables high availability.
-
Specifies the use of RPM packages.
-
Specifies the location of the
singlestoredb-server
package which will be used directly instead of downloaded. -
Installs
singlestoredb-server
v7.3. 10 on all hosts. -
On Host 1, creates a node on port
3306
, bootstraps it as the Master Aggregator using the specified license, and sets the password to the value provided in the cluster file. -
On Host 2, creates a node on the default port (
3306
), assign it as a Child Aggregator, and sets the password to the value provided in the cluster file. -
On Host 3, creates a node on the default port (
3306
), assign it as a Child Aggregator, and sets the password to the value provided in the cluster file. -
On Host 4, creates a node on the default port (
3306
), assign it as a leaf node, sets the node’s availability group to1
, and sets the password to the value provided in the cluster file. -
On Host 5, creates a node on the default port (
3306
), assign it as a leaf node, sets the node’s availability group to1
, and sets the password to the value provided in the cluster file. -
On Host 6, creates a node on the default port (
3306
), assign it as a leaf node, sets the node’s availability group to2
, and sets the password to the value provided in the cluster file. -
On Host 7, creates a node on the default port (
3306
), assign it as a leaf node, sets the node’s availability group to2
, and sets the password to the value provided in the cluster file.
Example 5
license: <license-from-portal.singlestore.com>memsql_server_version: 8.0.5package_type: debsync_variables:geo_sphere_radius: 10000.66cluster_name: <cluster-name>hosts:- hostname: <hostname1>localhost: truenodes:- register: falserole: Masterconfig:password: <secure-password>port: 3306- register: falserole: Leafconfig:password: <secure-password>port: 3307- hostname: <hostname2>ssh:host: <cluster-addressable-hostname>private_key: <path/to/private-key>nodes:- register: falserole: Aggregatorconfig:password: <secure-password>port: 3306- register: falserole: Leafconfig:password: <secure-password>port: 3307variables:http_api_pool_capacity: 4048allow_unsupported_filesystem: ON
Using this cluster file, sdb-deploy setup-cluster
:
-
Registers a local host to the cluster.
-
Register a remote host to the cluster using the provided
ssh
credentials. -
Installs
singlestoredb-server
v8.0. 5 on both hosts. -
On Host 1, creates a node on port
3306
, bootstraps it as the Master Aggregator using the specified license, and sets the password to the value provided in the cluster file. -
On Host 1, creates a node on port
3307
, assign it as a leaf node, and sets the password to the value provided in the cluster file. -
On Host 2, creates a node on port
3306
, assign it as a Child Aggregator, and sets the password to the value provided in the cluster file. -
On Host 2, creates a node on port
3307
, assign it as a leaf node, setshttp_
andapi_ pool_ capacity allow_
variables to values provided in the cluster file, and sets the password to the value provided in the cluster file.unsupported_ filesystem -
Sets
geo_
andsphere_ radius cluster_
sync variables to values provided in the cluster file.name
Definitions
The following is a detailed explanation of each field that can be configured in the cluster file.
CLUSTER
Option |
Description |
---|---|
|
When set to |
|
When set to |
|
This enables high availability on your cluster. |
|
The license that will be used for the cluster. |
|
When set to |
|
Includes two fields that can change how the cluster is optimized. |
|
The default password for all the nodes if a password is not specified in the node password field. |
|
When set to |
|
When set to |
|
The list of sync variables that will be set on the Master Aggregator node. |
The following two options are related to the version of SingleStoreDB that will be installed for your cluster.singlestoredb-server
is installed.
Option |
Description |
---|---|
|
The desired version of |
|
Installs version of |
|
The package type to use for the installation: |
HOST
Option |
Description |
---|---|
|
The cluster-addressable hostname of this host. |
|
Use this option to specify that the target host is |
|
The path to the |
|
The path to the |
|
When set to |
|
The directory to use for tarball-based installs on the target host. |
|
The path to the |
The following ssh
-related fields are required for remote hosts:
Option |
Description |
---|---|
|
The |
|
The |
|
The identity file for the host (typically found in |
|
The |
NODE
Option |
Description |
---|---|
|
The availability group to assign to the node when |
|
When set to |
|
If set to |
|
Set the value of this field to |
|
The role of the current node, which is one of |
|
The list of local variables that will be set on the node before the node starts. |
CONFIG
Option |
Description |
---|---|
|
The absolute path to the |
|
The absolute path to the base install directory. |
|
The absolute path to the node’s configuration file. |
|
The absolute path to the node’s |
|
When a node is created, its associated data and directories will be tied to a base install directory known as the |
|
If set to |
|
The SingleStoreDB password for the node. |
|
The absolute path to the |
|
The port that this node uses on its host. |
|
When set to |
|
The absolute path to the |
Host File Structure
For large clusters, you can specify all of your hosts' SSH information through an INI file that you reference using the --host-file
flag.
[master][ssh_user@]<master_host_IP_address|DNS_name>[:ssh_port][aggregators][ssh_user@]<agg_host1_IP_address|DNS_name>[:ssh_port]...[leaves][ssh_user@]<leaf_host1_IP_address|DNS_name>[:ssh_port]...
Usage
Usage:sdb-deploy setup-cluster [flags]For flags that can accept multiple values (indicated by VALUES after the name of the flag),separate each value with a comma.Flags:--aggregator-hosts strings SSH addresses of the child aggregator hosts--allow-duplicate-host-fingerprints Whether to allow hosts with identical SSH host keys. (ADVANCED)--allow-ipv6 Sets allow_ipv6 to true for all nodes in cluster--base-install-dir ABSOLUTE_PATH The absolute path to the base install directory for all the nodes--cluster-file FILE_PATH The path to the cluster definition file--data-dir ABSOLUTE_PATH The absolute path to the data directory for all the nodes--file-path ABSOLUTE_PATH The full path, including the filename, of the singlestoredb-server package to install--force-package-format {deb, rpm, tar} Specify 'deb', 'rpm', or 'tar' package format. Required if you have both dpkg and rpm package managers found on the host-h, --help Help for setup-cluster--high-availability Indicate whether to deploy the cluster in high-availability mode (default true)--hosts-file ABSOLUTE_PATH The file specifying the host addresses/hostnames for the Master Aggregator, child aggregators, and leaf nodes, in INI format with [master], [aggregators], and [leaves] sections-i, --identity-file ABSOLUTE_PATH The identity file to SSH to each host--leaf-hosts strings SSH addresses/hostnames of each leaf node host. At least one must be specified--license LICENSE The license for the cluster--load-balanced Use load_balanced mode for partitioning leaf nodes when high availability is enabled--master-host string SSH address/hostname of the Master Aggregator host--memsql-port PORT The port each node will run on--password STRING The database root user's password. If a password is specified on the command line, it must not contain an unescaped '$' character as it will be replaced by the shell--preinstalled-path ABSOLUTE_PATH The path to the folder (must be of the form 'singlestoredb-server-<version>*' or 'memsql-server-<version>*') that contains the singlestoredb-server binaries. Depending on the mode of cluster deployment, the folder contains either the unpacked tarball file or the installed package file (Debian or RPM) of singlestoredb-server. (ADVANCED)--skip-auto-config Skip automatic operating system configuration--skip-validate-env Skip environment validation--ssl-fips-mode string Whether the cluster operates in OpenSSL/FIPS mode--version VERSION The version of singlestoredb-server to install (default production:latest)Global Flags:--backup-cache FILE_PATH File path for the backup cache--cache-file FILE_PATH File path for the Toolbox node cache-c, --config FILE_PATH File path for the Toolbox configuration--disable-colors Disable color output in console, which some terminal sessions/environments may have difficulty with--disable-spinner Disable the progress spinner, which some terminal sessions/environments may have issues with-j, --json Enable JSON output--parallelism POSITIVE_INTEGER Maximum number of operations to run in parallel--runtime-dir DIRECTORY_PATH Where to store Toolbox runtime data--ssh-max-sessions POSITIVE_INTEGER Maximum number of SSH sessions to open per host, must be at least 3--ssh-strict-host-key-checking Enable strict host key checking for SSH connections--ssh-user-known-hosts-file FILE_PATH Path to the user known_hosts file for SSH connections. If not set, /dev/null will be used--state-file FILE_PATH Toolbox state file path-v, --verbosity count Increase logging verbosity: valid values are 1, 2, 3. Usage -v=count or --verbosity=count-y, --yes Enable non-interactive mode and assume the user would like to move forward with the proposed actions by default
Remarks
This command is interactive unless you use either --yes
or --json
flag to override interactive behavior.
Output
The following example uses the setup-cluster
command to set up a simple cluster on two different hosts using a hosts
file.
hosts file
[master]172.18.1.4[leaves]172.18.1.5
Setting up a cluster with a hosts file
sdb-deploy setup-cluster -i clusterkey --license <license> --hosts-file hosts --password SecureKey
✓ Registered hosts
sdb-deploy will perform the following actions:
· Install singlestoredb-server 7.3.10 on hosts
- 172.18.1.4
- 172.18.1.5
· Deploy a master aggregator on 172.18.1.4:3306
- Enable high availability mode
· Deploy a leaf node on 172.18.1.5:3306
· Set root password on all nodes
Would you like to continue? [y/N]: y
✓ Downloaded singlestoredb-server 7.3.10
Installing SingleStoreDB on all hosts...
✓ Installed singlestoredb-server7.3.10-6c220bccfc on host 172.18.1.4 (1/2)
✓ Installed singlestoredb-server7.3.10-6c220bccfc on host 172.18.1.5 (2/2)
✓ Successfully installed on 2 hosts
✓ Created master node
✓ Successfully set license
✓ Bootstrapped master aggregator
✓ Enabled high availability mode
✓ Created aggregator nodes
✓ Added aggregators nodes to cluster
✓ Created leaf nodes
✓ Added leaf nodes to cluster
To view your cluster, run 'sdb-admin list-nodes'
Last modified: October 6, 2023