CLI Online Deployment - Red Hat Distribution

Prerequisites

For this tutorial you will need:

• One (for single-host cluster-in-a-box for development) or four physical or virtual machines (hosts) with the following:

  • Each SingleStore node requires at least four (4) x86_64 CPU cores and eight (8) GB of RAM per host

  • Eight (8) vCPU and 32 GB of RAM are recommended for leaf nodes to align with license unit calculations

  • Running a 64-bit version of RHEL/AlmaLinux 7 or later, or Debian 8 or later, with kernel 3.10 or later

    For SingleStore 8.1 or later, glibc 2.17 or later is also required.

  • Port 3306 open on all hosts for intra-cluster communication. Based on the deployment method, this default can be changed either from the command line or via cluster file.

  • Port 8080 open on the main deployment host for the cluster

  • A non-root user with sudo privileges available on all hosts in the cluster that be used to run SingleStore services and own the corresponding runtime state

• SSH access to all hosts

  • Installing and using ssh-agent is recommended for SSH keys with passwords. Refer to ssh-agent and ssh-add and Use ssh-agent to Manage Private Keys for more information.

  • If your environment does not support the use of ssh-agent , make sure the identity key used on the main deployment host can be used to log in to each host in the cluster. Refer to How to Setup Passwordless SSH Login for more information.

• A connection to the Internet to download required packages

If running this in a production environment, it is highly recommended that you follow our host configuration recommendations for optimal cluster performance.

✘ Host check failed.host 172.26.212.166 has the same ssh
host keys as 172.16.212.165, toolbox doesn't support
registering the same host twice

sudo root
ls -al /etc/ssh/
rm /etc/ssh/<your-ssh-host-keys>
ssh-keygen -f /etc/ssh/<ssh-host-key-filename> -N '' -t rsa1
ssh-keygen -f /etc/ssh/<ssh-host-rsa-key-filename> -N '' -t rsa
ssh-keygen -f /etc/ssh/<ssh-host-dsa-key-filename> -N '' -t dsa

Online Installation - Red Hat Distribution

Run the following commands to install SingleStore Tools.

sudo yum-config-manager --add-repo https://release.memsql.com/production/rpm/x86_64/repodata/memsql.repo && \
sudo yum install -y singlestoredb-toolbox singlestore-client

Troubleshooting

If SingleStore Tools cannot be installed using the above commands, verify the following and re-run the above commands to install SingleStore Tools.

• Verify that the SingleStore repo information is listed under repolist.

  Shell

  Copy

  sudo yum repolist

  repo id            repo name            status
  memsql             MemSQL               125

• Verify that the which package is installed. This is used during the install process to identify the correct package type for your installation.

  Shell

  Copy

  rpm -q which

  • If which is not installed, you must install it before proceeding.

    Shell

    Copy

    sudo yum install -y which

  • If you cannot install which, you will need to specify the package as rpm during the deployment phase.

• If you receive an error that the GPG check failed, which resembles the following:

  Copy

  Importing GPG key <gpg-key>:
   Userid     : "MemSQL Release Engineering <security@memsql.com>"
   Fingerprint: <fingerprint>
   From       : https://release.memsql.com/release-aug2018.gpg
  Key import failed (code 2). Failing package is: singlestore-client-1.0.7-1.x86_64
   GPG Keys are configured as: https://release.memsql.com/release-aug2018.gpg
  Public key for singlestoredb-toolbox-1.13.13-9fa4ef2d34.x86_64.rpm is not installed.
  Failing package is: singlestoredb-toolbox-1.13.13-1.x86_64
   GPG Keys are configured as: https://release.memsql.com/release-aug2018.gpg
  The downloaded packages were saved in cache until the next successful transaction.
  You can remove cached packages by executing 'yum clean packages'.
  Error: GPG check FAILED

  • View the current crypto policies.

    Shell

    Copy

    update-crypto-policies --show

  • If SHA1 is not present, update the crypto policies to work with SingleStore Tools.

    Shell

    Copy

    sudo update-crypto-policies --set DEFAULT:SHA1

    Refer to Using system-wide cryptographic policies for more information.

Prerequisites

CLI Online Deployment

You can deploy SingleStore onto each host from the main deployment host and create the SingleStore nodes for your cluster.

From the main deployment host, deploy the SingleStore on all of your hosts using the setup-cluster command. Hosts are specified via the --master-host, --aggregator-hosts, and --leaf-hosts flags as comma-separated host names. The --password flag specifies the password for the root database user.

Other than the main deployment host being specified as the --master-host, the other hosts in your cluster can be used as hosts for the child aggregator or leaf nodes.

sdb-deploy setup-cluster -i /path/to/yourSSHkey \
    --license <license> \
    --master-host <main_IP_address_or_hostname> \
    --aggregator-hosts <child_agg_IP_address_or_hostname> \
    --leaf-hosts <leaf1_IP_address_or_hostname>,<leaf2_IP_address_or_hostname> \
    --password <secure_password> \
    --version 8.9

For large clusters with many hosts, it may be inconvenient to have to input all the host names in the command line. In place of the individual flags, you may instead use a cluster file. See the Cluster File Deployment for more information.

Note: If your license is not shown in the code block above, you can retrieve it from the Cloud Portal.

The setup-cluster command does several things for you:

• Installs the latest singlestoredb-server package on all hosts in your cluster.

• Deploys SingleStore engine across all of the hosts in your cluster.

• Creates the master aggregator. In this tutorial, the master aggregator resides on the main deployment host.

• Creates any child aggregators specified in either the host file or in the command-line.

• Creates leaf nodes for your cluster. Note: The setup-cluster command only creates one node per host. If your host is NUMA capable and has more than one NUMA node, you can install additional leaf nodes.

• By default, the setup-cluster command will also enable High Availability. To disable High Availability, use the flag --high-availability=false in the setup-cluster command.

After you have deployed your cluster, run sdb-admin optimize. This command checks your current cluster configuration against a set of best practices and either makes changes to maximize performance or provides recommendations for you. For hosts with NUMA support, this command will bind the leaf nodes to specific NUMA nodes.

sdb-admin optimize

If you encounter errors running either of these commands, verify that your deployment environment satisfies the following conditions:

• You can SSH to every host in the cluster using the IPs or hostnames specified in the setup-cluster step above.

• Your deployment user has root or sudo privileges in order to install packages on all hosts:

  Shell

  Copy

  sudo apt-get install ...

• Port 3306 on all hosts is open to all other hosts in the cluster.