DB Software v7.8

UI Online Deployment - Red Hat Distribution

Introduction

Installing SingleStore DB on bare metal, on virtual machines, or in the cloud can be done through the use of popular configuration management tools or through SingleStore’s management tools.

In this guide, you will deploy a SingleStore DB cluster onto physical or virtual machines and connect to the cluster using our monitoring, profiling, and debugging tool, SingleStore DB Studio Overview.

A four-node cluster is the minimal recommended cluster size for showcasing SingleStore DB as a distributed database with high availability; however, you can use the procedures in this tutorial to scale out to additional nodes for increased performance over large data sets or to handle higher concurrency loads. To learn more about SingleStore’s design principles and topology concepts, see Distributed Architecture.

Notice

There are no licensing costs for using up to four license units for the leaf nodes in your cluster. If you need a larger cluster with more/larger leaf nodes, please create an Enterprise License trial key.

Prerequisites

For this tutorial you will need:

  • One (for single-host cluster-in-a-box for development) or four physical or virtual machines (“hosts”) with the following:

    • Each SingleStore DB node requires at least four (4) x86_64 CPU cores and eight (8) GB of RAM per host

    • Eight (8) vCPU and 32 GB of RAM are recommended for leaf nodes to align with license unit calculations

    • Running 64-bit version of RHEL/CentOS 6 or higher or Debian 8 or higher, with kernel 3.10 or higher

    • Port 3306 open on all hosts for intra-cluster communication. This default can be changed in the cluster file.

    • Port 8080 open on the main deployment host for the cluster

    • A non-root user with sudo privileges available on all hosts in the cluster that be used to run SingleStore DB services and own the corresponding runtime state

  • SSH access to all hosts (installing and using ssh-agent is recommended for SSH keys with passwords).

    • If using SSH keys, make sure the identity key used on the main deployment host can be used to log in to the other hosts.

    • Refer to How to Setup Passwordless SSH Login for more information on using SSH without a password.

  • A connection to the Internet to download required packages

If running this in a production environment, it is highly recommended that you follow our host configuration recommendations for optimal cluster performance.

Duplicate Hosts

As of SingleStore DB Toolbox 1.4.4, a check for duplicate hosts is performed before SingleStore DB is deployed, and will display a message similar to the following if more than one host has the same SSH host key:

✘ Host check failed.host 172.26.212.166 has the same ssh
host keys as 172.16.212.165, toolbox doesn't support
registering the same host twice

Confirm that all specified hosts are indeed different and aren’t using identical SSH host keys. Identical host keys can be present if you have instantiated your host instances from images (AMIs, snapshots, etc.) that contain existing host keys. When a host is cloned, the host key (typically stored in /etc/ssh/ssh_host_<cipher>_key) will also be cloned.

As each cloned host will have the same host key, an SSH client cannot verify that it is connecting to the intended host. The script that deploys SingleStore DB will interpret a duplicate host key as an attempt to deploy to the same host twice, and the deployment will fail.

The following steps demonstrate a potential remedy for the “duplicate hosts” message. Please note these steps may slightly differ depending on your Linux distribution and configuration.

$ sudo root
# ls -al /etc/ssh/
# rm /etc/ssh/<your-ssh-host-keys>
# ssh-keygen -f /etc/ssh/<ssh-host-key-filename> -N '' -t rsa1
# ssh-keygen -f /etc/ssh/<ssh-host-rsa-key-filename> -N '' -t rsa
# ssh-keygen -f /etc/ssh/<ssh-host-dsa-key-filename> -N '' -t dsa

For more information about SSH host keys, including the equivalent steps for Ubuntu-based systems, refer to Avoid Duplicating SSH Host Keys.

As of SingleStore DB Toolbox 1.5.3, sdb-deploy setup-cluster supports an --allow-duplicate-host-fingerprints option that can be used to ignore duplicate SSH host keys.

Network Configuration

Depending on the host and its function in deployment, some or all of the following port settings should be enabled on hosts in your cluster.

These routing and firewall settings must be configured to:

  • Allow database clients (e.g. your application) to connect to the SingleStore DB aggregators

  • Allow all nodes in the cluster to talk to each other over the SingleStore DB protocol (3306)

  • Allow you to connect to management and monitoring tools

Protocol

Default Port

Direction

Description

TCP

22

Inbound and Outbound

For host access. Required between nodes in SingleStore DB tool deployment scenarios. Also useful for remote administration and troubleshooting on the main deployment host.

TCP

443

Outbound

To get public repo key for package verification. Required for nodes downloading SingleStore APT or YUM packages.

TCP

3306

Inbound and Outbound

Default port used by SingleStore DB. Required on all nodes for intra-cluster communication. Also required on aggregators for client connections.

TCP

8080

Inbound and Outbound

Default port for SingleStore DB Studio. (Only required for the host running Studio.)

The service port values are configurable if the default values cannot be used in your deployment environment. For more information on how to change them, see:

We also highly recommend configuring your firewall to prevent other hosts on the Internet from connecting to SingleStore DB.

Install SingleStore Tools

Caution

SingleStore DB Studio should be installed on a host that is open to local (internal) network traffic only, and not open to the Internet.

The first step in deploying your cluster is to download and install the SingleStore Tools on one of the hosts in your cluster. This host will be designated as the main deployment host for deploying SingleStore DB across your other hosts and setting up your cluster.

These tools perform all major cluster operations including downloading the latest version of SingleStore DB onto your hosts, assigning and configuring nodes in your cluster, and other management operations. For the purpose of this guide, the main deployment host is the same as the designated Master Aggregator of the SingleStore DB cluster.

Note: If SingleStore DB is installed as a sudo user via packages, systemd will automatically start the associated SingleStore DB processes when a host is rebooted.

Online Installation - Red Hat Distribution

Run the following commands to install SingleStore Tools.

sudo yum-config-manager --add-repo https://release.memsql.com/production/rpm/x86_64/repodata/memsql.repo && \
sudo yum install -y singlestore-client singlestoredb-toolbox singlestoredb-studio

Troubleshooting

If SingleStore Tools cannot be installed using the steps above, verify that the SingleStore repo information is listed under repolist.

sudo yum repolist
****
repo id            repo name            status
memsql             MemSQL               125

Verify that the which package is installed. This is used during the install process to identify the correct package type for your installation.

rpm -q which

If which is not installed, you must install it before proceeding.

sudo yum install -y which

If you cannot install which, you will need to specify the package as rpm during the deployment phase.

After verifying the above, re-run the commands to install SingleStore Tools.

Deploy SingleStore DB

Prerequisites

Warning

Before deploying a SingleStore DB cluster in a production environment, please review and follow the host configuration recommendations.

Failing to follow these recommendations will result in sub-optimal cluster performance.

Notes on Users and Groups

The user that deploys SingleStore DB via SingleStore DB Toolbox must be able to SSH to each host in the cluster. When singlestoredb-server is installed via an RPM or Debian package when deploying SingleStore DB, a memsql user and group are also created on each host in the cluster.

This memsql user does not have a shell, and attempting to log in or SSH as this user will fail. The user that deploys SingleStore DB is added to the memsql group. This group allows most Toolbox commands to run without sudo privileges, and members of this group can perform many Toolbox operations without the need to escalate to sudo. Users who desire to run SingleStore DB Toolbox commands must be added to the memsql group on each host in the cluster. They must also be able to SSH to each host.

Manually creating a memsql user and group is only recommended in a sudo-less environment when performing a tarball-based deployment of SingleStore DB. In order to run SingleStore DB Toolbox commands against a cluster, this manually-created memsql user must be configured so that it can SSH to each host in the cluster.

Minimal Deployment

SingleStore DB has been designed to be deployed with at least two nodes:

  • A Master Aggregator node that runs SQL queries and aggregates the results, and

  • A single leaf node, which is responsible for storing and processing data

These two nodes can be deployed on a single host (via the cluster-in-box option), or on two hosts, with one SingleStore DB node on each host.

While additional aggregators and nodes can be added and removed as required, a minimal deployment of SingleStore DB always consists of at least these two nodes.

UI Online Deployment

Notice

The user that deploys SingleStore DB via the UI must also be able to SSH into each host in the cluster without using a password.

As of SingleStore DB Toolbox 1.6, SingleStore DB can be deployed via browser-based UI. This option describes how to deploy SingleStore DB using this UI. Please review the prerequisites prior to deploying SingleStore DB.

In order to use this method of UI deployment, the user (and the user account that will deploy SingleStore DB) must:

  • Be able to install SingleStore DB and SingleStore DB Toolbox 1.6 using RPM or Debian packages.

  • Deploy a “standard” SingleStore DB configuration. Advanced options, such as those available with a cluster deployment via a YAML file, are also available in the UI.

Start the UI

Run the following command to start the UI.

  1. Run the following on the command line.

    sdb-deploy ui

This command will display a link with a secure token that you can use to deploy SingleStore DB via the UI.

For additional options that can be used with sdb-deploy ui, refer to the associated reference page.

Access the UI

Copy and paste this link into a Chrome or Firefox browser to access the UI.

Note: You may need to modify the URL by changing localhost to a hostname or IP address depending on how and where you installed SingleStore Tools. The hostname or IP address must be that of the main deployment host, which is typically the Master Aggregator.

Deploy SingleStore DB

Follow installation instructions in the UI to deploy SingleStore DB.

Troubleshooting

  • Message: unknown command "ui" for "sdb-deploy"

    Solution: Confirm that SingleStore DB Toolbox v1.6 or later has been installed on the main deployment host.

  • Message: sdb-deploy ui is not currently supported by SingleStore DB.

    Solution: The installed version of SingleStore DB Toolbox does not support deploying SingleStore DB via the UI. Please select another deployment option.

  • Message: Registered hosts detected. SingleStore DB Toolbox supports managing only one cluster per instance. To view them, run 'sdb-toolbox-config list-hosts'. To remove them, run 'sdb-toolbox-config unregister-host'

    Solution: SingleStore DB Toolbox can only manage a single instance of SingleStore DB.

Additional Deployment Options

Notice

If this deployment method is not ideal for your target environment, you can choose one that fits your requirements from the Deployment Options.

Interact with your Cluster

Start Studio

On your main deployment host, run the following command to use SingleStore DB Studio to monitor and interact with your cluster.

Enable the SingleStore DB Studio service to start SingleStore DB Studio at system boot (recommended).

sudo systemctl enable singlestoredb-studio.service
****
Created symlink /etc/systemd/system/multi-user.target.wants/memsql-studio.service → /lib/systemd/system/memsql-studio.service.

SSH into your main deployment host and run the following:

sudo systemctl start singlestoredb-studio

If your Linux distribution does not use systemd, you can run SingleStore DB Studio directly instead.

sudo singlestoredb-studio &

The Studio Web server will now be running on port 8080, which can be accessed via Web browser at http://<main-deployment-host>:8080.

Add a New Cluster to Studio

  1. With SingleStore DB Studio running, go to http://<main_deployment_host>:8080 and click Add New Cluster to set up a cluster.

    Important

    SingleStore DB Studio is only supported on Chrome and Firefox browsers at this time.

    To run Studio on a different port, add port = <port_name> to /etc/singlestore/singlestoredb-studio.hcl and restart Studio.

  2. Paste the main deployment host IP address or hostname into Hostname.

  3. Set Port to 3306.

  4. Specify root as the Username.

  5. In the Password field, provide the Superuser password that was set during cluster deployment.

  6. Click Create Cluster Profile and set Type as Development.

  7. Fill in Cluster Name and Description to your preference.

After you have successfully logged in, you will see the dashboard for your cluster. To run a query against your cluster, navigate to the SQL Editor through the navigation in the left pane.

Next Steps After Deployment

Now that you have installed SingleStore DB and connected to SingleStore DB Studio, check out the following resources to continue your learning:

In this section: