Configure Monitoring with Toolbox
Note
Using Toolbox to configure cluster monitoring is the preferred and default method. Toolbox largely automates the monitoring configuration by using standard metrics
database and pipeline DDLs and predefined values. To customize these DDLs and values, you may configure monitoring manually by using SQL.
Enable the Exporter Process
The memsql-exporter
process (or simply “the exporter”) collects data about a running cluster. The user that starts the exporter (other than the SingleStoreDBroot
user) must have the following permissions at a minimum:
GRANT CLUSTER on *.* to <user> GRANT SHOW METADATA on *.* to <user> GRANT SELECT on *.* to <user> GRANT PROCESS on *.* to <user>
Start the Exporter Process
Perform the following steps on the Source cluster Master Aggregator.
Note that the exporter process is restarted when the exporter_port
variable is set to any non-zero value, including the current value. The exporter_port
variable can be set by running sdb-admin configure-monitoring --exporter-port <port>
.
Run either of the following to start the exporter. This command will start the exporter using the SingleStoreDB root
user and default port (9104
). The exporter username and password are used to access the Source cluster. Refer to sdb-admin configure-monitoring for more information.
HTTP Connections
sdb-admin configure-monitoring \ --exporter-user root \ --exporter-password <secure-password>
HTTPS Connections
Copy the server certificate and key to the Master Aggregator host of the Source cluster. This certificate will be used by the exporter process and must be readable by the user under which the nodes are running on the host (typically the
memsql
user).On the Source cluster, run the following command to start the exporter.
sdb-admin configure-monitoring \ --exporter-user root \ --exporter-password <secure-password> \ --exporter-use-https \ --exporter-ssl-cert=/path/to/server-cert.pem \ --exporter-ssl-key=/path/to/server-key.pem \ --exporter-ssl-passphrase=<passphrase>
Note: The
--exporter-ssl-passphrase
option should only be included if the server key has a passphrase.Refer to configure-monitoring for additional options, including
--exporter-user
and--exporter-password
.
How to Stop the Exporter Process
You may stop the exporter at any time by running the following command on the Source cluster Master Aggregator.
sdb-admin configure-monitoring --stop-exporter
Configure the metrics Database
The metrics
database can either reside in the Source cluster, in a dedicated Metrics cluster, or in a remote SingleStoreDB cluster that the Metrics cluster can access.
Note that the user that creates the metrics
database may differ from the one that started the exporter. This user must be able to create tables, databases, and pipelines. At a minimum, the following permissions are required:
The following command creates a database named metrics with a 10-day retention period for the data, and will also create the associated metrics
and blobs
pipelines. These pipelines are responsible for extracting data from the exporter and storing it in the metrics
database.
The exporter-host
value must be the Master Aggregator host in the Source cluster. The user
and password
are the credentials for the Metrics cluster. The variable retention-period
defines the number of days that data is retained before being deleted. Refer to sdb-admin start-monitoring for more information.
Please note the following:
The database that is created to store monitoring data is named
metrics
by default. You may change the database name by updating the value of the--database-name
option.The database user is
root
by default. When creating themetrics
database with a database user other thanroot
, theSUPER
privilege must be granted to this user. For example, for adbmon
user:GRANT SUPER ON *.* TO 'dbmon'@'%';
The IP address provided in the
exporter-host
argument must be resolvable by all hosts in the cluster. If the cluster you are monitoring has hosts that contain leaf nodes (as is common in a multi-host vs. single-host deployment), do not uselocalhost
. Instead, provide the explicit IP address or hostname that the other hosts in the cluster can resolve.This command must be run on the Metrics cluster.
Note
For Metrics clusters running SingleStoreDB 7.3.12 and earlier that are managed by Toolbox 1.11.7 and earlier, when a new Source cluster is monitored, the specified --retention-period
will apply to all existing Source clusters.
For Metrics clusters running SingleStoreDB 7.3.13 and later that are managed with Toolbox 1.11.8 and later, a unique --retention-period
can be specified for each Source cluster. Therefore, when a new Source cluster is monitored, the specified --retention-period
will only apply to that Source cluster.
HTTP Connections
Run the following command to start monitoring.
sdb-admin start-monitoring \ --database-name metrics \ --exporter-host <exporter-hostname-or-IP-address> \ --user root \ --password <secure-password> \ --retention-period 10
HTTPS Connections
Copy the CA certificate to the same file path on each host of the Metrics cluster.
Alternatively, a directory containing multiple CA certificates can be provided, and this path must be the same on all hosts. This directory will be used by the monitoring pipelines and must be readable by the user under which the nodes are running on the host (typically the
memsql
user).Run the following command to start monitoring.
sdb-admin start-monitoring \ --database-name metrics \ --exporter-host=<exporter-hostname-or-IP-address> \ --user root \ --password=<secure-password> \ --retention-period 10 \ --ssl-ca=/path/to/ca-cert.pem --or-- --ssl-capath=/ca-directory/including/path
Refer to start-monitoring for additional options.
How to Stop the Monitoring Process
You may stop the monitoring process at any time by performing the following command.
Note: Be sure to substitute the database name and/or the associated database user if you've changed them from the defaults of the metrics
database and the root
user.
sdb-admin stop-monitoring \ --exporter-host <exporter-hostname-or-IP-address> \ --user root \ --password <secure-password>