calibrate-blob-storage
On this page
Description
Run calibrate-blob-storage
on S3-compatible object stores to measure blob storage performance.
Note that this command will not work in conjunction with other blob storage-related activities, such as when a user database is attached to the cluster.
This command measures the upload and download throughput of the cluster.
A calibrate_
database is created using the unlimited storage settings that are specified via command-line arguments and environment variables.
The workload consists of the following steps:
-
Upload test
-
Create the
calibrate_
databasestorage -
Generate the dataset
-
Introduce a high load on the cluster by maintaining multiple concurrent loads against the aggregator nodes for 10 minutes
-
-
Download test
-
Run aggregate queries against the data loaded during the upload test
-
-
Retrieve the results of the run and save them in a file on the Master Aggregator host
The maximum wait time for the upload queue during the upload test is divided into three main categories:
-
FAST: under 2 minutes
-
INTERMEDIATE: between 2 and 3 minutes
-
SLOW: over 3 minutes
The download test uses data that was loaded into the database during the upload test and calculates the average download throughput.
Examples
A simple example that runs a calibrate-blob-storage
workload with the root user of the cluster.
sdb-report calibrate-blob-storage -r 's3://test/bucket?region=eu-central-1'
Runs a calibrate-blob-storage
workload with a user user
and a password pass
.
sdb-report calibrate-blob-storage -r 's3://test/bucket?region=eu-central-1' -u user -p pass
Usage
Usage:sdb-report calibrate-blob-storage [flags]Flags:--blob-cache-eviction MINUTES Set a timeout for the blob cache cleanup performed during the download test (ADVANCED) (default 20)-h, --help Help for calibrate-blob-storage-p, --password STRING The database user's password for connecting to SingleStoreDB. If a password is specified on the command line, it must not contain an unescaped '$' character as it will be replaced by the shell-r, --repository STRING Path to the repository in the URL format. See https://docs.singlestore.com/toolbox-redir/memsql-backup-repository for examples-u, --user string The database user for connecting to SingleStoreDB (default "root")Global Flags:--backup-cache FILE_PATH File path for the backup cache--cache-file FILE_PATH File path for the Toolbox node cache-c, --config FILE_PATH File path for the Toolbox configuration--disable-colors Disable color output in console, which some terminal sessions/environments may have difficulty with--disable-spinner Disable the progress spinner, which some terminal sessions/environments may have issues with-j, --json Enable JSON output--parallelism POSITIVE_INTEGER Maximum number of operations to run in parallel--runtime-dir DIRECTORY_PATH Where to store Toolbox runtime data--ssh-control-persist SECONDS Enable SSH ControlPersist and set it to the specified duration in seconds--ssh-max-sessions POSITIVE_INTEGER Maximum number of SSH sessions to open per host, must be at least 3--ssh-strict-host-key-checking Enable strict host key checking for SSH connections--ssh-user-known-hosts-file FILE_PATH Path to the user known_hosts file for SSH connections. If not set, /dev/null will be used--state-file FILE_PATH Toolbox state file path-v, --verbosity count Increase logging verbosity: valid values are 1, 2, 3. Usage -v=count or --verbosity=count-y, --yes Enable non-interactive mode and assume the user would like to move forward with the proposed actions by default
Remarks
This command is interactive unless you use either --yes
or --json
flag to override interactive behavior.
Last modified: April 25, 2023