calibrate-blob-storage

Description

Run calibrate-blob-storage on S3-compatible object stores to measure blob storage performance.

During testing, the maximum_blob_cache_size_mb value is reduced to 1024 on each leaf node in the cluster. This could potentially lead to performance-related and other issues, particularly when querying unlimited storage (bottomless) databases during the calibration process. The previous maximum_blob_cache_size_mb value is restored after testing.

Note that this command will not work in conjunction with other blob storage-related activities, such as when a user database is attached to the cluster. To prevent this operation from timing out, please detach all user databases before running this command.

This command measures the upload and download throughput of the cluster.

A calibrate_storage database is created using the unlimited storage settings that are specified via command-line arguments and environment variables. The database is then populated with the dataset generated by this command.

The workload consists of the following steps:

Upload test
- Create the calibrate_storage database
- Generate the dataset
- Introduce a high load on the cluster by maintaining multiple concurrent loads against the aggregator nodes for 10 minutes
Download test
- Run aggregate queries against the data loaded during the upload test
Retrieve the results of the run and save them in a file on the Master Aggregator host

The maximum wait time for the upload queue during the upload test is divided into three main categories:

FAST: under 2 minutes
INTERMEDIATE: between 2 and 3 minutes
SLOW: over 3 minutes

The download test uses data that was loaded into the database during the upload test and calculates the average download throughput.

Examples

A simple example that runs a calibrate-blob-storage workload with the root user of the cluster.

Shell

sdb-report calibrate-blob-storage -r 's3://test/bucket?region=eu-central-1'

Runs a calibrate-blob-storage workload with a user user and a password pass.

Shell

sdb-report calibrate-blob-storage -r 's3://test/bucket?region=eu-central-1' -u user -p pass

Usage

Usage:
  sdb-report calibrate-blob-storage [flags]

  For flags that can accept multiple values (indicated by VALUES after the name of the flag),
  separate each value with a comma.

Flags:
      --blob-cache-eviction MINUTES   Set a timeout for the blob cache cleanup performed during the download test (ADVANCED) (default 20)
  -h, --help                          Help for calibrate-blob-storage
  -p, --password STRING               The database user's password for connecting to SingleStore.
                                      If a password is specified on the command line, it must not contain an unescaped '$' character as it will be replaced by the shell
  -r, --repository STRING             Path to the repository in the URL format. See https://docs.singlestore.com/toolbox-redir/memsql-backup-repository for examples
  -u, --user string                   The database user for connecting to SingleStore (default "root")

Global Flags:
      --backup-cache FILE_PATH                File path for the backup cache
      --cache-file FILE_PATH                  File path for the Toolbox node cache
  -c, --config FILE_PATH                      File path for the Toolbox configuration
      --disable-colors                        Disable color output in console, which some terminal sessions/environments may have difficulty with
      --disable-spinner                       Disable the progress spinner, which some terminal sessions/environments may have issues with
  -j, --json                                  Enable JSON output
      --parallelism POSITIVE_INTEGER          Maximum number of operations to run in parallel
      --runtime-dir DIRECTORY_PATH            Where to store Toolbox runtime data
      --ssh-control-persist SECONDS           Enable SSH ControlPersist and set it to the specified duration in seconds
      --ssh-max-sessions POSITIVE_INTEGER     Maximum number of SSH sessions to open per host, must be at least 3
      --ssh-strict-host-key-checking          Enable strict host key checking for SSH connections
      --ssh-user-known-hosts-file FILE_PATH   Path to the user known_hosts file for SSH connections. If not set, /dev/null will be used
      --state-file FILE_PATH                  Toolbox state file path
  -v, --verbosity count                       Increase logging verbosity: valid values are 1, 2, 3. Usage -v=count or --verbosity=count
  -y, --yes                                   Enable non-interactive mode and assume the user would like to move forward with the proposed actions by default

Remarks

This command is interactive unless you use either --yes or --json flag to override interactive behavior.