Sign inTry Free

Dealing with Cluster Failures

On this page

Assess the Situation

This topic is designed to help resolve common failure scenarios that one may encounter during cluster operation.

The most important step in resolving any cluster failure is to assess your cluster and determine:

  • Are there any network issues that are preventing the cluster from operating properly?

  • Are there any recent hardware issues?

  • Are any of the drives suddenly experiencing issues?

  • Is your cluster utilizing high availability, and, if so, which replication mode is it using (redundancy-1 or redundancy-2)?

Knowing the answers to these questions will help you decide which of the following sections will help you resolve your issue(s) and whether any data loss will be incurred.

In this section

Last modified: November 22, 2022

Was this article helpful?

On this page

Was this article helpful?

Verification instructions

Note: You must install cosign to verify the authenticity of the SingleStore file.

Use the following steps to verify the authenticity of singlestoredb-server, singlestoredb-toolbox, singlestoredb-studio, and singlestore-client SingleStore files that have been downloaded.

You may perform the following steps on any computer that can run cosign, such as the main deployment host of the cluster.

  1. (Optional) Run the following command to view the associated signature files.

    curl undefined

  2. Download the signature file from the SingleStore release server.

    • Option 1: Click the Download Signature button next to the SingleStore file.

    • Option 2: Copy and paste the following URL into the address bar of your browser and save the signature file.

    • Option 3: Run the following command to download the signature file.

      curl -O undefined

  3. After the signature file has been downloaded, run the following command to verify the authenticity of the SingleStore file.

    echo -n undefined |
        cosign verify-blob --certificate-oidc-issuer https://oidc.eks.us-east-1.amazonaws.com/id/CCDCDBA1379A5596AB5B2E46DCA385BC \
          --certificate-identity https://kubernetes.io/namespaces/freya-production/serviceaccounts/job-worker \
          --bundle undefined \
          --new-bundle-format -
    Verified OK