Sign inTry Free

MV_EVENTS

On this page

This view contains information about events and is useful for monitoring events across clusters over time.

Information on the most recent 1028 events is stored in the view. The view is flushed upon database restart, but otherwise there is no set retention period.

Events from all nodes are stored in the view and when a node is restarted, events specific to that node are removed from the view. For example, if the Master Aggregator is restarted, events related to it are removed from the view, while events from other nodes will remain.

Learn more about interpreting this view in the Components to Monitor guide.

Column Name

Description

ORIGIN_NODE_ID

The ID of the node from where a given event occurred.

EVENT_TIME

The timestamp of a given event.

SEVERITY

The severity of a given event: NOTICE, WARNING, or ERROR

EVENT_TYPE

The type of event that occurred.

DETAILS

Additional information about a given event in JSON format.

MV_EVENTS.EVENT_TYPE

Provides descriptions for each potential result of SELECT DISTINCT EVENT_TYPE FROM information_schema.MV_EVENTS. Each result is one type of event occurring on the related node.

Event Type

Description

AGGREGATOR_ADD

An aggregator is being added to the cluster.

AGGREGATOR_REMOVE

An aggregator is being removed from the cluster.

ALTER_NODE_SET_HOST

The host (network address) of a node in a cluster has changed.

ASYNC_UPGRADE_STARTED

Marks the start of an asynchronous upgrade on a cluster.

ASYNC_UPGRADE_FINISHED

Marks the end of an asynchronous upgrade on a cluster.

ASYNC_UPGRADE_STEP_STARTED

Marks the start of a specific asynchronous upgrade step on a cluster.

ASYNC_UPGRADE_STEP_FINISHED

Marks the end of a specific asynchronous upgrade step on a cluster.

ATTACH_DATABASE_START

The system has started attaching an existing database to the workspace.

ATTACH_DATABASE_FINISH

The database has been attached and is accessible.

BACKUP_DB

The related database, and therefore the given node, has been backed up.

RESTORE_DB

A backup of the related database, and therefore the given node, has been restored.

BLOBCACHE_DISK_OK

The previous cause of the BLOBCACHE_LOW_DISK event no longer applies, where SingleStore is now above the "redline" for disk usage.

BLOBCACHE_LOW_DISK

The blob cache on a node is at or near its disk usage limit.

BOTTLE_SERVICE_UP

The Bottle Service has successfully transitioned from a "down" state to an "up" state. This occurs when a successful heartbeat is detected after one or more consecutive failures or during the first successful heartbeat after initialization.

BOTTLE_SERVICE_DOWN

The Bottle Service has transitioned from an "up" state to a "down" state due to a detected heartbeat failure.

BOTTLE_SERVICE_ERROR

There is an error in the Bottle Service.

BOTTLE_SERVICE_WRONG_TERM

The Bottle Service detected an unexpected or mismatched "term" value—potentially connected to leader election, replication, or consensus protocol irregularities.

BOTTOMLESS_ALTER_COMPUTE

An even triggered by a periodic background task involved in bottomless storage management and, where supported, migration to the Bottle Service.

BOTTOMLESS_INGEST_THROTTLED

Incoming writes have been slowed or halted because the system is unable to offload data to remote storage fast enough or local resources (disk and cache) are threatened. This throttling is necessary to maintain cluster stability and data durability, but is a signal that ingest configuration or resource provisioning may need optimization.

BOTTOMLESS_RATE_LIMITED

The system has activated rate limiting for API calls to the bottomless storage system due to excessive requests or throttling signals. It provides details about the affected database and the number of slowdown exceptions encountered.

BOTTOMLESS_STORAGE_LOCKED_BY_OTHER

Operations involving the database's remote storage cannot proceed because a required lock is being held by another session—frequently arising after PITR or when certain processes are not properly cleaned up.

BOTTOMLESS_UPLOAD_DELAYED

Triggered when the system detects that uploads (of log chunks and blobs) are taking longer than expected. It provides details about the delay, including the affected database and the duration of the delay.

BOTTOMLESS_UPLOAD_VERIFICATION_FAILURE

A failure occurred during the verification step of an upload to bottomless storage.

CONSENSUS_STATE_TRANSITION

Represents changes to a node's state in the distributed consensus protocol (e.g., leadership elections, failovers). It provides the prior and new state, as well as the reason for transition.

CREATE_DATABASE_START

The system has started processing the creation of a new database.

CREATE_DATABASE_FINISH

The database has been created and is ready for use.

DATABASE_ATTACH_START

Marks the start of a database attach operation.

DATABASE_ATTACH_FINISH

Marks the end of a database attach operation.

DATABASE_CREATE_START

Marks the start of a database creation operation.

DATABASE_CREATE_FINISH

Marks the end of a database creation operation.

DATABASE_DETACH_START

Marks the start of a database detach operation.

DATABASE_DETACH_FINISH

Marks the end of a database detach operation.

DATABASE_DROP_START

Marks the start of a database drop operation.

DATABASE_DROP_FINISH

Marks the end of a database drop operation.

DATABASE_REPLICATION_START

Replication of the database is started.

DATABASE_REPLICATION_STOP

Replication of the database is stopped.

DATABASE_REPROVISION

The related database is being reprovisioned.

DETACH_DATABASE_START

The system has started detaching a database from the workspace.

DETACH_DATABASE_FINISH

The database has been detached and is no longer accessible.

DROP_DATABASE_START

The system has started removing the database.

DROP_DATABASE_FINISH

The database has been removed from the system.

HEARTBEAT_QUERY_FAILURE

A heartbeat query failed.

INGEST_ERRORS_OUT_OF_DISK

Ingest is failing due to low disk space available.

LEAF_ADD

A leaf node is being added to the cluster.

LEAF_REMOVE

A leaf node is being removed from the cluster.

LOW_DISK_SPACE

SingleStore ran out of cache space while attempting to create a blob, stopping replay.

MAX_BLOB_CACHE_SIZE_ON_REPLAY

SingleStore ran out of cache space while attempting to create a blob, stopping replay.

MAX_MEMORY

Maximum server memory has been reached.

MAX_MEMORY_ON_REPLAY

A node has run out of available memory specifically during a "replay" operation. Replay refers to processes where the system must load, reconstruct, or reapply database state from persisted logs or snapshots—this commonly happens during database attach, restore, or node startup.

MAX_TABLE_MEMORY

Maximum table memory has been reached.

NODE_ATTACHING

The related node is attaching.

NODE_DETACHED

The related node is detaching.

NODE_DISK_VIOLATION

A node has encountered a disk-related error or has exceeded a disk usage threshold—potentially resulting in error conditions, warnings, or operational limitations for the node.

NODE_MEMORY_VIOLATION

A node has encountered a memory-related error or has exceeded a memory usage threshold—potentially resulting in error conditions, warnings, or operational limitations for the node.

NODE_ONLINE

The related node is online.

NODE_OFFLINE

The related node is offline.

NODE_PING_VIOLATION

Node failed to respond to ping/heartbeat within expected timeframe; possible connectivity or health issue.

NODE_STARTING

The related node is starting up.

NODE_REACHABLE

The related node has become reachable.

NODE_UNREACHABLE

The related node has become unreachable.

NOTIFY_AGGREGATOR_PROMOTED

The related node has been notified of an aggregator being promoted from child to master.

PARTITION_UNRECOVERABLE

A partition is lost due to failure and no longer can be recovered.

PIPELINE_STOPPED

A pipeline stopped.

REBALANCE_STARTED

The rebalance process—which redistributes data partitions to optimize resource usage, recover from failures, or adjust for topology changes—began.

REBALANCE_FINISHED

A partition rebalance has finished.

REPAIR_JOB_STALLED

A "repair job"—a background process responsible for maintaining data consistency, integrity, or recoverability—has stalled, meaning it is taking longer to make progress than expected.

SYSTEM_VAR_CHANGED

An engine variable was reconfigured.

WORKLOAD_THROTTLE

Event emitted when write workload starts or stops getting throttled if a sync replica's replay falls behind.

Last modified: September 29, 2025

Was this article helpful?

On this page

Was this article helpful?

Verification instructions

Note: You must install cosign to verify the authenticity of the SingleStore file.

Use the following steps to verify the authenticity of singlestoredb-server, singlestoredb-toolbox, singlestoredb-studio, and singlestore-client SingleStore files that have been downloaded.

You may perform the following steps on any computer that can run cosign, such as the main deployment host of the cluster.

  1. (Optional) Run the following command to view the associated signature files.

    curl undefined

  2. Download the signature file from the SingleStore release server.

    • Option 1: Click the Download Signature button next to the SingleStore file.

    • Option 2: Copy and paste the following URL into the address bar of your browser and save the signature file.

    • Option 3: Run the following command to download the signature file.

      curl -O undefined

  3. After the signature file has been downloaded, run the following command to verify the authenticity of the SingleStore file.

    echo -n undefined |
        cosign verify-blob --certificate-oidc-issuer https://oidc.eks.us-east-1.amazonaws.com/id/CCDCDBA1379A5596AB5B2E46DCA385BC \
          --certificate-identity https://kubernetes.io/namespaces/freya-production/serviceaccounts/job-worker \
          --bundle undefined \
          --new-bundle-format -
    Verified OK