Sign inTry Free

MV_EVENTS

On this page

This view contains information about events and is useful for monitoring events across clusters over time.

Information on the most recent 1,028 events is stored in the view. The view is flushed upon database restart, but otherwise there is no set retention period.

Events from all nodes are stored in the view and when a node is restarted, events specific to that node are removed from the view. For example, if the Master Aggregator is restarted, events related to it are removed from the view, while events from other nodes will remain.

Learn more about interpreting this view in the Components to Monitor guide.

Column Name

Description

ORIGIN_NODE_ID

The ID of the node from which a given event occurred.

EVENT_TIME

The timestamp of a given event.

SEVERITY

The severity of a given event: NOTICE, WARNING, or ERROR

EVENT_TYPE

The type of event that occurred.

DETAILS

Additional information about a given event in JSON format.

MV_EVENTS.EVENT_TYPE

Provides descriptions for each potential result of SELECT DISTINCT EVENT_TYPE FROM information_schema.MV_EVENTS. Each result is one type of event occurring on the related node.

Event Type

Description

AGGREGATOR_ADD

An aggregator is being added to the cluster.

AGGREGATOR_REMOVE

An aggregator is being removed from the cluster.

ALTER_NODE_SET_HOST_PORT

Logged when a cluster node's hostname or port is changed via the ALTER NODE command.

This tracks changes to node network configuration in the cluster topology.

ASYNC_UPGRADE_STEP_FINISHED

Marks the end of a specific asynchronous upgrade step on a cluster.

ASYNC_UPGRADE_STEP_STARTED

Marks the start of a specific asynchronous upgrade step on a cluster.

BACKUP_DB

The related database, and therefore the given node, has been backed up.

BLOBCACHE_DISK_OK

The previous cause of the BLOBCACHE_LOW_DISK event no longer applies, where SingleStore is now above the "redline" for disk usage.

BLOBCACHE_LOW_DISK

The blob cache on a node is at or near its disk usage limit.

BOTTOMLESS_ALTER_COMPUTE

An even triggered by a periodic background task involved in bottomless storage management and, where supported, migration to the Bottle Service.

BOTTOMLESS_INGEST_THROTTLED

Incoming writes have been slowed or halted because the system is unable to offload data to remote storage fast enough, or local resources (disk and cache) are threatened. This throttling is necessary to maintain cluster stability and data durability, but is a signal that ingest configuration or resource provisioning may need optimization.

BOTTOMLESS_RATE_LIMITED

The system has activated rate limiting for API calls to the bottomless storage system due to excessive requests or throttling signals. It provides details about the affected database and the number of slowdown exceptions encountered.

BOTTOMLESS_STORAGE_LOCKED_BY_OTHER

Operations involving the database's remote storage cannot proceed because a required lock is being held by another session—frequently arising after PITR or when certain processes are not properly cleaned up.

BOTTOMLESS_UPLOAD_DELAYED

Triggered when the system detects that uploads (of log chunks and blobs) are taking longer than expected. It provides details about the delay, including the affected database and the duration of the delay.

BOTTOMLESS_UPLOAD_VERIFICATION_FAILURE

A failure occurred during the verification step of an upload to bottomless storage.

DATABASE_ATTACH_FINISH

Marks the end of a database attach operation.

The database has been attached and is accessible.

DATABASE_ATTACH_START

Marks the start of a database attach operation.

The system has started attaching an existing database to the cluster.

DATABASE_CREATE_FINISH

Marks the end of a database creation operation.

The database has been created and is ready for use.

DATABASE_CREATE_START

Marks the start of a database creation operation.

The system has started processing the creation of a new database.

DATABASE_DETACH_FINISH

Marks the end of a database detach operation.

The database has been detached and is no longer accessible.

DATABASE_DETACH_START

Marks the start of a database detach operation.

The system has started detaching a database from the cluster.

DATABASE_DROP_FINISH

Marks the end of a database drop operation.

The database has been removed from the system.

DATABASE_DROP_START

Marks the start of a database drop operation.

The system has started removing the database.

Logged at the beginning of a database drop operation on the Master Aggregator. Includes the database name and drop action type (e.g., regular drop, drop unrecoverable, etc.). This is paired with DATABASE_DROP_FINISH to track the full lifecycle of the drop operation.

DATABASE_REPLICATION_START

Replication of the database has started.

DATABASE_REPLICATION_STOP

Replication of the database has stopped.

DATABASE_REPROVISION

The related database is being reprovisioned.

HEARTBEAT_QUERY_FAILURE

A heartbeat query failed.

INGEST_ERRORS_OUT_OF_DISK

Ingest is failing due to low available disk space.

LEAF_ADD

A leaf node is being added to the cluster.

LEAF_REMOVE

A leaf node is being removed from the cluster.

LOW_DISK_SPACE

SingleStore ran out of cache space while attempting to create a blob, stopping replay.

MAX_MEMORY

Maximum server memory has been reached.

MAX_MEMORY_ON_REPLAY

A node has run out of available memory specifically during a "replay" operation.

Replay refers to processes where the system must load, reconstruct, or reapply database state from persisted logs or snapshots. This can typically happen during database attach, restore, or node startup.

MAX_TABLE_MEMORY

Maximum table memory has been reached.

NODE_ATTACHING

The related node is attaching.

NODE_DETACHED

The related node is detaching.

NODE_DISK_VIOLATION

A node has encountered a disk-related error or has exceeded a disk usage threshold.

This can potentially result in error conditions, warnings, or operational limitations for the node.

NODE_MEMORY_VIOLATION

A node has encountered a memory-related error or has exceeded a memory usage threshold.

This can potentially result in error conditions, warnings, or operational limitations for the node.

NODE_OFFLINE

The related node is offline.

NODE_ONLINE

The related node is online.

NODE_PING_VIOLATION

Node failed to respond to a ping / heartbeat check within the expected timeframe.

This can potentially indicate a connectivity or health issue.

NODE_REACHABLE

The related node has become reachable.

NODE_STARTING

The related node is starting up.

NODE_UNREACHABLE

The related node has become unreachable.

NOTIFY_AGGREGATOR_PROMOTED

The related node has been notified of an aggregator being promoted from child to master.

PARTITION_UNRECOVERABLE

A partition is lost due to failure and can no longer be recovered.

PIPELINE_STOPPED

A pipeline stopped.

REBALANCE_FINISHED

A partition rebalance has finished.

REBALANCE_STARTED

The rebalance process—which redistributes data partitions to optimize resource usage, recover from failures, or adjust for topology changes—began.

REPAIR_JOB_STALLED

A "repair job"—a background process responsible for maintaining data consistency, integrity, or recoverability—has stalled, which means it is taking longer to progress than expected.

RESTORE_DB

A backup of the related database, and therefore the given node, has been restored.

SYSTEM_VAR_CHANGED

An engine variable was reconfigured.

WORKLOAD_THROTTLE

Event emitted when write workload throttling starts or stops because a sync replica’s replay falls behind.

Last modified:

Was this article helpful?

On this page

Was this article helpful?

Verification instructions

Note: You must install cosign to verify the authenticity of the SingleStore file.

Use the following steps to verify the authenticity of singlestoredb-server, singlestoredb-toolbox, singlestoredb-studio, and singlestore-client SingleStore files that have been downloaded.

You may perform the following steps on any computer that can run cosign, such as the main deployment host of the cluster.

  1. (Optional) Run the following command to view the associated signature files.

    curl undefined

  2. Download the signature file from the SingleStore release server.

    • Option 1: Click the Download Signature button next to the SingleStore file.

    • Option 2: Copy and paste the following URL into the address bar of your browser and save the signature file.

    • Option 3: Run the following command to download the signature file.

      curl -O undefined

  3. After the signature file has been downloaded, run the following command to verify the authenticity of the SingleStore file.

    echo -n undefined |
        cosign verify-blob --certificate-oidc-issuer https://oidc.eks.us-east-1.amazonaws.com/id/CCDCDBA1379A5596AB5B2E46DCA385BC \
          --certificate-identity https://kubernetes.io/namespaces/freya-production/serviceaccounts/job-worker \
          --bundle undefined \
          --new-bundle-format -
    Verified OK

Try Out This Notebook to See What’s Possible in SingleStore

Get access to other groundbreaking datasets and engage with our community for expert advice.

Sign UpLog In