Troubleshooting Audit Logging

Warning

As of the time of this publication, strict mode is available and licensed only as part of the SingleStore Premium edition.

You may encounter problems with the audit logging feature during initial configuration, log file rotation, or general use. This section provides troubleshooting information for common error scenarios.

Note

When SingleStore is unable to write log entries for any reason, it will retry the operation every 10 seconds for 30 attempts. Additionally, all errors are written in a node’s tracelog. The tracelog should be used as the primary source of information while troubleshooting.

Configuration Issues

If you specify any invalid configuration value in your memsql.cnf file, the node won’t be able to start, and an error message will be printed to the console.

To fix a configuration problem, review your memsql.cnf file on the node and ensure that a valid audit logging variable has been specified.

Failed to Write Audit Log Record

The most common audit logging error you may encounter is a failure to write an entry in the audit log. When SingleStore is unable to write an entry to a log for any node in a cluster, the database will not process incoming queries until the problem is resolved. Therefore, the causes of this error are particularly important to understand.

This error writes the following message in the affected node’s tracelog: Failed to write audit log record., followed by a more detailed message of the cause. If your database becomes unresponsive and this error message is present, ensure that the following conditions are not present.

Available Disk Space

If your destination directory runs out of available disk space, SingleStore will be unable to write new log files. To fix this problem, either specify a new destination directory that has sufficient space, or make space available on the current destination directory.

Destination Directory Access

The specified destination directory may not be accessible for many reasons. The most common cases are described below:

  • If your destination directory’s permissions are misconfigured, SingleStore will not be able to write log files. Either specify a new destination directory that has sufficient permissions, or change the permissions on your current destination directory.

  • If your destination directory doesn’t exist or was removed, SingleStore will not be able to write log files. This problem may also be the root cause of more specific log file errors. Ensure that the destination directory actually exists.

  • If your destination directory is located on a network share and the node loses network connectivity, SingleStore will not be able to write log files. This condition usually resolves itself when network access is restored. If a node reconnects to the cluster but the destination directory is still inaccessible, you may need to restart the node.

Last modified: April 3, 2023

Was this article helpful?