Rotating Trace Log to Manage its Size
Without a log rotation policy in place, the tracelog can keep growing in size until it consumes all remaining disk space. Hence, it is important to implement log rotation. The memsql.log
file can be rotated by moving the memsql.log
file and then sending SIGHUP
to the memsqld
process. This will trigger the server to reopen memsql.log
and continue writing. As tracelog files are text files, you may want to compress them after rotation to free up disk space.
Using logrotate
To automate tracelog
rotation, you may use the logrotate
utility. The logrotate
utility runs with the default /etc/logrotate.conf
configuration. If you have custom configurations, you can place them in /etc/logrotate.d/
; the include
directive in the logrotate.conf
file makes sure that the custom configuration files are read as well. Alternatively, you can specify the necessary configuration files as arguments to the logrotate
command. When multiple configuration files are specified, files that appear later in the list override the options in the earlier ones, so the order in which the files are listed matters. For more information about logrotate
, refer to the logrotate
man page.
Example logrotate
Configuration File
The following logrotate
configuration file rotates two log files: memsql.log
and query.log
. It represents a generic configuration, which can be modified to suit your requirements.
Note: The example paths use the default folder location for SingleStoreDB with an example ID value in the folder name for a Master Aggregator node. You will need to replace them with the values for your node. Also, this example assumes you are logged in as the Linux root
user when running logrotate
.
cat > /var/tmp/tracelog-logrotate.conf <<EOF /var/lib/memsql/master-3306-MI4478312f/tracelogs/memsql.log /var/lib/memsql/master-3306-MI4478312f/tracelogs/query.log { daily rotate 7 missingok compress sharedscripts postrotate # Send SIGHUP to both memsqld processes killall -q -s1 memsqld endscript } EOF
The directives used in the above example indicate the following:
daily
: Log files are rotated every day; alternately, you can rotate them weekly or monthly.rotate
[count]: Specifies the number of times log files are rotated before being deleted. In the example, log files are rotated 7 times before being deleted. You can specify the number of times you want to rotate the old log files before deleting them. If run with the--mail
option, the log files will be rotated the specified number of times before being emailed, rather than being removed.missingok
: If a specified log file is missing, this directive ensures thatlogrotate
rotates the next file in the queue without throwing an error.compress
: Old versions of log files are compressed with gzip by default.sharedscripts
: With this directive specified, thepostrotate
script is run only one time (after the old logs have been compressed), not each time a log file is rotated.postrotate/endscript
: The lines between these directives are executed after the log file is rotated. The example uses thekillall -q -s1 memsqld
command, which allows for a new tracelog to be generated and written to the next trace.
Scheduling logrotate
The logrotate
process is typically run as a daily cron
job by the cron
daemon. The package manager used by the system creates a default schedule in /etc/cron.daily/logrotate
that runs logrotate
with the default /etc/logrotate.conf
configuration. If it is not automatically created, you can manually create one and add the following code snippet.
#! /bin/sh /user/sbin/logrotate /etc/logrotate.conf
You can also create cron
jobs in the cron.hourly
, cron.weekly
, or cron.monthly
folders. The jobs located in these folders and the cron.daily
folder are run by the cron
service, per the instructions in the /etc/crontab
file. If you want to run logrotate
with a custom schedule, create a cron
job and place it under /etc/cron.d/
.
Note that the cron
jobs stored in /etc/cron.d/
and /etc/crontab
directories are system cron
jobs. System cron
jobs are defined in shared directories on the filesystem. You need sudo
privileges to define system cron
jobs.
Cron
jobs are defined using the unix-cron format in both /etc/cron.d/
and /etc/crontab
directories. The schedule consists of a string of five values indicating when the cron
job must be executed. These values represent date and time fields, including minute, hour, day of month, and day of week, in that order. The string value is followed by the username under which the cron
job must run and then the command to execute. For example, the following cron
job issues logrotate
using the /etc/tracelog-logrotate.conf
configuration every Saturday at five minutes past midnight.
5 0 * * 6 root /usr/sbin/logrotate /etc/tracelog-logrotate.conf
Manually Running logrotate
To manually run logrotate
, use the following command and specify the configuration file.
logrotate -f /var/tmp/tracelog-logrotate.conf
To list the compressed log files, run the following command.
ls /var/lib/memsql/master-3306-MI4478312f/tracelogs/*.gz
Alternatives to logrotate
As an alternative to logrotate
, you can configure SingleStoreDB to log using the syslog
daemon, such as journald
, and then implement log rotation. Since logging is a host-level activity, you can use a logging system that is ideal for your environment.