Rotating Trace Log to Manage its Size

Built-In Trace Log Rotation

The SingleStore provided trace log rotation feature includes settings for:

• rotating based on log size

• rotating based on time

• deleting logs based on age

You can enable log rotation based on log size or time on disk or both. You can separately configure whether to delete old logs and the number of days on the system at which to delete them.

To enable trace log rotation, set either or both of the tracelog_rotation_size and tracelog_rotation_time engine variables to a valid value other than 0 -- the default values are 0 (disabled).

If you set both engine variables, rotation will happen on either of triggers;, i.e. either the file exceeds a particular size or the time has come to rotate. For most use cases, setting either one or other is sufficient.

Using logrotate

The memsql.log file can be rotated by moving the memsql.log file and then sending SIGHUP to the memsqld process. This will trigger the server to reopen memsql.log and continue writing.

To automate tracelog rotation, you may use the logrotate utility. The logrotate utility runs with the default /etc/logrotate.conf configuration. If you have custom configurations, you can place them in /etc/logrotate.d/; the include directive in the logrotate.conf file makes sure that the custom configuration files are read as well. Alternatively, you can specify the necessary configuration files as arguments to the logrotate command. When multiple configuration files are specified, files that appear later in the list override the options in the earlier ones, so the order in which the files are listed matters. For more information about logrotate, refer to the logrotate man page.

cat > /var/tmp/tracelog-logrotate.conf <<EOF
/var/lib/memsql/master-3306-MI4478312f/tracelogs/memsql.log /var/lib/memsql/master-3306-MI4478312f/tracelogs/query.log {
         daily
         rotate 7
         missingok
         compress
         sharedscripts
         postrotate
             # Send SIGHUP to both memsqld processes
             killall -q -s1 memsqld
         endscript
}
EOF

Scheduling logrotate

The logrotate process is typically run as a daily cron job by the cron daemon. The package manager used by the system creates a default schedule in /etc/cron.daily/logrotate that runs logrotate with the default /etc/logrotate.conf configuration. If it is not automatically created, you can manually create one and add the following code snippet.

#! /bin/sh
/user/sbin/logrotate /etc/logrotate.conf

You can also create cron jobs in the cron.hourly, cron.weekly, or cron.monthly folders. The jobs located in these folders and the cron.daily folder are run by the cron service, per the instructions in the /etc/crontab file. If you want to run logrotate with a custom schedule, create a cron job and place it under /etc/cron.d/.

Note that the cron jobs stored in /etc/cron.d/ and /etc/crontab directories are system cron jobs. System cron jobs are defined in shared directories on the filesystem. You need sudo privileges to define system cron jobs.

Cron jobs are defined using the unix-cron format in both /etc/cron.d/ and /etc/crontab directories. The schedule consists of a string of five values indicating when the cron job must be executed. These values represent date and time fields, including minute, hour, day of month, and day of week, in that order. The string value is followed by the username under which the cron job must run and then the command to execute. For example, the following cron job issues logrotate using the /etc/tracelog-logrotate.conf configuration every Saturday at five minutes past midnight.

5 0 * * 6 root /usr/sbin/logrotate /etc/tracelog-logrotate.conf

Manually Running logrotate

To manually run logrotate, use the following command and specify the configuration file.

logrotate -f /var/tmp/tracelog-logrotate.conf

To list the compressed log files, run the following command.

ls /var/lib/memsql/master-3306-MI4478312f/tracelogs/*.gz

Alternatives to logrotate

As an alternative to logrotate, you can configure SingleStore to log using the syslog daemon, such as journald, and then implement log rotation. Since logging is a host-level activity, you can use a logging system that is ideal for your environment.