Recently, our server was down, the main reason because the log files burned out our storage. We must find a way to restart the server, the temporary solution at that time was zip the logs and create new logs and then restart the server.
After that, I spend the time to research about Logrotate and init the Logrotate configurations for our servers. By the way, Logrotate is a system utility that manages the automatic rotation and compression of log files. This means by using the Logrotate, your log files can be compressed after a specific time and Logrotate will init an empty log file for you and you don’t need to get your server to switch over to the new log file.
Logrotate is installed by default on Ubuntu 16.04, and is set up to handle the log rotation needs of all installed packages, including rsyslog, the default system log processor.
/etc/logrotate.conf file and remove the comment sign for
# see "man logrotate" for details
Create a new file called
logrotate.conf from your server.
There are few options for the logrotate:
daily: rotate daily. You can use
missingok: ignore if the log file doesn’t exist.
rotate 90: only keep 90 days of logs around.
notifempty: don’t rotate if the log file is empty.
compress: gzip the log rotation. This uses gzip by default and results in files ending in .gz.
delaycompressis active, an archived log is compressed the next time that the log is rotated. This can be important when you have a program that might still write to its old log file for a time after a fresh one is rotated in. Note that
delaycompressworks only if you have compress in your configuration.
copytruncate: copy the log files and then empties it. This makes sure the log files always exists so you don’t need to reload the server.
My Logrotate configuration will be run daily at 3:00 AM so my cron:
0 3 * * * logrotate /home/myapp/logrotate.conf --state /home/myapp/logrotate-state --verbose
The Logrotate is cool, it will help us save storage from the server. After the rotate run, you can trigger another cron (maybe 4:00 AM) to upload the gzip file to clould (S3 for example).