rdiff-backup is a very neat tool to perform incremental backups on *nix machines. You can learn more about it on GitHub.
logrotate is a standard *nix utility to… well, rotate logs 🙂
If you do backups of your logs with rdiff-backup, the default configuration is going to take a lot of useless space, so let’s see a better way to use them. In the post we will assume a daily rotation.
The default behaviour of logrotate
logrotate by default use progressive numbers to identify previous logs. So if you have a log:
in a few days you will have
access.log access.log.1 access.log.2.gz access.log.3.gz
The issue with rdiff-backup
In the default configuration each file gets renamed each day. Today access.log will be access.log.1 tomorrow, and the access.log.2.gz, and so on.
But rdiff-backup is not able to pickup file renaming, so from its point of view all files are completely changed each day. And since .gz are binaries, partial diffs are not available and the only thing rdiff-backup can do is save the whole file.
Result: you have multiple copies of the same file, one for each day of the logrotate “keep” period.
The solution: dateext
In the logrotate configuration file (either the main one /etc/logrotate.conf or any additional configuration in /etc/logrotate.d/) you can add a parameter dateext which means file will be renamed according to date. E.g.
access.log access.log-20170803 access.log-20170802.gz access.log-20170801.gz
This means the file will not be renamed as it slides behind in the rotation queue, and rdiff-backup will not produce additional copies of it.