I am using Duplicity to automate off-site backups. What I wasn't doing before - checking integrity of remote backup. Due nature of backing up to remote location, it quite possible to have issues like half uploaded files, corrupted files, missing files, etc. I wanted to have something in place to check backup right after backup procedure is finished and email me if there are any differences.

Duplicity provides "verify" command to do comparison of backup repository with actual files on the disk. What was missing - email notifications if something went wrong. I decided to leverage Logwatch. Logwatch uses filters to apply to logfiles. And if there is any output - include in to daily email. Below are steps to setup Logwatch.

Start with installing logwatch:

apt-get install logwatch

Next new Logwatch group needs to be defined. This basically tells what log files should be tracked.

cd /etc/logwatch/conf/logfiles
vim duplicity.conf

add:

LogFile = /var/log/backup/backup.log
*ApplyStdDate
Next Logwatch service filter config should be defined:

vim /etc/logwatch/conf/services/duplicity.conf

add:

Title = "Duplicity Backup"
LogFile = duplicity

Create new Executable filter to parse duplicity's log files

vim /etc/logwatch/scripts/services/duplicity

add:

#!/bin/bash
 
cat | grep "Difference found:"

and make it executable:

chmod +x /etc/logwatch/scripts/services/duplicity

Once done, we need to create duplicity log file. Duplicity log file is just output of "duplicity verify" command which is modified to include date/time for each line. This is required for Logwatch to filter only lines for previous day (otherwise I would receive updates for the whole life of the log file which is not what I want.) Put something like this into crontab. Notice "sed" command - this adds timestamp to the log:

30 0 * * * /root/bin/webbackup.sh | sed "s/^/$(date +'%b %d %T') $(hostname) backup: /" >> /var/log/backup/backup.log 2>&1

And below sample of my backup script /root/bin/webbackup.sh:

#!/bin/bash
export PASSPHRASE=verysecurepassword
 
duplicity --exclude **/tmp --exclude **/session --exclude **/sessions --full-if-older-than 1M --archive-dir /srv/backup/cache /mnt/www/ file:///srv/backup/www/
duplicity remove-all-but-n-full 3 --force --archive-dir /srv/backup/cache file:///srv/backup/www/
duplicity remove-all-inc-of-but-n-full 1 --force --archive-dir /srv/backup/cache file:///srv/backup/www/
duplicity verify --exclude **/tmp --exclude **/session --exclude **/sessions file:///srv/backup/www/ /mnt/www/
 
unset PASSPHRASE