After the disaster with the Ukraine hosting data center hosting.ua I started thinking about backup options for my own dedicated server that I use to host yepcorp.com and dozens of my other projects and a few clients' sites. So far it has been a plain backup on the second drive, but this accident showed that even having a hosting company do backups in the same building is not really a safe solution.

Later I came across the Backup Manager (http://www.backup-manager.org) and its ability to upload backups to Amazon S3. This looks really promising since Amazon claims that they are keeping several copies of each file and they are distributed - exactly what I need for good backup storage.

Installing the Backup Manager was pretty straightforward. I had a few problems, so I'd like to share how to solve it.

After messing with some odd errors in script, I decided to go with a development version and check it out from Git repository. In general, installation looks like this:

$ git clone git://github.com/sukria/Backup-Manager.git 
$ cd Backup-Manager
$ sudo make install
$ sudo cp /usr/share/backup-manager/backup-manager.conf.tpl /etc/backup-manager.conf

Now you need to edit default /etc/backup-manager.conf - this file has a lot of comments so it should be pretty straightforward. I opted for monthly tar-incremental backup to s3.

Before you proceed with configuring backup-manager to backup to Amazon S3 - you need to:

1) Register on http://aws.amazon.com 2) Get some S3 client - I am using S3Fox plugin for Firefox 3) Create bucket for storing backup files

Now update backup-manager.conf S3 section with bucket name, access key and secret key.

Now the tricky part:

The thing I didn't like about the default behavior - the way backup-manager handles MySQL backups. It just dumps ALL databases in one huge file. Or you have an option to manually enter DB names you want to backup - but you need to update config file every time you add a new DB into MySQL.. I wanted something more simple and I've found it - just three lines of code to get a list of DB names and feed them into config variable:

Instead of:

  export BM_MYSQL_DATABASES="__ALL__"

I've put:

  LIST="$(mysql -u root -pPASSWORD -Bse 'show databases')"
  LISTJOIN="${LIST[*]}"
  export BM_MYSQL_DATABASES=$LISTJOIN

Really simple and works flawless so far. Now I am getting a dump of each database in its own file - much easier to do the restore of database backup.

Another tricky part - the backup-manager didn't want to upload backups to Amazon S3. It failed with:

Error reported by backup-manager-upload for method "s3", check "/tmp/bmu-log.P29000".

So, I've looked into /tmp/bmu-log.P29000:

Net::Amazon::S3 is not available, cannot use S3 service : Can't locate Net/Amazon/S3.pm in @INC (@INC contains: /usr/lib/perl5/5.8.8/i686-linux /usr/lib/perl5/5.8.8 /usr/lib/perl5/site_perl/5.8.8/i686-linux /usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/site_perl/5.8.5 /usr/lib/perl5/site_perl/5.8.4 /usr/lib/perl5/site_perl/5.8.3 /usr/lib/perl5/site_perl/5.8.2 /usr/lib/perl5/site_perl/5.8.1 /usr/lib/perl5/site_perl/5.8.0 /usr/lib/perl5/site_perl .) at (eval 12) line 2.
BEGIN failed--compilation aborted at (eval 12) line 2.
The upload transfer "s3" failed.

This happened because the backup-manager uses Perl's module Net::Amazon::S3 to do this job. So you need to install it:

In Ubuntu

$ sudo apt-get install libnet-amazon-s3-perl 

In other linuxes:

$ sudo cpan Net::Amazon::S3

If you install from CPAN - be prepared - there is TONs of dependencies, so it will take a while =)

After all that you should try to run backup manager in the command line to see if everything works alright:

$ sudo /usr/sbin/backup-manager

If there are no errors, backups are created in /var/archives and then uploaded to Amazon S3 - it's time to include backup-manager into crontab:

$ sudo crontab -e

0 0 * * * /usr/sbin/backup-manager > /dev/null 2>> /var/log/backup-manager.log

this will run backup-manager every day at 00:00 am

ps: backup-manager.conf from my own machine - http://pavel.karoukin.us/sites/default/files/attachments/backup-manager_... . Feel free to use it as reference.