I made a backup script that is working pretty nicely for me so far. It has several function, the last one being the essence of it.
- Make a dump of all databases (postgres & mysql) to zipped sql files. It deletes any that haven't changed since the last backup.
- Bring a list of git repositories up-to-date with with their remotes on the current branch, by cloning or pulling. (This is useful to have a backup of a non-bare repository).
- Make a copy of each in a list of remote directories. It does this by taking the last backup, hardlinking all the files to a new location and then bringing that one up-to-date with the remote.
This way, you'll end up with a copy of each directory you want to preserve, for each time you run the script. If your machine has any problem, you just grab any backup and you have your data back in the original format. And because they are all hardlinks, files only take space if they're changed.
It'd be best to run this every day from a remote computer that can login to (all) the machine(s) you want to backup. If those machines get compromised, the attacker can't get to your backups or logging. And you can easily have the backup computer elsewhere so that your data is safe even if your room burns down. Maybe it's better if you don't burn your room down though.
You can define your own logging functions, so that you'll noticed if backups stop for any reason. Have it send emails on success and failure or make a quick POST request to your logging server.
My setup looks like this:
There are some logging functions from shcripts but you can easily replace them if you wish. And conf_logging.sh has functions like this:
There are some requirements though. Like a basic understanding of bash and linux. All your machines should also be reachable through ssh. You should have some machine that can run the backups. A Raspberry Pi is ideal if you can get one to run stably, which I can't (it always gets corrupted after some weeks). In my case it runs on a VPS from Contabo, whose service I am very happy with. Furthermore the backup machine needs git, ssh, pax and rsync. They're all in apt-get and except for pax I think they're very common.
[It's called Dory because my original backup machine was called dory. All of them are named after Disney characters, and this character seemed particularly appropriate.]