Snapshot databases & files

Imagine that the filesystem is book and each entry in the table of contents (TOC) is a reference to a chapter. In the filesystem this “chapter” is referred to as an inode and the TOC entry is called a hard link.
you may call it a file ;)

Now the cool thing about this is that we can have several links to the same inode, in fact we can have 65.000 hard links to the same inode without using up any extra space.

“OK, now let’s look at four dimensions on the blackboard.”

Dr. Joy

It means that we can run incremental backups of the files that have changed since the last run, the other files (which are the same as last time) can then be hard linked into new directories using the same inodes as previous backup.

This way we can do backups for each hour, day, week, month and even year and each of these folders will have a complete set of “files”.

Since they are linking to the same inodes they wont occupy any extra space on the drive. Except of course for the space a hard link takes up which is about 500 Megabytes per 500 Gigabytes.

Note that a hard link differs from a symbolic link which can be seen as a reference to another reference instead of a reference to the same inode.

Auto Mysql Backup

Before we dive into the wonders of file backup using hard links, we’re going to setup scheduled backups of our databases using the utility “automysqlbackup”

# apt update && apt install automysqlbackup
# nvim /etc/defaults/automysqlbackup.conf
DBNAMES=`mysql --defaults-file=/etc/mysql/debian.cnf --execute="SHOW DATABASES" | awk '{print $1}' | grep -v ^Database$ | grep -v ^mysql$ | grep -v ^performance_schema$ | grep -v ^information_schema$ | tr \\\r\\\n ,\ `

Rsync Snapshots

# apt update && apt install rsnapshot
# nvim /etc/rsnapshot.conf

Make sure that you are using tabs and not spaces in the rsnapshot configuration!

config_version	1.2
snapshot_root	/storage/rsnapshot/
cmd_cp		/bin/cp
cmd_rm		/bin/rm
cmd_rsync	/usr/bin/rsync
cmd_ssh	/usr/bin/ssh
cmd_logger	/usr/bin/logger
retain	hourly	6
retain	daily	7
retain	weekly	4
retain	monthly	12
verbose		3
loglevel	3
lockfile	/var/run/
ssh_args	-i /root/.ssh/id_rsa_backup
exclude	.git
exclude	sess_
exclude	chroot_irc
exclude	chroot_mail
exclude	chroot_public
exclude	chroot_toni
exclude	proc
exclude	sys
exclude	dev
exclude	opt/debootstrap

backup	/etc/				localhost/
backup	/var/www/			localhost/
backup	user@server:/etc		server/

i am using the ssh_args parameter to specify which rsa key to use for authentication. (see my chapter on this: securing-you-rsync-backups-with-sshd-restrictions)

automysqlbackup adds itself to cron.daily, weekly & monthly but rsnapshot does not.

# nvim /etc/cron.d/rsnapshot
# This is a sample cron file for rsnapshot.
# The values used correspond to the examples in /etc/rsnapshot.conf.
# There you can also set the backup points and many other things.
# To activate this cron file you have to uncomment the lines below.
# Feel free to adapt it to your needs.

0 */4         * * *           root    /usr/bin/rsnapshot hourly
30 3          * * *           root    /usr/bin/rsnapshot daily
0  3          * * 1           root    /usr/bin/rsnapshot weekly
30 2          1 * *           root    /usr/bin/rsnapshot monthly

Log report

First of, Lorem ipsum

# cp /usr/share/doc/rsnapshot/examples/utils/ /usr/local/bin
# chmod +x /usr/local/bin/

Then change your rsnapshot.conf to add –stats. I also chose to change hourly to 2.

rsync_long_args    --delete --numeric-ids --relative --delete-excluded --stats

retain	hourly	2
retain	daily	7
retain	weekly	4
retain	monthly	3

And modify the crontab, 0 */12 means zero minutes passed every twelfth hour.

0 */12         * * *           root    /usr/bin/rsnapshot hourly 2>&1 | /usr/local/bin/ | /usr/bin/mail -s"rsnapshot daily" root

No actual syncing is performed on levels above hourly, just rotations.