Using LVM Snapshots to do a Backup

Posted by mwguy on Wed 23 December 2015

Using LVM Snapshots to do a Backup

At my former job we had a process that required the main system to be stopped before the backup could begin. The model the built in backup script followed was:

  1. Shut down Software
  2. Backup Software Location(s)
  3. Start Software Back Up

This piece of software wasn't something where it was critical that it be available 24x7x365 so it was okay that it was down for a short bit of time. However as the software continued to grow and the files below the software continued to get larger it was taking longer and longer to backup. So we decided to change how we backed up the software by using LVM Snapshots to accomplish our goals. This changed our model to:

  1. Shut down Software
  2. Take Snapshot of Location(s)
  3. Start Up Software
  4. Backup Snapshotted Files
  5. Destroy Snapshot

I can share the exact script that we used (given that it's owned by my previous company). But I can share a generic example of how we accomplished it. First we stop the service;

systemctl stop myservice.service

systemctl is-active myservice.service

is_running=$?
# Will be 0 if active or non-zero if not

if [[ is_running -eq 0 ]] ; then
    # Issues
    # Log
else
    # It's stopped
    take_snapshot_function
fi

# No Matter what restart
systemctl start myservice.service

systemctl is-active myservice.service

is_running-$?

if [[ is_running -ne 0 ]] ; then
    # Alarm per your normal methods
fi

# Take Backups
take_backup_function

exit 0

Elsewhere you can have a function that takes your snapshot. Just your snapshot.

take_snapshot_function(){
    # Log It
    /usr/sbin/lvcreate --size $datachange_allow --snapshot --name $name_of_snapshot $data_lv
    snapgood=$?
    # Check for errors
}

The things to note here are the variables. $datachange_allow contains how much space is alloted to track changes between the main lv and this new one before the snapshot auto destroys (EG 10G). $name_of_snapshot is the common name you're going to use for your snapshot. $data_lv is the full name of your source lv (E.G. /dev/mapper/vg1-appdata. It should contain the data for your application.

Then you're going to have your take backup function. It's going to mount the snapshot, do your backup and then destroy the snapshot.

take_backup_function(){
    # Make sure your target dir exists
    mkdir -p /path/to/mount
    # Mount the Snapshot
    mount -o nouuid,rw /dev/$data_vg/$name_of_snapshot /path/to/mount
    snap_mount_good=$?
    # Check if it's good and do stuff with it.

    # Now Run your Backup Script
    your_custom_backup_script.sh --sourcedir /path/to/mount

    # Now that your backup is complete Unmount and Destroy
    umount /path/to/mount

    # Use -f so it doesn't prompt you.
    /usr/sbin/lvremove -f /dev/$data_vg/$name_of_snapshot
    is_destroyed=$?
    # Check it

    # Should be good now
}

For the last function you will need a backup script. I assume you have one and that you can pass it the proper directory. It should also be noted that the nouuid section is only needed if you're using xfs (generally CentOS or RHEL styled systems). You're final script should be much more complex as it should at every point ensure that things are working as designed. However this will hopefully be enough to get you started.

If you're having some problems with your backup script toss it up on github and leave a comment below. When I get a chance I'll try to give it a look. Otherwise hope that this is a good intro into LVM Snapshots in the context of application backups.