How do we back up data on Unix machines? Most often, system administrators use whatever comes to hand first for this - simply copying the needed files to a remote server.
One way to do this is to use the SCP utility - to transfer files over SSH. But there's a more interesting option, especially if you're planning to set up differential file copying (i.e. copying only new or changed files, not everything at once) - the RSYNC utility.
InstallationUnlike SCP, it doesn't come out of the box, so we'll need to install it.
Linux Debian/ubuntu:
$ sudo aptitude install rsync
Linux CentOS/RedHat:
$ sudo yum install rsync
FreeBSD:
$ cd /usr/ports/net/rsync
$ sudo make install clean
here you'll be asked to enable the options you need. Go by your own needs, but leave the SSH option enabled - we'll need it.
What else is neededBesides the Rsync client, you also need its server. It's installed together with the client, so the same steps described above need to be carried out on the machine that copying will be done to.
After that
FreeBSD:
open the /etc/rc.conf file on the remote server for editing and add the following line to it:
rsyncd_enable="YES"
after that, on the remote machine, start the Rsync server:
$ sudo /usr/local/etc/rc.d/rsyncd start
Debian/ubuntu:
There's more hassle here. First open the /etc/default/rsync file and change the line "RSYNC_ENABLE=false" to "RSYNC_ENABLE=true":
RSYNC_ENABLE=true
Then go to /etc and create an Rsync config (rsyncd.conf) roughly like this:
log file=/var/log/rsyncd
pid file=/var/run/rsyncd.pid
socket options = SO_KEEPALIVE
and finally, start Rsync:
$ sudo /etc/init.d/rsync start
CentOS/RedHat:
This one also needs some tinkering. Open the /etc/xinetd.d/rsync file for editing and change the line "disable = yes" to "disable = no".
After that, restart the xinetd daemon:
$ sudo /etc/init.d/xinetd restart
Usage
$ sudo rsync -e ssh --progress -lzuogthvr --compress-level=9 --delete-after /mydir/sourcedir username@remoteserver:/backupdir/
Let's explain the parameters being passed:
- -e ssh : Use SSH for the connection
- --progress : Show copy progress
- -l : Recreate symlinks — i.e., symlinks will be created on the remote server pointing to the right location. Even if that location doesn't exist on the remote server, it's better to do it this way, since when the backup is restored on this machine, the symlinks will become functional!
- -z : Use compression (to save bandwidth). If you compress too aggressively, copying can take a very long time, and you can drop this option if the CPU isn't powerful enough
- -u : Update mode — i.e., differential copying. The utility will only copy files that don't yet exist on the remote server or that have been changed.
- -o : Set the owner of the copied file to be the same as the original
- -g : Set the owning group of the copied file to be the same as the original
- -t : Set the creation/modification time of the file on the remote server to match the original. This option is mandatory for differential copying — otherwise the file's time on the remote machine will be set to the current time
- -h : Display information on screen in a human-readable format
- -v : Verbose output (i.e., extended information during the copy process)
- -r : Recursive copying (i.e., everything found at the specified path, including subdirectories).
- --compress-level=9 : Compression level
- --delete-after : Changes the utility's behavior. By default the utility first deletes the target file if it exists, and only then copies its replacement, which is useful when transferring large files to a server that's low on free space. If you specify --delete-after, the new file is first downloaded to a temp directory, and only then is the old one deleted.
- /mydir/sourcedir : the directory that will be copied
- username : the username on the remote server
- remoteserver : the IP address or hostname of the remote server
- /backupdir/ : the directory the copied folder will be placed into
When using this utility in an automated backup script, there's no point specifying the --progress, -h, and -v options.
You can copy the whole directory back like this (when restoring from a backup):
$ sudo rsync -e ssh --progress -lzuogthvr --compress-level=9 --delete-after username@remoteserver:/backupdir/ /mydir/sourcedir
i.e., we simply swapped the last 2 parameters.
PS. When using the utility for copying in automatic mode, you need to set up passwordless SSH access. How to do that was described earlier
Comments