Synchronising files with rsync

The rsync command is a very powerful and versatile command for synchronising your files across two disks mounted on the same machine, or synchronising with another Mac/Unix/Linux machine on the Internet. I have provided some examples of its usage below, email me if you want more examples.

Let's say you're been away from the department and want to synchronise work that you've done in the papers directory on your laptop with your papers directory on aquila, the command you would use is:

rsync -uavz ~/papers/ aquila:papers/

you may have to use aquila.star.bris.ac.uk depending on how your
laptop is set up.

or if you are in your home directory:
rsync -uavz papers/ aquila:papers/
The placement of the '/' is important and must be placed at the end of the remote and local directory specifications as above. Leaving out the / would copy the whole of the first directory into the second.

It is a good idea to test which files will be overwritten before actually running the command in which case we add a '-n' to the above command as below.

rsync -nuavz  papers/ aquila:papers/
To synchronise files the opposite way, ie to bring your laptop up to date before going on a trip, then the command you need is:
rsync -uavz  aquila:papers/ papers/
If you want to synchronise your files across 2 disks, the command to synchronise your home directory with a copy on, say, /data/pavo1 would be as below. You would of course use your own directories instead of mine!
rsync -uavz /export/home/rahm/ /data/pavo1/rahm/
If you want to copy just programs and scripts, but not data files, the command below could be used to back up the hst directory on a remote machine with the home directory on aquila (which should not contain data files).
rsync -uavz --exclude='*.fits' hst/ aquila:hst/
Note that copying to Windows file systems, such as a FAT32 partition on a USB removable disk, may cause problems as their time-stamp accuracy is not good and may cause rsync to copy the whole lot over again. In this case, the command would be as below.
rsync -uavz --modify-window=1 papers/ aquila:papers/
Finally, if you're brave, rsync can make two directories identical by deleting files as well as copying them, this is done using the --delete option. If you are rsyncing from machine A to machine B, the --delete option will delete files on machine B if they do not exist on machine A.

Running rsync automatically with a cron job

First find and test an rsync command that works for you. Then it can be automated using the cron facility. Let us say that your username is amelia and you have data stored on /data/lyra1/amelia which you would like to back up on a regular basis to /data/gemini1/amelia, then the command would be
rsync -uav /data/lyra1/amelia/ /data/gemini1/amelia/
(remember the slashes are vital). Then in order to run this command every night at 3.12 am, do the following:
# Set your default editor.
setenv EDITOR emacs

# Open an editor to edit your crontab file
crontab -e

# Insert the following lines into the editor window.

# Run at 12 mins past 3, every day, every month, every year
12 3 * * *  rsync -uav /data/lyra1/amelia/ /data/gemini1/amelia/

# close the editor, do a 

crontab -l

to list your cron job, then check the first few backups to make sure
they are behaving as expected.

Rhys Morris
Last modified: Thu Nov 20 10:25:03 GMT 2014