Rsync

Rsync¶

rsync -ruiP –ignore-existing –log-file=”~/vautl10T-01-sync.txt” /media/vault10T-01/ /media/tom/80FA-526C

Basic Options¶

-g, --group                 preserve group
-l, --links                 copy symlinks as symlinks
-o, --owner                 preserve owner (super-user only)
-p, --perms                 preserve permissions
-t, --times                 preserve modification times
-r, --recursive             recurse into directories
-u, --update                skip files that are newer on the receiver
-c, --checksum              skip based on checksum, not mod-time & size
    --del                   an alias for --delete-during
    --delete-during         receiver deletes during the transfer
-h, --human-readable        output numbers in a human-readable format
    --progress              show progress during transfer
-i, --itemize-changes       output a change-summary for all updates
-P                          same as --partial --progress
    --log-file=FILE         log what we're doing to the specified FILE
    --log-file-format=FMT   log updates using the specified FMT
-n, --dry-run               perform a trial run with no changes made

z compress file data during the transfer (do this when transferring over the network)
exclude=PATTERN exclude files matching PATTERN

Options for scheduled backup i output a change-summary for all updates --ignore-existing skip updating files that exist on receiver

Compound Options -a is equal to -rlptgoD (no -H,-A,-X)

-D same as –devices –specials

Examples¶

Copy files from /foo/bar/ into /backups/bar skipping files that already exist on /backups/bar

    rsync -avu --ignore-existing "/foo/bar/" "/backups/bar"

Comparing files based on size and time (the default) is faster than using a checksum.

Copy files from with progress¶

Copy all files (books) from /foo/bar/books/ to /backups/bar/books skipping files that already exist on /backups/bar/books

rsync -ru --progress --ignore-existing \
    "/foo/bar/books/" \
    "/backups/bar/books"

Copy files from external server¶

rsync -avz username@server:/home/a/b/target_folders/ target_folder

This will copy the contents from target_folder on the remote machine into target_folder on local machine.

Monitor rsync¶

See what RSYNC is currently doing strace -e open $(ps -o lwp= -LC rsync | sed 's/^/-p/')

See what files are currently open lsof -ad3-999 -c rsync

Remove files that were deleted on SOURCE¶

In other words: Delete files on TARGET that do not exist on SOURCE

delete options are just for TARGET, if some files are removed from source, rsync --delete remove them from TARGET

Example

rsync -avh source/ dest/ --delete

`--delete-before` receiver deletes before transfer, not during
                  1. rsync delete the file from TARGET which are removed from
                     SOURCE.
                  2. rsync start syncing files.
`--delete-after`  receiver deletes after transfer, not during
                  1. rsync start syncing files
                  2. rsync delete the file from TARGET which are removed from
                     SOURCE after syncing

If not using -c to compare based on checksum, the modified time on the destination can be used to figure out which files to delete.

--delete-before receiver deletes before transfer, not during Means that:

rsync delete the file from TARGET which are removed from SOURCE.
rsync start syncing files.

--delete-after receiver deletes after transfer, not during Means that:

rsync start syncing files.
rsync delete the file from TARGET which are removed from SOURCE after syncing.

--partial just tells the receiving end to keep partially transferred files if the sending end disappears as though they were completely transferred.

While transferring files, they are temporarily saved as hidden files in their target folders (e.g. .TheFileYouAreSending.lRWzDC)

When a transfer fails and --partial is not set, this hidden file will remain in the target folder under this cryptic name, but if --partial is set, the file will be renamed to the actual target file name (in this case, TheFileYouAreSending), even though the file isn’t complete. The point is that you can later complete the transfer by running rsync again with either --append or --append-verify.

With regards to the --append switch mentioned above, this is the actual “resume” switch, and you can use it whether or not you’re also using --partial

Actually, when you’re using --append, no temporary files are ever created. Files are written directly to their targets. In this respect, --append gives the same result as --partial on a failed transfer, but without creating those hidden temporary files.

You probably always want the behaviour of --append-verify

--append-verify isn’t dangerous: It will always read and compare the data on both ends and not just assume they’re equal. It does this using checksums, so it’s easy on the network, but it does require reading the shared amount of data on both ends of the wire before it can actually resume the transfer by appending to the target.

With --checksum, rsync will examine files that exist on both ends of the wire. It does this in chunks, compares the checksums on both ends, and if they differ, it transfers just the differing parts of the file. But, as @Jonathan points out below, the comparison is only done when files are of the same size on both ends — different sizes will cause rsync to upload the entire file, overwriting the target with the same name.

Backup to media server¶

today="$(date +%Y-%m-%d)"
output_file="~/rsync-log-${today}.txt"
rsync -avuizP \
    --ignore-existing \
    --exclude=".*" \
    --exclude=".*/" \
    --exclude="*env/lib/python2.7/*" \
    --exclude="*/node_modules/" \
    --exclude="*lib/python*.*/site-packages/*" \
    --log-file="$output_file" \
    /media/vault10T-01/ \
    tom@192.168.88.144:/media/vault

Similar Tools¶

restic