Commands tagged rsync (51)

  • If your user has sudo on the remote box, you can rsync data as root without needing to login as root. This is very helpful if the remote box does not allow root to login over SSH (which is a common security restriction).


    18
    rsync --rsync-path 'sudo rsync' username@source:/folder/ /local/
    Alioth · 2009-03-25 21:18:55 11
  • The command copies a file from remote SSH host on port 8322 with bandwidth limit 100KB/sec; --progress shows a progress bar --partial turns partial download on; thus, you can resume the process if something goes wrong --bwlimit limits bandwidth by specified KB/sec --ipv4 selects IPv4 as preferred I find it useful to create the following alias: alias myscp='rsync --progress --partial --rsh="ssh -p 8322" --bwlimit=100 --ipv4' in ~/.bash_aliases, ~/.bash_profile, ~/.bash_login or ~/.bashrc where appropriate. Show Sample Output


    17
    rsync --progress --partial --rsh="ssh -p 8322" --bwlimit=100 --ipv4 user@domain.com:~/file.tgz .
    ruslan · 2011-02-10 14:25:22 9
  • This will backup the _contents_ of /media/SOURCE to /media/TARGET where TARGET is formatted with ntfs. The --modify-window lets rsync ignore the less accurate timestamps of NTFS.


    13
    rsync -rtvu --modify-window=1 --progress /media/SOURCE/ /media/TARGET/
    0x2142 · 2009-07-05 07:40:10 11
  • This creates an archive that does the following: rsync:: (Everyone seems to like -z, but it is much slower for me) -a: archive mode - rescursive, preserves owner, preserves permissions, preserves modification times, preserves group, copies symlinks as symlinks, preserves device files. -H: preserves hard-links -A: preserves ACLs -X: preserves extended attributes -x: don't cross file-system boundaries -v: increase verbosity --numeric-ds: don't map uid/gid values by user/group name --delete: delete extraneous files from dest dirs (differential clean-up during sync) --progress: show progress during transfer ssh:: -T: turn off pseudo-tty to decrease cpu load on destination. -c arcfour: use the weakest but fastest SSH encryption. Must specify "Ciphers arcfour" in sshd_config on destination. -o Compression=no: Turn off SSH compression. -x: turn off X forwarding if it is on by default. Flip: rsync -aHAXxv --numeric-ids --delete --progress -e "ssh -T -c arcfour -o Compression=no -x" [source_dir] [dest_host:/dest_dir]


    12
    rsync -aHAXxv --numeric-ids --delete --progress -e "ssh -T -c arcfour -o Compression=no -x" user@<source>:<source_dir> <dest_dir>
    somaddict · 2012-12-26 13:46:23 7
  • This command works by rsyncing the target directory (containing the files you want to delete) with an empty directory. The '--delete' switch instructs rsync to remove files that are not present in the source directory. Since there are no files there, all the files will be deleted. I'm not clear on why it's faster than 'find -delete', but it is. Benchmarks here: https://web.archive.org/web/20130929001850/http://linuxnote.net/jianingy/en/linux/a-fast-way-to-remove-huge-number-of-files.html


    11
    rsync -a --delete empty-dir/ target-dir/
    malathion · 2016-06-07 16:56:55 18
  • -r for recursive (if you want to copy entire directories) src for the source file (or wildcards) dst for the destination --progress to show a progress bar


    10
    rsync -rv <src> <dst> --progress
    fecub · 2011-08-05 09:29:12 14
  • connect to a remote server using ftp protocol over FUSE file system, then rsync the remote folder to a local one and then unmount the remote ftp server (FUSE FS) it can be divided to 3 different commands and you should have curlftpfs and rsync installed


    9
    curlftpfs ftp://YourUsername:YourPassword@YourFTPServerURL /tmp/remote-website/ && rsync -av /tmp/remote-website/* /usr/local/data_latest && umount /tmp/remote-website
    nadavkav · 2009-03-31 18:01:00 10
  • Assumed dir A, B, C are subdirs of the current dir Exact syntax of the command is: rsync -v -r --size-only --compare-dest=/path_to_A/A/ /path_to_B/B/ /path_to_C/C/ (do not omit end-slashes, since that would copy only the names and not the contents of subdirs of dir B to dir C) You can replace --size-only with --checksum for more thorough file differences validation Useful switch: -n, --dry-run perform a trial run with no changes made


    9
    rsync -v -r --size-only --compare-dest=../A/ B/ C/
    knoppix5 · 2013-09-10 21:41:16 7
  • Yes, rsync(1) supports local directories. And, should anything change, it's trivial to run the command again, and grab only the changes, instead of the full directory.


    8
    rsync -a /etc /destination
    atoponce · 2011-10-18 13:07:55 4
  • Copies the complete root-dir of a linux server to another one, where the new harddisks formated and mountet. Very useful to migrate a root-server to another one.


    7
    rsync -ayz -e ssh --exclude=/proc --exclude=/sys --exclude=/dev / root@NEWHOST:/MNTDIR
    bones · 2012-11-06 09:43:42 700
  • With this cron, rsync begins to sinchronize the contents of the local directory on /[VIPdirectory] with the directory /backup/[VIPdirectory] on the remote server X.X.X.X. Previously we need working on public/private-keys ssh to guarantee the acces to the remote server on X.X.X.X Show Sample Output


    5
    0 10 * * * rsync -rau /[VIPdirectory] X.X.X.X:/backup/[VIPdirectory]
    mack · 2010-03-02 17:48:54 3
  • Apart from an exact copy of your recent contents, also keep all earlier versions of files and folders that were modified or deleted. Inspired by EVACopy http://evacopy.sourceforge.net Show Sample Output


    5
    backup() { source=$1 ; rsync --relative --force --ignore-errors --no-perms --chmod=ugo=rwX --delete --backup --backup-dir=$(date +%Y%m%d-%H%M%S)_Backup --whole-file -a -v $source/ ~/Backup ; } ; backup /source_folder_to_backup
    pascalvaucheret · 2018-08-02 21:27:29 331

  • 4
    rsync -rv --exclude .svn src/dir/ dest/dir/
    pykler · 2011-11-15 21:42:22 8
  • rsync'ing an empty directory over a directory to be deleted recursively is much faster than using rm -rf, for various reasons. Relevant only for directories with really a lot of files.


    4
    blank=$(mktemp -d); rsync --delete "$blank/" "bigdir/"; rmdir "$blank"
    Natureshadow · 2016-02-15 11:23:15 12
  • Using the double dash before the source and target makes the command work fine with weird filenames.


    3
    rsync [options] -- * target
    l0b0 · 2010-11-18 23:09:17 6
  • Zsync is an implementation of rsync over HTTP that allows updating of files from a remote Web server without requiring a full download. For example, if you already have a Debian alpha, beta or RC copy downloaded, zsync can just download the updated bits of the new release of the file from the server. This requires the distributor of the file to have created a zsync build control file (using zsyncmake).


    2
    zsync -i existing-file-on-disk.iso http://example.com/new-release.iso.zsync
    rkulla · 2010-04-20 07:02:37 8
  • 'data' is the directory to backup, 'backup' is directory to store snapshots. Backup files on a regular basis using hard links. Very efficient, quick. Backup data is directly available. Same as explained here : http://blog.interlinked.org/tutorials/rsync_time_machine.html in one line. Using du to check the size of your backups, the first backup counts for all the space, and other backups only files that have changed. Show Sample Output


    2
    rsync -av --link-dest=$(ls -1d /backup/*/ | tail -1) /data/ /backup/$(date +%Y%m%d%H%M)/
    dooblem · 2010-08-05 19:36:24 6
  • '-mtime -10' syncs only files newer 10 days (-mtime is just one example, use whatever find expressions you need) printf %P: File's name with the name of the command line argument under which it was found removed. this way, you can use any src directory, no need to cd into your src directory first. using \\0 in printf and a corresponding --from0 in rsync ensures that even filenames with newline characters work (thanks syssyphus for #3808). both, #1481 and #3808 just work if you either copy the current directory (.) , or the filesystem root (/), otherwise the output from find and the source dir from rsync just don't match. #7685 works with an arbitrary source directory.


    2
    find /src/dir/ -mtime -10 -printf %P\\0|rsync --files-from=- --from0 /src/dir/ /dst/dir/
    pauli · 2011-01-18 22:23:47 3

  • 2
    rsync -avzhP <[[user@]host1:]directory1> <[[user@]host2:]directory2>
    funkjedi · 2013-04-27 03:14:23 8
  • "Sample output" shows a minimalistic configuration file. Show Sample Output


    2
    rsync --daemon --port 1234 --no-detach -v --config rsyncd.conf
    denilsonsa · 2016-08-21 22:10:17 15
  • --delete will delete copies on remote to match local if deleted on local --stats will output the results -z zip -a archive -A preserve ACL -x don't cross filesystem boundaries -h human readable -e specify the remote shell to use


    2
    rsync --delete --stats -zaAxh -e ssh /local_directory/ username@IP_of_remote:/Remote_Directory/ > /Text_file_Directory/backuplog.txt
    chicagonpg · 2019-09-10 21:00:29 89
  • Manage partial uploads using append option.


    2
    rsync --archive --recursive --compress --partial --progress --append root@123.123.123.123:/backup/somefile.txt.bz2 /home/ubuntu/
    shantanuo · 2020-02-11 13:48:22 122
  • Copying only wmv and mpg files recursively from to


    1
    rsync -rvtW --progress --include='*.wmv' --include='*.mpg' --exclude='*.*' <sourcedir> <destdir>
    MauricioVieira · 2009-10-19 10:38:11 4

  • 1
    rsync -vau --exclude='.*' SOURCE-PATH/myfold TARGET-PATH
    kev · 2011-07-20 17:49:50 3
  • Copies a directory structure from /home/ to /backups/home (notice that the destination does not have a trailing slash)


    1
    BEGIN=`date`; rsync -avxW /home/ /backups/home ; echo "Begin time: $BEGIN" ; echo "End time..: `date`"
    ryanchapman · 2013-07-06 08:24:45 25
  •  1 2 3 > 

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands


Check These Out

execute a shell with netcat without -e
how to execute a shell on a server with a netcat binary which doesn't support -e option

Set laptop display brightness
Run as root. Path may vary depending on laptop model and video card (this was tested on an Acer laptop with ATI HD3200 video). $ cat /proc/acpi/video/VGA/LCD/brightness to discover the possible values for your display.

Get AWS temporary credentials ready to export based on a MFA virtual appliance
You might want to secure your AWS operations requiring to use a MFA token. But then to use API or tools, you need to pass credentials generated with a MFA token. This commands asks you for the MFA code and retrieves these credentials using AWS Cli. To print the exports, you can use: `awk '{ print "export AWS_ACCESS_KEY_ID=\"" $1 "\"\n" "export AWS_SECRET_ACCESS_KEY=\"" $2 "\"\n" "export AWS_SESSION_TOKEN=\"" $3 "\"" }'` You must adapt the command line to include: * $MFA_IDis ARN of the virtual MFA or serial number of the physical one * TTL for the credentials

convert pdf to graphic file format (jpg , png , tiff ... )
need imagemagick package

Convert seconds to [DD:][HH:]MM:SS
Converts any number of seconds into days, hours, minutes and seconds. sec2dhms() { declare -i SS="$1" D=$(( SS / 86400 )) H=$(( SS % 86400 / 3600 )) M=$(( SS % 3600 / 60 )) S=$(( SS % 60 )) [ "$D" -gt 0 ] && echo -n "${D}:" [ "$H" -gt 0 ] && printf "%02g:" "$H" printf "%02g:%02g\n" "$M" "$S" }

move you up one directory quickly
In bash, this turns on auto cd. If a command is just a directory name, it cd's into that directory.

get all Google ipv4/6 subnets for a iptables firewall for example (updated version)
google has added 2 more netblocks...

Spawn a retro style terminal emulator.
Just note that ctrl+shift+t to make new tabs will not work with . Pair it with a terminal multiplexer like for the best experience.

Vectorize xkcd strips
Uses ImageMagick and potrace to vectorize the input image, with parameters optimized for xkcd-like pictures.

Copy an element from the previous command
You can specify a range via '-'.


Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: