Commands tagged sed (376)

  • Checks the Gmail ATOM feed for your account, parses it and outputs a list of unread messages. For some reason sed gets stuck on OS X, so here's a Perl version for the Mac: curl -u username:password --silent "https://mail.google.com/mail/feed/atom" | tr -d '\n' | awk -F '<entry>' '{for (i=2; i<=NF; i++) {print $i}}' | perl -pe 's/^<title>(.*)<\/title>.*<name>(.*)<\/name>.*$/$2 - $1/' If you want to see the name of the last person, who added a message to the conversation, change the greediness of the operators like this: curl -u username:password --silent "https://mail.google.com/mail/feed/atom" | tr -d '\n' | awk -F '<entry>' '{for (i=2; i<=NF; i++) {print $i}}' | perl -pe 's/^<title>(.*)<\/title>.*?<name>(.*?)<\/name>.*$/$2 - $1/' Show Sample Output


    47
    curl -u username:password --silent "https://mail.google.com/mail/feed/atom" | tr -d '\n' | awk -F '<entry>' '{for (i=2; i<=NF; i++) {print $i}}' | sed -n "s/<title>\(.*\)<\/title.*name>\(.*\)<\/name>.*/\2 - \1/p"
    postrational · 2009-09-07 21:56:40 46
  • In this case it's better do to use the dedicated tool


    45
    ssh-keygen -R <the_offending_host>
    bunam · 2010-07-11 19:37:24 16

  • 30
    sed -i 8d ~/.ssh/known_hosts
    prayer · 2010-07-10 14:22:34 11
  • recursively traverse the directory structure from . down, look for string "oldstring" in all files, and replace it with "newstring", wherever found also: grep -rl oldstring . |xargs perl -pi~ -e 's/oldstring/newstring'


    25
    $ grep -rl oldstring . |xargs sed -i -e 's/oldstring/newstring/'
    netfortius · 2009-03-03 20:10:19 26
  • Yeah I know it's been up here a million times, but this service is a really clean and nice one. Nothing but your IP address on it. Actually I was to write something like this, and noticed this on appspot... ;) Show Sample Output


    25
    curl ip.appspot.com
    ktoso · 2009-10-31 21:11:10 12
  • If you have used bash for any scripting, you've used the date command alot. It's perfect for using as a way to create filename's dynamically within aliases,functions, and commands like below.. This is actually an update to my first alias, since a few commenters (below) had good observations on what was wrong with my first command. # creating a date-based ssh-key for askapache.github.com ssh-keygen -f ~/.ssh/`date +git-$USER@$HOSTNAME-%m-%d-%g` -C 'webmaster@askapache.com' # /home/gpl/.ssh/git-gplnet@askapache.github.com-04-22-10 # create a tar+gzip backup of the current directory tar -czf $(date +$HOME/.backups/%m-%d-%g-%R-`sed -u 's/\//#/g' <<< $PWD`.tgz) . # tar -czf /home/gpl/.backups/04-22-10-01:13-#home#gpl#.rr#src.tgz . I personally find myself having to reference date --help quite a bit as a result. So this nice alias saves me a lot of time. This is one bdash mofo. Works in sh and bash (posix), but will likely need to be changed for other shells due to the parameter substitution going on.. Just extend the sed command, I prefer sed to pretty much everything anyways.. but it's always preferable to put in the extra effort to go for as much builtin use as you can. Otherwise it's not a top one-liner, it's a lazyboy recliner. Here's the old version: alias dateh='date --help|sed "/^ *%%/,/^ *%Z/!d;s/ \+/ /g"|while read l;do date "+ %${l/% */}_${l/% */}_${l#* }";done|column -s_ -t' This trick from my [ http://www.askapache.com/linux-unix/bash_profile-functions-advanced-shell.html bash_profile ] Show Sample Output


    21
    alias dateh='date --help|sed -n "/^ *%%/,/^ *%Z/p"|while read l;do F=${l/% */}; date +%$F:"|'"'"'${F//%n/ }'"'"'|${l#* }";done|sed "s/\ *|\ */|/g" |column -s "|" -t'
    AskApache · 2010-04-21 01:22:18 16
  • Delete a range of line


    19
    sed -i <file> -re '<start>,<end>d'
    Zulu · 2012-02-02 17:59:18 15
  • This command find all files in the current dir and subdirs, and replace all occurances of "oldstring" in every file with "newstring".


    16
    find . -type f -exec sed -i s/oldstring/newstring/g {} +
    SlimG · 2009-12-09 00:46:13 15
  • Use the following variation for FreeBSD: openssl rand 6 | xxd -p | sed 's/\(..\)/\1:/g; s/:$//'


    16
    openssl rand -hex 6 | sed 's/\(..\)/\1:/g; s/.$//'
    putnamhill · 2010-09-23 02:31:12 5

  • 16
    check(){ curl -sI $1 | sed -n 's/Location: *//p';}
    putnamhill · 2010-09-30 12:29:02 8
  • echo "http%3A%2F%2Fwww.google.com" | sed -e's/%\([0-9A-F][0-9A-F]\)/\\\\\x\1/g' | xargs echo -e http://www.google.com Works under bash on linux. just alter the '-e' option to its corresponding equivalence in your system to execute escape characters correctly.


    14
    sed -e's/%\([0-9A-F][0-9A-F]\)/\\\\\x\1/g' | xargs echo -e
    mohan43u · 2009-05-25 05:37:44 41
  • This one uses dictionary.com


    13
    pronounce(){ wget -qO- $(wget -qO- "http://dictionary.reference.com/browse/$@" | grep 'soundUrl' | head -n 1 | sed 's|.*soundUrl=\([^&]*\)&.*|\1|' | sed 's/%3A/:/g;s/%2F/\//g') | mpg123 -; }
    matthewbauer · 2010-03-13 04:23:56 12
  • Though without infinite time and knowledge of how the site will be designed in the future this may stop working, it still will serve as a simple straight forward starting point. This uses the observation that the only item marked as strong on the page is the single logical line that includes the italicized fact. If future revisions of the page show failure, or intermittent failure, one may simply alter the above to read. wget randomfunfacts.com -O - 2>/dev/null | tee lastfact | grep \<strong\> | sed "s;^.*<i>\(.*\)</i>.*$;\1;" The file lastfact, can then be examined whenever the command fails.


    13
    wget randomfunfacts.com -O - 2>/dev/null | grep \<strong\> | sed "s;^.*<i>\(.*\)</i>.*$;\1;"
    tali713 · 2010-03-30 23:49:30 81
  • Prints the 4th line and then quits. (Credit goes to flatcap in comments: http://www.commandlinefu.com/commands/view/6031/print-just-line-4-from-a-textfile#comment.)


    12
    sed -n '4{p;q}'
    kniht · 2010-07-10 00:26:22 5
  • The -i option in sed allows in-place editing of the input file. Replace myexpression with any regular expression. /expr/d syntax means if the expression matches then delete the line. You can reverse the functionality to keep matching lines only by using: sed -i -n '/myexpression/p' /path/to/file.txt


    11
    sed -i '/myexpression/d' /path/to/file.txt
    jgc · 2009-11-09 11:40:45 7
  • seq -s ' ' 1 9 | sed -n ':a;p;s/ *\w$//;h;/^$/t;b a;q' | tac | awk '{for(i=1;i 1x1=1 1x2=2 2x2=4 1x3=3 2x3=6 3x3=9 1x4=4 2x4=8 3x4=12 4x4=16 1x5=5 2x5=10 3x5=15 4x5=20 5x5=25 1x6=6 2x6=12 3x6=18 4x6=24 5x6=30 6x6=36 1x7=7 2x7=14 3x7=21 4x7=28 5x7=35 6x7=42 7x7=49 1x8=8 2x8=16 3x8=24 4x8=32 5x8=40 6x8=48 7x8=56 8x8=64 1x9=9 2x9=18 3x9=27 4x9=36 5x9=45 6x9=54 7x9=63 8x9=72 9x9=81


    11
    seq 9 | sed 'H;g' | awk -v RS='' '{for(i=1;i<=NF;i++)printf("%dx%d=%d%s", i, NR, i*NR, i==NR?"\n":"\t")}'
    kev · 2011-10-22 18:41:09 31
  • I needed a way to search all files in a web directory that contained a certain string, and replace that string with another string. In the example, I am searching for "askapache" and replacing that string with "htaccess". I wanted this to happen as a cron job, and it was important that this happened as fast as possible while at the same time not hogging the CPU since the machine is a server. So this script uses the nice command to run the sh shell with the command, which makes the whole thing run with priority 19, meaning it won't hog CPU processing. And the -P5 option to the xargs command means it will run 5 separate grep and sed processes simultaneously, so this is much much faster than running a single grep or sed. You may want to do -P0 which is unlimited if you aren't worried about too many processes or if you don't have to deal with process killers in the bg. Also, the -m1 command to grep means stop grepping this file for matches after the first match, which also saves time. Show Sample Output


    10
    sh -c 'S=askapache R=htaccess; find . -mount -type f|xargs -P5 -iFF grep -l -m1 "$S" FF|xargs -P5 -iFF sed -i -e "s%${S}%${R}%g" FF'
    AskApache · 2009-10-02 05:03:10 7
  • Not my script. Belongs to mathewbauer. Used without his permission. This script gives a single line as shown in the sample output. NOTE: I have blanked out the IP address for obvious security reasons. But you will get whatever is your IP if you run the script. Tested working in bash. Show Sample Output


    10
    curl -s "http://www.geody.com/geoip.php?ip=$(curl -s icanhazip.com)" | sed '/^IP:/!d;s/<[^>][^>]*>//g'
    getkaizer · 2009-11-04 07:15:02 11

  • 10
    color()(set -o pipefail;"$@" 2>&1>&3|sed $'s,.*,\e[31m&\e[m,'>&2)3>&1
    bkmeneguello · 2013-01-30 13:42:03 7

  • 9
    echo Print text vertically|sed 's/\(.\)/\1\n/g'
    axelabs · 2009-03-04 23:58:02 15

  • 9
    sed '/^#.*DEBUG.*/ s/^#//' $FILE
    svg · 2009-07-30 09:30:17 18

  • 9
    curl -s 'http://checkip.dyndns.org' | sed 's/.*Current IP Address: \([0-9\.]*\).*/\1/g'
    kulor · 2009-08-06 11:54:31 10

  • 8
    sed -i 's/^.*DEBUG.*/#&/' $file
    svg · 2009-07-30 09:20:37 77
  • Having to escape forwardslashes when using sed can be a pain. However, it's possible to instead of using / as the separator to use : . I found this by trying to substitute $PWD into my pattern, like so sed "s/~.*/$PWD/" file.txt Of course, $PWD will expand to a character string that begins with a / , which will make sed spit out an error such as "sed: -e expression #1, char 8: unknown option to `s'". So simply changing it to sed "s:~.*:$PWD:" file.txt did the trick. Show Sample Output


    8
    sed "s:/old/direcory/:/new/directory/:" <file>
    thebillywayne · 2009-08-06 00:37:45 6
  • I often need to extract a function from a bash script and this command will do it. Show Sample Output


    8
    sed -n '/^function h\(\)/,/^}/p' script.sh
    haivu · 2009-10-19 07:55:35 33
  •  1 2 3 >  Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands


Check These Out

Find and display most recent files using find and perl
This pipeline will find, sort and display all files based on mtime. This could be done with find | xargs, but the find | xargs pipeline will not produce correct results if the results of find are greater than xargs command line buffer. If the xargs buffer fills, xargs processes the find results in more than one batch which is not compatible with sorting. Note the "-print0" on find and "-0" switch for perl. This is the equivalent of using xargs. Don't you love perl? Note that this pipeline can be easily modified to any data produced by perl's stat operator. eg, you could sort on size, hard links, creation time, etc. Look at stat and just change the '9' to what you want. Changing the '9' to a '7' for example will sort by file size. A '3' sorts by number of links.... Use head and tail at the end of the pipeline to get oldest files or most recent. Use awk or perl -wnla for further processing. Since there is a tab between the two fields, it is very easy to process.

Put public IP address in a variable

list files recursively by size

Scans for open ports using telnet

Efficient count files in directory (no recursion)
$ time perl -e 'if(opendir D,"."){@a=readdir D;print $#a - 1,"\n"}' 205413 real 0m0.497s user 0m0.220s sys 0m0.268s $ time { ls |wc -l; } 205413 real 0m3.776s user 0m3.340s sys 0m0.424s ********* ** EDIT: turns out this perl liner is mostly masturbation. this is slightly faster: $ find . -maxdepth 1 | wc -l sh-3.2$ time { find . -maxdepth 1|wc -l; } 205414 real 0m0.456s user 0m0.116s sys 0m0.328s ** EDIT: now a slightly faster perl version $ perl -e 'if(opendir D,"."){++$c foreach readdir D}print $c-1,"\n"' sh-3.2$ time perl -e 'if(opendir D,"."){++$c foreach readdir D}print $c-1,"\n"' 205414 real 0m0.415s user 0m0.176s sys 0m0.232s

Create a thumbnail from a video file

Monitor memory fine-grained usage (e.g. firefox)
Sometimes top/htop don't give the fine-grained detail on memory usage you might need. Sum up the exact memory types you want

Clear terminal Screen

user 'tr' to convert mixed case in a file to lower case
convert mixed case in a file to lower case

drop first column of output by piping to this


Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: