SED

Introduction to SED

When to use SED Stream Editor

SED 特點

SED Characteristics

Operation Model

Execution Sequence

Drawback

Invoke

Options

Command Format

Line Number Addressing

Context Addressing

Function

Substitute Function

Substitute Function

Input-Output Functions

Multiple Input-Line Functions

Hold and Get Functions

Flow Control Functions

Miscellaneous Functions

Semicolon ';'

Index of sed commands

Squeezing blank lines (like cat -s)

Centering lines

Delete comments from C code

Increment a number

Search make targets

Rename to lower case

chars of lines

lines of files

Transform text into a C 'printf'able string

Prefix non blank lines with their numbers (cat -b)

Prefix lines by their number (cat -n)

Count chars of input (wc -c)

Count lines of input (wc -l)

Count words of input (wc -w)

Print the filename component of a path (basename)

Print directory component of a path (dirname)

Print the first few (=10) lines of input

Print last few (=10) lines of input

The tee(1) command in sed

Print uniq lines of input (uniq)

duplicated lines of input (uniq -d)

Print only and only duplicated lines (uniq -u)

References


Untitled Document
Introduction to SED
SED 是一種非互動式的編輯器
   
在作者的個人經驗中,sed 是 Shell Programming 中最常用到的 Unix 附屬工具軟體。 sed 是一種非互動式的編輯器,常常用在 shell script 中的一個小步驟, 使用者無法親眼看到編輯指令的結果, 完全顛覆了一般人對編輯器的認知。事實上,如果沒有sed 這樣的工具, Unix的威力會打一個大大的折扣。 而sed 的編輯指令與 ed/ex 差不多, 可以視為 ed/ex 的另一版本,專門設計來配合Shell Programming的需求。
 
除了搭配Shell Programming之外,非互動式的編輯器還有幾個不得不用的情景。
 
可以編輯超級巨大的檔案
   
其一是可以編輯超級巨大的檔案。 一般互動式編輯器所能編輯的檔案大小有其上限, 無法編輯超過上限的檔案, 而且編輯的檔案越大時,編輯器 跑得越慢,因為編輯器要不斷的從磁碟機讀取檔案的片段 (亦即太多page fault)。 反之,sed 只需從頭到尾讀取檔案一次即可, 而且檔案可以大到跟作業系統的檔案大小之上限。 換言之,只要是合規的檔案,就能編輯, 而且編輯的速度不受檔案大小之影響。
 
Offline 編輯可降低錯誤
   
第二種情況是,當編輯的指令太複雜, 操作互動式編輯器來編輯檔案時,容易出錯,不如 以offline 方式撰寫 sed 的編輯指令,可以減少錯誤。
 
可重複使用於大批檔案的編輯
   
第三種情況是, 同時要以相同或類似方式編輯一批檔案。例如,台中縣與台中市合併為台中市時, 很多台中縣的文件要修改機關全銜,這時 sed 或ex 等非互動式的編輯器就非常好用了, 使用者只要製作一個編輯指令檔,就可以用來修改一大批檔案,節省無數的工時。
Fri Oct 27 21:34:54 CST 2023 Untitled Document
When to use SED Stream Editor
   
To edit files too large for comfortable interactive editing
   
To edit any size file when the sequence of editing commands is too complicated to be comfortably typed in interactive mode
   
To perform multiple 'globle' editing functions efficiently in one pass through the input
Fri Oct 27 21:34:55 CST 2023 Untitled Document
SED 特點
   
既然 sed 編輯指令跟ed/ex 差不多, 所以只要熟習了任何一種, 就能很快上手其他的。根據作者多年的經驗,超過九成的機會會用到字串代換這個編輯指令, 所以只要學習幾個基本的編輯指令就可以開始運用在Shell Programming 中了,並不難學習。 列的例子,是將一批檔案之副檔名從 foo 改為 bar。
for i in *.foo
do
   mv $i `echo $i | sed -e 's/\.foo$//'`.bar
done
Fri Oct 27 21:34:55 CST 2023 Untitled Document
SED Characteristics
 
Most editing commands are the same as ed/ex
 
Good for in-pipe editing
accept target object from STDIN
does not generate temporary files
good for shell script
for i in *.foo
do
   mv $i `echo $i | sed -e 's/\.foo$//'`.bar
done
Fri Oct 27 21:34:55 CST 2023 Untitled Document
Operation Model

sed Command    Man Page

Fri Oct 27 21:34:55 CST 2023 Untitled Document
Execution Sequence
Scripts are concatenated as they appear, forming a big script.
That script is compiled into a sed program.
That program is then applied to each line of given files (the script itself can change this behavior).
The results are always written to stdout, although same commands can send stuff to specific files
Input files are seen as one to sed, i.e. `sed -n $= *' gives the number of lines of ALL *, something like `cat * | wc -l'
Fri Oct 27 21:34:55 CST 2023 Untitled Document
Drawback
 
Compared with interactive editor:
   
No relative addressing
   
lack of immediate verification that a command has done what was intended
Fri Oct 27 21:34:56 CST 2023 Untitled Document
Invoke
sed "10,20p" inputfile
sed -f sed_script inputfile
cat inputfile | sed "10,20p"
cat inputfile | sed -e "10,20p"
cat inputfile | sed -f sed_script
Fri Oct 27 21:34:56 CST 2023 Untitled Document
Options
 -n   
not to copy all lines, but only those specified by p functions or p flags after s functions
by default, sed writes each line to stdout when it reaches the end of the script (being whatever on the line); this option prevents that. i.e. no output unless there is a command to order SED specifically to do it (like p)
 -e 
take next argument as an editing command
 -f 
take the next argument as a file name; the file should contain editing commands, one to a line
 
Be careful with '-n' option.
   
Without an '-n' option, sed will print every line by default. With it, sed will only print the lines specified by p command.
   
Sed Script   意義
sed "10q" inputfile
head -10 inputfile
Fri Oct 27 21:34:56 CST 2023 Untitled Document
Command Format
Sed Command 格式
[address1,address2][function][arguments]
Pattern Space - The range of pattern matches
   
Default: One line
   
Multiple line pattern space is allowed by using N command
Fri Oct 27 21:34:56 CST 2023 Untitled Document
Line Number Addressing
   
Lines in the input file to which editing commands are to be applied can be selected by address.
位址欄數目 作用的標的
      0
每一行
      1
單一行
      2
一個區塊
 
    30=    
will write "30" if there are at least 30 lines on input, because the `=' command (print current line) will only be executed on line 30
    30,60=    
will write "30", "31"... "60" with the same conditions as above. i.e. input must contain more than or equal to N lines, to the number N to be written
    $=    
will write down the number of the last line, a kind of `wc -l'
Fri Oct 27 21:34:57 CST 2023 Untitled Document
Context Addressing
 
Operation applied to matched lines
   
例:
Sed Script   意義
/.\{73,\}/d
will delete all lines that have more than 72 characters
/^$/d
will delete all empty lines
/^$/,/^$/d    
delete from first empty line seen to the next empty, eating everything appearing in the middle (not very useful)
1,/^$/d
delete leading blank lines, i.e. the first output line will be non empty
Fri Oct 27 21:34:57 CST 2023 Untitled Document
 
Conver letter phone number to digit phone number
# -------------------------------------
# Script: phone v1 : 將英文字母電話號碼轉成數字
# Usage:  phone letter-phone-number
# ------------------------------------- 
echo $1 | sed \ -e 's/[abc]/2/g' -e 's/[def]/3/g' \ -e 's/[ghi]/4/g' -e 's/[jkl]/5/g' \ -e 's/[mno]/6/g' -e 's/[prs]/7/g' \ -e 's/[tuv]/8/g' -e 's/[wxy]/9/g'
   
Use   結果
phone 634laws
6345297
Fri Oct 27 21:34:57 CST 2023 Untitled Document
Function
Whole-Line Oriented Functions
  d  
delete lines
 
  n  
next line
jumps to next line. i.e. pattern space is replaced with the contents of the next line
execution is prosecuted in the command following the `n' command
  i\
<text>
insert lines
works like the append command, but <text> will be inserted before specified line
  c\
<text>
change lines
this will delete current pattern space, and replace it with 'text'
  a\
<text>
append lines
add after the specified line
(if address isn't given, then <text> will be added after EACH line of input that executes this)
   

<text> can have any number of lines,
 
  a\
  1st line\
  2nd\
  .\  last line
  `next command'

suppose that we have
    sed -e '$a\' -e '<the end>'
then a single line containing "the end" is appended to the file.
 
Note 1 : sed doesn't honor leading spaces, so the leading spaces in <text> will be removed
   
To avoid this behavior, a `\' can be placed before the first space that one wants to see written. That way the space is conveniently escaped and will be treated like a normal char.
 
note2: <text> in not processed by the sed program,
   
we insert/change/append raw text directly to output
Fri Oct 27 21:34:57 CST 2023 Untitled Document
Substitute Function
 
字串代換 's'
   
's'字串代換,與 ex 中的代換功能差不多,格式如下:
  s/<pattern>/<replacement>/<flags>
在格式上的差別,就是 's'之前無任何 line number addressing ,那就會 作用在讀進來的每一行資料。而 ex 中若無任何 line number addressing 那所下的指令僅作用在游標所在那一行。
 
在 Shell script 中使用 Sed, 可以將shell 變數用於 sed script 中,很有彈性;例如
  sed "s/$var1/$var2/g"   #將所有的Old 改成 New 
sed 中的字串代換指令所用的分隔符號,可以使用任意字符。
   
習慣上最常用的符號是"/",如果要替換的字串中含有此符號就很不方便,例如下面的例子 要把字串 Dir1/Old 改成字串 Dir2/New:
  sed "s/Dir1\/Old/Dir2\/New/g"
   
如果改用其他符號,如下例,就很方便了:
  sed "s%Dir1/Old%Dir2/New%g"
Multiple Editing Scripts in Sed
cat file |
    sed "s/Old1/New1/g" |
    sed "s/Old2/New2/g" |
    while read LINE
    do
      some commands
    done
.
cat file |
    sed -e "s/Old1/New1/g" \
        -e "s/Old2/New2/g" |
    while read LINE
    do
      some commands
    done
.
the '&' in the replacement part means reusing 'pattern' here
the '/' separator, in fact could be ANY character.
<flags> are optional, and can be combined
  g replace all occurrences of RE by <replacement> (the default is to replace only the first)
  p write the pattern space only if the substitution was successful
  w <file> work as `p' flag, but the pattern space is written to <file>
  d where `d' is a digit, replace the d-th occurrence, if any, of RE by <replacement>

Regular Expression Web Page Fri Oct 27 21:34:57 CST 2023 Untitled Document
Substitute Function
 
字串代換 't'
   
  y/<list1>/<list2>/ 
remaps all characters presents on <list1> by the character with the same index on <list2>
the size of <list1> must be the same as <list2>
all characters are literals. i.e. no ranges, etc...
the separator `/' may be replaced by any other char

 
# -----------------------------------------------
# Script: u2l 將大寫英文字母改成小寫  
# ----------------------------------------------- 
y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/
 
Conver letter phone number to digit phone number
# -------------------------------------
# Script: phone v2 : 將英文字母電話號碼轉成數字
# Usage:  phone letter-phone-number
# ------------------------------------- 
echo $1 | sed \ 'y/abcdefghijklmnoprstuvwxy/222333444555666777888999/'
Fri Oct 27 21:34:58 CST 2023 Untitled Document
Input-Output Functions

Sed Command   意義
  p  
print
  w <filename>  
write on <filename>
  r <filename>  
read the content of <filename>
Fri Oct 27 21:34:58 CST 2023 Untitled Document
Multiple Input-Line Functions
   
Deal with pattern spaces containing imbedded newlines.
They are intended principally to provide pattern matches across lines in the input.
Sed Command   意義
  N  
Next line
The next input line is appended to the current line in the pattern space; the two input lines are separated by an imbedded newline. Pattern matches may extend across the imbedded newline(s).
  D  
Delete first part of the pattern space
Delete up to and including the first newline character in the current pattern space.
If the pattern space becomes empty, read another line from the input. In any case, begin the list of editing commands again from its beginning.
  P  
Print first part of the pattern space
Print up to and including the first newline in the current pattern space.
Fri Oct 27 21:34:58 CST 2023 Untitled Document
Hold and Get Functions
Sed Command   意義
  h  
hold pattern space
Copy the contents of the pattern space into a hold area
destroying the previous contents of the hold area.
  H  
Hold pattern space
Append the contents of the pattern space into a hold area.
The former and new contents are separated by a newline.
  g  
get content of hold area
Copy the contents of the hold area into the pattern space
destroying the previous contents of the pattern space.
  G  
Get content of hold area
Append the contents of the hold area to the contents of the pattern space.
The former and new contents are separated by a newline.
  x  
Exchange
Interchanges the contents of the pattern space and the hold area.
Fri Oct 27 21:34:58 CST 2023 Untitled Document
Flow Control Functions
  !  
Don't;
negate address specification of next command
sed '!s/foo/bar/' is doing nothing
sed '/./!d' delete all empty lines
  {  
Grouping
Same as C
groups a set of commands that are executed on the specified lines
the closing `}' must appear on one line by itself
`{...}' can be nested
  :<label>  
Place a label
trailing space is sensitive
  b<label>  
branch to a label
If a label is not given, the default is to jump to the end of the script.
  t<label>  
Test substitutions
works like `b', but the jump is only done if a previous substitution has been successfully done (on current pattern space) - the flag that determines if the jump is given on not is: - set on a successful substitution (whatever it was) and reset - reset after `t' been executed - reset after reading a new line
Fri Oct 27 21:34:59 CST 2023 Untitled Document
Miscellaneous Functions
  =  
Write to the STDOUT the line number of the line matched by its address
  q  
Write the current line and terminate.
Fri Oct 27 21:34:59 CST 2023 Untitled Document
Semicolon ';'
 
Commands may be separated by semi-colons `;', with some exceptions.
 
例:
# -----------------------------------------------
# Script sed-semicolon
# ----------------------------------------------- 
sed '/^#/d;/^$/d;:b;/\\$/{;N;s/\n//;bb;}'
is equivalent to
/^#/d   #delete all lines beginned with `#' 
/^$/d   #delete all empty lines (/./!d could be used instead) 
:b
/\\$/{
N
s/\n//
bb
}
# -----------------------------------------------
would join all lines ended with `\', 
after deleting the `\' it self 
Fri Oct 27 21:34:59 CST 2023 Untitled Document
Index of sed commands
Operand
數量
Func
tion
  意義
(2)
!
!<cmd> Don't apply to specified addresses
(0)
#
comment
(0)
:
:<label> place a label
(1)
=
display line number
(2)
D
delete first part of the pattern space
(2)
G
append contents of hold area
(2)
H
append pattern space on buffer
(2)
N
append next line
(2)
P
print first part of the pattern space
(1)
a
append text
(2)
b
b<label> branch to label
(2)
c
change lines
(2)
d
delete lines
(2)
g
get contents of hold area
(2)
h
hold pattern space
(1)
i
insert lines
(2)
l
list lines
(2)
n
next line
(2)
p
print
(1)
q
quit
(1)
r
r <file> read the contents of <file>
(2)
t
t<label> test substitutions and branch on successful substitution
(2)
w
w <file> write to <file>
(2)
x
exchange buffer space with pattern space
(2)
{
group commands
(2)
s
s/OLD/<New>/[flags] substitute
(2)
y
y/<list1>/<list2>/ translates <list1> into <list2>
Fri Oct 27 21:34:59 CST 2023 Untitled Document
Script ID     Script
sed-eg-1
sed -e 'd' /etc/services
sed-eg-2
sed -e '1d' /etc/services
sed-eg-3
sed -e '1,10d' /etc/services
sed-eg-4
sed -e '/^#/d' /etc/services
sed-eg-5
sed -e '/regexp/d' /path/to/my/test/file
sed-eg-6
sed -n -e '/BEGIN/,/END/p' /my/test/file
sed-eg-7
sed -n -e '/main[[:space:]]*(/,/^}/p' sourcefile.c
# ---------------------------------------------------------
# to print out only the main() function in a C source file
Fri Oct 27 21:34:59 CST 2023 Untitled Document
Squeezing blank lines (like cat -s)
 
Leaves a blank line at the beginning and end, if there are there some already.
 #
# on empty lines, join with next 
# --------------------------------------------------------- 
:x
/^\n*$/{
N
bx
}
   # now, squeeze all '\n', this can be also done by: s/^\(\n\)*/\1/ 
s/\n*/\
/
   #leaves only at end 

#!/usr/bin/sed -f #delete all leading empty lines 1,/^./{ /./!d }
# find an empty line, keep it, and remove all following empty lines
:x /./!{ N s/^\n$// tx }
#Squeeze all, and remove all leading and trailing blank lines. This is also the fastest.
#!/usr/bin/sed -nf # delete all blanks /./!d # get here: so there is a non empty :x # print it p # get next n # got chars? print it again, etc... /./bx # no, don't have chars: another empty line :z # get next n # also empty? then ignore it, and get next... this will remove ALL empty # lines, if we get to end, sed script will finish on n(ext) command # so no trailing empty lines are written /./!bz # all empty lines were deleted/ignored, but we have a non empty, as # what we want to do is to squeeze, insert a blank line artificially i\ bx
Fri Oct 27 21:35:00 CST 2023 Untitled Document
Centering lines
   #!/usr/bin/sed -f 
   # center all lines of a file, on a 80 columns width 
   # 
   # to change that width, the number in \{\} must be replaced, and the number 
   # of added spaces also must be changed 
   # 
   # del leading and trailing spaces 
y/      / /
s/^ *//
s/ *$//
   # add 80 spaces to end of line 
s/$/          /
s/ *$/&&&&&&&&/
   # keep 1st 80 chars 
s/^\(.\{80\}\).*$/\1/
   # split trailing spaces, into two halves, 1st for beg, 2nd to end of line 
s/\( *\)$/#\1%\1/
s/^\(.*\)#\(.*\)%\(.*\)$/\2\1\3/
Fri Oct 27 21:35:00 CST 2023 Untitled Document
Delete comments from C code
   #!/usr/bin/sed -f 
   # if no /* get next 
/\/\*/!b
   # here we've got an /*, append lines until get the corresponding 
   # */ 
:x
/\*\//!{
N
bx
}
   # delete /*...*/ 
s/\/\*.*\*\///
Fri Oct 27 21:35:00 CST 2023 Untitled Document
Increment a number
   #!/usr/bin/sed -f 
   # algorithm by : 
   # Bruno <Haible@ma2s2.mathematik.uni-karlsruhe.de> 
   # incrementing one number, is just add 1 to first digit, i.e. replacing 
   # it by the following digit 
   # 
   # there is one exception, when carry does happen, on that case, all 
   # following digits must be added with one 
   # 
   # now this solution by `Bruno <Haible@ma2s2.mathematik.uni-karlsruhe.de>' 
   # is very clever and smart 
   # 
   # the only way to happen carry, is when the first digit is a 9 
   # all others cases are just fine 
   # 
   # for a number beginning with any digit except 9, just replace it (the digit) 
   # by the next digit, for each number beginning with a 9, just "remove" it and 
   # proceed as above for all others, i.e. all leadings 9s are "removes" until 
   # a non-9 is found, if any 9 did not remain, a 0 is insert 
   # replace all leading 9s by _ (any other char except digits, could be used) 
   # 
:d
s/9\(_*\)$/_\1/
td
   # if there aren't any digits left, add a MostSign Digit 0 
   # 
s/^\(_*\)$/0\1/
   # incr last digit only - there is no need for more 
   # 
s/8\(_*\)$/9\1/
s/7\(_*\)$/8\1/
s/6\(_*\)$/7\1/
s/5\(_*\)$/6\1/
s/4\(_*\)$/5\1/
s/3\(_*\)$/4\1/
s/2\(_*\)$/3\1/
s/1\(_*\)$/2\1/
s/0\(_*\)$/1\1/
   # replace all _ to 0s 
   # 
s/_/0/g
Fri Oct 27 21:35:00 CST 2023 Untitled Document
Search make targets
   #!/usr/bin/sed -nf 
   # make-targets 
   # 
   # tries to catch all targets on a Makefile 
   # 
   # the purpose of this is to be used on the complete [make] feature 
   # of tcsh... so it should be simple and fast 
   # 
   # this is not a shell script, exactly for that reason... hopefully 
   # the kernel will interpret this executable as a sed script and 
   # feed it directly to it 
   # 
   # the name of the makefile, unfortunelly, must be hard coded on the 
   # complete code, and it is "Makefile" 
   # take care of \ ended lines 
:n
/\\$/{
        N
        s/\\\n//
        bn
}
y/      / /
   # delete all comments 
/^ *#/d
s/[^\\]#.*$//
   # register variables, the only ones in here are the ones of form 
   # 
   #       VAR = one_word_def 
   # 
   # in that way, most vars will be skipped, and things like 
   # 
   #       SED_TARGET = sed 
   # 
   # will still work 
   # 
/\([A-Za-z_0-9-]\+\) *= *\([A-Za-z_0-9./-]\+\) *$/{
        s/ //g
        s/$/ /
        H
        b
}
   # now, perform the substitution 
/\$[({][A-Za-z_0-9-]\+[)}]/{
        s/$/##/
        G
        s/\(\$[{(]\)\([A-Za-z_0-9-]\+\)\([)}]\)\(.*\)##.*\2=\([A-Za-z_0-9./-]\+\).*/\5\4/g
}
   # finally, print the targets 
tt
:t
s/^\([A-Za-z_0-9./-]\+\)\(\( \+[A-Za-z_0-9./-]\+\)*\) *:\([^=].*\)\?$/\1\2/
tx
d
   # now, this a final selection of targets to be print 
   # kind of 'prog | grep -v ...' 
   # don't print *.[hco] targets cause in most cases that makes very long output 
:x
/\.[och]$/!p
Fri Oct 27 21:35:01 CST 2023 Untitled Document
Rename to lower case
   #!/bin/sh - 
   # rename files to lower/upper case... 
   # 
   # usage: 
   #       move-to-lower * 
   #       move-to-upper * 
   # or 
   #       move-to-lower -r . 
   #       move-to-upper -r . 
   # 
help()
{
        cat << eof
Usage: $0 [-n] [-r] [-h] files...
-n      do nothing, only see what would be done
-r      recursive (use find)
-h      this message
files   files to remap to lower case
例
       $0 -n *              (see if everything is ok, then...)
       $0 *
       $0 -r
eof } apply_cmd='sh' finder='echo $* | tr " " "\n"' files_only= while : do case "$1" in -n) apply_cmd='cat' ;; -r) finder='find $* -type f';; -h) help ; exit 1 ;; *) break ;; esac shift done [ "$1" ] || { echo Usage: $0 [-n] [-r] files... exit 1 } LOWER='abcdefghijklmnopqrstuvwxyz' UPPER='ABCDEFGHIJKLMNOPQRSTUVWXYZ' case `basename $0` in *to-lower*) FROM=$UPPER; TO=$LOWER ;; *to-upper*) TO=$UPPER; FROM=$LOWER ;; *lower*) FROM=$UPPER; TO=$LOWER ;; *upper*) TO=$UPPER; FROM=$LOWER ;; *) FROM=$UPPER; TO=$LOWER ;; esac eval $finder | sed -n ' # remove all trailing /s s/\/*$// # add ./ if there are no path, only filename /\//!s/^/.\// # save path+filename h # remove path s/.*\/// # do conversion only on filename y/'$FROM'/'$TO'/ # swap, now line contains original path+file, hold space contains conv filename x # add converted file name to line, which now contains something like # path/file-name\nconverted-file-name G # check if converted file name is equal to original file name, if it is, do # not print nothing /^.*\/\(.*\)\n\1/b # now, transform path/fromfile\ntofile, into mv path/fromfile path/tofile # and print it s/^\(.*\/\)\(.*\)\n\(.*\)$/mv \1\2 \1\3/p ' | $apply_cmd
Fri Oct 27 21:35:01 CST 2023 Untitled Document
chars of lines
 
reverse all chars of each line, keep line ordering
   #!/usr/bin/sed -f 
   # ignore empty lines, i.e. nothing to reverse 
/./!b
   # escape ! by doubling it, place markers at beginning and end of line 
   # the markers are -!- which can never happen after the escaping of ! 
s/!/!!/g
s/^/-!-/
s/$/-!-/
   # swaps first char after first maker, with first char before last marker 
   # and then advance the markers through the swapped chars 
ta
:a
s/-!-\([^!]\|!!\)\(.*\)\([^!]\|!!\)-!-/\3-!-\2-!-\1/
ta
   # delete markers, and then unescape the !s 
s/-!-//g
s/!!/!/g
Fri Oct 27 21:35:01 CST 2023 Untitled Document
lines of files
   #!/usr/bin/sed -nf 
   # reverse all lines of input, i.e. first line became last, ... 
   # first line is pasted into buffer 
1{h;b;}
   # for all other lines, the buffer (which contains all previous) 
   # is appended to current line, so, the order is being reversed 
   # on the buffer, after that is done, store everything on the buffer 
   # again 
G;h
   # the last line (after have done the above job) get the contents 
   # of buffer, and print it 
${g;p;}
Fri Oct 27 21:35:01 CST 2023 Untitled Document
Transform text into a C 'printf'able string
   #!/usr/bin/sed -f 
   # The purpose of this script is to construct C programs like this 
   # 
       printf("\
   # common text 
   # ... 
   # 
   # 
   # ... 
   # last line of text 
   # 
   # and then pipe trought this filter the portion between printf and the last 
   # line of text, and get a valid C statement 
   # 
   # That's why, " is placed on last line, and not in first, for eg 
   # escape all special chars " and \ inside a string... 
s/["\\]/\\&/g
   # adds a \n\ to the end of each line, except the last, which gets \n" 
s/$/\\n/
$!s/$/\\/
$s/$/"/
Fri Oct 27 21:35:02 CST 2023 Untitled Document
Prefix non blank lines with their numbers (cat -b)
   #!/usr/bin/sed -nf 
   # copy all lines of input, prefixing only non blank lines by its number, 
   # kind of `cat -b' 
   # init counter 
1{
        x
        s/^/0/
        x
}
   # for blanks, don't incr count, but print 
/./!{
        p
        b
}
   # for the rest is the same as a `cat -n' 
x
:d
s/9\(_*\)$/_\1/
td
s/^\(_*\)$/0\1/
s/8\(_*\)$/9\1/
s/7\(_*\)$/8\1/
s/6\(_*\)$/7\1/
s/5\(_*\)$/6\1/
s/4\(_*\)$/5\1/
s/3\(_*\)$/4\1/
s/2\(_*\)$/3\1/
s/1\(_*\)$/2\1/
s/0\(_*\)$/1\1/
s/_/0/g
s/^/      /
s/^.*\(......\)/\1/
G
s/\n/  /p
s/^ *//
s/  .*//
h
Fri Oct 27 21:35:02 CST 2023 Untitled Document
Prefix lines by their number (cat -n)
   #!/usr/bin/sed -nf 
   # copy all lines of input, prefixed by its number, kind 
   # of `cat -n' 
   # switch to buffer 
x
   # init the counting 
1{
        s/^/0/
}
   # increment the count: first line == number 1 
:d
s/9\(_*\)$/_\1/
td
s/^\(_*\)$/0\1/
s/8\(_*\)$/9\1/
s/7\(_*\)$/8\1/
s/6\(_*\)$/7\1/
s/5\(_*\)$/6\1/
s/4\(_*\)$/5\1/
s/3\(_*\)$/4\1/
s/2\(_*\)$/3\1/
s/1\(_*\)$/2\1/
s/0\(_*\)$/1\1/
s/_/0/g
   # format the number like printf's `"%6d"' 
s/^/      /
s/^.*\(......\)/\1/
   # append the line to the number, and write: "<number>  <line>" 
   # note: this is the format of gnu-cat 
G
s/\n/  /p
   # after printing the line, transform the line into the number, and 
   # store it on buffer again 
s/^ *//
s/  .*//
h
Fri Oct 27 21:35:02 CST 2023 Untitled Document
Count chars of input (wc -c)
   #!/usr/bin/sed -nf 
   # count all chars of input, kind of `wc -c' 
   # the buffer hold the count 
x
1{
        s/^/0/
}
   # we have a line, so at least there is one char: the `\n' 
tx
:x
s/9\(_*\)$/_\1/
tx
s/^\(_*\)$/0\1/
s/ \(_*\)$/0\1/
s/8\(_*\)$/9\1/
s/7\(_*\)$/8\1/
s/6\(_*\)$/7\1/
s/5\(_*\)$/6\1/
s/4\(_*\)$/5\1/
s/3\(_*\)$/4\1/
s/2\(_*\)$/3\1/
s/1\(_*\)$/2\1/
s/0\(_*\)$/1\1/
s/_/0/g
   # get back to the line 
x
   # for each char in the line, increment the count 
tc
:c
s/.//
x
tx
   # on last line, all is done, so print the count, and quit 
${p;q;}
   # put current line (which has been swapped with the count) to the buffer 
h
Fri Oct 27 21:35:02 CST 2023 Untitled Document
Count lines of input (wc -l)
   #!/usr/bin/sed -nf 
   # count lines of input, kind of `wc -l' 
$=
Fri Oct 27 21:35:03 CST 2023 Untitled Document
Count words of input (wc -w)
   #!/usr/bin/sed -nf 
   # count all words on input 
   # words are separated by tabs, newlines and spaces 
   # the buffer hold the count 
1{;x;s/^/0/;x;}
s/^[     ]*/\
/
ts
:t
s/^/w/
ts
:s
s/^\(.*\n\)[^   ]\+[    ]*/\1/
tt
s/\n.*$//
   # the above, replaced all words by `w', and delete everything else 
   # except newlines, so, now the job to do, is only of counting chars 
   # 
   # from this on, this is the same os count-chars, by first we must 
   # delete one char (to keep up with the extra newline) 
/./!{;${;g;p;q;};d;}
s/.//
x
   # we have a line, so at least there is one char: the `\n' 
tx
:x
s/9\(_*\)$/_\1/
tx
s/^\(_*\)$/0\1/
s/ \(_*\)$/0\1/
s/8\(_*\)$/9\1/
s/7\(_*\)$/8\1/
s/6\(_*\)$/7\1/
s/5\(_*\)$/6\1/
s/4\(_*\)$/5\1/
s/3\(_*\)$/4\1/
s/2\(_*\)$/3\1/
s/1\(_*\)$/2\1/
s/0\(_*\)$/1\1/
s/_/0/g
   # get back to the line 
x
   # for each char in the line, increment the count 
tc
:c
s/.//
   # put count on line 
x
tx
   # update buffer with count 
h
   # on last line, all is done, so print the count 
$p
Fri Oct 27 21:35:03 CST 2023 Untitled Document
Print the filename component of a path (basename)
   #!/usr/bin/sed -f 
   # usage: fbasename file 
   # or 
   # usage: find path -print | fbasename 
   # 
   # 
   # this is a basename, but read filenames from stdin, each line 
   # contains the path and a possible suffix 
   # 
   # this will produce one output line per input line, with 
   # the filename component of path, with the (possible) suffix 
   # removed 
s/^[    ]*//
s/[     ]*$//
tc
:c
s/[     ][      ]*/\
/
ta
s/\/*$//
s/.*\///
b
:a
h
s/.*\n//
x
s/\n.*//
s/\/*$//
s/.*\///
tb
:b
G
s/^\(.*\)\(.*\)\n\2$/\1/
t
P
d
Fri Oct 27 21:35:03 CST 2023 Untitled Document
Print directory component of a path (dirname)
   #!/usr/bin/sed -f 
   # usage: find path -print | fdirname 
   # 
   # fdirname acts like dirname, but read files from stdin 
   # print the directory component of a path 
   # special case: `/' is given 
/^\/$/c\
/
   # strip trailing `/'s if any 
s/\/*$//
   # strip trailing filename 
s/[^/]*$//
   # if get no chars after these, then we have current dir (things like 
   # `bin/ src/' were given 
/./!c\
  • # delete the trailing `/' # ("/usr/bin/ls" --> "/usr/bin/", this makes "/usr/bin") s/\/$//
  • Fri Oct 27 21:35:03 CST 2023 Untitled Document
    Print the first few (=10) lines of input
       #!/usr/bin/sed -f 
       # display first 10 lines of input 
       # the number of displayed lines can be changed, by changing the number 
       # before the `q' command to `n' where `n' is the number of lines wanted 
    10q
    
    Fri Oct 27 21:35:04 CST 2023 Untitled Document
    Print last few (=10) lines of input
       #!/usr/bin/sed -f 
       # this is a tail command, it displays last 10 lines of input 
       # if there are 10 or more, if less than that, displays all 
       # to change number of displayed lines, the "$b;N" number of 
       # statements after the "1{" must be changed to `n-2', where `n' 
       # is the number of pretended lines, e.g. if want 10 lines, 
       # should have 8 `$b;N' 
       # to do that with vi, just goto the first `$b,N' and do `d/^}/-2 dd 8p' 
    1{
            $b;N
            $b;N
            $b;N
            $b;N
            $b;N
            $b;N
            $b;N
            $b;N
    }
    $b;N
    $!D
    
    Fri Oct 27 21:35:04 CST 2023 Untitled Document
    The tee(1) command in sed
       #!/bin/sh - 
       # emulation of tee using sed, and a sh(1) for cycle 
    cmd=
    for i
    do
            cmd="$cmd -e 'w $i'"
    done
    eval sed $cmd
    
    Fri Oct 27 21:35:04 CST 2023 Untitled Document
    Print uniq lines of input (uniq)
       #!/usr/bin/sed -f 
       # print all uniq lines on a sorted input-- only one copy of a duplicated 
       # line is printed 
       # like `uniq' 
    :b
    $b
    N
    /^\(.*\)\n\1$/{
            s/.*\n//
            bb
    }
    $b
    P
    D
    
    Fri Oct 27 21:35:04 CST 2023 Untitled Document
    duplicated lines of input (uniq -d)
       #!/usr/bin/sed -nf 
       # print all duplicated uniq lines on a sorted input 
       # like `uniq -d' 
    $b
    N
    /^\(.*\)\n\1$/{
            s/.*\n//
            p
            :b
            $b
            N
            /^\(.*\)\n\1$/{
                    s/.*\n//
                    bb
            }
    }
    $b
    D
    
    Fri Oct 27 21:35:05 CST 2023 Untitled Document
    Print only and only duplicated lines (uniq -u)
       #!/usr/bin/sed -f 
       # print all uniq lines on a sorted input-- no copies of duplicated 
       # lines are printed 
       # like `uniq' 
    $b
    N
    /^\(.*\)\n\1$/!{
            P
            D
    }
    :c
    $d
    s/.*\n//
    N
    /^\(.*\)\n\1$/{
            bc
    }
    D
    
    Fri Oct 27 21:35:05 CST 2023 Untitled Document
    References
    Reference Book sed & awk, by Dougherty, Dale and Robbins, Arnold from Oreilly & Associates, ISBN 1-56592-225-5
    GNU sed ftp://ftp.gnu.org/gnu/sed/.
    An excellent source http://sed.sourceforge.net/.
    FAQ http://www.dreamwvr.com/sed-info/sed-faq.html.
    FAQ ftp://rtfm.mit.edu/pub/faqs/editor-faq/sed.
    Fri Oct 27 21:35:05 CST 2023