Filter

General Form

Sed

Summary of Filter

Replace String

Remove String

Remove String From Front

Remove String At the End

Insert String in Front

Append String to End

Truncate String

Extract String by Columns

convert upper case letters to lower case, and vise versa

Change Tabs to Spaces

Change Multiple Spaces to a Single Space

Change Whitespaces to a Single Space

Delete Leading Whitespaces

Delete Trailing Whitespaces

Delete Lines

Delete Empty Lines

Delete Lines

Print First Line

Print Last Line

Delete Comments

Delete Lines Between Keywords

Extract Lines Between Keywords

Extract Odd/Even Lines

Read a File Backwards

Convert < > into HTML code

Convert lower case letters to upper case

Convert upper case letters to lower case

Untitled Document
General Form
   
軟體程式在解決一個複雜的計算問題時， 通常會將一個問題分解成數個小問題分別解決，再合併成一個 完整的程式。 Shell programming 最常見的一種分工方式是使用一連串的 過濾器串接成為一種流水線的分工方式。 任何程式只要是從 STDIN 讀進資料而從 STDOUT 輸出結果就可作為過濾器。過濾器之間則使用管線 (Pipe)串接。 例如:下面的Script就是一個由三節過濾器組成的程式。 
  cat myfile | sed "s/Old/New/g" | sort
流水線分工方式的一般型態如下: 
 
cat file |
    filter_1 |
    filter_2 |
    filter_3 |
    while read var1  var2 var3 
    do
      some commands on variable $var1 $var2 $var3
    done
   
最前面的 cat 負責讀取輸入的檔案並傳到後方的過濾器， 最後可以接一個迴圈將從過濾器輸出的資料逐一處理。 當然，過濾器的數量是有彈性的，最前端與最後端的兩個步驟也都是 可有可無的。
Thu Nov 2 10:14:14 CST 2023 Untitled Document
Sed
Sed
 
sed 是最常見的製造過濾器的工具, 例如： 
 sed "s/Old/New/g" myfile   #將myfile 中所有的Old 改成 New，輸出於螢幕
 
上面的 script 可寫成過濾器的形式: 
  cat myfile | sed "s/Old/New/g"
使用數個過濾器形成一個流水線工作形式: 
#------------------------------------------
# txt2htm:  將所有的 *.txt 檔的副檔名改成 htm
#------------------------------------------

ls *.txt  | sed 's/.txt//' |
    while read name
    do
      mv $name.txt $name.htm
    done
Thu Nov 2 10:14:14 CST 2023 Untitled Document
Summary of Filter
Script ID Script 說明
filter-1
# replace text
# 將所有的 Old 代換成 New
#-------------------------------------------------------------
sed "s/Old/New/g
filter-2
# remove "XXX"
# 將所有的 "XXX" 刪除
#-------------------------------------------------------------
sed "s/XXX//g"
filter-3
# remove "XXX" from front
# 將每一行開頭的 "XXX" 刪除
#-------------------------------------------------------------
sed "s/^XXX//"
filter-4
# remove "XXX" at the end
# 將每一行結尾的 "XXX" 刪除
#-------------------------------------------------------------
sed "s/XXX\$//"
filter-5
# remove "XXX" at the end
# 將每一行結尾的 "XXX" 刪除
#-------------------------------------------------------------
sed 's/XXX$//'
filter-6
# insert "XXX" in front
# 在每一行前面插入 "XXX"
#-------------------------------------------------------------
sed "s/^/XXX/"
filter-7
# append "XXX" to the end
# 在每一行後面接上 "XXX"
#-------------------------------------------------------------
sed "s/\$/XXX/g"
filter-8
# append "XXX" to end
# 在每一行後面接上 "XXX"
#-------------------------------------------------------------
sed 's/$/XXX/g'
filter-9
# truncate text from "XXX" to the end
# 將每一行中的 XXX 字串及之後所有文字全部刪除
#-------------------------------------------------------------
sed "s/XXX.*//"
filter-10
# extract text by columns
# 將每一行中第5至8的字元抽取出來，其餘全刪掉
#-------------------------------------------------------------
cut -c5-8
filter-11
# convert upper case letters to lower case
# 將所有英文字母變成小寫
#-------------------------------------------------------------
tr '[A-Z]' '[a-z]'
filter-12
# convert lower case letters to upper case
# 將所有英文字母變成大寫
#-------------------------------------------------------------
tr '[a-z]' '[A-Z]'
filter-13
# change tabs to spaces
# 將每一個"跳格"取代成為"空格"
#-------------------------------------------------------------
sed "s/<tab>/ /g"
filter-14
# change multiple spaces to a single space
# 將多個連續的"跳格"取代成為一個"空格"
#-------------------------------------------------------------
sed "s/  */ /g"
filter-15
# change whitespaces to a single space
# 將多個連續的"跳格"或"空格"取代成為一個"空格"
#-----------------------------------------------------------------------------------------------------
sed "s/[<tab><space>][<tab><space>]*/<space>/g"
filter-16
# delete leading whitespaces
# 刪除每一行最前端的多個連續"跳格"或"空格"
#-------------------------------------------------------------
sed "s/^[<tab><space>]*//g"
filter-17
# delete trailing whitespaces
# 刪除每一行最末端的多個連續"跳格"或"空格"
#-------------------------------------------------------------
sed 's/[<tab><space>]*$//'
filter-18
# delete the lines that contain "XXX"
# 將含XXX 字串的行整行全部刪除
#-------------------------------------------------------------
sed "/XXX/d"
filter-19
# delete the lines that contain "XXX"
# 將含有XXX 字串的行整行全部刪除
#-------------------------------------------------------------
grep -v "XXX"
filter-20
# delete empty lines
# 刪除空行
#-------------------------------------------------------------
sed '/^$/d'
filter-21
# delete empty lines
# 刪除空行，行內若只含空格，視為空行
#-------------------------------------------------------------
sed '/^[<tab><space>]*$/d'
filter-22
# delete first line
# 刪除第一行
#-------------------------------------------------------------
sed '1d'
filter-23
# delete first seven lines
# 刪除前七行
#-------------------------------------------------------------
sed '1,7d'
filter-24
# delete last line
# 刪除最後一行
#-------------------------------------------------------------
sed '$d'
filter-25
# print first line
# 印出第一行
#-------------------------------------------------------------
sed -n '1p'
filter-26
# print first line
# 印出第一行
#-------------------------------------------------------------
sed '2,$d'
filter-27
# print first line
# 刪除第一行
#-------------------------------------------------------------
sed '1q'
filter-28
# print last line
# 印出最後一行
#-------------------------------------------------------------
sed -n '$p'
filter-29
# delete comments
# 刪除最前端是"#"的行
#-------------------------------------------------------------
sed '/^#/d'
filter-30
# delete comments
# 刪除最前端是"#"的行
#-------------------------------------------------------------
sed 's/#.*//'
filter-31
# delete text between lines that contain specific keywords
# 從含有字串"Keyword1"的行到含有字串"Keyword2"的行之間全部刪除
#-------------------------------------------------------------------------------------------
sed '/Keyword1/,/Keyword2/d'
filter-32
# delete text between lines that contain specific keywords
# 從只含有字串"Keyword1"的行到只含有字串"Keyword2"的行之間全部刪除
#-----------------------------------------------------------------------------------------------------
sed '/^Keyword1$/,/^Keyword2$/d'
filter-33
# extract text between keywords
# 從只含有字串"Keyword1"的行到只含有字串"Keyword2"的行之間全部印出
#-----------------------------------------------------------------------------------------------------
sed '/^Keyword1$/,/^Keyword2$/!d'
filter-34
# print odd lines
# 印出奇數行
#-------------------------------------------------------------------------------------------
cat -n file | sed -e '/^.....[02468]/d' -e 's/^.......//'
filter-35
# print even lines
# 印出偶數行
#-------------------------------------------------------------------------------------------
cat -n file | sed -e '/^.....[13579]/d' -e 's/^.......//'
filter-36
# read a file backwards
# 倒捲一個檔案
#-------------------------------------------------------------
cat -n file | sort -nr | sed 's/^.......//'
filter-37
# convert < > into HTML code
#-------------------------------------------------------------
cat $* | sed -e 's/</\&lt;/g' -e 's/>/\&gt;/g'
Thu Nov 2 10:14:14 CST 2023 Untitled Document
Replace String
 
Replace text

將所有的 Old 代換成 New
  sed "s/Old/New/g"
Thu Nov 2 10:14:15 CST 2023 Untitled Document
Remove String
 
Remove "XXX"

將所有的 "XXX" 刪除
  sed "s/XXX//g"
Thu Nov 2 10:14:15 CST 2023 Untitled Document
Remove String From Front
 
Remove "XXX" from front

將每一行開頭的 "XXX" 刪除
  sed "s/^XXX//"
Thu Nov 2 10:14:15 CST 2023 Untitled Document
Remove String At the End
 
Remove "XXX" at the end

將每一行結尾的 "XXX" 刪除
  sed "s/XXX\$//"

  sed 's/XXX$//'
Thu Nov 2 10:14:15 CST 2023 Untitled Document
Insert String in Front
 
Insert "XXX" in front

在每一行前面插入 "XXX" 
  sed "s/^/XXX/"
Thu Nov 2 10:14:15 CST 2023 Untitled Document
Append String to End
 
Append "XXX" to the end

在每一行後面接上 "XXX" 
  sed 's/$/XXX/'

  sed "s/\$/XXX/"
Thu Nov 2 10:14:16 CST 2023 Untitled Document
Truncate String
 
Truncate text from "XXX" to the end

將每一行中的 XXX 字串及之後所有文字全部刪除 
  sed "s/XXX.*//"
Thu Nov 2 10:14:16 CST 2023 Untitled Document
Extract String by Columns
 
Extract text by columns

將每一行中第5至8的字元抽取出來，其餘全刪掉 
  cut -c5-8
Thu Nov 2 10:14:16 CST 2023 Untitled Document
convert upper case letters to lower case, and vise versa
 
Convert upper case letters to lower case

將所有英文字母變成小寫
  tr '[A-Z]' '[a-z]'
 
Convert lower case letters to upper case
將所有英文字母變成大寫
  tr '[a-z]' '[A-Z]'
Thu Nov 2 10:14:16 CST 2023 Untitled Document
Change Tabs to Spaces
 
Change tabs to spaces
將每一個"跳格"取代成為"空格"
  sed "s/<tab>/ /g"
Thu Nov 2 10:14:16 CST 2023 Untitled Document
Change Multiple Spaces to a Single Space
 
Change multiple spaces to a single space
將多個連續的"跳格"取代成為一個"空格" 
  sed "s/  */ /g"
Thu Nov 2 10:14:16 CST 2023 Untitled Document
Change Whitespaces to a Single Space
 
Change whitespaces to a single space 將多個連續的"跳格"或"空格"取代成為一個"空格" 
  sed "s/[<tab><space>][<tab><space>]*/<space>/g"
Thu Nov 2 10:14:17 CST 2023 Untitled Document
Delete Leading Whitespaces
 
刪除每一行最前端的多個連續"跳格"或"空格" 
  sed "s/^[<tab><space>]*//g"
Thu Nov 2 10:14:17 CST 2023 Untitled Document
Delete Trailing Whitespaces
 
Delete trailing whitespaces
刪除每一行最末端的多個連續"跳格"或"空格"
  sed 's/[<tab><space>]*$//'
Thu Nov 2 10:14:17 CST 2023 Untitled Document
Delete Lines
 
Delete the lines that contain "XXX"
將含有XXX 字串的行整行全部刪除 
  sed "/XXX/d" 

  grep -v "XXX"
Thu Nov 2 10:14:17 CST 2023 Untitled Document
Delete Empty Lines
 
Delete empty lines
刪除空行
  sed '/^$/d'
Delete empty lines
刪除空行，行內若只含空格，視為空行
  sed '/^[<tab><space>]*$/d'
Thu Nov 2 10:14:17 CST 2023 Untitled Document
Delete Lines
 
刪除第一行 
  sed '1d'
刪除前七行 
  sed '1,7d'
刪除最後一行 
  sed '$d'
Thu Nov 2 10:14:18 CST 2023 Untitled Document
Print First Line
 
印出第一行 
  sed -n '1p'

  sed '2,$d'

  sed '1q'
Thu Nov 2 10:14:18 CST 2023 Untitled Document
Print Last Line
 
印出最後一行 
  sed -n '$p'
Thu Nov 2 10:14:18 CST 2023 Untitled Document
Delete Comments
 
刪除最前端是"#"的行 
  sed '/^#/d'

  sed 's/#.*//'
Thu Nov 2 10:14:18 CST 2023 Untitled Document
Delete Lines Between Keywords
 
從含有字串"Keyword1"的行到含有字串"Keyword2"的行之間全部刪除 
  sed '/Keyword1/,/Keyword2/d'
從 只 含有字串"Keyword1"的行到 只 含有字串"Keyword2"的行之間全部刪除 
  sed '/^Keyword1$/,/^Keyword2$/d'
Thu Nov 2 10:14:18 CST 2023 Untitled Document
Extract Lines Between Keywords
 
從 只 含有字串"Keyword1"的行到 只 含有字串"Keyword2"的行之間全部印出 
  sed '/^Keyword1$/,/^Keyword2$/!d'
Thu Nov 2 10:14:19 CST 2023 Untitled Document
Extract Odd/Even Lines
印出偶數行 (Print Even Lines)
 
當檔案中每一行只有一欄時，可簡單的script 
aaaaaa
bbbbbb
cccccc
dddddd
eeeeee
ffffff

# -----------------------------------------------
# Script: even
# ----------------------------------------------- 

  
cat $1 | while read odd
do
   read even
   echo $even
done
 
結果 
bbbbbb
dddddd
ffffff
 
當檔案中每一行有不止一欄時，無法用 上面簡單的 script 
aaaaaa aaaaaa aaaaaa
bbbbbb bbbbbb 
cccccc cccccc cccccc 
dddddd 
eeeeee eeeeee 
ffffff 

# -----------------------------------------------
# Script: even2
# ----------------------------------------------- 

  cat -n file |  sed -e '/^.....[13579]/d' -e 's/^.......//'
印出奇數行 (Print Odd Lines) 
# -----------------------------------------------
# Script: odd2
# ----------------------------------------------- 

  
cat $1 | while read odd
do
   read even
   echo $odd
done

# -----------------------------------------------
# Script: odd
# ----------------------------------------------- 

  cat -n file |  sed -e '/^.....[02468]/d' -e 's/^.......//'
Thu Nov 2 10:14:19 CST 2023 Untitled Document
Read a File Backwards 
  cat -n file | sort -nr | sed 's/^.......//'
Thu Nov 2 10:14:19 CST 2023 Untitled Document
Convert < > into HTML code 
 
  cat $* | sed -e 's/</\&lt;/g' -e 's/>/\&gt;/g'
Thu Nov 2 10:14:19 CST 2023 Untitled Document
Convert lower case letters to upper case
Thu Nov 2 10:14:19 CST 2023 Untitled Document
Convert upper case letters to lower case
Thu Nov 2 10:14:20 CST 2023