Linux 命令之 sed

sed 的全称是 stream editor，是支持正则表达式的流编辑器。其作用便是将文本文件或来自于管道符传入的输入流进行文本处理，如替换、增加内容、删除内容等等。

一、基本参数

下面对各个参数举例说明，以此文件为例：

shiyanlou:~/ $ nl a.txt  
     1  Nice to meet you p2p
     2  Where you've been?
     3  I can show you incredible things
     4  Magic, madness, heaven, (sins)
     5  Saw you there and I thought oh my_god
     6  Look at that face, you lookokok like my next mistake404
     7  Look at that face, you loook like my next mistake404
     8  Love's a game, wanna play
     9  New money, suit and tie
    10  I can read you like a magazine
shiyanlou:~/ $

1.1 d 删除指定行：

shiyanlou:~/ $ nl a.txt | sed '2,5d'   # 删除 2~5 行
     1  Nice to meet you p2p
     6  Look at that face, you lookokok like my next mistake404
     7  Look at that face, you loook like my next mistake404
     8  Love's a game, wanna play
     9  New money, suit and tie
    10  I can read you like a magazine
shiyanlou:~/ $

shiyanlou:~/ $ nl a.txt | sed '7,$d'   # 删除 7 至最后一行
     1  Nice to meet you p2p
     2  Where you've been?
     3  I can show you incredible things
     4  Magic, madness, heaven, (sins)
     5  Saw you there and I thought oh my_god
     6  Look at that face, you lookokok like my next mistake404
shiyanlou:~/ $ 

shiyanlou:~/ $ nl a.txt | sed '$d'   # 删除最后一行
     1  Nice to meet you p2p
     2  Where you've been?
     3  I can show you incredible things
     4  Magic, madness, heaven, (sins)
     5  Saw you there and I thought oh my_god
     6  Look at that face, you lookokok like my next mistake404
     7  Look at that face, you loook like my next mistake404
     8  Love's a game, wanna play
     9  New money, suit and tie
shiyanlou:~/ $

1.2 -i 修改原文件：

shiyanlou:~/ $ sed -i '1d' a.txt   # 删除第一行
shiyanlou:~/ $ nl a.txt   # 打印文件，第一行没了
     1  Where you've been?
     2  I can show you incredible things
     3  Magic, madness, heaven, (sins)
     4  Saw you there and I thought oh my_god
     5  Look at that face, you lookokok like my next mistake404
     6  Look at that face, you loook like my next mistake404
     7  Love's a game, wanna play
     8  New money, suit and tie
     9  I can read you like a magazine
shiyanlou:~/ $ 

1.3 a 在某行后面添加一行字符串：

shiyanlou:~/ $ nl a.txt | sed '2a 今天天气不错。'   # 在第 2 行后面添加一行字符串
     1  Where you've been?
     2  I can show you incredible things
今天天气不错。
     3  Magic, madness, heaven, (sins)
     4  Saw you there and I thought oh my_god
     5  Look at that face, you lookokok like my next mistake404
     6  Look at that face, you loook like my next mistake404
     7  Love's a game, wanna play
     8  New money, suit and tie
     9  I can read you like a magazine
shiyanlou:~/ $ 

1.4 i 在某行前面添加一行字符串：

shiyanlou:~/ $ nl a.txt | sed '2i 哈哈'   # 在第 2 行前面添加一行
     1  Where you've been?
哈哈
     2  I can show you incredible things
     3  Magic, madness, heaven, (sins)
     4  Saw you there and I thought oh my_god
     5  Look at that face, you lookokok like my next mistake404
     6  Look at that face, you loook like my next mistake404
     7  Love's a game, wanna play
     8  New money, suit and tie
     9  I can read you like a magazine
shiyanlou:~/ $ 

1.5 c 将指定行替换为字符串：

shiyanlou:~/ $ nl a.txt | sed '6,$c hello world'   # 将 6 至尾行替换为字符串
     1  Where you've been?
     2  I can show you incredible things
     3  Magic, madness, heaven, (sins)
     4  Saw you there and I thought oh my_god
     5  Look at that face, you lookokok like my next mistake404
hello world
shiyanlou:~/ $ 

1.6 -n 只打印受影响的行；p 复制某行并打印，通常配合 -n 一起用：

shiyanlou:~/ $ nl a.txt | sed '5,7p'
     1  Where you've been?
     2  I can show you incredible things
     3  Magic, madness, heaven, (sins)
     4  Saw you there and I thought oh my_god
     5  Look at that face, you lookokok like my next mistake404
     5  Look at that face, you lookokok like my next mistake404
     6  Look at that face, you loook like my next mistake404
     6  Look at that face, you loook like my next mistake404
     7  Love's a game, wanna play
     7  Love's a game, wanna play
     8  New money, suit and tie
     9  I can read you like a magazine
shiyanlou:~/ $ nl a.txt | sed -n '5,7p'
     5  Look at that face, you lookokok like my next mistake404
     6  Look at that face, you loook like my next mistake404
     7  Love's a game, wanna play
shiyanlou:~/ $

1.7 G 使文件中每一行下面添加一个空行：

shiyanlou:~$ nl a.txt | sed 'G'
     1  Where you've been?

     2  I can show you incredible things

     3  Magic, madness, heaven, (sins)

     4  Saw you there and I thought oh my_god

     5  Look at that face, you lookokok like my next mistake404

     6  Look at that face, you loook like my next mistake404

     7  Love's a game, wanna play

     8  New money, suit and tie

     9  I can read you like a magazine

1.8 ! 反操作：

shiyanlou:~$ nl a.txt | sed '4,$!G'   # 4 行及以下不执行
     1  Where you've been?

     2  I can show you incredible things

     3  Magic, madness, heaven, (sins)

     4  Saw you there and I thought oh my_god
     5  Look at that face, you lookokok like my next mistake404
     6  Look at that face, you loook like my next mistake404
     7  Love's a game, wanna play
     8  New money, suit and tie
     9  I can read you like a magazine

二、高级用法

以此文件为例：

shiyanlou:~/ $ nl words.txt
     1  syl
     2  I hate shiyanlou
     3  syl

2.1 s 行内替换，多重命令使用 ; 隔开，-e 和“分行”亦可实现多重命令：

shiyanlou:~/ $ sed 's/syl/shiyanlou/; s/hate/love/' words.txt
shiyanlou
I love shiyanlou
shiyanlou
shiyanlou:~/ $

shiyanlou:~/ $ sed -e 's/syl/haha/' -e 's/hate/love/' words.txt
haha
I love shiyanlou
haha
shiyanlou:~/ $

shiyanlou:~/ $ sed '   # 分行
quote> s/syl/haha/
quote> s/hate/love/' words.txt 
haha
I love shiyanlou
haha
shiyanlou:~/ $

2.2 g 无它，只替换每行第一个字段；有它，替换全部：

shiyanlou:~$ nl words.txt | sed 's/a/A/'
     1  sys
     2  I hAte shiyanlou.
     3  sys
shiyanlou:~$ nl words.txt | sed 's/a/A/g'
     1  sys
     2  I hAte shiyAnlou.
     3  sys
shiyanlou:~$ 

2.3 -f 选项执行 sed 脚本文件：

shiyanlou:~/ $ echo "s/syl/haha/\ns/hate/love/" > sfile
shiyanlou:~/ $ nl sfile
     1  s/syl/haha/
     2  s/hate/love/
shiyanlou:~/ $ sed -f sfile words.txt
haha
I love shiyanlou
haha
shiyanlou:~/ $ 

2.4 将 shebang 写入脚本文件，赋予其可执行权限，即可直接执行：

shiyanlou:~/ $ which sed   # 查找 sed 命令的可执行文件
/bin/sed
# 将 shebang 插入到文件的第一行
shiyanlou:~/ $ sed -i '1i #!/bin/sed -f' sfile   
shiyanlou:~/ $ nl sfile
     1  #!/bin/sed -f
     2  s/syl/haha/
     3  s/hate/love/
shiyanlou:~/ $ chmod +x sfile
shiyanlou:~/ $ ./sfile words.txt
haha
I love shiyanlou
haha

2.5 寻址

上文中使用 s 选项对文件进行修改，默认是全文修改。实际上我们在使用 sed 命令处理流数据时，经常需要对特定的行进行修改，这就用到了“寻址”之术。

使用下面的文件进行举例：

shiyanlou:~/ $ cat -n xunzhi
     1  a b
     2  b
     3  c
     4  
     5  d b
     6  
     7  e
     8  
     9  f

2.5.1 将字符 a 所在的行的字符 b 替换为 haha：

shiyanlou:~/ $ nl xunzhi | sed '/a/s/b/haha/'
     1  a haha
     2  b
     3  c
       
     4  d b
       
     5  e
       
     6  f
shiyanlou:~/ $ 

2.5.2 /^&/ 表示空行：

shiyanlou:~/ $ cat xunzhi   # nl 命令虽然不给空行编号，但打印结果空行不是没有字符的
a b
b
c

d b

e

f
shiyanlou:~/ $ cat xunzhi | sed '/^$/d'   # 删掉全部空行
a b
b
c
d b
e
f
shiyanlou:~/ $ 

shiyanlou:~/ $ cat xunzhi | sed '2,/^$/d'   # 删掉第 2 行到第 1 个空行
a b
d b

e

f
shiyanlou:~/ $ 

2.5.3 { } 大括号可实现分组
先将 xunzhi 文件中包含 b 字符又包含 hello 字符串的行（即第 1、3 行）中的 world 替换为 shiyanlou，再将包含 b 和 d 字符的行（即第 5 行）中的 world 替换为 louplus：

shiyanlou:~/ $ cat -n xunzhi   # 修改后的 xunzhi 文件
     1  a b hello world
     2  b world
     3  c b d hello world
     4  
     5  d b world good
     6  
     7  e hello world
shiyanlou:~/ $ sed '/b/{
quote> /hello/s/world/shiyanlou/
quote> /d /s/world/louplus/   # 此处的 d 为特殊字符，且不能被转义
quote> }' xunzhi | cat -n     # 所以在其后面加个空格，成为 'd '
     1  a b hello shiyanlou   # 否则只匹配前面的 'b' 字符，那么第 2 行也将被修改
     2  b world
     3  c b d hello shiyanlou
     4  
     5  d b louplus good
     6  
     7  e hello world
shiyanlou:~/ $

2.6 -r 参数使用标准正则表达式，上面 2.5.3 的例子可以这样写：

shiyanlou:~/ $ sed -r '/b/{
quote> /hello/s/world/shiyanlou/
quote> /\d/s/hello/louplus/   # d 为特殊字符，须转义
quote> }' xunzhi | cat -n
     1  a b hello shiyanlou
     2  b world
     3  c b d louplus shiyanlou
     4  
     5  d b world good
     6  
     7  e hello world
shiyanlou:~/ $