jump to navigation

The basics of sed November 13, 2010

Posted by Tournas Dimitrios in Linux.

sed is the unix `stream editor’. Its functionality follows its name: sed operates on an input file or pipe , line by line, and edits each line according to the instructions you give it. sed is a great way to edit lots of files simultaneously. It is available with every unix system I’ve seen. A similar unix command that may or may not be better suited to your applications is awk. The format I usually use for sed is
cat inputfile | sed [sed options] sed command‘ > outputfile
sed commands tend to look like those in vi.

Sed editing commands :

Command Result
a\ Append text below current line.
c\ Change text in the current line with new text.
d Delete text.
i\ Insert text above current line.
p Print text.
r Read a file.
s Search and replace text.
w Write to a file.

Apart from editing commands, you can give options to sed. An overview is in the table below:

Sed options :

Option Effect
-e SCRIPT Add the commands in SCRIPT to the set of commands to be run while processing the input.
-f Add the commands contained in the file SCRIPT-FILE to the set of commands to be run while processing the input.
-n Silent mode.(suspend non-matching lines
-V Print version information and exit.

The sed info pages contain more information; I only list the most frequently used commands and options here.
Lets practice with some commands and options:

Sed Find and Display Text Between Two Strings or Words
To output all the text from file called test.xt’ between ‘FOO’ and ‘BAR’, type the following command at a shell prompt. The -n option suppress automatic printing of pattern space:

sed -n ‘/FOO/,/BAR/p’ test.txt

You can easily find out all virtual host entries from httpd.conf, type

sed  -n ‘/^<VirtualHost>/  ,  /<^\/VirtualHost>/p’ /etc/httpd/conf/httpd.conf

Remove/Delete All Leading Blank Spaces/Tabs ( whitespace ) From Each Line
To remove all blank lines, enter:

cat /etc/rssh.conf | sed ‘/^$/d’ > /tmp/output.file

To remove all whitespace (including tabs) from left to first word, enter:

echo ”     This is a test” | sed  -e ‘s/^[ \t]*//’

To delete trailing whitespace from end of each line, enter:

cat input.txt | sed ‘s/[ \t]*$//’ > output.txt

Better remove all leading and trailing whitespace from end of each line:

cat input.txt | sed ‘s/^[ \t]*//;s/[ \t]*$//’ > output.txt>

Append Lines Using Sed Command
Sed provides the command “a” which appends a line after every line with the address or patter :

sed ‘ADDRESS a\    Line which you want to append’ filename.txt

sed ‘/PATTERN/ a\Line which you want to append’ filename.txt

Add a line after the 3rd line of the file.

sed ‘3 a\Cool gadgets and websites’ filename.txt

Deleting lines of input containing a pattern
The d command results in excluding lines from being displayed.

sed ‘/erors/d’ example.txt

Matching lines starting with a given pattern and ending in a second pattern are showed like this:

sed -n ‘/^This.*errors.$/p’ example.txt

Note that the last dot needs to be escaped in order to actually match. In our example the expression just matches any character, including the last dot.

First specify the range of lines we want to address , then apply the d (delete)command

sed ‘2,4d’ example.txt

To delete the file starting from a certain line until the end of the file :

sed ‘3,$d’ example.txt

To delete all Virtual Hosts from httpd configuration file

sed   ‘/<VirtualHost>/  ,  /<\/VirtualHost>/d ’ /etc/httpd/conf/httpd.conf

Find and replace with sed
search and replace the word “erors” with the word “errors”

sed ‘s/erors/errors/’ example.txt

If this is not exactly the desired effect: only the first occurrence of the search string has been replaced

Use the g command to indicate to sed that it should examine the entire line instead of stopping at the first occurrence of your string:

sed ‘s/erors/errors/g’ example.txt

To insert a string at the beginning of each line of a file, for instance for quoting:

sed ‘s/^/> /’ example.txt

Insert some string at the end of each line:

sed ‘s/$/EOL/’ example.txt

Multiple find and replace commands are separated with individual -e options:

sed -e ‘s/erors/errors/g’ -e ‘s/last/final/g’ example.txt


s/string1/string2/ substitutes string2 for the first occurence of string1 in each line
s/string1/string2/g substitutes string2 for string1 everywhere
in each line
2 s/limits.*/hello/ in line 2, looks for the string ‘limits.*’
where * is any string, and replaces with ‘hello’
2,4 s/junk/try/ substitutes ‘try’ for the first occurence of ‘junk’
in lines 2 through 4 only
8 a\set c=818. appends the string ‘set c=818.’  after line 8
8 a\set i1=(i+c)\/c – 1 appends the string ‘set i1=(i+c)/c – 1′ after line 8. Note the use of the \ is needed to print out
`special characters’, like /, &, %, $, and of course, \ itself
8 i\set c=818. inserts the string ‘set c=818.’ before line 8

Suppose you have 50 files, and they all need to have ‘junk’ replaced by ‘try’ in lines 2 through 10. You could edit all 50 files individually (what a pain), or do it quickly with sed. If the files are named file1, file2, etc. you could do the following:
vi a.sed, and create a file that looks like:

2,10 s/junk/try/

Back at the unix prompt type:
ls file*

You get:


Now type:
ls file* | sed ‘s/.*/cat & | sed -f a.sed > tmp; mv -f tmp &/’ > doit

which is of the form:
inputlines | sed ‘sed-command’ > outputfile

sed stores each line (the .* in the above command) in a variable called & (it does this
by default), and substitutes

cat & | sed -f a.sed > tmp; mv -f tmp &

for the line.

So the file ‘doit’ looks like:
cat file1 | sed -f a.sed > tmp; mv -f tmp file1
cat file10 | sed -f a.sed > tmp; mv -f tmp file10
cat file11 | sed -f a.sed > tmp; mv -f tmp file11

cat file50 | sed -f a.sed > tmp; mv -f tmp file50

Execute these commands by typing:
cat doit | csh

You can do all of this in one line by typing:
ls file* | sed ‘s/.*/cat & | sed -f a.sed > tmp; mv -f tmp &/’ | csh

Another way to accomplish this renaming (courtesy J. Salk) is:
for f in file*; do mv $f tmp; sed ‘2,10 s/junk/try/g’ tmp >$f; done



No comments yet — be the first.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s