A Quick Introduction to sed

sed, the stream editor, lets you edit a file by giving commands from the command line or from within a script. This allows you to do a “batch edit” of multiple files, or to perform common editing operations without ever having to open vi or emacs.

The sed Processing Cycle

The most important thing to know about sed is the steps it takes when processing a file.

1. Read a line from the file into a buffer.
2. Execute the command (or commands) on the buffer.
3. Output the buffer to standard output.

Because sed outputs to standard output, your original file will always remain untouched.

Command Line sed

The format for using sed from the command line is either:

sed somecommand inputfile
sed -e somecommand inputfile  # same as previous line
sed -e firstcommand -e secondcommand inputfile # multiple commands

What’s in a Command?

A command consists of:

1. An optional line number or line number range
2. A one-letter command to sed
3. Any parameters that command might require.

Substitution

To start with, let’s use possibly the most common sed command, s for substitution. If I needed to change every occurence of UNIX to Linux in file info.txt, I would issue this command:

sed -e 's/UNIX/Linux/g' info.txt

Because I started out with the command (not giving a line number or line number range), sed will apply the command to every line in the original file. The s command is followed by:

• A delimiter (in this case, /)
• The pattern for the thing you want to change (and this can be a regular expression)
• The delimiter again
• The replacement for the matched text
• The delimiter again–and it is required!
• Any options for the substitute command. In this case, we have used g (global) to change all occurrences on the line.

Here’s another, more complex pattern that uses saving of matched characters to change the format of lines that start with a telephone number in the form 408.555-1212 to read like this: (408)555-1212

sed -e 's/^\([0-9]\{3\}\)\./(\1)/' info.txt

Line Numbers

A line number or line number range can be any of the following:

• A single number
• ., meaning the current line
• $, meaning the last line
• A beginning and ending number, separated with a comma:
  • 1,5 (lines one through five)
  • 7,$ (lines seven through the end of the file)
  • .,20 (current line through line 20)

So let’s say I wanted to delete lines three through seven of file info.txt:

sed -e '3,7d' info.txt

Important: sed still processes every line in the file. It reads the first line, sees that it is not in the line range, and passes it to the output unchanged. It reads the second line, tests it against the line range, and, because it is not, passes it to the output unchanged. The third through seventh lines do pass the test of being in the line range, so the d command deletes them and passes nothing to the output. Lines eight through end of file are processed, found to be outside the line range, and are passed to the output unchanged.

Patterns as “Line Numbers”

It is possible to use a regular expression in place of a line number. For example, let us say I wanted to delete any line that had the text # temporary on it I could do it this way:

sed -e '/# temporary/d' info.txt

This is not a substitution. Instead, I am saying:

/# temporary/

For lines that match this regular expression...

d

delete the entire line

I can use a pattern as a beginning and end of a range. The following sed command deletes lines. It uses as its starting point a line that begin with the word FIRST and as its ending point a line that begins with the word LAST.

sed -e '/^FIRST/,/^LAST/d' info.txt

Needless to say, things can get pretty tricky with the slashes. If I wanted to substitute the word JPEG with JPG in the same line range as before, the command would be:

sed -e '/^FIRST/,/^LAST/s/JPEG/JPG/' info.txt

If you get confused by all the slashes, you may replace the / in the substitute by another punctuation mark, for example, % or |:

sed -e '/^FIRST/,/^LAST/s%JPEG%JPG%' info.txt

sed from a Script

Let’s say I need to do the following three things to a file:

1. Delete lines 1 and 2
2. On lines 3 through the end, replace perl with Perl
3. Before lines containing the word test, insert a line reading
   System.out.println("Test");

I could do this from the command line, but it's hard to type, and if I ever need to do it again, I have to start from scratch:

sed -e '1,2d' -e '3,$s/perl/Perl/g' -e '/test/i\
System.out.println("Test");' info.txt

Instead, it is easier to put all three commands into a file called make_edits.sed (or whatever name I like). Note that the words sed -e do not appear in the sed script file!

1,2d
3,$s/perl/Perl/g
/test/i\
System.out.println("Test")

Then, from the command line, I just tell sed to get its information from the file I created:

sed -f make_edits.sed info.txt

Multiple Commands

Sometimes it is necessary to do several actions in one line range. To make this easier, you can group commands together with { and }. This script file will do three separate changes on any lines that contain the words change me:

1. Change digits to number signs
2. Replace the word "This" with "That"
3. Append a blank and the word "done" to the end of the line.

/change me/y/0123456789/##########/
/change me/s/This/That/g
/change me/s/$/ done/

By grouping, we get the more readable:

/change me/{
	y/0123456789/##########/
	s/This/That/g
	s/$/ done/
}

In this case, we do not need the braces; they simply improve readability. Sometimes, though, the braces are necessary to achieve what you want. (For example, in the first script, if my first substitution had eliminated the words change me, then the next two lines would not have been able to find that pattern to do the remaining changes.)