The Stream Editor - sed - PowerPoint Presentation

1. The Stream Editor - sed Readings: Chapter 15

2. sedStream Editor (sed) A batch editorTransforms input stream from a file or standard inputFrequently used as filter in a pipesed can test, branch, substitute, hold, and exchange dataOf course, so can Perl or almost any scripting language But the beauty of sed is its simplicity (and powerfulness at its size)2

3. Basic SyntaxInvoking sedsed [options] script [file-list]sed [options] –f script-file [file-list]Another way to run a sed script is toAdd #!/bin/sed –f as the first lineUse chmod to make the script executableLines in a sed script-file[address[,address]] command [argument-list]# is the comment character in sed scriptsSee sed/sed1 (on input file test1.txt)sed –f sed1 test1.txtsed1 test1.txt3

4. How sed Process Input LinesRead one line (from file or standard input)Read command one by one, and apply them one by one if the line matches the addressA command without address will be applied to all linesNote that a line may be transformed by a previous instruction before passing to the next instructionAfter all commands have been tried, read next line and repeat4

5. Addressing BasicsA sed command can specify zero, one, or two addressesAn address can bea line numbera line addressing symbol ($)a regular expression that describes a patternIf no address is specified, The command is applied to each lineFor example, “d” # delete all linesIf there is only one address, The command is applied to any line matching the addressFor example, “2 d” # delete 2nd lineSpace between address and instruction is optional5

6. Addressing BasicsIf two comma-separated addresses are specified,The command is performed on the first matching line and all succeeding lines up to and including a line matching the second addressThis range may match multiple times throughout the inputFor example, “2,3 d” # delete lines between 2 and 3If an address is followed by an exclamation mark (!), the command is applied to all lines that do not match the addressFor example, “2! d” # delete all lines except line 2Although the space is optional, it can create problems because of history expansion, which use “!”Use single quote ‘’ is safer. However, it can still be a problem with single quote, for example, in tcsh. History expansion is not prevented by single quote in tcsh6

7. Addressing using RE and Special SymbolsWhen a regular expression is supplied as an address, the command affects only the lines matching that patternThe regular expression is enclosed by slashes (/)/^$/d # deletes all blank lines1,/^$/d # mixing different addressingSpecial symbol $Last line of input file50,$d # line 50 to last lineSee sed/sed2 (on input file coureses.html)7

8. Common sed CommandsSubstitution command ss/regexp/replacement/flagsThe delimiter can be any single character (here we use /)If a match is found for regexp, replace the part of text matched with replacement stringSome common flagsg: apply replacement to all matches (in the line)p: print the matched linew filename: write the matched lines to filename(see example sed/sed4, with test1.txt)Quit command qOnly accepts a single addressQuit sed when the specified address is reached8

9. Common sed CommandsDelete command dDelete the corresponding lines specified using addressesGroup of commands {commands}You can apply multiple commands to corresponding addressed linesFor example, to search every line of a list, capitalize the word Caution on any of those lines, and delete any line with <br />:/^<ul>/,/^<\/ul>/{ s/Caution/CAUTION/g/^<br \/>/d } Comment command ## indicates a comment line (no address allowed)9

10. Common sed CommandsNext line command nOutput current line (if allowed), read in and start processing next lineExample /iaeo/ npSee sed/next1.sed (with test1.txt)Using sed –n (silent)10

11. Less Common sed CommandsTransform command (y)y/source-chars/dest-chars/Replace every source character by the corresponding destination characterEffectively, it performs a similar function to Unix command tr Exmaples y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/ y/abcdefghijklmnopqrstuvwxyz/nopqrstuvwxyzabcdefghijklm/ rot13 transformation - a simple form of encryption in which each alphabetic character is replaced by the character halfway through the alphabet. See sed/rot13.sed and derot13.sed (with test1.txt)11

12. Less Common sed CommandsChange command cDelete matched lines, replace them by text, where text can be multiple lines (separated with \)Example See sed/change.sed (with test2.txt)12c\text/WORD/ c\Replace the current line with the line\add another line

13. Less Common sed CommandsInsert command iOutput the text above the matched line(s), where text can be multiple lines (separated with \)Example See sed/insert.sed (with test2.txt)13i\text/WORD/ i\Insert this line above the matched line\add another line

14. Less Common sed CommandsAppend command aOutput the text after the matched line(s), where text can be multiple lines (separated with \)Example See sed/append.sed (with test2.txt)14a\text/WORD/ a\Add this line after the matched line\add another line

15. Less Common sed CommandsNext line command NRead next line and append it to the current line, after adding NEWLINE characterExampleSee examples/sed/next2.sedUsing sed –n (with file test1.txt)15/iaeo/ Ns/\n/ /p

16. Hold SpaceThe pattern space is a buffer that contains the current input lineThere is also a set-aside buffer called the hold space The contents of the pattern space can be copied to the hold space, and vise versaA group of commands allows you to move data between the hold space and the pattern space The hold space is used for temporary storage Individual commands can't address the hold space or alter its contentsHold (h or H ) - Copy or append contents of pattern space to hold spaceGet (g or G ) - Copy or append contents of hold space to pattern spaceUppercase does the append (after adding a newline)Exchange ( x ) - Swap contents of hold space and pattern space16

17. Hold Space: ExampleCopying all headings in an HTML file to the end of the fileSee examples/sed/sed3 (with courses.html)17/^<h[12]>/H/^<\/body>/{ i\<strong>Summary:</strong> x G s/<\/*h[12]>//g}

18. More about sedManual page of sedsed Gnu Projecthttp://www.gnu.org/software/sed/sed.htmlSed – An Introductionhttp://www.grymoire.com/Unix/Sed.html18

19. Some Unix CommandstrTranslate or delete characterstr –d ‘[0-9]’ < test1.txt # delete all digits tr ‘[a-z]’ ‘[A-Z]’ < test1.txt # replace all lower-case character #by the corresponding upper-case character.TimeTime a command to show running timetime example1.xCpCopy filescp file1.txt file2.txtcp file1.txt test # where test is directoryLnCreate links (hard and symbolic)Ln file1.txt file1_link.txtLn –s file1.txt file1_sym.txt19

20. Reading AssignmentChapter 14 on awk20