Cs288 Intensive Programming in Linux - PowerPoint Presentation

Slide1

Cs288Intensive Programming in Linux

Instructor: C F YurkoskiChristopher.f.yurkoski@njit.eduSection web site: https://web.njit.edu/~yurkowsk/cs288.html

6-9-15

1

Slide2

6-9-15

2

Slide3

1st Homework results

average grade 70%indent your code properly!Don’t submit .txt fileDon’t submit .tgzTest on afs

Questions after class

6-9-15

3

…………………………………………………..

Slide4

Webex test

https://njit.webex.com/mw3100/mywebex/default.do?siteurl=njit6-9-154

Slide5

Project revu

project using cut, grep, wget, loops etc.In https://web.njit.edu/~yurkowsk/project1/ there are 4 files:https://web.njit.edu/~yurkowsk/project1/names

https://web.njit.edu/~yurkowsk/project1/idshttps://web.njit.edu/~yurkowsk/project1/hours

 andhttps://web.njit.edu/~yurkowsk/project1/names/ratenames contains a list of names, one per line.

id

 contains a list of semicolon separated tuples, one per line consisting of a name and the literal ID followed by a numeric id.

hours

  contains list of semicolon separated tuples, one per line consisting of a numeric id and a number indicating the number hours worked that week.

rate

  contains list of semicolon separated tuples, one per line consisting of a numeric id and an hourly rate for that id.  Assume the rate is USD per hour.

Your goal is to write a script which calculates the compensation for the names in first file for the week and output the names followed by a tab followed by a dollar sign followed by that name’s pay.

6-9-15

5

Slide6

Output one name and pay amount per line.  The output should be in alphabetical order by name regardless of the order in the “names” file.

Write the results to standard out, output any error messages to standard error.Your script should accept 5 command line arguments.   The first will be the name of a directory like the one above, the next 4 are assumed be file names in that directory whose contents are formatted as the four URLs above and are in the same order as above.   Base on the contents of the files calculate the pay for name in the names file pay by using it to retrieve the id from the id file and using that in turn to obtain the pay rate and hours worked. 

 E.g. I may call your script  (where

Yourscript is the name of the file you submit to moodle) like this:

Yourscript

  

https://web.njit.edu/~yurkowsk/project1

 names ids hours rate 2> errors

or I may call it like this:

Yourscript

  

https://web.njit.edu/~yurkowsk/some/otherdirectory

 Names IDs hours Rate 2> errors

6-9-15

6

Slide7

So you can be guaranteed of the order of the arguments and of the format of the files but 

not their actual names or actual contents. Your output should be formatted like this:

Al

Aho     $4567

Brian Kernighan $0

Peter Weinberger $1234

Be sure to write your error messages to standard error because your standard output will automatically be compared to the expected output.  You should check for (at least) the follow error conditions:

The wrong number of arguments.

There is a name in the names file for which there is no id in the id file.

If you encounter the first condition you should exit with a false exit value, for the other you should log a message to

stderr

and continue.

 

If you find no hours for a name in the hours file, assume she worked 0 hours that week.   If you find no rate for a worker assume she is working pro bono.

If there are no errors, you should exit with a true exit code.

 

Do your own work!   Otherwise it won’t be counted.

6-9-15

7

Slide8

Two part quiz

You’ll find thes 2 URLs:https://web.njit.edu/~yurkowsk/hourlyrate andhttps://web.njit.edu/~yurkowsk/employees

hourlyrate contains 1 line, a number in USD.employees contains several lines with tab separated fields of the format:

status<TAB>name <TAB>social security number<TAB> hours

6-9-15

8

Slide9

Write a small bash script which accepts as arguments those two URLs and which calculates the pay for each hourly employee and print their names and the amount of their pay assuming they all work 30 hours per week. The names and the amount should be tab separated.

E.g:>: ./

cfyscript

https://web.njit.edu/~yurkowsk/hourlyrate https://web.njit.edu/~yurkowsk/employees

Jane

Deardon

$450.00

Jill

Deardon

$450.00

J

Deardon

$450.00

Jill Jackson $450.00

>:

Complete in 15 minutes.

6-9-15

9

Slide10

quiz revu

Given this input file: afsconnect1-59

quizes >: cat input

ab,cd,efghijk,lmn,opqurstuvwxyz

ab,cd,efghijk,lmn

,

efghijk,efghijk,opqurstuvwxyz

,,

efghijk,lmn,opqurstuvwxyz

,,

efghijk,,lmn,z

,

a,b,c,n

,

afsconnect1-60

quizes

>:

what is the output of this command line:

afsconnect1-61

quizes

>: cat input |

grep

"^e.*n" | cut -f4 -d"," | while read x; do echo ${x} $x

x

"x"; done

6-9-15

10

Slide11

>: cat input |

grep "^e.*n" | cut -f4 -d"," | while read x ; do echo ${x} $x x

"x"; donex

x

z

z

x

x

6-9-15

11

Slide12

cat input |

grep "^e.*n" | cut -f4 -d"," | while read x ; do echo ${x} $x x

"x"; done

ab,cd,efghijk,lmn,opqurstuvwxyz

ab,cd,efghijk,lmn

,

efghijk,efghijk,opqurstuvwxyz

,,

efghijk,lmn,opqurstuvwxyz

,,

efghijk,,lmn,z

,

a,b,c,n

,

6-9-15

12

Slide13

cat input |

grep "^e.*n" | cut -f4 -d"," | while read x ; do echo ${x} $x

x

"x"; done

efghijk,lmn,opqurstuvwxyz

,,

efghijk,,lmn,

z

,

6-9-15

13

Slide14

cat input |

grep "^e.*n" | cut -f4 -d"," | while read x ; do echo ${x} $x x

"x";

done

echo ${x} $x

x

"x";

6-9-15

14

Slide15

Quiz 1, Part 2

rate=`wget -q -O - $1 `

let pay=$rate*30

wget

-q -O - $2 | grep "^hourly" | grep -v "hourly status" | cut -f 2 |while read name

do

echo -e ${name} \\t \$${pay}.00

done

6-9-15

15

Slide16

Ascii table

http://www.ascii-code.com/6-9-1516

Slide17

other Linux tools and built ins

shiftsortuniqawk Aho, Weinstein & Kernighan.sed stream editor.

6-9-15

17

Slide18

Basic awk

6-9-15

18

Slide19

6-9-15

19Usage: sort [OPTION]... [FILE]...

-b, --ignore-leading-blanks ignore leading blanks

-d, --dictionary-order consider only blanks and alphanumeric characters -f, --ignore-case fold lower case to upper case characters

-g, --general-numeric-sort compare according to general numerical value

-

i

, --ignore-nonprinting consider only printable characters

-M, --month-sort compare (unknown) < `JAN' < ... < `DEC'

-h, --human-numeric-sort compare human readable numbers (e.g., 2K 1G)

-n, --numeric-sort compare according to string numerical value

-R, --random-sort sort by random hash of keys

--random-source=FILE get random bytes from FILE

-r, --reverse

reverse

the result of comparisons

Slide20

Other reminders

to redirect stdout to stderr: 1>&2 to redirect stderr to stdout: 2>&1 to test for null strings: –zto test for non-null strings: -n

6-9-15

20

Slide21

6-9-15

21sed

Slide22

sed examples

sed -e "/pool-17/d" hostssed -e "s/pool-1[1-9]/poolA/" hosts

sed -f script hosts

cat scripts/pool-1[1-9]/

poolA

/

s/pool-[1-9]/

poolB

/

6-9-15

22

Slide23

environment

. “pathname”source “pathname”e.g.:. ./.envsource /home/.env

6-9-15 cfy

23

Slide24

More useful bash commands

tailtouchfindmansource6-9-15 cfy

24

Slide25

in class assignment

Write a script that takes 3 arguments;The first arg is a pathname of a file that contains a list of script names, the 2nd is a pathname of file that is to be passed as input to each of those scripts, and the 3rd is the expected output when those scripts when passed that input as it first arg. Your script should assume that each line in arg1 is a script in its PATH, it should execute each script passed to it with its arg1 your scripts arg2.

Your script should output the name of each script it runs followed by the output that script produces when run with the input specified.

6-9-15

25

Slide26

e.g.

>: yourscript students input1 outputstudent output

------- ------

cifer,l input1

emlin,g

42

lem,s

2

assuming:

afsconnect1-43

quizes

>: cat input1

21

afsconnect1-44

quizes

>:

6-9-15

26

Slide27

6-9-15

27afsconnect1-45 quizes >: cat students

cifer,l

emlin,glem,s

afsconnect1-46

quizes

>:

and afsconnect1-47

quizes

>: cat students | while read x

> do

> echo $x

> cat $x

> echo

> done

cifer,l

echo $1

emlin,g

y=`cat $1`

let x=y*2

echo $x

lem,s

echo 2

Slide28

6-9-15

28>: cat

yourscript

echo student output

echo ------- ------

cat $1 | while read

testscript

do

echo -e -n "$

testscript

\t"

bash $

testscript

$2

done

>:

Slide29

A better solution

Compare the output of each script to the contents of the third file.Output the name of the script followed by a tab and a indication as to whether the results are correct (the same as the contents of the output file) or not.6-9-15

29

Slide30

assuming:

afsconnect1-50 quizes >: cat answertoeverything

42afsconnect1-51

quizes >:

6-9-15

30

Slide31

extra credit example

>: bash extracredit students input1 ./answertoeverything

student grade

------- ------

cifer,l

is wrong

emlin,g

is correct

lem,s

is wrong

6-9-15

31

Slide32

echo student grade

echo ------- ------

cat $1 | while read

testscript

do

rm

/

tmp

/xyzzy

echo -e -n "$

testscript

\t"

bash $

testscript

$2 > /

tmp

/xyzzy

diff /

tmp

/xyzzy $3 >/dev/null

if [ "$?" -

eq

0 ]

then

echo is correct

else

echo is wrong

fi

done

6-9-15

32

Slide33

6-9-15

33echo student output

echo ------- ------cat $1 | while read

testscript

do

echo -e -n "$

testscript

\t"

bash $

testscript

$2

done

Slide34

6-9-15

34cat $1 | while read testscript

do

rm

/

tmp

/

xyzzy

echo -e -n "$

testscript

\t"

bash $

testscript

$2 > /

tmp

/

xyzzy

diff /

tmp

/

xyzzy

$3 >/dev/null

if [ "$?" -

eq

0 ]

then

echo is correct

else

echo is wrong

fi

done

Slide35

homework

Write a script that accepts as it two command line argument two urls, for example, but not necessarily:https://web.njit.edu/~yurkowsk/29sept/competitorAandhttps://web.njit.edu/~yurkowsk/29sept/competitorB

These files contain 3 comma separated fields:customer,sales,product

where sales in a number and customer and product are strings.Find any customers that appear in both files that have total sales of more than 100000 and print the customer name and products they purchased.

6-9-15

35

Slide36

https://web.njit.edu/~yurkowsk/29sept/competitorA

customerA,1234567,routercustomerC,7234567,routercustomerE,2345,switchcustomerF,234567,servercustomerG,123456,router6-9-15

36