UMass Lowell Computer Science 91.503 - PowerPoint Presentation

Slide1

UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels

Design Patterns

for

Optimization Problems

Dynamic

Programming

Matrix Parenthesizing

Longest Common Subsequence

Activity SelectionSlide2

Algorithmic Paradigm Context

Subproblem solution order

Make choice, then solve subproblem(s)

Solve subproblem(s), then make choiceSlide3

Dynamic Programming Approach to Optimization ProblemsCharacterize structure of an optimal solution.Recursively define value of an optimal solution.

Compute value of an optimal

solution, typically

in bottom-up fashion.

Construct an optimal solution from computed information

.

(separate slides for rod cutting)

source: 91.503 textbook Cormen, et al.Slide4

Dynamic Programming

Matrix ParenthesizationSlide5

Example: Matrix Parenthesization Definitions

Given “chain” of n matrices: <A

1

, A

2

, … An, >Compute product A1

A2… An efficientlyMultiplication order matters!Matrix multiplication is associative

Minimize

“cost” = number of scalar

multiplications

source: 91.503 textbook Cormen, et al.Slide6

Example: Matrix Parenthesization Step 1: Characterizing an Optimal Solution

Observation:

Any parenthesization of

A

i

A

i+1

… A

j

must split it between

A

k

and

A

k+1

for some k.

THM: Optimal Matrix Parenthesization:

If an optimal parenthesization of

AiAi+1… Aj splits at k, then parenthesization of prefix AiAi+1… Ak must be an optimal parenthesization.Why? If existed less costly way to parenthesize prefix, then substituting that parenthesization would yield less costly way to parenthesize AiAi+1… Aj , contradicting optimality of that parenthesization.

source: 91.503 textbook Cormen, et al.

common DP proof technique: “cut-and-paste” proof by contradictionSlide7

Example: Matrix Parenthesization Step 2: A Recursive SolutionRecursive definition of minimum parenthesization cost:

m[i,j]=

min{m[i,k] + m[k+1,j] + p

i-1

p

k

p

j

} if i < j

0 if i = j

How many distinct subproblems?

i <= k < j

each matrix A

i

has dimensions p

i-1

x p

i

source: 91.503 textbook Cormen, et al.Slide8

Example: Matrix Parenthesization Step 3: Computing Optimal Costs

0

2,625

2,500

1,000

s: value of k that achieves optimal cost in computing m[i, j]

source: 91.503 textbook Cormen, et al.Slide9

Example: Matrix Parenthesization Step 4: Constructing an Optimal SolutionPRINT-OPTIMAL-PARENS(s, i, j)

if

i

==

j print “A”i

else print “(“ PRINT-OPTIMAL-PARENS(s, i, s[i

, j])

PRINT-OPTIMAL-PARENS(s, s[

i

, j]+1, j)

print “)“

source: 91.503 textbook Cormen, et al.Slide10

Example: Matrix Parenthesization MemoizationProvide Dynamic Programming efficiency

But

with

top-down

strategy

Use recursionFill in m table

“on demand”(can modify to fill in s table)

source: 91.503 textbook Cormen, et al.

MEMOIZED-MATRIX-CHAIN(p)

n =

p.length

– 1

let m[1…n,1…n] be a new table.

for

i

= 1

to

n4 for j = i to n5 m[i,j] = 6 return LOOKUP-CHAIN(m, p,1,n)

LOOKUP-CHAIN(

m,p,i,j) 1

if

m[

i,j

] <

2

return

m[

i,j

]

3

if

i

==

j

4

m[

i,j

]

= 0

5

else

for

k

=

i

to j-1

6

q = LOOKUP-CHAIN(m,p,i,k) + LOOKUP-CHAIN(m,p,k+1,j) + pi-1 pk pj7 if q < m[i,j] 8 m[i,j] = q9 return m[i,j] Slide11

Dynamic Programming

Longest Common SubsequenceSlide12

Example: Longest Common Subsequence (LCS): MotivationStrand of DNA: string over finite set {A,C,G,T}each element of set is a base: adenine, guanine, cytosine or thymine

Compare DNA similarities

S

1

=

ACCGGTCG

AGTGCGCGGAAGCCGGCCGAA

S

2

=

GTCGT

T

CGGAA

T

GCCG

TT

GC

T

C

TGTAAAOne measure of similarity:find the longest string S3 containing bases that also appear (not necessarily consecutively) in S1 and S2S3 = GTCGTCGGAAGCCGGCCGAAsource: 91.503 textbook Cormen, et al.Slide13

Example: LCS Definitions

Sequence is a

subsequence

of if

(strictly increasing indices of X)

such thatexample: is subsequence of with index sequence

Z is common subsequence of X and Y if Z is subsequence of both X and Yexample:common subsequence but not longest

common subsequence. Longest?

Longest Common Subsequence Problem

: Given 2 sequences X, Y, find maximum-length common subsequence Z.

source: 91.503 textbook Cormen, et al.Slide14

Example: LCS

Step 1: Characterize an LCS

THM 15.1: Optimal LCS Substructure

Given sequences:

For any LCS of X and Y:

1

if then and Z

k-1

is an LCS of X

m-1

and Y

n-1

2

if then Z is an LCS of X

m-1

and Y

3 if then Z is an LCS of X and Y

n-1

PROOF: based on producing

contradictions

1 a)

Suppose . Appending

to

Z contradicts

longest

nature of Z.

b) To establish

longest

nature of Z

k-1

, suppose common subsequence W of

X

m-1

and

Y

n-1

has

length

> k-1. Appending to W yields common subsequence of length > k = contradiction.

2

To establish optimality (

longest

nature), common

subsequence W of

X

m-1

and

Y

of length > k would also be common subsequence of

Xm, Y, contradicting longest nature of Z. 3 Similar to proof of (2)source: 91.503 textbook Cormen, et al.(using prefix notation)Slide15

Example: LCS Step 2: A Recursive SolutionImplications of Theorem 15.1:

?

yes

no

Find LCS(X

m-1

, Y

n-1

)

Find LCS(X

m-1

, Y)

Find LCS(X, Y

n-1

)

LCS

1

(X, Y) = LCS(X

m-1

, Yn-1) + xm

LCS2(X, Y) = max(LCS(Xm-1, Y), LCS(X, Y

n-1))

An LCS of 2 sequences contains, as a prefix, an LCS of prefixes of the sequences.Slide16

Example: LCS Step 2: A Recursive Solution (continued)Overlapping subproblem structure:

Recurrence for length of optimal solution:

Conditions of problem can exclude some subproblems!

c[i,j]= c[i-1,j-1]+1 if i,j > 0 and x

i

=y

j

max(c[i,j-1], c[i-1,j]) if i,j > 0 and x

i

=y

j

0 if i=0 or j=0

Q

(mn) distinct subproblems

source: 91.503 textbook

Cormen

, et al.Slide17

Example: LCS Step 3: Compute Length of an LCSsource: 91.503 textbook Cormen, et al.

c table

(represent b table)

0

1

2

3

4

What is the asymptotic worst-case time complexity?Slide18

Example: LCS Step 4: Construct an LCS

source: 91.503 textbook Cormen, et al.

8Slide19

Dynamic Programming…leading to a Greedy Algorithm…

Activity SelectionSlide20

Activity Selection Optimization Problem

Problem Instance

:

Set

S

= {a1,a

2,...,an} of n activitiesEach activity i

has:

start time:

s

i

finish time:

f

i

Activities require exclusive use of a common resource.

Activities

i

, j

are compatible

iff non-overlapping: Objective:select a maximum-sized set of mutually compatible activities source: 91.404 textbook Cormen, et al.Slide21

Activity Selection

1

2

3

4

5

6

7

8

9

10

12

11

13

14

15

16

1

2

3

4

8

7

6

5

Activity Time Duration

Activity Number

What is

an answer

in this case?Slide22

Activity Selection

Solution to

S

ij

including

a

k produces 2 subproblems:1)

S

ik

(start after

a

i

finishes; finish before

a

k

starts)

2)

S

kj

(start after ak finishes; finish before aj starts) source: 91.404 textbook Cormen, et al.c[i,j]=size of maximum-size subset of mutually compatible activities in Sij.