Lower Bounds - PowerPoint Presentation

Slide1

Lower Bounds in Greedy Model

Sashka Davis

Advised by Russell Impagliazzo

(Slides modified by Jeff)

UC San Diego

October 6, 2006Slide2

Suppose you have to solve a problem Π…

Is there a Greedy algorithm that solves

Π

?

Is there a Backtracking algorithm that solves Π?

Is there a Dynamic Programming algorithm that solves Π?

Eureka! I have a DP Algorithm!

No Backtracking

agl.

exists? Or I didn’t think of one?

Is my DP algorithm optimal or a better one exists?

No Greedy

alg.

exists

? Or I didn’t think of one?Slide3

Suppose we a have formal model of each algorithmic paradigm

Is there a Greedy algorithm that solves

Π

?

No Greedy

algorithm can solve Π exactly.

Is there a Backtracking algorithm that solves Π?

No Backtracking algorithm can solve Π

exactly.

Is there a Dynamic Programming alg. that solves Π?

DP helps!

Is my algorithm optimal, or a better DP algorithm exists?

Yes, it is! Because NO DP alg. can solve

Π

more efficiently.Slide4

The goalTo build a formal model

of each of the basic algorithmic design paradigms which should

capture the strengths

of the paradigm.To develop lower bound technique, for each formal model, that can prove negative results for all algorithms in the class.Slide5

Using the framework we can answer the following questions

1. When solving problems exactly:

What algorithmic design paradigm can help?

No algorithm within a given formal model can solve the problem exactly.We find an algorithm that fits a given formal model.2. Is a given algorithm optimal?

Prove a lower bound matching the upper bound for all algorithms in the class.3. Solving the problems approximately:What algorithmic paradigm can help?Is a given approximation scheme optimal within the formal model?Slide6

Some of our results

ADAPTIVE

PRIORITY

FIXED

Greedy

Backtracking & Simple DP

(tree)

Dynamic

Programming

pBT

pBP

OnlineSlide7



is a set of data items;



is a set of options

Input

: instance I={1 ,2 ,…,n }, I

 Output: solution S={(i , i) | i= 1,2,…,d}; i 

 1. Order: Objects arrive in worst case order chosen by adversary.2. Loop considering 

i in order.Make a irrevocable decision i  

On-line algorithmsSlide8



is a set of data items;



is a set of options

Input

: instance I={1 ,2 ,…,n }, I

 Output: solution S={(i , i) | i= 1,2,…,d}; i 

 1. Order: Algorithm chooses fixed π : 

→R+ without looking at I. 2. Loop considering 

i in order.Make a irrevocable decision i   Fixed priority algorithmsSlide9



is a set of data items;



is a set of options

Input

: instance I={1 ,2 ,…,n }, I

 Output: solution S={(i , i) | i= 1,2,…,d}; i 

 2. Loop - Order: Algorithm reorders π

: →R+ without looking at rest of I. - Considering next

i in current order.Make a irrevocable decision i  

Adaptive priority algorithmsSlide10



is a set of data items;



is a set of options

Input

: instance I={1 ,2 ,…,n }, I

 Output: solution S={(i , i) | i= 1,2,…,d}; i 

 1. Order: Algorithm chooses π : →

R+ without looking at I. 2. Loop considering i

in order.Make a set of decisions i   (one of which will be the final decision.)

Fixed priority “Back Tracking”Slide11

Some of our results

PRIORITY

pBT

pBP

ADAPTIVE

PRIORITY

FIXED

Shortest Path in negative graphs no cycles

Bellman-Ford

Shortest Path in no-negative graphs

Dijkstra’s

Online

Maximum Matching

in Bipartite graphs

Flow Algorithms

Maximum Matching

in Bipartite graphs

Minimum Spanning Tree

Prim’s

Kruskal’s

Kruskal’sSlide12

Some of our results

PRIORITY

pBT

pBP

ADAPTIVE

PRIORITY

FIXED

Dijkstra’s

Shortest Path in no-negative graphs

Online

Prim’s

Kruskal’s

Minimum Spanning Tree

Kruskal’sSlide13

Kruskal algorithm for MST is a Fixed priority algorithm

Input (G=(V,E),

ω

: E →R)Initialize empty solution TL = Sorted list of edges in non-decreasing order according to their weight

while (L is not empty)e = next edge in LAdd the edge to T, as long as T remains a forest and remove e from LOutput TSlide14

Prim’s algorithm Input G=(V,E), w: E →R

Initialize an empty tree T ←

; S

← Pick a vertex u; S={u}; for (i=1 to |V|-1)(u,v) = min(u,v)

cut(S, V-S)w(u,v)S←S  {v}; T←T{(u,v)}Output TPrims algorithm for MST

is an adaptive priority algorithmSlide15

Dijkstra’s Shortest Paths Alg is an adaptive priority algorithm

Dijkstra algorithm

(G=(V,E), s

 V)T←∅; S←{s};Until (S≠V)Find e=(u,x) | e = mineCut(S, V-S){path(s, u)+ω(e)}T← T{e}; S

← S {x}Slide16

Some of our results

PRIORITY

pBT

pBP

ADAPTIVE

PRIORITY

FIXED

Dijkstra’s

Shortest Path in no-negative graphs

Online

Prim’s

Kruskal’s

Minimum Spanning Tree

Kruskal’sSlide17

ShortPath Problem: Given a graph G=(V,E), ω

: E →R

+

; s, t V. Find a directed tree of edges,rooted at s, such that the combined weight of thepath from s to t is minimal

Theorem

:

No Fixed priority algorithm can achieve any constant approximation ratio for the ShortPath problemSome of our results

Data items are edges of the graph

Decision options = {accept, reject}Slide18

Fixed priority game

Solver

Adversary

γ

d

γ

i

γ

3

γ

j

γ

k

γ

2

γ

1

Γ

0

S_sol = {(

γ

i2

,

σ

i2

)}

σ

i2

S_sol = {(

γ

i2

,

σ

i2

),

(

γ

i4,σi4)}

γ

i2

γi9,…

γi1

γi3

γi4

γi5

γ

i6

γ

i7

γ

i8

Γ

0

Γ

1

Γ

2

σ

i4

Γ

3

End Game

S_adv = {(

γ

i2

,

σ

*

i2

),

(

γ

i4

,

σ

*

i4

)

}

Solver is awarded

=∅Slide19

Adversary selects 0

t

b

s

a

u(k)

w(k)

x(1)

v(1)

y(1)

z(1)Slide20

Solver selects an order on 0

If then the Adversary presents:

t

b

s

a

u(k)

w(k)

x(1)

v(1)

y(1)

z(1)Slide21

Adversary’s strategyWaits until Solver considers edge

y(1)

Solver will consider

y(1) before z(1)

Event 1

σy

=acceptEvent 2σ

y=rejectSlide22

Event 1: Solver accepts y(1)

t

u(k)

x(1)

y(1)

z(1)

b

a

s

The Solver constructs a path {u,y}

The Adversary outputs solution {x,z}Slide23

Event 2: Solver rejects y(1)

The Solver fails to construct a path.

The Adversary outputs a solution {u,y}.

t

u(k)

x(1)

y(1)

z(1)

b

a

sSlide24

The outcome of the game:

The Solver either fails to output a solution or achieves an approximation ratio

(k+1)/2

The Adversary can set k arbitrarily large and thus can force the Algorithm to claim arbitrarily large approximation ratioSlide25

Some of our results

PRIORITY

pBT

pBP

ADAPTIVE

PRIORITY

FIXED

Dijkstra’s

Shortest Path in no-negative graphs

OnlineSlide26

Some of our results

PRIORITY

pBT

pBP

ADAPTIVE

PRIORITY

FIXED

Interval Scheduling

value is width

Factor of 3

Online

Factor

of 3Slide27

Interval scheduling on a single machineInstance

:

Set of intervals I=(i

1, i2,…,in), j ij=[rj, dj]Problem: schedule intervals on a single machineSolution

: S  IObjective function: maximize iS(dj - rj)Slide28

A simple solution (LPT)Longest Processing Time algorithm

input I=(i

1, i2,…,in) Initialize S ← Sort the intervals in decreasing order (dj – r

j)while (I is not empty)Let ik be the next in the sorted orderIf ik can be scheduled then S ← S U {ik};I ← I \ {ik

}Output SSlide29

LPT is a 3-approximation

LPT sorts the intervals in decreasing order according to their length

3 LPT

≥ OPT

OPTOPT

OPT

LPT

r

i

diSlide30

Example lower bound [BNR02]Theorem1:

No

adaptive

priority algorithm can achieve an approximation ratio better than 3 for the interval scheduling problem with proportional profit for a single machine configurationSlide31

Proof of Theorem 1Adversary’s move

Algorithm’s move: Algorithm selects an ordering

Let i be the interval with highest priority

1

2

3q

q-1

q-1

1

2

3

eSlide32

Adversary’s strategyIf Algorithm decides

not

to schedule

i During next round Adversary removes all remaining intervals and schedules interval i

12

3i

j

k

1

2

3

i

Alg’s value = 0

Adv’s value = iSlide33

Adversary’s strategyIf

i

= and Algorithm schedules

iDuring next round the Adversary restricts the sequence:i

j

k

1

2

3

i

i

i+1

i-1

Alg’s value = i

Adv’s value = (i-1)+3(i/3)+(i+1)=3iSlide34

Adversary’s strategyIf

i

= and Algorithm schedules

iDuring next round the Adversary restricts the sequence:1

Alg’s value = 1Adv’s value = 3(1/3)+(2)=3

12

3

ij

k

1

2

3

1

2Slide35

Adversary’s strategyIf

i

= and Algorithm schedules

iDuring next round the Adversary restricts the sequence:1

2

3i

jk

1

2

3

q

q

q-1

q-1

Alg’s value = q

Adv’s value = (q-1)+3(q/3)+(q-1)=3q-1

But q is bigSlide36

Adversary’s strategyIf

i

= and Algorithm schedules

iDuring next round Adversary restricts the sequence:1

2

3m

jk

1

2

3

i

i

m

Alg’s value = i

Adv’s value = (3i) =3iSlide37

PRIORITY

pBT

pBP

ADAPTIVE

PRIORITY

FIXED

Interval Scheduling

value is width

Factor of 3

Online

Factor

of 3

The algorithm was missed up before

it got a chance to reorder things.

?

Some of our resultsSlide38

Some of our results

PRIORITY

pBT

pBP

ADAPTIVE

PRIORITY

FIXED

Weighted

Vertex Cover

Factor of 2

OnlineSlide39

[Joh74] greedy 2-approximation for WVC

Input

: instance

G with weights on nodes.Output: solution S  V covers all edges and minimizes weight taken nodes.Repeat until all edges covered.Take v minimizing

ω(v)/(# uncovered adj edges)

Weighted Vertex Cover Slide40

With Shortest Path,a data item is an edge of the graph

 = (<u,v>,

ω(<u,v>) ) With weighted vertex cover,A data item is a vertex of the graph  = (v, ω

(v), adj_list(v))(Stronger than having the items be edges,because the alg gets more info from nodes.

Weighted Vertex Cover

Theorem

:

No Adaptive priority algorithm can

achieve an approximation ration better than 2Slide41

Adaptive priority game

Solver

Adversary

γ

3

γ

5

γ

6

γ

1

γ

4

γ

7

γ

2

S_sol = {(

γ

7

,

σ

7

)}

σ

4

S_sol = {(

γ

7

,

σ

7

),

(

γ

4

,

σ4)}Γ3

Γ1

Γ

2

σ7

The Game Ends

:S_adv = {(

γ7,σ*7

),

(

γ

4

,

σ

*

4

),(

γ

2

,

σ

*

2

)

}

Solver is awarded payoff

f

(S_sol)/

f

(S_adv)

γ

8

γ

9

γ

10

γ

11

γ

12

Γ

0

σ

2

S_sol = {(

γ

7

,

σ

7

),

(

γ

4

,

σ

4

),(

γ

2

,

σ

2

)

}

Slide42

The Adversary chooses instances to be graphs Kn,n

The weight function

ω

:V→ {1, n2}

n

2

1

n

2

n

2

n

2

1

1

1Slide43

The gameData items

each node appears in



o as two separate data items with weights 1, n2 Solver movesChoses a data item, and commits to a decisionAdversary moveRemoves from the next 

t the data item, corresponding to the node just committed and.. Slide44

Adversary’s strategy is to wait unitl

Event 1

: Solver

accepts a node of weight n2 Event 2: Solver rejects a node of any weightEvent 3: Solver has committed to all but one nodes on either side of the bipartite

1

1

1

1

1Slide45

Event 1: Solver accepts a node ω

(v)=n

2

The Adversary chooses part B of the bipartite as a cover, and incurs cost

n

The cost of a cover for the Solver is at least

n2+n-1

1

1

n

2

1

1

1

1Slide46

Event 2: Solver rejects a node of any weight

The Adversary chooses part A of the bipartite as a cover.

The Solver must choose part B of the bipartite as a cover.

n

2

n

2Slide47

Event 3: Solver commits to n-1 nodes w(v)=1, on either side of Kn,n

The Adversary chooses part B of the bipartite as a cover, and incurs cost

n

The cost of a cover for the Solver is

2n-1

1

1

1

1

1

1

1

n

2Slide48

Some of our results

PRIORITY

pBT

pBP

ADAPTIVE

PRIORITY

FIXED

Weighted

Vertex Cover

Factor of 2

OnlineSlide49

Some of our results

PRIORITY

pBT

pBP

ADAPTIVE

PRIORITY

FIXED

Facility Location

Factor of logn

OnlineSlide50

Facility location problem

Instance

is a set of cities and set of facilities

The set of cities is C={1,2,…,n}Each facility fi has an opening cost cost(fi) and connection costs for each city: {ci1, c

i2,…, cin}Problem: open a collection of facilities such that each city is connected to at least one facilityObjective function: minimize the opening and connection costs min(ΣfS

cost(fi) + ΣjCmin fiScij ) Slide51

[AB02] resultTheorem:

No adaptive priority algorithm can achieve an approximation ratio better than

log

(n) for facility location in arbitrary spacesSlide52

Adversary presents the instance:

Cities

: C={1,2,…,n}, where

n=2kFacilities: Each facility has opening cost nCity connection costs are 1 or ∞ Each facility covers exactly

n/2 citiescover(fj) = {i | i  C,cji=1} Cu denotes the set of cities not yet covered by the solution of the AlgorithmSlide53

Adversary’s strategy

At the beginning of each round

t

The Adversary chooses St to consist of facilities f such that fSt

iff |cover(f) ∩ Cu| = n/(2t)The number of uncovered cities Cu is

n/(2t-1)Two facilities are complementary if together they cover all cities in C. For any round t St consists of complementary facilitiesSlide54

The game

Uncovered cities

C

uSlide55

End of the gameEither Algorithm opened log(n) facilities or failed to produce a valid solution

Cost of Algorithm’s solution is n.log(n)+n

Adversary opens two facilities incurs total cost 2n+nSlide56

Some of our results

PRIORITY

pBT

pBP

ADAPTIVE

PRIORITY

FIXED

Facility Location

Factor of logn

Online