Adam Smith Algorithm Design and Analysis L ECTURE 8 Greedy Graph Algs II Implementing Dijkstra MST CSE 565 91010 A Smith based on slides by E Demaine C Leiserson S Raskhodnikova K Wayne ID: 760526
Download Presentation The PPT/PDF document "9/10/10 A. Smith; based on slides by E. ..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Adam Smith
Algorithm Design and Analysis
L
ECTURE 8Greedy Graph Alg’s IIImplementing DijkstraMST
CSE
565
Slide29/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Rough algorithm (Dijkstra)
Maintain a set of explored nodes S whose shortest path distance d(u) from s to u is known.Initialize S = { s }, d(s) = 0.Repeatedly choose unexplored node v which minimizesadd v to S, and set d(v) = (v).
shortest path to some u in explored part, followed by a single edge (u, v)
s
v
u
d(u)
S
(e)
Slide3Review Question
Is Dijsktra’s algorithm correct with negative edge weights?
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Slide49/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Proof of Correctness (Greedy Stays Ahead)
Invariant. For each node u S, d(u) is the length of the shortest path from s to u.Proof: (by induction on |S|)Base case: |S| = 1 is trivial.Inductive hypothesis: Assume for |S| = k 1.Let v be next node added to S, and let (u,v) be the chosen edge.The shortest s-u path plus (u,v) is an s-v path of length (v).Consider any s-v path P. We'll see that it's no shorter than (v).Let (x,y) be the first edge in P that leaves S,and let P' be the subpath to x.P + (x,y) has length · d(x)+ (x,y)· (y)· (v)
S
s
y
v
x
P
u
P'
Slide59/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Implementation
For unexplored nodes, maintainNext node to explore = node with minimum (v).When exploring v, for each edge e = (v,w), updateEfficient implementation: Maintain a priority queue of unexplored nodes, prioritized by (v).
Priority Queue
Slide6Priority queues
Maintain a set of items with priorities (= “keys”)Example: jobs to be performedOperations:InsertIncrease keyDecrease keyExtract-min: find and remove item with least keyCommon data structure: heapTime: O(log n) per operation
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Slide79/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Pseudocode
for Dijkstra(G, )
d[s] 0for each v Î V – {s}do d[v] ¥; p[v] ¥S Q V ⊳ Q is a priority queue maintaining V – S, keyed on [v]
while Q ¹ do u EXTRACT-MIN(Q)S S È {u}; d[u] p[u] for each v Î Adjacency-list[u]do if [v] > [u] + (u, v) then [v] d[u] + (u, v)
explore edges leaving v
Implicit DECREASE-KEY
Slide89/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Analysis of Dijkstra
degree
(
u)times
n
times
Handshaking Lemma
·
m implicit DECREASE-KEY’s.
while
Q ¹ do u EXTRACT-MIN(Q)S S È {u}for each v Î Adj[u]do if [v] > [u] + w(u, v) then [v] [u] + w(u, v)
† Individual ops are amortized bounds
PQ Operation
ExtractMin
DecreaseKey
Binary heap
log n
log n
Fib heap †
log n
1
Array
n
1
Total
m log n
m + n log n
n2
Dijkstra
n
m
d-way Heap
HW3
HW3
m log
m/n
n
Slide99/15/2008
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Minimum spanning tree (MST)
Input: A connected undirected graph G = (V, E) with weight function w : E R. For now, assume all edge weights are distinct.
.
Output:
A
spanning tree
T
— a tree that connects all vertices — of minimum weight:
Slide109/15/2008
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Example of MST
6
12
5
14
3
8
10
15
9
7
Slide119/15/2008
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Example of MST
6
12
5
14
3
8
10
15
9
7
Slide129/15/2008
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Greedy Algorithms for MST
Kruskal's:
Start with T =
. Consider edges in ascending order of weights. Insert edge e in T unless doing so would create a cycle.
Reverse-Delete:
Start with T = E. Consider edges in descending order of weights. Delete edge e from T unless doing so would disconnect T.
Prim's:
Start with some root node s. Grow a tree T from s outward. At each step, add to T the cheapest edge e with exactly one endpoint in T.
Slide139/15/2008
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Cycles and Cuts
Cycle:
Set of edges the form (a,b),(b,c),(c,d),…,(y,z),(z,a). Cut: a subset of nodes S. The corresponding cutset D is the subset of edges with exactly one endpoint in S.
Cycle C = (1,2),(2,3),(3,4),(4,5),(5,6),(6,1)
1
3
8
2
6
7
4
5
Cut S = { 4, 5, 8 }
Cutset D = (5,6), (5,7), (3,4), (3,5), (7,8)
1
3
8
2
6
7
4
5
S
Slide149/15/2008
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Cycle-Cut Intersection
Claim. A cycle and a cutset intersect in an even number of edges.Proof: A cycle has to leave and enter the cut the same number of times.
S
V - S
C
Slide159/15/2008
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Cut and Cycle Properties
Cut property. Let S be a subset of nodes. Let e be the min weight edge with exactly one endpoint in S. Then the MST contains e.Cycle property. Let C be a cycle, and let f be the max weight edge in C. Then the MST does not contain f.
f
C
S
e is in the MST
e
f is not in the MST
Slide169/15/2008
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Proof of Cut Property
Cut property: Let S be a subset of nodes. Let e be the min weight edge with exactly one endpoint in S. Then the MST T* contains e.Proof: (exchange argument)Suppose e does not belong to T*.Adding e to T* creates a cycle C in T*.Edge e is both in the cycle C and in the cutset D corresponding to S there exists another edge, say f, that is in both C and D.T' = T* { e } - { f } is also a spanning tree.Since ce < cf, cost(T') < cost(T*). Contradiction. ▪
f
T*
e
S
Slide179/15/2008
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Proof of Cycle Property
Cycle property: Let C be a cycle in G. Let f be the max weight edge in C. Then the MST T* does not contain f.Proof: (exchange argument)Suppose f belongs to T*.Deleting f from T* creates a cut S in T*.Edge f is both in the cycle C and in the cutset D corresponding to S there exists another edge, say e, that is in both C and D.T' = T* { e } - { f } is also a spanning tree.Since ce < cf, cost(T') < cost(T*). Contradiction. ▪
f
T*
e
S
Slide189/15/2008
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Greedy Algorithms for MST
Kruskal's:
Start with T =
. Consider edges in ascending order of weights. Insert edge e in T unless doing so would create a cycle.
Reverse-Delete:
Start with T = E. Consider edges in descending order of weights. Delete edge e from T unless doing so would disconnect T.
Prim's:
Start with some root node s. Grow a tree T from s outward. At each step, add to T the cheapest edge e with exactly one endpoint in T.
Slide199/15/2008
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Prim's Algorithm: Correctness
Prim's algorithm. [Jarník 1930, Prim 1959]Apply cut property to S.When edge weights are distinct, every edge that isadded must be in the MSTThus, Prim’s alg. outputs the MST
S
Slide209/15/2008
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Correctness of Kruskal
[Kruskal, 1956]: Consider edges in ascending order of weight.Case 1: If adding e to T creates a cycle, discard e according to cycle property.Case 2: Otherwise, insert e = (u, v) into T according to cut property where S = set of nodes in u's connected component.
Case 1
e
v
u
Case 2
e
S
Slide219/15/2008
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Review Questions
Let G be a connected undirected graph with distinct edge weights. Answer true or false:
Let e be the cheapest edge in G. Some MST of G contains e?
Let e be the most expensive edge in G. No MST of G contains e?
Slide229/15/2008
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Review Questions
Let G be a connected undirected graph with distinct edge weights. Answer true or false:
Let e be the cheapest edge in G. Some MST of G contains e?
(Answer:
Yes, by the Cut Property
)
Let e be the most expensive edge in G. No MST of G contains e?
(Answer:
False. Counterexample: if G
is a tree, all its edges are in the MST)
Slide23Non-distinct edges?
9/15/2008
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Slide249/15/2008
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Implementation of Prim(G,w)
IDEA: Maintain V – S as a priority queue Q (as in Dijkstra). Key each vertex in Q with the weight of the least-weight edge connecting it to a vertex in S.
Q Vkey[v] ¥ for all v Î V key[s] 0 for some arbitrary s Î Vwhile Q ¹ do u EXTRACT-MIN(Q)for each v Î Adjacency-list[u]do if v Î Q and w(u, v) < key[v] then key[v] w(u, v) ⊳ DECREASE-KEY p[v] u
At the end,
{(
v
,
p
[
v
])}
forms the MST.
Slide259/15/2008
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Handshaking Lemma
Q(m) implicit DECREASE-KEY’s.
Q
Vkey[v] ¥ for all v Î V key[s] 0 for some arbitrary s Î Vwhile Q ¹ do u EXTRACT-MIN(Q) for each v Î Adj[u]do if v Î Q and w(u, v) < key[v] then key[v] w(u, v) p[v] u
Analysis of Prim
degree
(
u)times
n
times
Q
(
n) total
Time:
as in Dijkstra
Slide269/15/2008
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Handshaking Lemma
Q(m) implicit DECREASE-KEY’s.
while
Q ¹ do u EXTRACT-MIN(Q) for each v Î Adj[u]do if v Î Q and w(u, v) < key[v] then key[v] w(u, v) p[v] u
Analysis of Prim
degree
(
u)times
n
times
† Individual ops are amortized bounds
PQ Operation
ExtractMin
DecreaseKey
Binary heap
log n
log n
Fib heap †
log n
1
Array
n
1
Total
m log n
m + n log n
n2
Prim
n
m
d-way Heap
HW3
HW3
m log
m/n
n
Slide279/15/2008
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Greedy Algorithms for MST
Kruskal's
:
Start with T =
. Consider edges in ascending order of weights. Insert edge
e
in T unless doing so would create a cycle.
Reverse-Delete:
Start with T = E. Consider edges in descending order of weights. Delete edge
e
from T unless doing so would disconnect T.
Prim's:
Start with some root node
s
. Grow a tree T from
s
outward. At each step, add to T the cheapest edge
e
with exactly one endpoint in T.
Slide28Union-Find Data Structures
Operation\ ImplementationArray + linked-lists and sizesBalanced TreesFind (worst-case)ϴ(1)ϴ(log n)Union of sets A,B (worst-case)ϴ(min(|A|,|B|) (could be as large as ϴ(n)ϴ(log n)Amortized analysis: k unions and k finds, starting from singletonsϴ(k log k)ϴ(k log k)
9/15/2008
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
With modifications, amortized time for tree structure is
O(n
Ack(n
)),
where
Ack(n
), the Ackerman function grows much more slowly than log
n
.
See KT Chapter 4.6