Introduction to Algorithms: - PowerPoint Presentation

Introduction to Algorithms:Greedy Algorithms (and Graphs)

Graphs ( R eview) Definition. A directed graph (digraph)G = (V, E) is an ordered pair consisting ofa set V of vertices (singular: vertex),a set E  V  V of edges.In an undirected graph G = (V, E), the edge set E consists of unordered pairs of vertices.In either case, we have | E | = O(V 2). Moreover, if G is connected, then | E |  | V | – 1, which implies that lg | E | = (lg V). CS 421 - Introduction to Algorithms 2

Adjacency- M atrix Representation The adjacency matrix of a graph G = (V, E), where V = {1, 2, …, n} is the matrix A[1 ... n, 1 . . n] given byA[i, j] =1 if (i, j)  E,0 if (i, j)  E.CS 421 - Introduction to Algorithms3

Adjacency- M atrix Representation The adjacency matrix of a graph G = (V, E), where V = {1, 2, …, n} is the matrix A[1 ... n, 1 . . n] given byA[i, j] =1 if (i, j)  E,0 if (i, j)  E.CS 421 - Introduction to Algorithms4  ( V 2 ) storage  dense representation. A 1 2 3 4 2 1 1 0 1 1 0 2 0 0 1 0 3 4 3 0 0 0 0 4 0 0 1 0

Adjacency- L is t RepresentationAn adjacency list of a vertex v  V is the list Adj[v] of vertices adjacent to v.Adj[1] = {2, 3}Adj[2] = {3}Adj[3] = {}Adj[4] = {3}2 1 3 4 CS 421 - Introduction to Algorithms 5

Adjacency- L is t RepresentationAn adjacency list of a vertex v  V is the list Adj[v] of vertices adjacent to v.Adj[1] = {2, 3}Adj[2] = {3}Adj[3] = {}Adj[4] = {3}2 1 3 4 CS 421 - Introduction to Algorithms 6 For undirected graphs, | Ad j [ v ] | = degre e ( v ) . Fo r digraphs, | Ad j [ v ] | = out-degree(v).

Adjacency- L is t RepresentationAn adjacency list of a vertex v  V is the list Adj[v] of vertices adjacent to v.Adj[1] = {2, 3}Adj[2] = {3}Adj[3] = {}Adj[4] = {3}2 1 3 4 7 For undirected graphs, | Ad j [ v ] | = degre e ( v ) . Fo r digraphs, | Ad j [ v ] | = out-degre e(v).Handshaking Lemma: vV = 2 | E | for undirected graphs  adjacency lists use  ( V + E ) storage — a sparse representation (for either type of graph).

Minimum S panning Trees (MSTs)Input: A connected, undirected graph G = (V, E) with weight function w : E  .For simplicity, assume that all edge weights are distinct.  CS 421 - Introduction to Algorithms8

Minimum S panning Trees (MSTs)Input: A connected, undirected graph G = (V, E) with weight function w : E  .For simplicity, assume that all edge weights are distinct.Output: A spanning tree T — a tree that connects all vertices — of minimum weight:  CS 421 - Introduction to Algorithms9w(T )  w(u, v) .(u,v)T

Example of MST 6 12 5 14 3810 9 15 7CS 421 - Introduction to Algorithms10

Example of MST 6 12 5 14 3810 9 15 7CS 421 - Introduction to Algorithms11

Optimal S ubstructure MST T : (Othe r edges o f G ar e no t shown.) CS 421 - Introduction to Algorithms 12

Optimal S ubstructure MST T : CS 421 - Introduction to Algorithms 13 u v Remove any edge ( u , v )  T .

Optimal S ubstructure MST T : CS 421 - Introduction to Algorithms 14 u v Then, T is partitioned into two subtree s T 1 and T 2 .

Optimal S ubstructure MST T : CS 421 - Introduction to Algorithms 15 u v Theorem. The subtree T 1 is an MS T of G 1 = ( V 1 , E 1 ) , the subgraph of G induced by the vertices of T1: V1 = vertices of T1,E1 = { (x , y)  E : x, y  V1 } . Similarl y fo r T 2 .

Proof of Optimal Substructure Proof. Cut and paste:w(T) = w(u, v) + w(T1) + w(T2).If T1 were a lower-weight spanning tree than T1 for G1, then T  = {(u, v)}  T1  T2 would be alower-weight spanning tree than T for G. CS 421 - Introduction to Algorithms16

Proof of Optimal Substructure Proof. Cut and paste:w(T) = w(u, v) + w(T1) + w(T2).Do we also have overlapping sub-problems?•Yes.Great, then dynamic programming may work!•Yes, but MST exhibits another powerful property which leads to an even more efficient algorithm.CS 421 - Introduction to Algorithms17

Hallmark for “Greedy” Algorithms Greedy-choice property A locally optimal choice is globally optimal.CS 421 - Introduction to Algorithms18

Hallmark for “Greedy” Algorithms Greedy-choice property A locally optimal choice is globally optimal.Theorem. Let T be the MST of G = (V, E), and let A  V. Suppose that (u, v)  E is the least-weight edge connecting A to V – A. Then, (u, v)  T.CS 421 - Introduction to Algorithms19

Proof o f Theorem  A V – Au(u, v) = least-weight edge connecting A to V – AProof. Suppose (u, v)  T. Cut and paste.T: vCS 421 - Introduction to Algorithms20

Proof o f Theorem  A V – Au(u, v) = least-weight edge connecting A to V – AProof. Suppose (u, v)  T. Cut and paste.T: vCS 421 - Introduction to Algorithms21Consider the unique simple path from u to v in T.Swap (u, v) with the first edge on this path that connects a vertex in A to a vertex in V – A.A lighter-weight spanning tree than T results .

Prim’s A lgorithm IDEA: Maintain V – A as a priority queue Q. Key each vertex in Q with the weight of the least- weight edge connecting it to a vertex in A. Q  V key[v]   for all v  V key[s]  0 for some arbitrary s  V while Q   do u  EXTRACT-MIN(Q) for each v  Adj[u] do if v  Q and w(u, v) < key[v] then key[v]  w(u, v )  [ v ]  u ⊳ D ECREAS E -K EY At the end, { ( v , [v])} forms the MST.22

Example of Prim’s A lgorithm  A V – A 6 12 5 14 3 8 10 9 15 7 CS 421 - Introduction to Algorithms 23      

Example of Prim’s A lgorithm  A V – A 6 12 5 14 3 8 10 9 15 7 CS 421 - Introduction to Algorithms 24      0 s

Example of Prim’s A lgorithm  A V – A10 6 12 5 14 3 8 10 9 15 7 CS 421 - Introduction to Algorithms 25  7  15  0

Example of Prim’s A lgorithm  A V – A10 6 12 5 14 3 8 10 9 15 7 CS 421 - Introduction to Algorithms 26   15  0 7

Example of Prim’s A lgorithm  A V – A10 6 12 5 14 3 8 10 9 15 7 CS 421 - Introduction to Algorithms 27 5 9 15 12 0 7

Example of Prim’s A lgorithm  A V – A10 6 12 5 14 3 8 10 9 15 7 CS 421 - Introduction to Algorithms 28 9 15 12 0 7 5

Example of Prim’s A lgorithm  A V – A14 8 6 12 5 14 3 8 10 9 15 7 CS 421 - Introduction to Algorithms 29 9 15 0 7 5 6

Example of Prim’s A lgorithm  A V – A14 6 12 5 14 3 8 10 9 15 7 CS 421 - Introduction to Algorithms 30 9 15 0 7 5 6 8 8

Example of Prim’s A lgorithm  A V – A 3 6 12 5 14 3 8 10 9 15 7 CS 421 - Introduction to Algorithms 31 9 15 0 7 5 6 8

Example of Prim’s A lgorithm  A V – A6 12 5 14 3 8 10 9 15 7 CS 421 - Introduction to Algorithms 32 9 15 0 7 5 6 8 3

Example of Prim’s A lgorithm  A V – A6 12 5 143 8 10 9 15 7 CS 421 - Introduction to Algorithms 33 15 0 7 5 6 8 3 9

Example of Prim’s A lgorithm  A V – A6 12 5 143810 9 15 7 CS 421 - Introduction to Algorithms 34 0 7 5 6 8 3 9 15

Analysi s of Prim ‘s AlgorithmQ  Vkey[v]   for all v  Vkey[s]  0 for some arbitrary s  Vwhile Q  do u  EXTRACT-MIN(Q)for each v  Adj[u]do if v  Q and w(u, v) < key[v]then key[v]  w(u, v)[v]  uCS 421 - Introduction to Algorithms35

Analysi s of Prim‘s AlgorithmQ  Vkey[v]   for all v  Vkey[s]  0 for some arbitrary s  Vwhile Q  do u  EXTRACT-MIN(Q)for each v  Adj[u]do if v  Q and w(u, v) < key[v]then key[v]  w(u, v)[v]  uCS 421 - Introduction to Algorithms36(V)total|V |times degree ( u ) times Handshakin g Lemm a   ( E ) implici t D ECREASE - KEY’s.

Analysi s of Prim ‘s AlgorithmQ  Vkey[v]   for all v  Vkey[s]  0 for some arbitrary s  Vwhile Q  do u  EXTRACT-MIN(Q)for each v  Adj[u]do if v  Q and w(u, v) < key[v]then key[v]  w(u, v)[v]  uCS 421 - Introduction to Algorithms37(V)total|V |timesdegree(u) times Tim e =  ( V )· T E XTRAC T - M IN +  ( E )· T DECREASE-KEY

Analysi s of Prim ( cont'd)Time = (V)·TEXTRACT-MIN + (E)·TDECREASE-KEYTotalQCS 421 - Introduction to Algorithms38TEXTRACT-MIN TDECREASE-KEY O(V) O(1)O(V2)arraybinary heapFibonacci heapO(lg V)O(lg V) amortizedO(lg V)O(1)amortizedO(E lg V) O(E + V lg V)worst case

MST A lgorithms Kruskal’ s algorithm (see CLRS):Uses the disjoint-set data structure Running time = O(E lg V).Best to date:Karger, Klein, and Tarjan [1993].Randomized algorithm.O(V + E) expected time.CS 421 - Introduction to Algorithms39