Transitive Closure Jeffrey D Ullman Stanford University Infolab Counting Triangles Bounds on Numbers of Triangles Heavy Hitters An Optimal Algorithm Counting Triangles Why Care Density of triangles measures maturity of a community ID: 755505
Download Presentation The PPT/PDF document "Graph Algorithms Counting Triangles" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Graph Algorithms
Counting TrianglesTransitive Closure
Jeffrey D. Ullman
Stanford University/InfolabSlide2
Counting Triangles
Bounds on Numbers of TrianglesHeavy Hitters
An Optimal AlgorithmSlide3
Counting TrianglesWhy Care?Density of triangles measures maturity of a community.As communities age, their members tend to connect.The algorithm is actually an example of a recent and powerful theory of optimal join computation.
3Slide4
Data Structures NeededWe need to represent a graph by data structures that let us do two things efficiently:Given nodes u and v, determine whether there exists an edge between them in O(1) time.
Find the edges out of a node in time proportional to the number of those edges.Question for thought: What data structures would you recommend?
4Slide5
First ObservationsLet the graph have N nodes and M edges.N < M < N2.
One approach: Consider all N-choose-3 sets of nodes, and see if there are edges connecting all 3.An O(N3) algorithm.Another approach: consider all edges e and all nodes u and see if both ends of e have edges to u.
An O(MN) algorithm.Therefore never worse than the first approach.
5Slide6
Heavy HittersTo find a better algorithm, we need to use the concept of a heavy hitter – a node with degree at least M.Note
: there can be no more than 2M heavy hitters, or the sum of the degrees of all nodes exceeds 2M.Impossible because each edge contributes exactly 2 to the sum of degrees.A heavy-hitter triangle is one whose three nodes are all heavy hitters.
6Slide7
Finding Heavy-Hitter TrianglesFirst, find the heavy hitters.Determine the degrees of all nodes.Takes time O(M), assuming you can find the incident edges for a node in time proportional to the number of such edges.Consider all triples of heavy hitters and see if there are edges between each pair of the three.
Takes time O(M1.5), since there is a limit of 2M on the number of heavy hitters.7Slide8
Finding Other TrianglesAt least one node is not a heavy hitter.Consider each edge e.If both ends are heavy hitters, ignore.Otherwise, let end node u not be a heavy hitter.For each of the at most M nodes v connected to u, see whether v is connected to the other end of e.
Takes time O(M1.5).M edges, and at most M work with each.
8Slide9
Optimality of This AlgorithmBoth parts take O(M1.5) time and together find any triangle in the graph.For any N and M, you can find a graph with N nodes, M edges, and (M
1.5) triangles, so no algorithm can do significantly better.Hint: consider a complete graph with M nodes, plus other isolated nodes.Note that M1.5
can never be greater than the running times of the two obvious algorithms with which we began: N3 and MN.
9Slide10
ParallelizationNeeds a constant number of MapReduce rounds, independent of N or M.Count degrees of each node.Filter edges with two heavy-hitter ends.1 or 2 rounds to join only the heavy-hitter edges.
Join the non-heavy-hitter edges with all edges at a non-heavy end.Then join the result of (4) with all edges to see if a triangle is completed.
10Slide11
Transitive Closure
Classical ApproachesArc + Path => Path
Path + Path => Path
“Smart” Transitive ClosureStrongly Connected ComponentsSlide12
Issues Regarding ParallelismDifferent algorithms for the same problem can be parallelized to different degrees.The same activity can (sometimes) be performed for each node in parallel.A relational join or similar step can be performed in one round of MapReduce.Parameters: N = # nodes, M = # edges, D = diameter.
12Slide13
The SettingA directed graph of N nodes and M arcs.Arcs are represented by a relation Arc(u,v) meaning there is an arc from node u to node v.Goal is to compute the transitive closure of Arc, which is the relation Path(
u,v), meaning that there is a path of length 1 or more from u to v.Bad news: TC takes (serial) time O(NM) in the worst case.Good news: But you can parallelize it heavily.
13Slide14
Why Transitive Closure?Important in its own right.Finding structure of the Web, e.g., strongly connected “central” region.Finding connections: “was money ever transferred, directly or indirectly, from the West-Side Mob to the Stanford Chess Club?”
Ancestry: “is Jeff Ullman a descendant of Genghis Khan?”Every linear recursion (only one recursive call) can be expressed as a transitive closure plus nonrecursive stuff to translate to and from TC.
14Slide15
Classical Methods for TC
Warshall’s AlgorithmDepth-First Search
Breadth-First SearchSlide16
Warshall’s AlgorithmPath := Arc;FOR each node u, Path(
v,w) += Path(v,u) AND Path(u,w); /*u is called the pivot */Running time O(N3
) independent of M or D.Can parallelize the pivot step for each u (next slide).But the pivot steps must be executed sequentially, so N rounds of MapReduce are needed.
16Slide17
Parallelizing the Pivot StepA pivot on u is essentially a join of the Path relation with itself, restricted so the join value is always u.Path(v,w) += Path(v,u) AND Path(u,w
).But (ick!) every tuple has the same value (u) for the join attribute.Standard MapReduce join will bottleneck, since all Path facts wind up at the same reducer (the one for key u).17Slide18
Skew JoinsThis problem, where one or more values of the join attribute are “heavy hitters” is called skew.It limits the amount of parallelism, unless you do something clever.But there is a cost: in MapReduce terms, you communicate each Path fact from its mapper to many reducers.As communication is often the bottleneck, you have to be clever how you parallelize when there is a heavy hitter.
18Slide19
Skew Joins – (2)The trick: Given Path(v,u) and Path(u,w) facts:
Divide the values of v into k equal-sized groups.Divide the values of w into k equal-sized groups.Can be the same groups, since v and w range over all nodes.Create a key (reducer) for each pair of groups, one for v and one for w.
Send Path(v,u) to the k reducers for key (g,h), where g is the group of v, and h is any group for w.
Send Path(u,w) to the k reducers for key (g,h), where h is the group of w and g is any group for v.k times the communication, but k2 parallelism
19Slide20
Mapping Path Facts to Reducers20
Path(
v,u
) group 1
Path(
v,u
) group 2
Path(
v,u
) group 3
Path(
u,w
)
group 1
Path(
u,w
)
group 2
Path(
u,w
)
group 3
k = 3
Notice:
every
Path(
v,u
)
meets every
Path(
u,w
) at
exactly one
reducer.Slide21
Depth-First SearchDepth-first search from each node.O(NM) running time.Can parallelize by starting at each node in parallel.But depth-first search is not easily parallelizable.Thus, the equivalent of M rounds of MapReduce needed, independent of N and D.
21Slide22
Breadth-First SearchSame as depth-first, but search breadth-first from each node.Search from each node can be done in parallel.But each search takes only D MapReduce rounds, not M, provided you can perform the breadth-first search in parallel from each node you visit.Similar in performance (if implemented carefully) to “linear TC,” which we will discuss next.
22Slide23
Linear Transitive ClosureLarge-scale TC can be expressed as the iterated join of relations.Simplest case is where weInitialize Path(U,V) = Arc(U,V).
Join an arc with a path to get a longer path, as:Path(U,V) += PROJECTUV(Arc(U,W) JOIN Path(W,V)) or alternatively
Path(U,V) += PROJECTUV
(Path(U,W) JOIN Arc(W,V))Repeat (2) until convergence (requires D iterations).23Slide24
Notation for Join-ProjectJoin-project, as used here is really the composition of relations.Shorthand: we’ll use R(A,B) S(B,C) for PROJECTAC(R(A,B) JOIN S(B,C)).
MapReduce implementation of composition is the same as for the join, except:You exclude the key b from the tuple (a,b,c) generated in the Reduce phase.You need to follow it by a second MapReduce job that eliminates duplicate (a,c
) tuples from the result.24Slide25
Seminaive AlgorithmJoining Path with Arc repeatedly redoes a lot of work.Once I have combined Arc(a,b) with Path(b,c) in one round, there is no reason to do so in subsequent rounds.I already know Path(
a,c).At each round, use only those Path facts that were discovered on the previous round. 25Slide26
Seminaive DetailsPath = ;
NewPath = Arc;while (
NewPath
!= ) {
Path +=
NewPath
;
NewPath
(U,V)=
Arc(U,W)
NewPath
(W,V));
NewPath
-= Path;
}
26Slide27
Example: Seminaive TC27
1
3
4
2
Arc
U
V
1
2
1
3
2
3
2
4
Initial: - 12, 13, 23, 24
Path
NewPath
Path +=
NewPath
12, 13, 23, 24 12, 13, 23, 24
Compute
NewPath
12, 13, 23, 24 13, 14
Subtract Path 12, 13, 23, 24 14
Path +=
NewPath
12, 13, 14, 23, 24 14
Compute
NewPath
12, 13, 14, 23, 24 -
DoneSlide28
Computation Time of SeminaiveEach Path fact is used in only one round.In that round, Path(b,c) is paired with each Arc(a,b
).There can be N2 Path facts.But the average Path fact is composed with M/N Arc facts.To be precise, Path(b,c) is matched with a number of arcs equal to the in-degree of node b.Thus, the total work, if implemented correctly, is O(MN).
28Slide29
How Many Rounds?Each round of seminaive TC requires two MapReduce jobs.One to join, the other to eliminate duplicates.Number of rounds needed equals the diameter.More parallelizable than classical methods (or equivalent to breadth-first search) when
D is small. 29Slide30
Nonlinear Transitive ClosureIf you have a graph with large diameter D, you do not want to run the Seminaive TC algorithm for D rounds.Why? Successive MapReduce jobs are inherently serial.Better approach:
recursive doubling = compute Path(U,V) += Path(U,W) Path(W,V) for log
2(D) number of rounds.After r rounds, you have all paths of length < 2
r.Seminaive works for nonlinear as well as linear.30Slide31
Nonlinear Seminaive DetailsPath =
;NewPath = Arc;
while (NewPath
!= ) {
Path +=
NewPath
;
NewPath
(U,V)=
Path(U,W)
NewPath
(W,V));
NewPath
-= Path;
}
31
Note
: in general,
seminaive
evaluation requires
the “new” tuples to be available for each use of
a relation, so we would need the union with another
term
NewPath
(U,W) o Path(W,V). However, in this case
it can be proved that this one term is enough.Slide32
Computation Time of Nonlinear + SeminaiveEach Path fact is in NewPath only once.There can be N2 Path facts.
When (a,b) is in NewPath, it can be joined with N other Path facts.Those of the form Path(x,a).Thus, total computation is O(N3
).Looks worse than the O(MN) we derived for linear TC.
32Slide33
A Problem With Nonlinear TCGood news: You generate the same Path facts as for linear TC, but in fewer rounds, often a lot fewer.Bad news: you generate the same fact in many different ways, compared with linear.Neither method can avoid the fact that if there are many different paths from u to v, you will discover each of those paths, even though one would be enough.
But nonlinear discovers the same exact path many times.33Slide34
34Example: Linear TC Arc + Path = PathSlide35
35Nonlinear TC Constructs Path + Path = Path in Many WaysSlide36
36Smart TC(Valduriez-Boral, Ioannides) Construct a path from two paths:
The first has a length that is a power of 2.The second is no longer than the first.Slide37
37 Example: Smart TCSlide38
Implementing Smart TCThe trick is to keep two path relations, P and Q.After the i-th round:P(U,V) contains all those pairs (u,v) such that the shortest path from u to v has length
less than 2i.Q(U,V) contains all those pairs (u,v) such that the shortest path from u to v has length exactly 2
i.For the next round:Compute P +=
Q P.Paths of length less than 2i+1.Compute Q = (Q Q) – P.P here is the new value of P; gives you shortest paths of length
exactly 2
i+1
.
38Slide39
Summary of TC Options39
Method
Total (Serial) Computation
Parallel RoundsWarshallO(N3)
O(N)
Depth-First
Search
O(NM)
O(M)
Breadth-First
Search
O(NM)
O(D)
Linear +
Seminaive
O(NM)
O(D)
Nonlinear
+
Seminaive
O(N
3
)
O(log D)
Smart
O(N
3
)
O(log
D)
Seems odd. But in the worst case, almost all shortest paths can have a
length that is a power of 2, so there is no guarantee of improvement for Smart.Slide40
Graphs With Large CyclesIn a sense, acyclic graphs are the hardest TC cases.If there are large strongly connected components (SCC’s) = sets of nodes with a path from any member of the set to any other, you can simplify TC.Example: The Web has a large SCC and other acyclic structures (see Sect. 5.1.3).
The big SCC and other SCC’s made it much easier to discover the structure of the Web. 40Slide41
The Trick: Collapse Cycles FastPick a node u at random.Do a breadth-first search to find all nodes reachable from u.Parallelizable in at most D rounds.Imagine the arcs reversed and do another breadth-first search in the reverse graph.The intersection of these two sets is the SCC containing u.
With luck, that will be a big set.Collapse the SCC to a single node and repeat.41Slide42
TC-Like ApplicationsInstead of just asking whether a path from node u to node v exists, we can attach values to arcs and extend those values to paths.Example: value is the “length” of an arc or path.Concatenate paths by taking the sum.Path(
u,v, x+y) = Arc(u,w
, x)
Path(w,v, y).Combine two paths from u to v by taking the minimum.Similar example: value is cost of transportation.
42