Dependency Analysis Recap - PowerPoint Presentation

1. Dependency Analysis

2. RecapProgram EquivalenceData dependencies across loop iterations

3. Data Dependencies in Nested LoopIteration Numberloop index: ILower and Upper Bound: L and UStep size: Siteration number i = (I – L+1) / SExample:L=1; U=10; S=2;for(int I=L; I<=U; I+=S) { …}Suppose, L= 1, U = 10, I = 7, S =2i = (7 –1 +1)/2 = 4

4. Data Dependencies in Nested LoopIteration VectorConsider a nest of n loops labelled by 1 <= k <=n from the outer to the inner loop. The iteration vector i = {i1, i2, …, in} of a particular iteration of the n-th loop is a vector of integers that contains the iteration numbers for each of the loops in order of nesting level.Example:for(int i=1;i<=2;i++) { // 1 for (int j=1;j<=2;j++) { // 2 S; }}S(2,1) is the instance of statement S in the - 2nd iteration of the 1-loop and - 1st iteration of the 2-loop

5. Data Dependencies in Nested LoopIteration SpaceAll possible iteration vectorsExample:for(int i=1;i<=2;i++) { // 1 for (int j=1;j<=2;j++) { // 2 S; }}{(1,1), (1,2), (2,1), (2,2)}

6. Data Dependencies in Nested Loopik is the kth element of the vector ii[1:k] is a k-vector consisting of the leftmost k elements of iIteration i precedes iteration j, denoted i < j, if and only if1) i[1:n-1] < j[1:n-1] or2) i[1:n-1] = j[1:n-1] and in < jni = (2,1) i1 = 2, i2 = 1i[1:2] = (2,1)Given i = (2,1) and j =(2,2) i < j(2,1) < (2,2)

7. Dependence in Loop NestThere exists a dependence from statement S1 to statement S2 in a common nest of loops if and only if there exist two iteration vectors i and j for the nest, such that (1) i < j or i = j and there is a path from S1 to S2 in the body of the loop, (2) statement S1 accesses memory location M on iteration i and statement S2 accesses location M on iteration j, and(3) one of these accesses is a write.

8. Dependence in Loop NestThere exists a dependence from statement S1 to statement S2 in a common nest of loops if and only if there exist two iteration vectors i and j for the nest, such that (1) i < j or i = j and there is a path from S1 to S2 in the body of the loop, (2) statement S1 accesses memory location M on iteration i and statement S2 accesses location M on iteration j, and(3) one of these accesses is a write.for(int i = 1, i<=N; i++){ // 1 for(int j=1; j<=2; j++){ // 2 S0: A[i][j] = A[i][j] + B } S1: T = A(i,1); S2: A(i,1) = A(i,2); S3: A(i,2) = T;}S0S1S2S3j=1j=1j=2j=2

9. Distance VectorIf there is a dependence from S1 on iteration i to S2 on iteration j; then the dependence distance vector d(i, j) is defined as d(i, j) = j - ifor(int i=1;i<=N;i++){ for(int j=1;j<=M;j++){ for(int k=1; k<=L; k++){ S1: a[i+1,j,k-1] = a[i,j,k]+1; } }}j = (2,2,2) reads a[2][2][2]i = (1,2,3) writes a[2][2][2] d(i,j) = j – i = (1,0,-1)

10. Direction VectorIf there is a dependence from S1 on iteration i and S2 on iteration j; then the dependence direction vector D(i, j) is defined asD(i, j)k = “<” if d(i, j)k > 0“=” if d(i, j)k = 0“>” if d(i, j)k < 0for(int i=1;i<=N;i++){ for(int j=1;j<=M;j++){ for(int k=1; k<=L; k++){ S1: a[i+1,j,k-1] = a[i,j,k]+1; } }}j = (2,2,2) reads a[2][2][2]i = (1,2,3) writes a[2][2][2] d(i,j) = j – i = (1,0,-1)D(i,j) = (<, =, >)

11. Direction Vector and Dependenciesfor(int i=1;i<=N;i++){ for(int j=1;j<=M;j++){ for(int k=1; k<=L; k++){ S1: a[i+1,j,k-1] = a[i,j,k]+1; } }}j = (2,2,2) reads a[2][2][2]i = (1,2,3) writes a[2][2][2] d(i,j) = j – i = (1,0,-1)D(i,j) = (<, =, >)There is a data dependenceA dependence cannot exist if it has a direction vector such that the leftmost non-“=” component is not “<”

12. Direction Vector and Dependenciesfor(int i=1;i<=N;i++){ for(int j=1;j<=M;j++){ for(int k=1; k<=L; k++){ S1: a[i+1,j,k-1] = a[i,j,k]+1; } }}Any anti-dependence?i = (2,2,2) reads a[2][2][2]j = (1,2,3) writes a[2][2][2] d(i,j) = j – i = (-1,0,1)D(i,j) = (>, =, <)No anti-dependenceA dependence cannot exist if it has a direction vector such that the leftmost non-“=” component is not “<”

13. Loop-carried DependencesThere is loop-carried dependence takes place across iterations in a loop.Statement S2 has a loop-carried dependence on statement S1 if and only if S1 references location M on iteration i, S2 references M on iteration j and d(i,j) > 0 (that is, D(i,j) contains a “<” as leftmost non “=” component).The level of a loop-carried dependence is the index of the leftmost non-“=” of D(i,j) for the dependence.for(int i = 1; i<10; i++) for(int j = 1; j<10; j++) for(int k = 1; k<10; k++) S1: A[i][j][k+1] = A[i][j][k];D(i,j) = (=,=,<) for all dependenciesDependencies level = 3

14. Loop Independent DependenciesStatement S2 has a loop-independent dependence on statement S1 if and only if there exist two iteration vectors i and j such that:Statement S1 refers to memory location M on iteration i, S2 refers to M on iteration j, and i = j.(2) There is a control flow path from S1 to S2 within the iteration.for(int i=0;i<N;i++){ S1: t = a[i]; S2: a[i] = b[i]; S3: b[i] = t;}

15. General Condition for Loop Dependency Let α and β be iteration vectors within the iteration space of the following loop nestfor(int i1 = L1; i1<=U1; i1+=S1) for(int i2 = L2; i2<=U2; i2+=S2) ... for(int in = Ln; in<=Un; in+=Sn) S1: A(f1(i1,...,in),...,fm(i1,...,in)) = ... S2 ... = A(g1(i1,...,in),...,gm(i1,...,in)) } ... }}A dependence exists from S1 to S2 if and only if there exist values of α and β such that (1) α is lexicographically less than or equal to β and(2) the following system of dependence equations is satisfied:fi(α) = gi(β) for all i, 1 ≤ i ≤ m

16. ReferencesChapter 2Optimizing compilers for modern architectures a dependence-based approach by Randy Allen, Ken Kennedy