Dependency Analysis Recap Program Equivalence Data

slide1. Dependency Analysis<br>
slide2. Recap Program Equivalence

Data dependencies across loop iterations<br>
slide3. Data Dependencies in Nested Loop Iteration Number loop index: I
Lower and Upper Bound: L and U
Step size: S

iteration number i = (I – L+1) / S Example:

L=1; U=10; S=2;
for(int I=L; I<=U; I+=S) {
…
} Suppose, L= 1, U = 10, I = 7, S =2

i = (7 –1 +1)/2 = 4<br>
slide4. Data Dependencies in Nested Loop Iteration Vector Consider a nest of n loops labelled by 1 <= k <=n from the outer to the inner loop.

The iteration vector i = {i1, i2, …, in} of a particular iteration of the n-th loop
is a vector of integers that contains the iteration numbers for each of the loops in order of nesting level. Example:

for(int i=1;i<=2;i++) { // 1
for (int j=1;j<=2;j++) { // 2
S;
}
} S(2,1) is the instance of statement S in the
- 2nd iteration of the 1-loop and
- 1st iteration of the 2-loop<br>
slide5. Data Dependencies in Nested Loop Iteration Space All possible iteration vectors Example:

for(int i=1;i<=2;i++) { // 1
for (int j=1;j<=2;j++) { // 2
S;
}
} {(1,1), (1,2), (2,1), (2,2)}<br>
slide6. Data Dependencies in Nested Loop ik is the kth element of the vector i

i[1:k] is a k-vector consisting of the leftmost k elements of i

Iteration i precedes iteration j,
denoted i < j, if and only if

1) i[1:n-1] < j[1:n-1] or

2) i[1:n-1] = j[1:n-1] and in < jn i = (2,1)

i1 = 2, i2 = 1

i[1:2] = (2,1)

Given i = (2,1) and j =(2,2)

i < j
(2,1) < (2,2)<br>
slide7. Dependence in Loop Nest There exists a dependence from statement S1 to statement S2 in a common nest of loops if and only if
there exist two iteration vectors i and j for the nest, such that

(1) i < j or i = j and there is a path from S1 to S2 in the body of the loop,

(2) statement S1 accesses memory location M on iteration i and statement S2 accesses location M on iteration j, and

(3) one of these accesses is a write.<br>
slide8. Dependence in Loop Nest There exists a dependence from statement S1 to statement S2 in a common nest of loops if and only if
there exist two iteration vectors i and j for the nest, such that
(1) i < j or i = j and there is a path from S1 to S2 in the body of the loop,
(2) statement S1 accesses memory location M on iteration i and statement S2 accesses location M on iteration j, and
(3) one of these accesses is a write. for(int i = 1, i<=N; i++){ // 1
for(int j=1; j<=2; j++){ // 2
S0: A[i][j] = A[i][j] + B
}
S1: T = A(i,1);
S2: A(i,1) = A(i,2);
S3: A(i,2) = T;
} S0 S1 S2 S3 j=1 j=1
j=2 j=2<br>
slide9. Distance Vector If there is a dependence from S1 on iteration i to S2 on iteration j;
then the dependence distance vector d(i, j) is defined as d(i, j) = j - i for(int i=1;i<=N;i++){
for(int j=1;j<=M;j++){
for(int k=1; k<=L; k++){
S1: a[i+1,j,k-1] = a[i,j,k]+1;
}
}
} j = (2,2,2) reads a[2][2][2]

i = (1,2,3) writes a[2][2][2]

d(i,j) = j – i = (1,0,-1)<br>
slide10. Direction Vector If there is a dependence from S1 on iteration i and S2 on iteration j;
then the dependence direction vector D(i, j) is defined as D(i, j)k = “<” if d(i, j)k > 0“=” if d(i, j)k = 0“>” if d(i, j)k < 0 for(int i=1;i<=N;i++){
for(int j=1;j<=M;j++){
for(int k=1; k<=L; k++){
S1: a[i+1,j,k-1] = a[i,j,k]+1;
}
}
} j = (2,2,2) reads a[2][2][2]

i = (1,2,3) writes a[2][2][2]

d(i,j) = j – i = (1,0,-1)

D(i,j) = (<, =, >)<br>
slide11. Direction Vector and Dependencies for(int i=1;i<=N;i++){
for(int j=1;j<=M;j++){
for(int k=1; k<=L; k++){
S1: a[i+1,j,k-1] = a[i,j,k]+1;
}
}
} j = (2,2,2) reads a[2][2][2]

i = (1,2,3) writes a[2][2][2]

d(i,j) = j – i = (1,0,-1)

D(i,j) = (<, =, >)

There is a data dependence A dependence cannot exist if it has a direction vector such that the leftmost non-“=” component is not “<”<br>
slide12. Direction Vector and Dependencies for(int i=1;i<=N;i++){
for(int j=1;j<=M;j++){
for(int k=1; k<=L; k++){
S1: a[i+1,j,k-1] = a[i,j,k]+1;
}
}
} Any anti-dependence?

i = (2,2,2) reads a[2][2][2]

j = (1,2,3) writes a[2][2][2]

d(i,j) = j – i = (-1,0,1)

D(i,j) = (>, =, <)

No anti-dependence A dependence cannot exist if it has a direction vector such that the leftmost non-“=” component is not “<”<br>
slide13. Loop-carried Dependences There is loop-carried dependence takes place across iterations in a loop. Statement S2 has a loop-carried dependence on statement S1 if and only if

S1 references location M on iteration i, S2 references M on iteration j and

d(i,j) > 0 (that is, D(i,j) contains a “<” as leftmost non “=” component).

The level of a loop-carried dependence is the index of the leftmost non-“=” of D(i,j) for the dependence. for(int i = 1; i<10; i++)
for(int j = 1; j<10; j++)
for(int k = 1; k<10; k++)
S1: A[i][j][k+1] = A[i][j][k]; D(i,j) = (=,=,<) for all dependencies

Dependencies level = 3<br>
slide14. Loop Independent Dependencies Statement S2 has a loop-independent dependence on statement S1 if and only if there exist two iteration vectors i and j such that:

Statement S1 refers to memory location M on iteration i,
S2 refers to M on iteration j, and i = j.
(2) There is a control flow path from S1 to S2 within the iteration. for(int i=0;i<N;i++){
S1: t = a[i];
S2: a[i] = b[i];
S3: b[i] = t;
}<br>
slide15. General Condition for Loop Dependency Let α and β be iteration vectors within the iteration space of the following loop nest

for(int i1 = L1; i1<=U1; i1+=S1)
for(int i2 = L2; i2<=U2; i2+=S2)
...
for(int in = Ln; in<=Un; in+=Sn)
S1: A(f1(i1,...,in),...,fm(i1,...,in)) = ...
S2 ... = A(g1(i1,...,in),...,gm(i1,...,in))
}
...
}
}

A dependence exists from S1 to S2 if and only if there exist values of α and β such that
(1) α is lexicographically less than or equal to β and
(2) the following system of dependence equations is satisfied:

fi(α) = gi(β) for all i, 1 ≤ i ≤ m<br>
slide16. References Chapter 2
Optimizing compilers for modern architectures
a dependence-based approach
by Randy Allen, Ken Kennedy<br>