and '
Transformations Dependences
$
&
1
%
'
Recall: Polyhedral algebra tools for determining emptiness of convex polyhedra enumerating integers in such a polyhedron. Central ideas: reduction of matrices to echelon form by unimodular column operations, Fourier-Motzkin elimination
$
&
Let us use these tools to determine (i) legality of permutation and (ii) generation of transformed code.
2
%
'
Organization of lecture: Using ILP to generate transformed code for loop permutation What is a dependence? Dependence abstractions (summaries): distance/direction Computing dependence abstractions using ILP How to avoid calling the ILP calculator: ZIV,SIV subscripts and separability GCD test Caching of results
$
&
3
%
' &
Loop permutation can be modeled as a linear transformation on iteration space:
J V
$
I 0 1 1 0 I U = J V
U
DO I = 1, N DO J = I,N X(I,J) = 5
DO U = 1, N DO V = 1,U X(V,U) = 5
Permutation of loops in n-loop nest: nxn permutation matrix P PI =U
Questions: (1) How do we generate new loop bounds? (2) How do we modify the loop body? (3) How do we know when loop interchange is legal?
4
%
' &
Code Generation for Transformed Loop Nest Two problems: (1) Loop bounds (2) Change of variables in body (1) New bounds: Original bounds: A I b where A is in echelon form Transformation: U = T I Note: for loop permutation, T is a permutation matrix => inverse is integer matrix So bounds on U can be written as A T ;1 U b Perform Fourier-Motzkin elimination on this system of inequalities to obtain bounds on U. (2) Change of variables: I = T ;1 U Replace old variables by new using this formula
5
$
%
'
J
Example:
V
$
I 01 10 I U = J V U DO U = 1, N DO V = 1,U X(V,U) = 5 Fourier-Motzkin elimination < -1 N 0 N -1 0 10 1 -1 01 01 10 U V < -1 N 0 N
DO I = 1, N DO J = I,N X(I,J) = 5
&
-1 0 10 1 -1 01
I J
6
%
'
-1 0 10 1 -1 01 01 10 0 -1 01 -1 1 10 U V < 1< U < N
U V
$
< -1 N 0 N
-1 N 0 N
Projecting out V from system gives
&
Bounds for V are 1<V < min(U,N)
These are loop bounds given by FM elimination. With a little extra work, we can simplify the upper bound of V to U.
7
%
'
Key points: Loop bounds determination in transformed code is mechanical. Polyhedral algebra technology can handle very general bounds with max's in lower bounds and min's in upper bounds. No need for pattern matching etc for triangular bounds and the like.
$
&
8
%
' &
When is permutation legal? Position so far: if there is a dependence between iterations, then permutation is illegal.
DO I = 1, 100 DO J = 1, 100 X(2I,J) = .... X(2I-1,J-1)...
$
Is there a ow dependence between di erent iterations? 1 Iw Ir Jw Jr 100 (Iw Jw) (Ir Jr) 2Iw = 2Ir ; 1 Jw = Jr ; 1
ILP decision problem: is there an integer in union of two convex polyhedra? No => permutation is legal.
9
%
' &
Permutation is legal only if dependence does not exist: too simplistic. Example:
DO I = 1, 100 DO J = 1, 100 X(I,J) = .... X(I-1,J-1)...
$
Only dependence is ow dependence:
Iw Jw Ir Jr 100 (Iw Jw) (Ir Jr) Iw = Ir ; 1 Jw = Jr ; 1
1
ILP problem has solution: for example, (Iw = 1 Jw = 1 Ir = 2 Jr = 2) Dependence exists but loop interchange is legal!
10
%
'
Point: Existence of dependence is a very \coarse" criterion to determine if interchange is legal. Additional information about dependence may let us conclude that a transformation is legal. To get a handle on all this, let is rst de ne dependence precisely.
$
&
11
%
' &
Consider single loop case rst:
DO I = 1, 100 X(2I+1) = ....X(I)... Flow dependences between iterations: Iteration 1 writes to X(3) which is read by iteration 3. Iteration 2 writes to X(5) which is read by iteration 5. .... Iteration 49 writes to X(99) which is read by iteration 99.
$
I
5 6 7 8 9 10
If we ignore the array locations and just think about dependence between iterations, we can draw this geometrically as follows:
0 1 2 3 4
Dependence arrows always go forward in iteration space. (eg. there cannot be a dependence from iteration 5 to iteration 2)
12
%
'
Intuitively, dependence arrows tell us constraints on transformations.
I
0 1 2 3 4 5 6 7 8 9 10
$
&
Suppose a transformed program does iteration 2 before iteration 1. OK! Transformed program does iteration 3 before iteration 1. Illegal!
13
%
'
Formal view of a dependence: relation between points in the iteration space.
DO I = 1, 100 X(2I+1) = ....X(I)...
$
I
0 1 2 3 4 5 6 7 8 9 10
Flow dependence = f(Iw 2Iw + 1)j1 Iw 49g (Note: this is a convex set)
&
In the spirit of dependence, we will often write this as follows: Flow dependence = f(Iw ! 2Iw + 1)j1 Iw 49g
14
%
' &
2D loop nest
DO 10 I = 1,100 DO 10 J = 1,100 10 X(I,J) = X(I-1,J+1) + 1
$
11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 11 111 11 11 00 00 000 00 00 11 11 111 11 11 00 00 000 00 00 11 11 111 11 11 00 00 000 00 00
1 2 34 5
Dependence: relation of the form (I1 J1 ) ! (I2 J2 ). Picture in iteration space:
J 5 4 3 2 1
11 00 11 00 11 00 11 00 11 00 11 00
111 000 111 000 111 000 111 000 111 000 111 000
11 00 11 00 11 00 11 00 11 00 11 00
11 00 11 00 11 00 11 00
111 000 111 000 111 000 111 000 111 000 111 000 111 000 111 000
I
source
target 111 111 000 000 111 111 000 000 11111111 00000000 (I1,J1) (I2,J2)
15
%
'
Legal and illegal dependence arrows:
J
$
legal dependence arrows illegal dependence arrows
&
If (A ! B) is a dependence arrow, then A must be lexicographically less than or equal to B.
16
%
'
Dependence relation can be computed using ILP calculator
DO 10 I = 1,100 DO 10 J = 1,100 10 X(I,J) = X(I-1,J+1) + 1
$
&
Flow dependence constraints: (Iw Jw ) ! (Ir Jr ) 1 Iw Ir Jw Jr 100 (Iw Jw ) (Ir Jr ) Iw = Ir ; 1 Jw = Jr + 1 Use ILP calculator to determine the following relation: D = f(Iw Jw) ! (Iw + 1 Jw ; 1)j(1 Iw 99) ^ (2 Jw 100)g
17
%
' &
If we have the full dependence relation, can we determine when permutation is legal? Let us look at geometric picture to understand when permutation is legal.
J 5 4 3 2 1
$
11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00
1
11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00
2
11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00
1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0
J
1 0 1 0 1 0 1 0 1 0 1 0 1 0
5
1 0 1 0 1 0 1 0 1 0 1 0 1 0
I
5 4 3 2 1
1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0
1
1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0
2
11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00
1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0
11 00 11 00 11 00 11 00 11 00 11 00
5
1 0 1 0 1 0 1 0 1 0 1 0
I
34
34
DO I = 1,N DO J = 1,N X(I,J) = X(I-1,J+1)...... Permutation is illegal
DO I = 1, N DO J = 1,N X(I,J) = X(I-1,J-1)...... Permutation is legal
Intuitively, if an iteration is dependent on an iteration in its "upper left hand corner", permutation is illegal. How do we express this formally?
18
%
' &
Legality of permutation can be framed as an ILP problem.
$
DO 10 I = 1,100 DO 10 J = 1,100 10 X(I,J) = X(I-1,J+1) + 1
Permutation is illegal if there exist iterations (I1 J1 ) (I2 J2 ) in source program such that ((I1 J1 ) ! (I2 J2 )) 2 D (dependent iterations) (J2 I2 ) (J1 I1 ) (iterations done in wrong order in transformed program)
This can obviously be phrased as an ILP problem and solved. One solution: (I1 J1 ) = (1 2), (I2 J2 ) = (2 1). Interchange is illegal.
19
%
' &
General picture: Permutation is co-ordinate transformation: U = P I where P is a permutation matrix. Conditions for legality of transformation: For each dependence D in loop nest, check that there do not exist iterations I 1 and I 2 such that (I 1 ! I 2 ) 2 D P(I 2 ) P(I 1 )
$
First condition: dependent iterations Second condition: iterations are done in wrong order in transformed program. Legality of permutation can be determined by solving a bunch of ILP problems.
20
%
' &
Problems with using full dependence sets:
$
Expensive (time/space) to compute full relations Need to solve ILP problems again to determine legality of permutation Symbolic loop bounds ('N') require parameterized sets ('N' is unbound variable in de nition of dependence set) Dependence abstractions: summary of dependence set D less information than full set of tuples in D more information than non-emptiness of D intuitively, \as much as is needed for transformations of interest"
21
%
' &
Distance/direction: Summarize dependence relation
J 5 4 3 2 1
Look at dependence relation from earlier slides: f(1 2) ! (2 1) (1 3) ! (2 2) ::(2 2) ! (3 1):::g
111 000 111 000 111 000 111 000 111 000 111 000 111 000
1
$
11 00 11 00 11 00 11 00 11 00 11 00 11 00
2
11 00 11 00 11 00 11 00 11 00 11 00 11 00
111 000 111 000 111 000 111 000 111 000 111 000 111 000
111 000 111 000 111 000 111 000 111 000 111 000 111 000 111 000
5
111 000 111 000 111 000 111 000 111 000 111 000 111 000 111 000
I
source
target 111 111 000 000 111 000 000 (I2,J2) 111 (I1,J1)
34
Di erence between dependent iterations = (1 ;1). That is, (Iw Jw ) ! (Ir Jr ) 2 dependence relation, implies Ir ; Iw = 1 Jr ; Jw = ;1 We will say that the distance vector is (1 ;1). Note: From distance vector, we can easily recover the full relation. In this case, distance vector is an exact summary of relation.
22
%
'
Set of dependent iterations usually is represented by many distance vectors.
DO I = 1, 100 X(2I+1) = ....X(I)...
$
I
0 1 2 3 4 5 6 7 8 9 10
Flow dependence = f(Iw ! 2Iw + 1)j1 Iw 49g Distance vectors: f(2), (3), (4), .... , (50)g Distance vectors can obviously never be negative (if (-1) was a distance vector for some dependence, there is an iteration I1 that depends on iteration I1 + 1 which is impossible.)
&
23
%
' &
Distance vectors are an approximation of a dependence: (intuitively, we know the arrows but we do not know their sources.) Example: D = f(Iw 2Iw + 1)j1 Iw 49g Distance vectors: f(2), (3), (4), .... , (50)g D1 = f(I1 I2 )j(1 I1 49) ^ (50 + I1 ) I2 (2I1 + 1)g is a (convex) superset of D that has the same distance vectors.
I
0 1 2 3 4 5 6 7 8 9 10
$
I
0 1 2 3 4 5 6 7 8 9 10
Both dependences have same set of distance vectors
24
%
' &
Computing distance vectors for a dependence
DO I = 1, 100 X(2I+1) = ....X(I)...
$
Flow dependence: 1 Iw < Ir 100 2Iw + 1 = Ir Flow dependence = f(Iw 2Iw + 1)j1 Iw 49g Computing distance vectors without computing dependence set: Introduce a new variable = Ir ; Iw and project onto 1 Iw < Ir 100 2Iw + 1 = Ir = Ir ; Iw = fdj2 d 50g
25
Solution:
%
' &
Example:2D loop nest
DO 10 I = 1,100 DO 10 J = 1,100 10 X(I,J) = X(I-1,J+1) + 1
$
;1)
Flow dependence constraints: (Iw Jw ) ! (Ir Jr ) Distance vector: ( 1 2 ) = (Ir ; Iw Jr ; Jw ) 1 Iw Ir Jw Jr 100 (Iw Jw ) (Ir Jr ) I w = Ir ; 1 Jw = Jr + 1 ( 1 2 ) = (Ir ; Iw Jr ; Jw ) Solution: (
1 2
) = (1
26
%
' &
General approach to computing distance vectors: Set of distance vectors generated from a dependence is itself a polyhedral set. Computing distance vectors without computing dependence set: To the linear system representing the existence of the dependence, add new variables corresponding to the entries in the distance vector and project onto these variables.
$
27
%
' &
Reality check: In general, dependence is some complicated convex set. In general, distance vectors of a dependence are also some complicated convex set! What is the point of \summarizing" one complicated set by another equally complicated set?!! Answer: We use distance vector summary of a dependence only when dependence can be summarized by a single distance vector (called a uniform dependence). How do we summarize dependence when we do not have a uniform dependence? Answer: use direction vectors.
$
28
%
'
DO I
Digression: When is a dependence a uniform dependence? That is, when can a dependence be summarized by a single distance vector? Conjecture: subscripts are of the following form
DO J X(I+a,J+b) = .... X(I+c,J+d)...
$
Check: ow dependence equations are
&
Iw + a = Ir + c Jw + b = Jr + d
So distance vector is (a ; c b ; d). Let us introduce some terminology to make the conjecture precise.
29
%
'
10
ZIV,SIV,MIV Subscripts Consider equalities for following dependence problem:
DO 10 I DO 10 J DO 10 K A(5,I+1,J) = ...A(N,I,K) + c
$
&
Subscripts in 1st dimension of A do not involve loop variables ) subscripts called Index Zero Variable (ZIV) subscripts Subscripts in 2nd dimension of A involve only one loop variable (I) ) subscripts called Single Index Variable (SIV) subscripts Subscripts in 3rd dimension of A involve many loop variables (J,K) ) subscripts called Multiple Index Variable (MIV) subscripts
30
%
'
10
Separable SIV Subscript
DO 10 I DO 10 J DO 10 K A(I,J,J) = ...A(I,J,K) + c
$
&
Subscripts in both the rst and second dimensions are SIV. However, index variable in rst subscript (I) does not appear in any other dimension ) separable SIV subscript Second subscript is also SIV, but its index variable J appears in 3rd dimension as well. ) coupled SIV subscript
31
%
'
Conjecture: Consider the ow dependence in following program
DO I DO J X(...,...,...) = ... X(...,...,...)
$
&
Conjecture: If ow dependence exists, it can be summarized by a distance vector i each subscript is a separable SIV subscript. This conjecture is false.
32
%
'
DO I DO J X(I+2J-1,3I+J+2) = ... X(I+2J,3I+J)
$
Ir ; Iw + 2Jr ; 2Jw = ;1 3Ir ; 3Iw + Jr ; Jw = 2
Both subscripts are MIV. Dependence equations:
&
Easy to verify that distance vector is (1,-1).
33
%
' &
Another example:
DO I DO J X(I-J+2,I+J,I+J) = ...X(I-J,-I-2J,3I+4J)...
$
2 + Iw ; Jw = Ir ; Jr Iw + Jw = ;Ir ; 2Jr Iw + Jw = 3Ir + 4Jr
Here, subscripts are MIV and the subscripts of reads and writes look quite di erent. Dependence equations:
Easy to verify that dependence distance is (1,-1).
34
%
' &
Modi ed conjecture: Consider the program
$
DO I X(A*I + a) = ... X(B*I + b) ..
Here, I is a vector, A and B are matrices etc. If A = B, columns of A (and of B) are linearly independent and dependence exists, then dependence is uniform dependence. Proof: Equality system is A Iw + a = B Ir + b.
A Iw + a = B Ir + b B (Ir ; Iw ) = a ; b(sinceA = B) B = a ; b( isdistancevector)
Since null space of B contains only the 0 vector, the equation has a unique solution if it has a solution at all.
35
%
'
Two caveats You must check inequalities to make sure dependence actually exists.
DO I = 1, 100 X(I+100) = ...X(I)
$
&
It is incorrect to conclude that distance vector is (100) since no dependence exists. As we will see later, separable SIV subscripts are very common MIV is very rare.
End of digression.
36
%
' &
Direction vectors Example:
DO 10 I = 1,100 10 X(2I+1) = X(I) + 1
$
Flow dependence equation: 2Iw + 1 = Ir . Dependence relation: f(1 ! 3) (2 ! 5) (3 ! 7) :::g (1). No xed distance between dependent iterations! But all distances are +ve, so use direction vector instead. Here, direction = (+). Intuition: (+) direction = some distances in range 1 1) In general, direction = (+) or (0) or (-). Also written by some authors as (<), (=), or (>). Direction vectors are not exact. (eg):if we try to recover dependence relation from direction (+), we get bigger relation than (1): f(1 ! 2) (1 ! 3) ::: (1 ! 100) (2 ! 3) (2 ! 4) :::g
37
%
'
J
Directions for Nested Loops
Assume loop nest is (I,J). If (I1 J1 ) ! (I2 J2 ) 2 dependence relation, then Distance = (I2 ; I1 J2 ; J1 ) Direction = (sign(I2 ; I1 ) sign(J2 ; J1 ))
Legal direction vectors: (+,+) (0,+) (+,-) (0,0) (+,0) The following direction vectors cannot exist: (0,-)
I
$
(+,-) (0,0) (+,0)
&
(0,+) (+,+)
(-,+) (-,0) (-,-)
Valid dependence vectors are lexicographically positive.
38
%
'
10
How to compute Directions: Use IP engine
DO 10 I = 1, 100 X(f(I)) = ... = ...X(g(I))..
$
&
Focus on ow dependences: f(Iw ) = g(Ir ) 1 Iw 100 1 Ir 100 First, use inequalities shown above to test if dependence exists in any direction (called (*) direction). If IP engine says there are no solutions, no dependence. Otherwise, determine the direction(s) of dependence. Test for direction (+): add inequality Iw < Ir Test for direction (0): add inequality Iw = Ir In a single loop, direction (;) cannot occur.
39
%
' &
Computing Directions: Nested Loops
(* , *)
Same idea as single loop: hierarchical testing
$
illegal directions
(+ , *)
(0 , *)
(- , *)
(+ , +) (+ , 0) (+, -)
(0 , +) (0 , 0) (0 , -)
Figure 1: Hierarchical Testing for Nested Loop
Key ideas: (1) Re ne direction vectors top down. (eg),no dependence in ( ) direction ) no need to do more tests. (2) Do not test for impossible directions like (; ).
40
%
'
It is also possible to compute direction vectors by projecting on the variables in the , the iteration di erence vector. Similar to what we did for distance vectors. Left as an exercise for you.
$
&
41
%
'
Big hairy example: Compute dependences for following program:
DO I = 1,N DO J = 1,N X(I,J) = ...X(I,I)...
J
anti-dependence flow dependence 0 + 0 0 0 + flow
$
anti
&
I
42
%
' &
0 0
Linear system for anti-dependence: Iw = Ir Jw = Ir 1 Iw Ir Jw Jr N (Ir Jr ) (Iw Jw ) 1 = (Iw ; Ir ) 2 = (Jw ; Jr ) Projecting onto 1 and 2, we get 1=0 0 2 (N ; 1) So directions for anti-dependence are
and 0 +
43
$
%
'
Similarly, you can compute direction for ow dependence
0 +
$
and also show that no output dependence exists.
&
44
%
'
Dependence matrix for a loop nest Matrix containing all dependence distance/direction vectors for all dependences of loop nest. In our example, the dependence matrix is
0 0 0 +
$
&
45
%
'
Dependence direction/distance are adequate for testing legality of permutation.
$
&
46
%
'
J I DO I = 1, N DO J = I,N .......... 0 1 1 0 I U = J V
V
$
U DO U = 1, N DO V = 1,U ......... T I1 J1 T I2 J2 Distance between iterations = T I2 J2 T I1 J1 =T I2 - I1 J2 - J1 = J2 - J1 I2 - I1
&
I1 J1
I2 J2 I2 - I1 J2 - J1
Dependence distance =
Check for legality: interchange positions in distance/direction vector & check for lex +ve If transformation P is legal and original dependence matrix is D, new dependence matrix is T*D.
47
%
' &
Correctness of general permutation Transformation matrix: T Dependence matrix: D Matrix in which each column is a distance/direction vector Legality: T:D 0 Dependence matrix of transformed program: T:D
$
48
%
'
Examples:
DO I = 1,N DO J = 1,N X(I,J) = X(I-1,J-1)....
$
Distance vector = (1,1) => permutation is legal Dependence vector of transformed program = (1,1)
&
DO I = 1,N DO J = 1,N X(I,J) = X(I-1,J+1)....
Distance vector = (1,-1) => permutation is not legal
49
%
'
Remarks on dependence abstractions A good dependence abstraction for a transformation should have the following properties. Easy to compute Easy to test for legality. Easy to determine dependence abstractions for transformed program.
$
&
Direction vectors are a good dependence abstraction for permutation.
50
%
'
Engineering a dependence analyzer In principle, we can use IP engine to compute all directions. Reality: most subscripts and loop bounds are simple! Engineering a dependence analyzer:
$
First check for simple cases. Call IP engine for more complex cases.
&
51
%
'
Important optimization: splitting of linear systems In practice, many dependence computations can be decomposed into two or more smaller, independent problems.
DO 10 I DO 10 J DO 10 K 10 A(I,J,J) = ...A(I,J,K) + c
$
&
occurs only in rst subscript and bounds on I are independent of other variables => inequalities/equalities for (Ir Iw ) for example can be separated from rest of system and solved separately.
I
52
%
'
10
Special case of splitting: separable S...