Proving Optimizations Correct

Proving Optimizations Correctusing Parameterized Program Equivalence

University of California, San Diego

Sudipta KunduZachary TatlockSorin Lerner

Compiler CorrectnessDifficult to develop reliable compilers:

large, complex programstake a long time to matureConsequence: buggy compilers, but also …hinders development of new languages, architectures

discourages user from extending compilerFocus on correctness of compiler optimizationsmany intricate opts, unexpected interactionsturning off optimizations often no longer an option

Existing TechniquesTranslation Validation

prove equivalencefor each transformationevery compiler execution

a priori

C

orrectness

prove correctness

before compiler runs

once and for all

TVOC

[

Zuck

et al.]

Rhodium

[Lerner et al.]

Compcert

[Leroy et al.]

[

Necula

00]

Focus on Automated Techniques

Verified TV

[Tristan et al.]

[

Pnueli

et al.]

Scope of Guarantee

Verify Compiler Run

Verify Compiler

TVOC

[

Zuck

et al.]

Rhodium

[Lerner et al.]

[

Necula

00]

Expressive Power

Complex

Loop OptsOne-to-one Rewrites

Focus on Automated Techniques

complex loop opts

+

once-and-for-all

correctness

Our Approach:

Adapt Translation Validation to once-and-for-all setting

Key Insight:

Generalize to Parameterized Progs

Equivalence

Checking

Generalize to Parameterized Programs

Output

Prog

Input

Prog

Output

PProg

Input

PProg

Translation Validation

k := 0

while(

k<

100){

a[k] += k

k++

}

Generalize to Parameterized Progs

Equivalence

Checking

Generalize to Parameterized Programs

I := 0

while(

I<

100){

a[k] += I

I++

}

Output

Prog

Input

Prog

Output

PProg

Input

PProg

Generalize to Parameterized Progs

Equivalence

Checking

Generalize to Parameterized Programs

I := 0

while(

I<

E){

a[k] += I

I++

}

Output

Prog

Input

Prog

Output

PProg

Input

PProg

Equivalence

Checking

I := 0

while(

I<

E){

S

I++

}

I := 0

while(

I<

E){

S

I++

}

Generalize to Parameterized

Progs

Output

Prog

Input

Prog

Generalize to Parameterized Programs

Output

PProg

Input

PProg

Optimization

Instance

Optimization

Generalize to Parameterized

Progs

Equivalence

Checking

Generalize to Parameterized Programs

Parameterized

Equivalence

Checking

Optimization

Instance

Optimization

Output

Prog

Input

Prog

Output

PProg

Input

PProg

Contributions

Parameterized

Equivalence

Checking

Parameterized Equivalence Checking (PEC)

proves opts correct statically and automatically

can reason about many-to-many opts

Expressed and proved a variety of opts correct

which Rhodium could not have proven correct

software pipelining and other complex loop opts

1

2

3

Optimization

Output

PProg

Input

PProg

I

:= 0

while(I<E-1){ S

I

++

}

SI++Parameterized Rewrite RulesLoop Peelingmove iter out of loopids range over:

I : variable

E

: expressionS

: statementShift final iteration after loopSide conditions indicate when rewrite safe

I

:= 0

while(I < E){

S

I

++

}

where:

E > 0

S

does not modify

I, E

Enable loop unrolling

Apply RewriteMatch parameters

Check side condsRewrite

where:

100 > 0

a[k] += k

does not modify k,100Applying Rewrite Rules

I := 0

while(I < E){

S

I

++

}

I

:= 0

while(I<

E-1){ S

I

++}

SI++where:

E > 0

S does not modify I,

E

k := 0

while(

k<

100){

a[k] += k

k++

}

k := 0

while(

k<

99){

a[k] += k

k++

}

a[k] += k

k++

Divisible by 3

Directly unroll by 3





Not divisible by 3

Hard to unroll by 3

Checking Correctness

OPT

Parameterized

Equivalence

Checking

OPT

Checking Correctness

Generalization of

[

Necula

00 ]

Generalization of

[

Zuck

et al. 04]

OR

Relate

Permute

Relate

I

:= 0

while(

I<

E-1){

S

I++}

SI++

I

:= 0

while(I < E){

S

I

++

}where:

E

> 0

S does not modify

I, E

Checking Rewrite Rules

I:=0

I<E

I≥E

S

I++

I:=0

I<E-1

I≥E-1

S

I++

S

I++

σ

1

=

σ

2

Programs equivalent:

Consider CFGs

Start

in equal states

End in equal states

σ

1

=

σ

2

Checking Rewrite Rules

Relate Executions:

Find

synchronization

points

Generate

invariants

Check invariants

preserved

I:=0

I<E

I≥E

S

I++

I:=0

I<E-1

I≥E-1

S

I++

S

I++

A

B

Use auto theorem

prover

Each invariant must imply successor invariants

Strengthen if inv too weak

σ

1

=

σ

2

σ

1

=

σ

2

1. Find Synchronization Points

I<E

S

I++

I:=0

I<E-1

I≥E-1

S

I++

S

I++

Traverse in lockstep

Stop

at stmt

params

Prune infeasible paths

From Path:

E≤0

Side

Conds

:

E>0



Path never executes

I:=0

I≥E

2. Generate Invariants

I:=0

I<E

I≥E

S

I++

I:=0

I<E-1

I≥E-1

S

I++

S

I++

Invariants:

predicates over

σ

1

,

σ

2

Gen initial invariant:

σ

1

=

σ

2

AND

strongest post

cond

σ

1

=

σ

2

σ

1

=

σ

2

B

A

A

(

σ

1

,

σ

2

)

...

B

(

σ

1

,

σ

2

)

...

A

(

σ

1

,

σ

2

)

σ

1

=

σ

2

∧

eval

(

σ

1

, I < E)

∧

eval

(

σ

2

, I < E-1)

B

(

σ

1

,

σ

2

)

...

B

A

I<E

I<E-1

I≥E-1

A

(

σ

1

,

σ

2

)

σ

1

=

σ

2

∧

eval

(

σ

1

, I < E)

∧

eval

(

σ

2

, I < E-1)

B

(

σ

1

,

σ

2

)

σ

1

=

σ

2

∧

eval

(

σ

1

, I < E)

∧

eval

(

σ

2

, I ≥ E-1)

3. Check Invariants

I:=0

I<E

I≥E

S

I++

I:=0

I<E-1

I≥E-1

S

I++

S

I++

σ

1

=

σ

2

σ

1

=

σ

2

B

A

Each invariant must imply successor invariants

Query Theorem

Prover

B

A

I<E

S

I++

I≥E-1

S

I++

S

I++

I<E

S

I++

I≥

E-1

A

B

A

(

σ

1

,

σ

2

)

σ

1

=

σ

2

∧

eval

(

σ

1

, I < E)

∧

eval

(

σ

2

, I < E-1)

B

(

σ

1

,

σ

2

)

σ

1

=

σ

2

∧

eval

(

σ

1

, I < E)

∧

eval

(

σ

2

, I ≥ E-1)

Entry

 A

Entry

 B

A

 B

A

 A

B

 Exit



σ

1 σ2 .

A

(

σ

1

,

σ2) ∧σ1’ = step(σ1, S;I++;

I<E)

∧σ

2’ = step(σ

2

, S;I++;I≥E-1)

B

(

σ1’,

σ2

’)

3. Check Invariants

A

B

S

I++

I<E

S

I++

I≥

E-1

σ

1

σ

2

σ

1

’

σ

2

’



σ

1

σ

2

.

A

(

σ

1

,

σ

2

)

∧

σ

1

’ = step(

σ

1

, S;I++;

I<

E)

∧

σ

2

’ = step(

σ

2

, S;I++;

I≥

E-1)

B

(

σ

1

’,

σ

2

’)

ATP Query:

ATP

A

(

σ

1

,

σ

2

)

σ

1

=

σ

2

∧

eval

(

σ

1

, I < E)

∧

eval

(

σ

2

, I < E-1)

B

(

σ

1

,

σ

2

)

σ

1

=

σ

2

∧

eval

(

σ

1

, I < E)

∧

eval

(

σ

2

, I ≥ E-1)



σ

1

σ

2

.

A

(

σ

1

,

σ

2

)

∧

σ1

’ = step(σ1

, S;I++;I<E)

∧

σ2’ = step(

σ2, S;I++;I≥E-1)

B

(σ1

’, σ

2

’)3. Check Invariants

A

B

S

I++

I<E

S

I++

I≥

E-1

σ

1

σ

2

σ

1

’

σ

2

’

ATP Query:

ATP



A

(

σ

1

,

σ

2

)

σ

1

=

σ

2

∧

eval

(

σ

1

, I < E)

∧

eval

(

σ

2

, I < E-1)

B

(

σ

1

,

σ

2

)

σ

1

=

σ

2

∧

eval

(

σ

1

, I < E)

∧

eval

(

σ

2

, I ≥ E-1)

σ

1

=

σ

2

∧

eval

(

σ

1

, I < E)

∧

eval

(

σ

2

, I < E-1)

σ

1

’

=

σ

2

’

∧

eval

(

σ

1

’

, I < E)

∧

eval

(

σ

2

’

, I ≥ E-1)



σ

1

σ

2

.

A

(

σ1,

σ

2) ∧

σ1’ = step(

σ

1, S;I++;I<

E)

∧

σ2

’ = step(σ2

, S;I++;

I≥E-1)

B(

σ1’, σ2’)

3. Check Invariants

B

S

I++

I<E

S

I++

I≥

E-1

σ

1

σ

2

σ

1

’

σ

2

’

ATP Query:

ATP



A

(

σ

1

,

σ

2

)

σ

1

=

σ

2

∧

eval

(

σ

1

, I < E)

∧

eval

(

σ

2

, I < E-1)

B

(

σ

1

,

σ

2

)

σ

1

=

σ

2

∧

eval

(

σ

1

, I < E)

∧

eval

(

σ

2

, I ≥ E-1)

σ

1

=

σ

2

∧

eval

(

σ

1

, I < E)

∧

eval

(

σ

2

, I < E-1)

σ

1

’

=

σ

2

’

∧

eval

(

σ

1

’

, I < E)

∧

eval

(

σ

2

’

, I ≥ E-1)

A

∧

B

(

σ

1

’,

σ

2

’)

Strengthen A if the theorem

prover

fails

σ

1

’ = step(

σ

1

, S;I++;

I<

E)

σ

2

’ = step(

σ

2

, S;I++;

I≥

E-1)

A

3. Check Invariants

I:=0

I<E

I≥E

S

I++

I:=0

I<E-1

I≥E-1

S

I++

S

I++

σ

1

=

σ

2

σ

1

=

σ

2

B

A

A

(

σ

1

,

σ

2

)

σ

1

=

σ

2

∧

eval

(

σ

1

, I < E)

∧

eval

(

σ

2

, I < E-1)

B

(

σ

1

,

σ

2

)

σ

1

=

σ

2

∧

eval

(

σ

1

, I < E)

∧

eval

(

σ

2

, I ≥ E-1)

Each invariant must imply successor invariants

Query Auto Theorem

Prover

Entry

 A

Entry

 B

A

 B

A

 A

B

 Exit











OPT

Checking Correctness

Generalization of

[

Necula

00 ]

Generalization of

[

Zuck

et al. 04]

OR

Relate

Permute

Relate

Permute

Works well for structure-preserving optimizations

Works well for non structure-preserving optimizations

Permute Module

for (I in R1){

for (J in R2){ S[I, J]; }}

for (N in R2){

for (M in R1){

S[M, N];

}

}

where: ⟨side condition⟩

Generate correspondence between loop indices

Ask ATP to show that

⟨

side condition⟩ implies:correspondence is one-to-one

For all I, I’

∊ R1 and J, J’

∊ R2 S[I, J] commutes with S[I’, J’]

Loop Interchange

Contributions

Parameterized

Equivalence

Checking

Parameterized Equivalence Checking (PEC)

proves opts correct statically and automatically

can reason about many-to-many opts

Expressed and proved a variety of opts correct

which Rhodium could not have proven correct

software pipelining and other complex loop opts

1

2

3

Optimization

Output

PProg

Input

PProg

Optimizations Proved Correct

Category 1: PEC and Rhodium formulation equivalent

Copy propagationConstant propagationCommon sub-expression elimPartial redundancy elimCategory

2:

PEC formulation easier, more general

Loop invariant code hoisting

Conditional speculation

Speculation

Category 3:Expressible in PECNo Rhodium formulation possibleSoftware pipelining Loop unswitching Loop unrolling Loop peelingLoop splitting Loop alignment Loop interchange Loop reversal Loop skewing Loop fusion

Loop distribution

Conclusion

Parameterized

Equivalence

Checking

Parameterized Equivalence Checking (PEC)

proves opts correct statically and automatically

can reason about many-to-many opts

Expressed and proved a variety of opts correct

which Rhodium could not have proven correct

software pipelining and other complex loop opts

Optimization

Output

PProg

Input

PProg