Contract-based Resource Verification for - PowerPoint Presentation

Slide1

Contract-based Resource Verification for Higher-order Functions with Memoization

Ravichandhran

Madhavan

EPFL

Sumith

Kulal

IIT Bombay

Viktor

Kuncak

EPFLSlide2

Resource VerificationProving upper bounds on resource usage of programs (e.g. time, space)Physical units e.g. time <= 10 ms, space <= 100 KBAlgorithmic units e.g. evaluation steps <= 10n + 10, heap-allocated objects <= 200

Asymptotic Bounds

e.g. evaluation steps <= O(n), heap-allocated objects <= O(1)Slide3

Resource VerificationProving upper bounds on resource usage of programs (e.g. time, space)Physical units e.g. time <= 10 ms, space <= 100 KBAlgorithmic units e.g. evaluation steps <= 10n + 10, heap-allocated objects <= 200

Asymptotic Bounds

e.g. evaluation steps <= O(n), heap-allocated objects <= O(1)Slide4

Vasconcelos et al., ESOP’15Expressivity,InteractionAutomation

Push-button Tools

Specification-based Verifiers

Interactive Theorem

Provers

Resource Verification Techniques

COSTA TCS’12

SPEED POPL’09

Alias et al. SAS’10

RAML, CAV’12

Le

Metayer

, TOPLAS’88

Navas

ICFP’07

Zuleger

et al., SAS’11

Sinn et al., CAV’14

Danner et al. PLPV’13

Sands ESOP’90

Alonso-Blas et al.

ESOP’90

Carbonneaux

et al.PLDI’14

Madhavan

et al. CAV’14

This Work

Nipkow

ITP’15

Chargueraud

, ITP’15

Danielsson

, POPL ‘08Slide5

Specifications for Resource VerificationResource bounds are as hard as correctness propertiesHigh-level properties and bounds can be specified by users Low-level constraint solving is automatedModular – context invariants are captured by contracts

Slide6

Memoization and Lazy EvaluationMemoizationFunction results are cached for each distinct argumentLazy EvaluationEvaluations are suspended until neededSuspended evaluations are memoized closures with unit parameter

@memoize

def

f

(

n

) =

{

…

f

(

1

)

+

f

(

2

)

}

Memo Table

fun

args

res

f

g

h

…

fun

args

res

f

g

h

…

Slide7

Importance of Memoization &Lazy Evaluation“Lazy evaluation is strictly more efficient - in asymptotic terms - than eager evaluation, and that there is no general complexity-preserving translation of lazy programs into an eager functional language.”Practical Examples:Okasaki’s lazy data structuresDynamic programming algorithms

- Richard Bird, Geraint Jones and 

Oege

de Moor

More Haste, Less Speed

Journal of Functional Programming 1997 Slide8

Functional Reasoning for Evaluation Results“Lazy evaluation, which applies functions to arguments whose evaluation is postponed until the first (but only the first) time their values are needed. This ensures that equational reasoning about the values of programs is sound ….”- Richard Bird, Geraint Jones and Oege de Moor More Haste, Less Speed Journal of Functional Programming 1997 Functions

are referentially transparent

Heap

can be assumed to be

immutable (No frame problem)

Fewer

specification overhead, more efficient verificationSlide9

Complicates Resource Usage“This ensures that equational reasoning about the values of programs is sound, at the expense of making it extremely hard to reason about when expressions are evaluated, and to estimate how much work is involved in doing so.”- Richard Bird, Geraint Jones and Oege de Moor More Haste, Less Speed Journal of Functional Programming 1997 Bad news: the state of the cache is visible in the resource behavior

Good news: the state is well-behaved and grows monotonicallySlide10

Expressive system for specifying and verifying resource bounds with memoization (and laziness)ApproachEvaluationImplementationleondev.epfl.chSoundness proofsSlide11

The Overall StrategyAllow users to provide resource bounds in contractsProvide special constructs for expressing properties about Cache stateFunction-valued parameters Verify the specifications using lossless encoding into decidable logicSlide12

Specification StrategySlide13

Streams1g(1)s

2

g(2)

3

g(3)

tail

tailSlide14

Streams1g(1)s

2

g(2)

3

g(3)

tail

tail

Say g(n) takes O(n) time

take(n, s)

takes O(n

2

) time

…

take(n, s)

takes

O(n) time

Invariant:

tail of first n elements of s is

memoized

Resource Bound:

take(

n,s

) runs in time O(n)Slide15

Specifying Cache Invariantscached(g x) - true iff the function g is memoized for the argument xcached(s.tail) concreteUntil(n,

s.tail) if n

>=

1

true otherwise

concreteUntil

(

n,s

)

= Slide16

Specifying Resource Bounds and Invariantsdef take(n, s) { require(concreteUntil(n, s))} ensuring{ steps ? * n + ? }

Resource bounds are specified as templates with numerical holes in the

postcondition

Invariants are specified in the contractsSlide17

Scheduling-based Data StructuresWe allow equating closures in the contracts[Conc-trees for functional and parallel programming. A. Prokopec and M. Odersky. LCPC 2015]Slide18

Semantics of Closure EqualitiesClosures are lambdas without free variablesWe use intensional or structural equalityBenefits: Can encode reference equality of closuresCan be checked efficiently using SMT theoriesSlide19

Recap of the Specification StrategyResource bounds are expressed in post conditions as templates with holesA construct cached specifies the state of memoizationThe behavior of closures is constrained using structural equality

Recursive functions and nonlinearity allowed in contractsSlide20

KnapsackStreams

Scheduling-based

Queues

Lazy merge sort

(

Kth

minimum)

Packrat Parsing

Knapsack

Streams

Scheduling-based

Queues

Lazy merge sort

(

Kth

minimum)

Packrat Parsing

Sample Benchmarks and BoundsSlide21

The Verification ApproachSlide22

Verification ProblemCheck that the invariants in the contracts are valid Infer values for holes that make the bound hold for all executionsMake the bound as strong as possible

def

take(n, s) {

require

(

concreteUntil

(n, s))

}

ensuring

{ steps

? * n + ?

}

 Slide23

Aspects of VerificationProof Principle/StrategyVerification Condition generationSolving VCs - Inferring values for holesSlide24

Function-level Modular Reasoningdef f(x) { require(pre) body}ensuring(post)

d

ef

main(x) {

f(x)

}

Requires extensions for closures and state-dependent specificationsSlide25

Key Idea: Exploit Monotonicity of CacheObservation 1: Cache increases monotonicallyObservation 2: Resource usage of a call is anti-monotonic w.r.t cacheSpecifications can use only cache monotonic propertiesP

is cache monotonic iff

P

holds

in a smaller cache

P

holds

in a

larger

cache

 Slide26

Creation-Dispatch ReasoningP(x) holds at creation point but not available at invocation pointCheck precondition of on the captured argument during creationAssume precondition of

on the

captured arguments

during application

d

ef

f(l){

//

x not

in scope

l(10)

}

d

ef

main(x) {

require

(

P

(x))

f(

)

}

d

ef

g(x

) {

require

(

P

(x)

)

x

}Slide27

Expressions to FormulasAlgebraic DatatypesResource TemplatesArithmeticRecursive functionsHigher-order functionsCache-dependent specsMemoizationAliasing of closuresLIAADTUFSets

Recursion

Contracts

Templates

Arithmetic

Sets

Model

g

eneration

VC

generation

A first-order functional

program

with sets and

datatypes

Instrumentation of

cache state and resource

usage

Encoding

of higher-order callsSlide28

Solving Verification ConditionsCounter-example guided solvingUnfolding of Recursive Functions

Solver

(uses

Farkas

’ Lemma)

Solver

(z3 + cvc4)

Nonlinearity

i

A

B

No

solution

Conflicting

disjunct

Minimized

Model

for

is

unsat

[

Symbolic Resource Bound Inference for Functional Programs, R.

Madhavan

, V.

Kuncak

, CAV’14]Slide29

Experimental EvaluationSlide30

BenchmarksBenchmarksLOCBytecodeTemplatesSpec funsAnalysis TimeLazy data structures

Lazy

sorting

360

0.13mb

10

10

2 min

Streams

1000

0.3mb

42

18

5 min

Real time queue

207

69kb

5

6

1 min

Deque

426

0.1mb

16

7

5 min

Numerical

Representation

(

Conqueue

)

8800.2mb

18585 min

Dynamic ProgrammingKnapsack, Packrat parsing, Viterbi etc.

8000.3mb3232

8 min

5k lines of Scala code 123 resource templatesSlide31

More Sample Templates InferredBenchmarksTemplates InferredLazy data structures Lazy sorting

Streams

Real time queue

Conqueue

Dynamic Programming

Viterbi

Benchmarks

Templates

Inferred

Lazy data structures

Lazy

sorting

Streams

Real time queue

Conqueue

Dynamic Programming

ViterbiSlide32

Bottom-up Lazy Merge Sort(kth Minimum) Slide33

ConclusionProving precise resource bounds with memoization and higher-order features is challenging but possibleDemonstrated on very complex benchmarksScheduling-based queuesLazy sorting, Numerical representations, Packrat parsing, ViterbiExtensible to other resources, amortized bounds and relational reasoningTry it online here: leondev.epfl.ch

(Memresources)Slide34
Slide35

Supplementary DetailsSlide36

Example 2: The Knapsack ProblemOptimally filling a sack of capacity C with items of weight and value

 Slide37

Bottom-up Iteration

Iterations

1

2

n

Invariant:

m < n.

ks

(m, items) is

memoized

Resource Bound:

ks

(

n,items

) runs in time O(

items.size

)

 Slide38

Specifying Cache Invariantscached(g x) - true iff the function g is memoized for the argument xcached(ks(n-1, items)) ksp(n-1,

items) if n

>= 2

true otherwise

ksp

(n,

items) = Slide39

Accuracy MeasurementsRatio of worst case dynamic resource usage to static estimate for thousands of inputs

 Slide40

Contributors to ImprecisionThe template which specifies the closed form solutionThe constants inferred by the toolPareto Optimal Bound:Obtained by minimizing one constant in the inferred bound until it under-approximates a run

 Slide41

Evaluation Summary