Gupta Bharath Hariharan Alex Aiken and Aditya Nori Stanford UC Berkeley Microsoft Research India Verification as Learning Geometric Concepts Invariants assume xlt0 while xlt0 ID: 711722
Download Presentation The PPT/PDF document "Rahul Sharma , Saurabh" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Rahul Sharma, Saurabh Gupta, Bharath Hariharan, Alex Aiken, and Aditya Nori(Stanford, UC Berkeley, Microsoft Research India)
Verification as
Learning Geometric ConceptsSlide2
Invariantsassume x<0;while ( x<0 ){ x = x+y; y = y+1;}
assert y>0;
while
Find invariants
Quantifier free arithmeticDisjunctive invariants
Slide3
Disjunctive invariantsassume n > 0;x = 0 ;while ( x < n ){ x = x+1;}
assert
x = n;
Blow up
Restrict disjunctionsTemplatesHeuristics
This work
No templates, no blowup
Slide4
Classification
positive
examples
n
egative
examples
+
+
+
+
+
+
+
+
+Slide5
From invariants to classifiersSafety properties define bad statesInvariants separate reachable states from bad statesPossible to obtain some examples of statesInvariants -> classifiersExamples of reachable/good states -> positive examplesExamples of bad states -> negative examplesUse a classifier to separate ALL good and bad statesSlide6
Sample, guess, and checkGenerate examples of good and bad concrete statesGuess an invariant using learnerCheck if verification succeedsIf yes, then doneIf no, then guess again with more examplesUse counter-examples to verification taskSlide7
Sample good statesassume x<0;while ( x<0 ){ print(x,y); x = x+y
;
y = y+1;
}
assert y>0;Reachable statesRun the programSlide8
Sample bad statesassume P;while ( B ){ S}assert
Q;
Unreachable for correct programs
Backward analysis
Slide9
From program to dataassume x<0;while ( x<0 ){ x = x+y; y = y+1;}assert y>0;
x = -1, y = 0
x>=0 && y<=0
+
+
+
x
ySlide10
LearnerBshouty, Goldman, Mathias, Suri, Tamaki in STOC’96Learn arbitrary boolean combinations of inequalitiesCreate a large enough candidate set of planesIntelligently select from candidatesSeparate given examples of good and bad statesUse only a few planesSlide11
Candidate planesCandidate set:One plane per partition planes
x
ySlide12
ExampleCandidate set:One plane per partition planesConnect every good with every bad stateBipartite graphSelect candidatesC
ut all edges
Minimize #selected candidates
+++
x
y
x
ySlide13
GuaranteesIf the invariant has planes in dimensions,and the candidate set is adequatethen produce an output of size
Independent of the number of samples!
(
Output is only logarithmically larger than invariant
Slide14
From planes to predicates
Planes tessellate space
Label regions
Return simplest predicate
Contains all good regionsNo bad regionsSome don’t caresLogic minimization
+
+
+
x
ySlide15
Efficiency?Candidate planes in numberAbstract interpretation over polyhedra is exponentialAssume that inequalities are of specific formIntervals:
, Octagons:
Restrict inequalities for efficient learners
But find arbitrary
boolean combinations Slide16
Small candidate setsInvariants are arbitrary boolean combinations of intervalsNeed adequate candidate setsIntervals through every state are sufficientSize of candidate set: Octagons:
, TCM, …
Slide17
Guarantees on generalizationPrograms have unbounded behaviorsAnalyze some finite behaviors and generalizeSLAM/BLAST: Ask for predicates to discard spurious cexsImpact: Unwind loops and interpolateAbstract interpretation: iterate and widenNeed a formal definition of generalizationNeed generalization guarantees for useful toolsSlide18
A step: PACProbably approximately correctAssume an oracle that knows the invariant Oracle draws samples from and labels using A PAC learner given enough samples (
)
With high probability outputs a classifier
Misclassifies a new sample with low probability (
) Slide19
Summary of resultsGiven sufficient good and bad samples,with high probability, the learner generates a predicate,that has high accuracy for unseen samplesThe generated classifier is expressive Arbitrary boolean combinations of linear inequalities#Planes in classifier independent of samplesWorst case only logarithmically more than invariantSlide20
Non-linear invariantsArbitrary boolean combinations of polynomial inequalities of a given degreeCreate a new variable for every monomial using , vars=
, introduce new
vars
for
The whole machinery carries overWith increased (number of variables/dimension) Slide21
Implementation100 lines of MATLAB for learnerHavoc+Boogie for checkingInput: annotated C programsBoogie internally uses SMT solver Z3Example invariants
Slide22
ExperimentsSlide23
Related workInvariant inferenceAbstract interpretation – disjunctive completionConstraint based (Sting, InvGen, GSV’ 08)Use tests to help static analysis: Yogi, InvGen, …Guess and check: Daikon, SAN’ 12, SGHALN’ 13Slide24
ConclusionConnections between verification and learningGeneralization is a fundamental problem for bothPossible to obtain invariant generators with guaranteesHandling disjunctions and non-linearities is easyDifficult for symbolic approachesNeed data, which is availableFuture work, beyond numerical