Assertion true 6 task6null task6null task6null 16 notif notif task6null task6null 27 tasknull false 16 727 28n true task6null fa ID: 362027
Download Pdf The PPT/PDF document "ErrorInvariant" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Assertion ErrorInvariant ::: true `6 ^task6=null^::: task6=null ^::: task6=null `16 ^[((notif.^(:::)) _(:notif.^(:::))] task6=null ^::: task6=null `27 ^(task=null)^::: false `1�6 `7�27 `28�n true task6=null false assume(task6=null) assert(task=null) Figure2:(a)First-orderlogicalformulaforFigure 1 (b)ErrorInvariantAutomatonforFigure 1 . Therearetwoequivalentwaystocreateaminimalerrorinvariantautomaton.Oneistostartwiththefullprogram,andtomergestatesuntilaxedpointisreached(thisisthealgorithmwepresentinx 4 ).Theother,whichwedescribehere,istostartwithanemptyprogramandaddabstractstatesuntilthedesiredpropertyisprovable.Notethatthisproceduregeneratesaminimalautomatonbutnotnecessar-ilyaminimumautomaton,sincewhichstatescanbemergeddependsonwhichinvariantsarechosenforthestatelabels.WeassumethatwehaveastaticanalysistoolsuchasJoogiethatprovestheexistenceofinconsistentcode[2].Inthisexample,Joogiereportsanerroronline27.Weusethisinformationtoextractaprogramfragmentwhichassertstheinconsistency.Inthiscase,weassertthattheprogramreachesline27,andthattaskisnullatline27.Fromthisprogramfragment,weconstructarst-orderlogicformulawhichisunsatisablebecauseoftheinconsistencyintheoriginalprogram.Thisencodingissimilartotheextendedpathformulasshownin[8],howeverourapproachencodesmultiplepathsintooneformula.Wecannowusethegeneratedrst-orderformulatoiden-tifythecauseoftheinconsistency.Ourprocedureproceedsrecursively.First,weselecta(sub)programandacandidateinvariantwhichholdsatthebeginningofthatsub-program(thisinvariantcouldbesuppliedbytheuser,orautomati-callygeneratedbyaninterpolatingtheoremprover).Ifthecandidateinvariantalsoholdsattheexitofthesub-program,thenitisaninductiveinvariant,andwecanreplacethatsub-programwiththeinvariant.Otherwise,wesplittheprogramintotwosub-programs.Wethencalculateanewcandidateinvariantforeachsub-program,andrepeat.Attheend,wehaveasimpleautomatonwhichconciselyrepresentsthecauseoftheinconsistency.TheresultforExample1canbeseeninFigure 2 .Webeginbyformulatingtheprogramasarst-orderlogicalformula(theleftcolumninFigure 2 (a)).Theright-handcolumnshowsinvariantswhichholdateachprogramloca-tion,andwhicharesucienttoprovetheinconsistencyofthecode.Figure 2 (b)showstheerrorinvariantautomatongeneratedusingourprocedure.Weconstructourerrorinvariantautomatonstartingfromasinglenodewhichislabeledwiththerstcandidateerrorinvarianttrue.Thisinvariantholdsupto,butnotpast,line6.Wethereforesplittheprogramintwo,withonenoderepresentinglines1{6andannotatedtrue,andtheotherrepresentingtherestoftheprogram.Wethenrepeatforlines6{end.Thepredicatetask!=nullisavalider-rorinvariantforlines7{27,sowesplittheprogramagain.Inparticular,thisinvariantisinductivefortheconditional choiceinline16,soneitherofbothbrancheshasane ectthatisrelevantfortheproof.Finally,falseisavaliderrorinvariantfortheremainderoftheprogram,sowearedone.Aprogrammerattemptingtolocatethecauseofthein-consistencybyanalyzingtheoriginalprogramwouldneedtoanalyzea28+lineprocedurecontainingconditionals.Aprogrammerusingourtoolwouldonlyneedtoanalyzeastraightforwardprocedurewithtwostatementslinkedbyonenon-trivialinvariant.2.2Example2Thisexampledemonstrateshowwedealwithnon-trivialbranches.Inthepreviousexample,wewereabletore-placetheconditionalwithaninvariant.IntheproceduretoyExample(Figure 3 ),thelocationoftheerrordependsonthevalueb.Ifbistrue,toyExampleattemptstoderefer-enceanullpointerandfailsinline4;ifbisfalse,itfailsonline6.Notethatthebranchesfailatdi erentlocations. /*Aconstructedexample*/1:publicvoidtoyExample(Booleanb){2:MyObjectx=null;3:if(b){4:x.foo();5:}6:x.bar();7:}Figure3:Branch-dependentinconsistencyincode Figure 5 shows(asimpliedversionof)theunsatisableformulacreatedfortheprograminFigure 3 .Multipleas-signmentstothesamevariablearehandledbyrepresent-ingtheprograminSSA(SingleStaticAssignment)form,withsuperscriptsusedtodistinguishbetweendi erentas-signments.Wehandleearlyterminationbyintroducinganauxiliaryvariableexit,whichissettotruewhenevertheprogramexitsearly.Allcodesubsequenttoapotentialexitise ectivelyguardedbythetest(!exit&&code).Figure 4 (b)showsacasewherewewereunabletondaninductiveinvariantfortheconditionalchoice.Wehandlethisbysplittingtheautomaton.Onebranchrepresentsthecasewheretheconditionalistaken,andtheotherrepresentsthebranchwhereitisnot.Wecanthenrecursivelyapplytheprocedureoneachbranch,asbefore.Figure 6 showstheresultofmakinganalternativechoiceforourinductiveinvariant.Thepredicatex=nullisaninductiveerrorin-variantwhichholdsacrosstheconditionalonlines3{5,andhenceallowsustocollapsethesizeoftheerrorinvariant owconstructrepresentedbyst(`ij).Wethenapplytheal-gorithmrecursivelytoallofthesesmallerautomata.Here,weexploitthepropertiesofthecomputederrorinvariantsIj,whichensurethateachsuchautomatonisinconsistentsubjecttoPre=Ij�1andPost=:Ij.ItremainstodeneformallythestatementsinthepathAthatconstitutethetop-levelbasicblockofA.Forthispurpose,lettoploc(A)betheorderedsequenceoflocationsofAthatarethestartingpointsofstatementsinthetop-levelbasicblock.Thatis,toploc(A)isthemaximalsequenceofdistinctlocations`0;`1;:::;`n+1suchthatforalli;j,suchthat0ijn+1,wehave`i A`j.Inparticular,wehave`n+1=`e.Foreverylocation`2toploc(A),wedenotebylp(`)thelengthofthelongestpathfrom`0to`inAandwedenelp(A)=lp(`e).Now,foreverylocation`i,withi2[0;n],letst(`i)beafreshstatementnotinandletYibeafreshcopyoftheprogramvariablesinX.ForaformulaFwedenotebyF[Yi=X]theformulathatisobtainedfromFbysubstitutingalloccurrencesof(primed)variablesinXbytheir(primed)versionsinYi.ThetransitionformulaTF(st(`i))ofthenewstatementst(`i)isdenedasfollows.If`iisafork,wemusthavejoin(`i)=`i+1.ThendeneTF(sti)asthedisjunctionsofthepathformulasoftheautomataforthebranchesbetween`iand`i+1.Thatis,let`i;0;:::;`i;kbetheimmediatesuccessorsof`iinA.Further,forallj2[0;k],letAj=A(`i;`i;j),mj=lp(Aj),anddenem=maxfmjj0jkg.Finally,wedeneTF(st(`i))=8�&#x]TJ ;� -1;.13; Td;&#x [00;:X=Yi^X0=Yhmii^W0jkPF(Aj)[Yi=X]^Yhmjii=YhmiiOtherwise,if`iisnotafork,then`i+1istheuniquedirectsuccessorof`iforsomestatementsti2.Inthiscase,simplydeneTF(st(`i))=TF(sti).ItiseasytoprovebyinductionthatPre^PF(A)^Posthniisunsatisablei AisinconsistentsubjecttoPreandPost.5.EXTENSIONSNext,wediscusshowwecanuseourbasicalgorithmfromtheprevioussectiontohandlecommonfeaturesfoundinactualprogramminglanguages.5.1LoopsandProcedureCallsTohandleloopsandprocedurecallsininconsistentcode,werelyonexistingtechniques.Forexample,inourpreviousworkoninconsistentcodedetection[22],wepresentedanap-proachthatwenamedabstractunrolling.Abstractunrollingover-approximatethebehaviorofaprogramwithloopsbyonewithoutloops.Thetechniqueunrollstherstandlastiterationofaloopandabstractsallintermediateiterationsbyasingletransitionthatassignsnon-deterministicvaluestothemodiedvariablesintheloop.Wehavefoundthatthistechniquescaleswellbecauseitisasimplesyntactictransformationoftheprogram,yetpreservescodeinconsis-tenciesinpractice.Inparticular,usingthistechniqueonecanstilldetectcommoncodeinconsistenciesinloopssuchaso -by-oneerrors.Sincetheabstractionover-approximatesthebehavioroftheoriginalprogramweguaranteethattheinputprogramisinconsistentiftheabstractionisinconsis-tent.Abstractunrollingcanbegeneralizedtohandleproce-durecallsbyinliningcalledproceduresintheanalyzedcode fragment,butabstractingsubsequentcallsinsidetheinlinedprocedurebodies.Itisalsopossibletocombinetheabovetechniqueswithmoreheavy-weightanalysesthatincreasethedetectionratebutaremoreexpensive.Notethattheproblemofdetect-inginconsistentcodecanbereducedtoverifyingasafetyproperty,namelythattheexitlocationoftheprogramisunreachable.Wecanthereforeuseexistingstaticanalysistechniquesforinferringloopinvariantsandproceduresum-mariestoincreasetheprecisionofabstractunrolling(re-spectively,abstractinlining).Techniquesthatarebasedoninterpolation[1,14,30]areparticularlywell-suitedbecauseourlocalizationalgorithmalreadyusesinterpolationproce-dures.Usingthecomputedinvariantsonecanthenobtainmoreprecisetransformationsintoloop-freeprograms.Insummary,theproblemofhowtodealwithloopsandprocedurecallsmustalreadyhavebeenaddressedinthede-tectionofcodeinconsistencies.Infact,a(Hoare)proofofinconsistencyofaprogramalwaysyieldsasyntactictrans-formationintoaloop-freeprogramthatisinconsistent.5.2NonstructuredControlFlowInSection4,weassumedthattheinputprogramau-tomatonhasstructuredcontrol owandouralgorithmforexplaininginconsistentcodereliesonthispropertytoen-codetheautomatone ectivelyintoaformula.Despitethisrestriction,wecanstillsupportcommonformsofunstruc-turedcontrol owthatcanbefoundinmanyprogramminglanguagessuchasreturn,break,andcontinuestatements,andexceptionmechanisms.Allthesemechanismshaveincommonthatcontroldoesnotjumparbitrarily.Instead,controlistransferredimmediatelytosomeprogramloca-tionthatisreachablebyfollowingtheregularcontrol owoftheprogram.Wecanthereforeencodethesemechanismbyintroducingauxiliaryvariables.Forexample,tomodelareturnstatement,weintroduceanauxiliaryBooleanvariablereturned.Initially,thisvari-ableissettofalseanditissettotrueifareturnstatementisexecuted.AllthetransitionformulasTF(st)ofthepro-gramarethenguardedbythisvariable,i.e.,theyareoftheform:returned)F(X;X0),whereF(X;X0)istheactualtransitionformulathatprovidesthesemanticsofstatementst.Hence,ifareturnstatementisexecuted,controlfollowsthenormal owoftheprogrambutallstatementsalongthepathareskipped.Alocationalongthepathisthenreach-ableintheoriginalprogramifitisreachableinthenewprograminastateinwhichreturnedisfalse.Othermechanismsfornon-structuredcontrol owcanbemodeledinasimilarmanner,includingassertstatementsthatcheckfortheoccurrenceofrun-timeerrorssuchasnull-pointerdereferences.Byusingdi erentauxiliaryvariablesforencodingthesemechanisms,wecanalsoclassifycodeinconsistencies,e.g.,todistinguishbetweeninconsistenciesthatarecausedbyguaranteederrors,andinconsistenciessuchascodethatisunreachablebecauseaprecedingreturnstatementisalwaysexecuted.6.EVALUATION6.1ConstructionofErrorInvariantAutomataWeevaluatedourapproachusingsixreal-worldexamplesofinconsistentcodefoundinopen-sourceprojects.ThreeexamplesweretakenfromthemindmappingtoolFreeMind, oneexample(theonefromFigure 1 )istakenfromRachota,andtheremainingtwoaretakenfromdevicedriversintheLinuxkerneldiscussedbyEngleretal.[13].Foreachoftheseexamples,weconstructedanerrorin-variantautomatonfollowingthealgorithmdiscussedinx 4 .Procedurecallswereabstractedascallinghavocontheirmodset,whichwassucienttoprovetheinconsistencyinallexamples.Noneoftheexamplescontainedloops,sowedidnothavetouseloopabstractiontechniquesinthepro-gramautomaton.Sincewewereabletoproveinconsistencyevengiventhisveryweakapproximation,allgeneratederrorinvariantautomatarepresentrealcodeinconsistencies,withnofalsealarms.Thegeneratedpathformulasfortheini-tialautomatarangedfrom70{142linesofsmt-lib2[5]code(includingcomments),withamedianof89lines.Thetrans-lationwasperformedmanually,butwasfairlymechanisticandwouldnotbediculttoautomate.Wegeneratedcandidateerrorinvariantsusingtheinter-polationproceduresimplementedintheSMTsolverMath-SAT[9].UsingrepeatedcallstotheSMTsolverwethenidentiedthecodefragmentsforwhichtheyareinductive.Insomecaseswesplitconjunctsbyaddingauxiliaryvari-ables,inordertoallowpreciseplacementoftheinterpolationpoints.Results.Runningtimetoproveunsatandgeneratetheinterpolantsrangedfrom0.008seconds(experiment4)to0.019seconds(experiment6),whichsuggeststhatthistechniqueisprac-ticalforuseinreal-timetoolssuchascodeeditors.6.2UsabilityTestingWeconductedanexperimenttoevaluatewhethererrorinvariantautomatacanbeusedtoprovidevisualassistancewhichallowsaprogrammertomorequicklyunderstandthecausesofcodeinconsistencies.Werecruited11program-mersandcomputerscientistsforthisstudy,5attheUnitedNationsUniversityinMacau,and7atNewYorkUniversity.Wegavea5minuteintroductiontoeachcandidatewhereweexplainedtheconceptofinconsistentcode,thepurposeoftheexperiment,andsomesamplesofinconsistentcode.Participantsweretoldthattheywouldbepresentedwithaseriesoffunctionswhichcontainedinconsistentcode,andthattheirjobwastoidentifythecauseoftheinconsistencyassoonaspossible.Halfoftheexamplestheywouldbeshownwouldcontaintheentirebodyoftherelevantfunc-tion,withthelinewheretheinconsistencymanifestedit-selfunderlinedinred.Theotherhalfoftheexamplesusedtheerrorinvariantautomatontoprovidevisualassistanceasfollows:allstatementsofthefunctionthatdonothaveacorrespondingedgeintheerrorinvariantautomatonarehiddenbehindsolidblueboxes.Theboxesarelabeledwiththeinvariantassociatedwiththenodeintheerrorinvariantautomatonthatsummarizesthehiddenstatementsunderit.Figure 8 givesanexampleofafunctionwithout(left)andwithvisualassistance(right).Foreachcandidatewealternatedthesnippetsforwhichweprovidedthevisualas-sistance.Foreachexample,halfoftheparticipants(chosenrandomly)wereshownthefullfunction;theotherhalfwereshowntheerrorinvariantautomaton.Assoonasacodesnippet(withorwithoutvisualassis-tance)wasonthescreen,westartedastopwatchandtoldthecandidatetosay\stop",once(s)heissurewhatthecause ofinconsistencyis.Iftheexplanationwaswrong,wecon-tinuedthestopwatch.Ifnocorrectanswerwasgivenwithin150seconds,westoppedthewatchandexplainedthesolu-tion.ThesetofslidesusedinourexperimentsisavailableonDropbox 2 .Results.Allcandidatesintotaltook1hourand6minutestoiden-tifytheproblemsinallcodesnippets.Forthecodesnippetswithoutexplanationtheytookatotalof51minutes,andforthecodesnippetswithexplanationtheytook17min-utes,whichroughlyisaspeedupbyafactorof3. Figure9:Averagetimepercandidateinsecondstospottheprobleminour6codesnippets.Theleftbarsindarkercolorrefertotheaveragetimewithoutvisualassistance,therightbarsinbrightercolorshowtheaveragetimewithassistance. Figure 9 showstheaveragetimeourcandidatestookperquestionwithandwithoutvisualassistance.Ingeneral,ourparticipantsperformedsignicantlybetterwhengivenvi-sualassistancethanwhentheywerenot.TheoneexceptionisExperiment4.Inthisexperiment,weshowedaprocedurefromFreemind,wherealocalvariableisinitializedtonull,andthenitischeckedatthreedi erentlocationsifthisvari-ableisnull,causingallthreeelseblockstobeunreachable.Thiswastheonlyexperimentwherewehighlightedmultiplelinesinthesameprogram,whichcausedconfusion.Forallotherexperiments,ourvisualassistancebasedonerrorinvariantautomatahelpedthecandidatestospottheproblemmorequickly.Weobservedthatthecandidatesgotfasterfromoneexperimenttoanother,astheygotusedtothepatternsofcodeinconsistencies.Thiscorrespondswiththefeedbackthatwegotfromourcandidatesthattheyarenotusedtolookingforinconsistencies,butaftertheyunderstoodtheproblemfounditeasiertondtherelevantstatements.Hence,forourfutureexperiments,weplantodoseveraltrainingroundswiththecandidates.6.3ThreatstoValidityThereareseveralthreatstovalidityinthisstudy.Therstisthattheparticipantsmaynotberepresentativepro-grammers,astheywereselectedbasedontheiravailabilityratherthanonstatisticallymeaningfulcriteria.This,com-binedwiththesmallsamplesizeused,makesitdiculttomakeanystatisticallyrigorousclaimsbasedonourdata. 2 http://goo.gl/FF9an