VaqUoT:AToolforVacuityDetectionMihaelaGheorghiuandArieGurnkelDepartme - PDF document

VaqUoT:AToolforVacuityDetectionMihaelaGheorghiuandArieGurnkelDepartmentofComputerScience,UniversityofToronto,Toronto,ONM5S3G4,Canada.Email:fmg,arieg@cs.toronto.eduAbstract.ThispaperpresentsVaqUoT–aUniversityofTorontotoolforvacu-itydetection,builtontopofNuSMV.Inonemodel-checkingpass,VaqUoTestablishesthetruthvalueofaCTLformulaaswellasthelargestsetofnon-overlappingsubformulasinwhichthatformulaisvacuous.Wedescribethetoolandevaluateitsperformance.Duringmodel-checking,propertiesaresometimessatisedbymodelsforthewrongreasons.SupposeaCTLformula =AG(r_y_g)ischeckedagainstamodelofatrafc-lightcontroller,whereatomicpropositionsr,y,andgstandforthecolorsofthelight:red,yellow,andgreen,respectively.Theformulaisintendedtoexpressthatineverystatethelighthasoneofthesecolors.Thisrequirementmaynotbesatised,evenif passesthecheck:itispossibleforthemodeltobeoverconstrainedsothatthelightalwaysstaysred.Insuchcases,ananswer“true”,givenusuallybymodel-checkers,isinsufcient;auserneedstoknowwhytheformulaissatised.Vacuitydetection[2]canhelp,bydeterminingwhethersomepartsoftheformuladonotmatterfortheverication,i.e.,arevacuous.Forinstance,yandgshouldbereportedasvacuousin .Althoughvariousapproachestovacuitydetectionhavebeenproposed(e.g.,[1,2,6]),fewimplementationshavebeenreported[1,7],andtoourknowledgenonearepubliclyavailable.OurtoolVaqUoTispubliclyavailableasapatchfortheopen-sourcemodel-checkerNuSMV.VaqUoTisbasedontechniquesdescribedin[5],whereamulti-valuedlatticeisintroducedforthedetectionofallvacuoussubformulas.Sincethislatticedoesnotimmediatelyleadtoanefcientimplementation,hereweconsiderasimplerlattice,butasimilarapproach.GivenamodelandaCTLformula,VaqUoTcheckswhethertheformulaistrueinthemodel,andreportsallthevacuousatomicpropositions.Following[6],weconsiderapropositionvacuousifitcanbereplacedbyaconstant(TrueorFalse)withoutaffect-ingthevalueoftheformulainthemodel.Wetreatdifferentoccurrencesofthesameatomicpropositionasdifferentpropositions.Whentheformulaistrue,VaqUoTreportswhetherallofitsatomicpropositionsarevacuous(VacuouslyTrue),noneofthemarevacuous(Non-VacuouslyTrue),orsomeoftheatomicpropositionsarevacuous(Vac-uouslyTruefollowedbyalistofthevacuouspropositions).Similaranswersaregivenwhentheformulaisfalse.Implementation.ThebasisofVaqUoTisamulti-valued“vacuity”latticeandatranslationofCTLformulasintothislattice.InsteadoftheformulasbegininterpretedovertheBooleanlattice(fTrue;Falseg;),theyareinterpretedoverthevacuitylatticeLV=(fTrue;Falseg2A;v),where2AisthepowersetofthesetAofatomicpropositions.LatticeLVisdeterminedonlybythenumberofatomicpropositions (True;fa;b;cg)(False;fa;b;cg)(False;fa;bg)(True;fa;bg)(False;fa;cg)(True;fb;cg)(True;fa;cg)(True;;)(False;;)(False;fb;cg)(True;fbg)(True;fcg)(False;fcg)(False;fbg)(False;fag)(True;fag)ModelFormulasBasicMCNaiveVDVaqUoTTotalVacuousMemoryTimeWitnessesTimeelevator3l452643.51690.163995228.941441.22guidance23161054.18244306.99274.81production15157.242.41187228.87184.08-cellabp104310.683.1826316.63304.51fgs562106189.5782239.04191.92msiwtrans15310.330.218153.6383.98luckySeven4012.9469.33201257.11842.48eisenberg54311.312535.7739.77ticTacToe4238.915.8136368.72102.51Fig.1.(a)Vacuitylatticeforthreeatomicpropositions;(b)Experimentalresults.intheformulabeingchecked.Anelement(t;s)2LVisapossibleresultofvacu-itydetection,showingthattheformulahastruthvaluet,andthelargestsubsetofitsatomicpropositionsthatarevacuousiss.Foranyu;v22A,(False;u)v(True;v),(True;u)v(True;v)iffuv,and(False;u)v(False;v)iffvu(notethere-versalofsetinclusion).ThetopelementofLVis(True;A),orVacuouslyTrue.Thebottomelementis(False;A),orVacuouslyFalse.Thevacuitylatticeforthreeatomicpropositionsa;b;cisdepictedinFigure1(a).InVaqUoT,wereplaceeachatomicpropositionaof'by((a^VTAna)_VFAna),whereVTAnaandVFAnadenotelatticevalues(True;Anfag),and(False;Anfag),respectively.Allreplacementsaredonesimultaneously.Theresultingmulti-valuedfor-mulaisthenmodelchecked.Thejusticationforthistranslationfollowsfrom[5].Forexample,VaqUoTreportsthatthetrafc-lightformula denedearlierholdsandisvacuousinbothyandg,byoutputting(True;fy;gg).Inourimplementation,weencodeeachvalueofthevacuitylatticeLVasa32-bitword.Theleast-signicantbitrepresentsthetruth:1forTrue,0forFalse.Theotherbitsrepresentthevacuity:0forvacuous,1fornon-vacuous.Forinstance,forthetrafc-lightformula ,thelatticevalue(True;fy;gg)isrepresentedbytheword00:::00011,wheretherightmost0011means,respectively,thatgandyarevacuous,risnot,andthetruthvalueisTrue.Thus,latticeoperationscanbeefcientlyimplementedbitwise.Thexedwordlength,whichcouldbeincreasedfrom32to64or128,limitsthenumberofatomicpropositionsintheformulaswecancheckefcientlyto31,63,127,respectively.Bitvectorsofarbitrarylengthcouldbeused,atthecostofincreasingthecomplexityoflatticeoperations.VaqUoTisbuiltontopofNuSMV,whichusestheCUDDpackagefortheimple-mentationofdecisiondiagrams(DDs)[4].Wehaveimplementedmulti-valuedDDsusingCUDDADDsandchangedtheinterfacebetweenNuSMVandCUDDsothatourmulti-valuedoperationsareperformedinsteadoftheirBDDcounterparts.Thesemod-icationsdonotaffectthecomplexityofdecisiondiagramoperationsorxpointcom-putations,buttheymayaffectperformance,sincethedecisiondiagramsmaybelarger.OurchangesarecompatiblewiththevariousNuSMVoptimizations(e.g.,coneofin-uence,dynamicreordering,partitioning).ThetoolisavailableasapatchforNuSMVv.2.1.2,fromwww.cs.toronto.edu/fm/vaquot.html.Experiments.AfewexperimentscomparingVaqUoTwithbasicmodelcheckingandwithanaiveapproachtovacuitydetectionarereportedinFigure1(b).Thenaive2 approachconsistsofseparatelyreplacingeachatomicpropositionbyTrueandthenbyFalseandchecktheresultingformulas,inadditiontotheoriginalformula;alltheseformulasarecalledwitnesses.ThenumberofwitnessesreportedinFigure1(b)istheactualnumberofformulascheckedinthenaiveapproach,whichweimplementedontopofNuSMVaswell.TheexperimentswereperformedonaDellPCwitha2.4GHzIntelCeleronCPUand512MBofRAM,runningLinux2.4.20.Modelsguidance,production-cell,abp10,andmsiwtrans,andmostoftheirpropertiesarefromtheNuSMVdistribution.elevator3lisamodelofathree-oorelevatorsys-temwrittenbystudentstakingtheAutomatedVericationclassatUniv.ofToronto,andfgs5isaproprietarymodelforaight-guidancesystem.ModelsluckySeven,eisenberg,andticTacToeareSMVtranslationsoftheirVerilogcounterpartsdis-tributedwiththeVISmodelchecker.Foreachmodel,wereportthetotalnumberofformulascheckedandhowmanywerefoundvacuous(Formulas),thetotalmemory(inMB)andtime(inseconds)usedbymodel-checkingwithoutvacuitydetection(BasicMC),thetotalnumberofwitnessesandthetimeusedbythenaivevacuitydetection(NaiveVD),andtherunningtimeofVaqUoT.Asitcanbeseen,VaqUoTperformsbetterthanthenaiveapproachinmostcases,andbyaconsiderablemargininsome:ouralgorithmavoidsmuchoftheredundantworkperformedbythenaiveapproach.InthecaseswhereVaqUoTperformsworse,weobservedthatthesizesofthedecisiondiagramsarethebottleneck,andweareinvestigatingwaystoovercomethis.Themethodof[7]istheclosesttoours,fromrelatedwork.In[7],witnessesaregeneratedandcheckedinparallelandcompositionally,byabottom-upexplorationoftheparsetreeofaformula,withexplicitcachingofintermediateresults.Therepre-sentationofwitnessesisexplicitaswell.Alltheseareimplicitinthemulti-valueddecisiondiagramsinourimplementation.Trueandfalseformulasaretreateddiffer-ently,whereasVaqUoThandlesbothuniformlyinonepass.Extensiveexperimentsandcomparisonsbetweenthetwomethodsremainforfuturework;however,theresultsshowninFigure1(b),specically,forthelastthreeexamples(usedalsoin[7]),indi-catethatbothtoolsexhibitasimilarimprovementoverthenaiveapproach.Inaddition,foreisenbergandticTacToe,VaqUoTfoundmorevacuouspasses.ComplementarytoourvacuitycheckingofCTLfromulasusingBDD-basedtech-niques,theworkof[8]addressesvacuitycheckingofLTLformulas,implementedusingSAT-basedmethods.Inaparalleldevelopment,[3]re-examinesthemeaningofvacuityintermsofsystemversusenvironmentbehavior,andarguesthatcurrentvacuitycheck-ingmethodologyproducestoomanyfalsepositives,thatis,casesofvacuitythatdonotindicateproblems.Asanalternative,[3]proposescheckingwhenformulaspass/failsolelyduetoerrorsintheenvironmentmodel,andshowsonarealisticcasestudythatthisnewmethodologydiscoverstrulyproblematiccasesofvacuity.References1.R.Armoni,L.Fix,A.Flaisher,O.Grumberg,N.Piterman,A.Tiemeyer,andM.Vardi.ªEn-hancedVacuityDetectioninLinearTemporalLogicº.InProc.ofCAV'03,volume2725ofLNCS,pages368–380,2003.2.I.Beer,S.Ben-David,C.Eisner,andY.Rodeh.ªEf®cientDetectionofVacuityinACTLFormulasº.InProc.ofCAV'97,volume1254ofLNCS,pages279–290,1997.3 3.M.Chechik,M.Gheorghiu,andA.Gur®nkel.ªFindingEnvironmentalGuaranteesª.CSRGTechnicalReport,submittedforpublication,UniversityofToronto,April2006.4.A.Cimatti,E.M.Clarke,,E.Giunchilia,F.Giunchiglia,M.Pistore,M.Roveri,R.Sebastiani,andA.Tacchella.ªNUSMVVersion2:AnOpenSourceToolforSymbolicModelCheckingº.InProc.ofCAV'02),volume2404ofLNCS,pages359–364,2002.5.A.Gur®nkelandM.Chechik.ªHowVacuousIsVacuous?º.InProc.ofTACAS'04,volume2988ofLNCS,pages451–466,2004.6.O.KupfermanandM.Vardi.ªVacuityDetectioninTemporalModelCheckingº.InProc.ofCHARME'99,volume1703ofLNCS,pages82–96,1999.7.M.PurandareandF.Somenzi.ªVacuumCleaningCTLFormulaeº.InProc.ofCAV'02,volume2404ofLNCS,pages485–499,2002.8.J.Simmonds,J.Davies,andA.Gur®nkel.ªVaqTree:ExlpoitingResolutionProofsforLTLVacuityDetectionª.acceptedatfm'06toolsession,UniversityofToronto,June2006.4