Proceedings of the TwentySecond International Joint Conference on Artificial Intelligence alsoabbreviatedtowherewhere1hasthemeaningofaprobabilityandconformstothesyntaxofanatomThemeaningofanatomi ID: 353617
Download Pdf The PPT/PDF document "GeneralisingtheInteractionRulesinProbabi..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
GeneralisingtheInteractionRulesinProbabilisticLogicArjenHommersomandPeterJ.F.LucasInstituteforComputingandInformationSciencesRadboudUniversityNijmegenNijmegen,TheNetherlands{arjenh,peterl}@cs.ru.nlThelasttwodecadeshasseentheemergenceofmanydifferentprobabilisticlogicsthatuselogi-callanguagestospecify,andsometimesreason,withprobabilitydistributions.Probabilisticlog-icsthatsupportreasoningwithprobabilitydistri-butions,suchasProbLog,useanimplicitde“nitionofaninteractionruletocombineprobabilisticev-idenceaboutatoms.Inthispaper,weshowthat Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence alsoabbreviatedto,wherewhere,1]hasthemeaningofaprobability,andconformstothesyntaxofanatom.Themeaningofanatomintermsofprobabilitytheoryisthatofasetofrandomvariables.Forexample,4::parentspeci“esthecollectionofrandomvariablesparent,foreachgroundinstanceofparent,obtainedbyapplyingparent.Forexample,parentparent,where.Theresult-inggroundatomsarecalledlogicalfactsInadditiontolabelledfacts,aProbLogprogramconsistsofrules,constitutingthebackgroundknowledgeofthepro-gram.Now,letbeaProbLogprogramandletbethesetofallpossiblesubstitutionsassociatedwiththelogicalfact.Then,isde“nedastheoflogicalfactsthatcanbeaddedtoapplyingthesetoftothefact,foreach.TheProbLogprogramthende“nesajointprobabilitydistributiononthelogicalfacts.Let,then:nowbeanyquerytotheProbLogprogram,thenitholdsthatandindicateslogicalentailment.EachiscalledanexplanationThesemanticsofProbLogiscalledthedistributionseman-;ithasbeenborrowedfromPRISMPRISMSato,1995.Basi-cally,inthedistributionsemanticsallfactsareassumedtobemutuallyindependent.However,thisdoesnotimplythatitisimpossibletoencodeprobabilisticdependences:alldepen-dencesarede“nedatthelogicallevel.Thisallowsde“ninganyjointprobabilitydistribution.Thedistributionsemanticsalsohasparticularconsequencesforobtainingprobabilisticinteractionsbetweenfacts,asillustratedbythefollowingex-Example1.Considerthefollowing(trivial)ProbLogpro-gramthatrepresentssomecausal,medicalknowledge:Inthisprogramaretwoindependentran-domvariablesaccordingtothedistributionsemantics.Wenowwishtocomputetheprobability.Notethatcanbeexplainedfromeither,orboth.Thus,accordingtoEquation(1)weget:+(114+006+0Asiswellknown,thisprobabilisticresultisidenticaltowhatwouldhavebeenobtainedbyoneofthemostpopularwaystomodeltheinteractionof(conditionally)independentevents:theso-calledso-calledPearl,1988.Withthenoisy-ORonemodelsanuncertain,disjunctiveinteractionbetweenevents.Asthenoisy-ORisbasedonlogicaldisjunction,justoneofthe16binaryBooleanoperators,onecanimaginethattheremightbeother,equallysound,waystocombineprobabilis-ticevidence.However,inordertobeabletocombinesuchevidence,oneneedsanalgebraicmethodtocombinesuchinformation.Ageneral,algebraicwaytocombineprobabilis-ticinformationisavailablefrombasicprobabilitytheoryal-thoughitisrarelyusedtomodelBooleaninteraction.Inthefollowing,webrie”yreviewthenecessarybasicsfromprob-abilitytheory,withconvolutionasaspecialcase,andtheninvestigatehowideasfromprobabilisticlogic,Booleanin-teractionanddefaultlogiccanbemergedtoobtainamoreexpressiveprobabilisticlogic,whichwecallprobabilisticin-teractionlogic,orProbILforshort.2.2ProbabilisticBooleanInteractionInthefollowing,aprobabilitymassfunctionofarandomvariableisreferredtobydenotestheassociatedprobabilitydistribution.Aclassicalresultfromprobabilitytheorythatisusefulwhenstudyingsumsofvariablesisthefollowingwell-knowntheorem(cf.(cf.GrimmettandStirzaker,Theorem1.beajointprobabilitymassfunctionoftherandomvariables,suchthat.Thenitholdsthatx,zProof..GrimmettandStirzaker,2001 areindependent,then,inaddition,thefollow-ingcorollaryholds.Corollary1.betwoindependentrandomvari-ables,thenitholdsthatTheprobabilitymassfunctionisinthatcasecalledconvolution,anditiscommonlydenotedas.Theconvolutiontheoremisveryuseful,assumsofindependentrandomvariablesoccurveryfrequently inprobabilitytheoryandstatistics.Itcanalsobeappliedre-cursively,i.e.,\b···\basfollowsfromtherecursiveapplicationofEquation(2).Theorem1doesnotonlyholdfortheadditionoftworan-domvariables,butalsoforBooleanfunctionsofrandomvari-ables.However,incontrasttothe“eldofrealnumberswhereavalueofarandomvariableisuniquelydeterminedbyarealnumber,inBooleanalgebravaluesofBooleanvariablesonlyconstrainthevaluesofotherBooleanvariables.Theseconstraintsmayyieldasetofval-ues,ratherthanasinglevalue,whichisstillcompatiblewiththetheorem.Inthefollowing,weusethenotationi,Jforsuchconstraints,wheretheBooleanvaluestoparticularvalues.Forexample,fori,qstandforhasthevaluetrue)andhasthevaluetrue),itholdsthat.Thus,isanabbreviationfori,jThetheoremcanthenbere-expressedasfollows.Theorem2.beajointprobabilitymassfunctionoftherandomvariablessuchthatI,J,withaBooleanfunction.Then,itholdsthatI,JI,Ji,bi,JProof.I,Jspacede“nedbyI,Jcanbede-composedasfollows:i,jwheretheexpressioni,jshouldbeinterpretedasalogicalconstraintontheBooleanvaluesofthevariableSincetheindividualsetsi,jmutuallyexclusive,theresultfollows. Thefollowingcorollaryforconvolutionisobtainedifareindependent.Corollary2.beajointprobabilitymassfunctionofindependentrandom,BooleanvariablesandletaBooleanfunctionde“nedon,thenitholdsthatI,Ji,JTheorem2nowappearstobethekeyinsighttogeneraliseExample2.ReconsiderExample1,andtheBooleanrelation,wherestandsfor”u,forpneumonia,andforfever.WeusethesameprobabilitydistributionsasinExample1:.ByapplyingTheorem2thefollowingresults:))+wherethetermresultsfromthelogicalcon-straintthat,i.e.,Thus,theexampledemonstratesthatthenoisy-ORcanbedescribedquitenaturallybyconvolution.SeeSeeLucasandHommersom,2010fordetails.Thecorrespondencebetweenthetwoapproachesisasfol-lows.TheBooleanfunctioncorrespondstothedetermin-isticprobabilitydistributionofEquation(1).Asisde“nedintermsoflogicalentailmentof,thequestionaddressedinthispaperiswhethertherearewaystoreplacelogicalentailmentbyareasoningmethodthatincorporatesprobabilisticBooleaninteractionasamethodtoexpressinteractionbetweenheadsofrules.SinceBooleaninteractioncanbelookeduponasaninferencerule,anaturalwaytoextendProbLogisbyreplacingstandardlogicbydefaultlogic.2.3DefaultLogicIndefaultlogiclogicReiter,1980oneaddsspecialinferencerules,calleddefaults,toordinary“rst-orderpredicatelogic.havethefollowingform:prerequisite whereprerequisiteisaconditionthat,ifitistrue,thencon-sequentcanbederived,however,whentheresultingtheorytogetherwiththeassumptionsdescribedbythejus-ti“cationsisconsistent.TheresultingtheoryisdenotedasD,W,whereinthispaperstandsforasetoffactsandrulesinHorn-clauselogicasinProbLog,andarede-faults.Inferenceindefaultlogicisdonebycomputingtheextensionsofthetheory,usinga“xedpointoperator.WewillD,W,whereisadefaultextensionD,WThenextsectionwilldeveloptheprobabilisticinteractionlogic,ProbIL,byusingdefaultstorepresentBooleaninterac-tion,which,whencombinedwithprobabilitiesyieldsproba-bilisticBooleaninteractionasdevelopedinSection2.2.3ProbabilisticInteractionLogic3.1GeneralIdeaAsExample1illustrates,probabilisticlanguagesasProbLogimplicitlycombineprobabilisticevidenceusinglogicaldis-junction,whichcorrespondstothenoisy-ORoperator.Thischoicegivesrisetoaparticularprobabilisticbehaviour,thatmaynotalwaysbejusti“ed.Theresultingprobabilitywhenusingthenoisy-ORisalwayslargerthanitscomponents,i.e.,theprobabilisticbehaviourismonotonicallyincreasing:)+(1p,q.Thus,itisnotpossibletomodelthatparticularevents,whentakentogether,canceleachotherout.Incontrast,non-monotoniclogicssuchasdefaultlogiccanbeusedtoimplementsuchreasoning.ThegeneralideaofProbabilisticInteractionLogic(Pro-bIL)istoreplacethebackgroundtheorybyadefaultlogictheory.Thislogichasthesamebene“tsasdefaultlogic,namelythatwecanspecifyproblem-dependentdefaultbe-haviourwithoutsacri“cingordinarylogicaldeduction.InthecontextofProbIL,defaultsmodelvalidinteractionsdepen-dentofthegenericproblem-solvingmethodthatneedstobeexpressedoroftheactualproblemathand. Formally,theprobabilityisadaptedasfollows.Weassumewehaveabackgroundtheoryconsistingofasetofdefaultsandasetofstandardlogicalrules.Thenwehave:D,Bi.e.,themonotoniclogicalentailmenthasbeenreplacedbythedefaultlogicentailmentExample3.Thepopularexamplefromnon-monotoniclog-icsbirdstypically”yŽismodelledbythedefaultset andweaddthestandardlogicruleex-pressingthatPenguinsdonot”y,i.e.,Penguin.Further,wemightknowthatinapetstore30%oftheanimalsarebirds,i.e.,.TocomputewhetherTweety()”ies,wecompute,))=))=0Thishasthesamerepresentationalbene“tasdefaultlogicincomparisontostandardlogic,i.e.,theexamplecouldalsobeformalisedbytheProbLogrule:PenguinHowever,if,asintheexample,wedonotknowwhetherTweety”iesandwedonothaveadistributionforTweetybe-ingapenguin,then))=0.Thisillustratesthein-terplaybetweennon-monotonicandprobabilisticreasoning,whichisnotavailableinmanyexistingprobabilisticlogics.InthefollowingweusethismechanismtostudytypesofinteractionsthatcanbemodelledandwaystorepresenttheresultingprobabilitymassexplicitlybyusingprobabilisticBooleaninteraction,andbyaconvolutionoperatorincaseofindependence.3.2BooleanInteractionsConsiderthefollowingProbLogclauses:c,a.Inordertomodeltheinteractionbetweentheprobabilisticevi-dencecontributingtotheprobabilityof,weneedtospec-ifyacombinationfunctionfortheseclauses,i.e.,wewishtointerprettheseclausesasc,e,withaBooleanfunction.Aswehaveseen,inthecaseofProbLog,c,ealwaysequalto.Defaultlogic,asanexpressivelogicalformalism,isusedheretoreasonabouttheseBooleaninter-actions.ThefollowingpropositionexpressesthatallpossibleinterpretationsofBooleaninteractionscanindeedbemod-elledindefaultlogic.Proposition1.ForallBooleanfunctionsandasetofatomsH,Bthereisadefaultlogictheory,whichdoesnotcontain,suchthat:D,BiffToillustratethis,welistanumberofcommonbinaryBooleanfunctionsinFigure1withtheirassociateddefaultlogictheory.Hence,foragivensetofinteractions,wemay NameDefaultrulesChoicesfor false true H({B1,B2}) ANDB1,B2 H{{1,B2}} ORB1 H,B2 H({B1,B2})\{} EQB1,B2 H,:¬B1,¬B2 H{,{B1,B2}} XORB1:¬B2 H,B2:¬B1 H{{1},{B2}} Figure1:RepresentationofthebinarycommutativeandassociativeBooleanfunctionsusingdefaultlogicforrulesuchthatD,Liff.Inthis,denotesthepowersetof.Other,lesscommonBooleanoperatorscanbede“nedsimilarly.replacetherelevantrulesbythecorrespondingtheorywheretheinteractionsaremodelledbydefaultrules.Ofcourse,theconverseofProposition1alsoholds:exten-sionsofadefaulttheorycanbecharacterisedusingaBooleanfunction.Inthefollowing,weassumethatwehaveaBooleanfunctiontomodeltheinteraction.Example4.Peoplehavingheadacheoftenuseover-the-counterpainmedicationtoobtainpainrelief().However,scienti“cevidenceindicatesthatheadachecanactuallybetriggeredbytheoveruseofcommonpainkillers.Supposewehavetwopainkillerswhichincombinationarein-effectiveagainstheadache.Alsoassumethatpainkillerthinstheblood().Finallyassumewehavetwoprobabilis-ticfactsthatmodeltheprobabilisticdecisionofwhetherornottotake.WecanformalisethisusingthefollowingProbLogtheory:undermineeachothereffecton,wewouldliketoincludeanXORinteractionfunctionfor.Thus,theprobabilisticrulesshouldbeinterpretedasthedefaulttheory: r,k2:¬k1 togetherwithapurelylogicaltheory:Weareinterestedinhypothesessuchthat:D,Br,t.For,theonlyexplanationsare,butnot.For,ontheotherhand,theexplana-tionsare.Hence,theonlycommonexpla-nationfor 3.3ProbabilisticBooleanInteractionDefaultlogicprovidesareasoningmechanismforBooleaninteractions.Inthissection,weexplorethecomplementaryprobabilisticperspective,bygeneralisingresultspresentedinSection2.2.Asmentioned,Theorem2canbeusedasabasisforprobabilisticinteractionlogic.Theorem3.beanatomthatappearsinthetheorybyasinglerule:I,JFortheprobabilitymassfunctioni,bi,JProof.I,Ji,bi,Ji,bi,J Example5.ReconsiderExample4.Astheexplanationsofare,wehave:)+(1UsingthestandardsemanticsofProbLog,notethat.ByapplyingTheorem3,weobtain)+(1andtheresultscorrespond.However,ifwehadusedthestan-dard(noisy-OR)semanticsofProbLog,wewouldhaveob-+(1,whichislargerthanorequaltoAnassumptionthatisoftenmadeinprobabilisticlogicslogicsPoole,1993),isthatexplanationsaremutuallyexclu-sive,i.e.,,withexplanations.Withoutthisassumption,theprobabilityofthedisjunctioniscomputedby“ndinganequivalentdisjunctionwherethedisjunctsaremutuallyexclusive(calledproblem).Similarly,ifweassumethatbod-iesofahead,say,areindependentofeachother,nodisjoint-sumproblemhastobesolvedasthenitholdsthat.Thiscanbeseenasasomewhatweakerrequirement,asthisdoesnotrestrictthestructureoflogicaltheories,butonlytheprobabil-itydistributionthatisgenerated.Inthecaseofindependentbodiesforaquery,convolutioncanbeexploited.Corollary3.beanatomthatappearsintheoryasasinglerule:I,Jindependent.Thentheprobabilitymassfunc-i,JExample6.ReconsideragainExample4.Itisnotdif“culttoseethat.So,wemayusetheconvolutionasfollows:)+(1whichyieldsagainthesameresultasbefore.However,inthiscasethecomputationofisindependentof3.4RepresentationIncasetheindependenceassumptiondoesnothold,adisjoint-sumproblemhastobesolved,e.g.,usingbinaryde-cisiondiagrams(BDDs)(BDDs)Kimmigetal.,2010.However,itcannotbedeterminedbeforehandwhenthisproblemhastobesolved,soarepresentationofsuchindependencecanavoidtheoverheadofBDDsandimprovetheinferenceinpractice.Syntactically,weintroducealabellingofrules,andusethistodenoteaBooleaninteractionbetweenruleswiththesamehead.Forexample:c,re,bInthisspeci“cationexpressesthat bf(e)where denotesconvolutionusingBooleanoperatorThisoperatorisusefulforalgebraicmanipulationofmodelswithcomplexinteractionsbetweenformulae,asthealgebraicpropertiesoftheBooleanoperatorcarryovertothe erator.Moreover,itdirectlysimpli“esinferenceasitpreventstheoverheadofsolvingthedisjoint-sumproblemfor.Fi-nally,therepresentationisgenericforbothdiscretemodels,continuousmodels,andmixturesofthesetheseLucasandHom-mersom,2010,sotheresultsgeneralisetoprobabilisticlog-icalmodelswithcontinuousdistributions,suchasproposedproposedGutmannetal.,20104RelatedWorkItiswell-knownthatdefaultlogiccanbeembeddedinalogicprogramminglanguagewithnegationasfailureailureKakas,1994].AprobabilisticvariantofsuchalogicprogramminglanguagewithsuchnegationsisICLICLPoole,1997.An-otherapproachisthelanguageofP-logP-logBaraletal.,2009whichextendstheAnswerSetProgramming(ASP)frame-workwithprobabilisticfacts.Bothapproacharecloselyre-latedtoProbILinthesensethattheybothareprobabilisticnon-monotoniclogics.Thegoalofthispaper,however,is complementaryaswearenotfocusedonnon-monotonicthe-oriesperse,butrather,theorieswithdifferenttypesofinter-actionsbetweenbodies.Infact,itholdsforbothICLandP-logthattheinteractionsbetweenbodiesfollowanoisy-ORinteraction.Incontrast,thispaperhasshowedthatinterac-tionscanbefaithfullyrepresentedinanon-monotoniclogicandcansometimesbedecomposedusingatypeofconvo-lutionoperator.ProbILisusedasabasiclanguagetogiveinsighttotheformergoalasitveryintuitivelycanrepresentcausesthatcanceleachotherout.Fromamorepracticalpointofview,othersystemscouldbeusedtoactuallyrepresentandcomputetheseprobabilities.5ConclusionsInthispaper,weproposedanewmechanismformodellinginteractionsinprobabilisticlogics.AssuggestedininPoole,1990],abductioncanbeseenasformalismforexplainingob-servations,whereasdefaultlogicisusedtomakepredictions.WhilelanguagessuchasProbLoguseanon-monotonicap-proach(abduction)forexplainingapossiblequery,predic-tionsarecompletelymonotonic.Wehaveuseddefaultrea-soninginthiscontextofpredictionasamethodformodellinginteractionsbetweenprobabilisticfacts.Defaultreasoningisdif“cultingeneral;initsfullgen-erality,defaultabductionisintractableintractableEiteretal.,1997Nonetheless,algorithmsforsolvingthisproblemforproposi-tionaldefaulttheorieshavebeendevelopedthatuseef“cientquanti“edBooleanformulasolversersTompits,2003.More-over,wehaveshownthatformanyclassesofproblems,theprobabilisticinteractionscanbedecomposedandeffectivelyrepresentedbyaconvolutionoperator.Thismaysigni“cantlyimprovethecomputationalcomplexityofreasoninginawaysimilartotheuseofcausalindependencemodelsinBayesiannetworksorksZhangandPoole,1996.AlthoughwehavenotpresentedapracticalimplementationofProbILinthispaper„ProbILisratheractingasagenerallanguageforrepresent-inginteractions„itcouldbeimplementedusingsomeoftheexistingapproachesinprobabilisticlogicsbasedonlogicprogramming.Actuallydevelopingsuchimplementationsisfutureresearch.Importantinthisworkisthatwehaveincorporateddiffer-entmethodsofreasoningintoa”exibleprobabilisticlogic,whilestillmaintainingtheoveralldesignaimofprovidinglogicalandprobabilisticreasoninghandinhand.Webelievethatthiscanbeofhelpformodellingandreasoningaboutawiderangeofactualproblems.AcknowledgmentsArjenHommersomwassupportedbyVENIGrant639.021.918fromTheNetherlandsOrganizationofScien-ti“cResearch.Wethankthereviewersfortheirconstructivecommentswhichhavesigni“cantlyimprovedthispaper.ReferencesencesBaraletal.,2009ChittaBaral,MichaelGelfond,andNel-sonRushton.Probabilisticreasoningwithanswersets.TheoryandPracticeofLogicProgramming,9:57…144,9:57…144,DeRaedtetal.,2008L.DeRaedtetal.Towardsdigestingthealphabet-soupofstatisticalrelationallearning.In2008WorkshoponProbabilisticProgramming,2008.2008.Eiteretal.,1997T.Eiter,G.Gottlob,andLeoneN.Se-manticsandcomplexityofabductionfromdefaulttheo-Arti“cialIntelligence,90(1-2):177…223,1997.1997.GrimmettandStirzaker,2001G.GrimmettandD.Stirza-ker.ProbabilityandRandomProcesses.OxfordUniver-sityPress,Oxford,2001.2001.Gutmannetal.,2010B.Gutmann,M.Jaeger,andL.DeRaedt.Extendingproblogwithcontinuousdistributions.InProcILP2010,2010.2010.Kakas,1994A.Kakas.Defaultreasoningvianegationasfailure.InG.LakemeyerandB.Nebel,editors,Founda-tionsofKnowledgeRepresentationandReasoning,vol-ume810of,pages160…178.SpringerBerlin/Hei-delberg,1994.1994.Kimmigetal.,2010A.Kimmig,B.Demoen,L.DeRaedt,V.SantosCosta,andR.Rocha.OntheimplementationoftheprobabilisticlogicprogramminglanguageProbLog.TheoryandPracticeofLogicProgramming,2010.2010.LucasandHommersom,2010P.J.F.LucasandA.J.Hom-mersom.ModellingtheinteractionsbetweendiscreteandcontinuouscausalfactorsinBayesiannetworks.InProc,pages185…192,2010.2010.Pearl,1988J.Pearl.ProbabilisticReasoninginInteligentSystems:NetworksofPlausibleInference.MorganKauf-mann,1988.1988.Poole,1990D.Poole.Amethodologyforusingadefaultandabductivereasoningsystem.InternationalJournalofIntelligentSystems,5(5):521…548,1990.1990.Poole,1993D.Poole.ProbabilisticHornabductionandBayesiannetworks.Arti“cialIntelligence,64:81…129,64:81…129,Poole,1997D.Poole.Theindependentchoicelogicformodellingmultipleagentsunderuncertainty.Arti“cialIn-telligence,94(1-2):7…56,1997.1997.Reiter,1980R.Reiter.Alogicfordefaultreasoning.“cialIntelligence,13(1-2):81…132,1980.1980.RichardsonandDomingos,2006M.RichardsonandP.Domingos.Markovlogicnetworks.MachineLearning62(1-2):107…136,2006.2006.Sato,1995T.Sato.Astatisticallearningmethodforlogicprogramswithdistributionsemantics.InL.Sterling,ed-itor,IntConfLogicProgramming,pages715…729.MITPress,1995.1995.Tompits,2003H.Tompits.Expressingdefaultabductionproblemsasquanti“edBooleanformulas.AICommun.16:89…105,June2003.2003.ZhangandPoole,1996N.L.ZhangandD.Poole.Exploit-ingcausalindependenceinBayesiannetworkinference.JAIR,5:301…328,1996.