Jeavons Computing Laboratory University of Oxford Wolfson Building Parks Road Oxford OX1 3QD United Kingdom stanislavzivnypeterjeavons comlaboxacuk Abstract Submodular constraints play an important role both in the ory and practice of valued constra ID: 86418
Download Pdf The PPT/PDF document "Classes of Submodular Constraints Expres..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
variablesinthescopeoftheconstraint.Thegoalistondanassignmentofvaluestoallofthevariableswhichhastheminimumtotalcost.Weremarkthatinnitecostscanbeusedtoindicateinfeasibleassignments(hardconstraints),andhencetheVCSPframeworkincludesthestandardCSPframeworkasaspecialcaseandisequivalenttotheConstraintOptimisationProblem(COP)framework[21],whichiswidelyusedinpractice.OnesignicantlineofresearchontheVCSPistoidentifyrestrictionswhichensurethatinstancesaresolvableinpolynomialtime.Therearetwomaintypesofrestrictionsthathavebeenstudiedintheliterature.Firstly,wecanlimitthestructureoftheinstances.Wewillnotdealwiththisapproachinthispaper.Secondly,wecanrestricttheformsofthevaluedconstraintswhichareallowedintheproblem,givingrisetoso-calledlanguagerestrictions.Severallanguagerestrictionswhichensuretractabilityhavebeenidentiedintheliterature,(seee.g.,[8]).Oneimportantandwell-studiedrestrictiononvaluedconstraintsissubmodularity.Infacttheclassofsubmodularconstraintsistheonlynon-trivialtractablecaseinthedichotomyclassicationoftheBooleanVCSP[8].Theconceptofsubmodularitynotonlyplaysanimportantroleintheory,butisalsoveryimportantinpractice.Forexample,manyoftheproblemsthatariseincomputervisioncanbeexpressedintermsofenergyminimisation[16].TheproblemofenergyminimisationisNP-hardingeneral,andthereforealotofresearchhasbeendevotedtoidentifyinginstanceswhichcanbesolvedmoreeciently.KolmogorovandZabihidentiedclassesofinstancesforwhichtheen-ergyminimisationproblemcanbesolvedeciently[16],andwhichareapplicabletoawidevarietyofvisionproblems,includingimagerestoration,stereovisionandmotiontracking,imagesynthesis,imagesegmentation,multi-camerascenereconstructionandmedicalimaging.Theso-calledregularitycondition,whichspeciestheecientlysolvableclassesin[16],isequivalenttosubmodularity.Thenotionofsubmodularityoriginallycomesfromcombinatorialoptimi-sationwheresubmodularfunctionsaredenedonsubsetsofagivenbaseset[14,18].ThetimecomplexityofthefastestknownalgorithmfortheproblemofSubmodularFunctionMinimisation(SFM)isroughlyO(n6)[19].How-ever,thereareseveralknownspecialclassesofSFMthatcanbesolvedmoreecientlythanthegeneralcase(see[3]forasurvey).Cohenetal.showedthatVCSPswithsubmodularconstraintsoveranarbi-trarynitedomaincanbereducedtotheSFMproblemoveraspecialfamilyofsetsknownasaringfamily[8].ThisproblemisequivalenttothegeneralSFMproblem[23],thusgivinganalgorithmoforderO(n6+n5L),whereListhelook-uptime(neededtoevaluateanassignmenttoallvariables),foranyVCSPwithsubmodularconstraints.Thistractabilityresulthassincebeengeneralisedtoawiderclassofvaluedconstraintsoverarbitrarynitedomainsknownastournament-pairconstraints[6].Analternativeapproachcanbefoundin[9].InthispaperwefocusonsubmodularconstraintsoveraBooleandomainf0;1g,whichcorrespondpreciselytosubmodularsetfunctions[8].WedescribeanalgorithmbasedongraphcutswhichcanbeusedtosolvecertainVCSPswithsubmodularconstraintsoveraBooleandomainmuchmoreecientlythan2 assignmentfortheinstancePisamappingsfromVtoD.Thecostofanassignmentsisdenedasfollows:CostP(s)=Xhhv1;v2;:::;vmi; i2C (hs(v1);s(v2);:::;s(vm)i):AsolutiontoPisanassignmentwithminimumcost.Anyset�ofcostfunctionsiscalledavaluedconstraintlanguage.TheclassVCSP(�)isdenedtobetheclassofallVCSPinstanceswherethecostfunc-tionsofallvaluedconstraintsliein�.InanyVCSPinstance,thevariableslistedinthescopeofeachvaluedcon-straintareexplicitlyconstrained,inthesensethateachpossiblecombinationofvaluesforthosevariablesisassociatedwithagivencost.Moreover,ifwechooseanysubsetofthevariables,thentheirvaluesareconstrainedimplicitlyinthesameway,duetothecombinede ectofthevaluedconstraints.Thismotivatestheconceptofexpressibilityforcostfunctions,whichisdenedasfollows:Denition2.ForanyVCSPinstanceI=hV;D;Ci,andanylistofvariablesofI,l=hv1;:::;vmi,theprojectionofIontol,denotedl(I),isthem-arycostfunctiondenedasfollows:l(I)(x1;:::;xm)=minfs:V!Djhs(v1);:::;s(vm)i=hx1;:::;xmigCostI(s):Wesaythatacostfunction isexpressibleoveravaluedconstraintlanguage�ifthereexistsaninstanceI2VCSP(�)andalistlofvariablesofIsuchthatl(I)= .WecallthepairhI;liagadgetforexpressing over�.VariablesfromVnlarecalledextraorhiddenvariables.Notethatinthespecialcaseofrelations(crispcostfunctions)thisnotionofexpressibilitycorrespondstothestandardnotionofexpressibilityusingcon-junctionandexistentialquantication(primitivepositiveformulas)[4].Wedenotebyh�itheexpressivepowerof�whichisthesetofallcostfunctionsexpressibleover�uptoadditiveandmultiplicativeconstants.2.2SubmodularfunctionsandpolynomialsAfunction :2V!QdenedonsubsetsofasetViscalledasubmodularfunction[18]if,forallsubsetsSandTofV, (S\T)+ (S[T) (S)+ (T):TheproblemofSubmodularFunctionMinimisation(SFM)consistsinndingasubsetSofVforwhichthevalueof (S)isminimal.Foranylattice-orderedsetD,acostfunction :Dk! Q+iscalledsub-modularifforeveryu;v2Dk, (min(u;v))+ (max(u;v)) (u)+ (v)wherebothminandmaxareappliedcoordinate-wiseontuplesuandv.Notethatex-pressibilitypreservessubmodularity:ifevery 2�issubmodular,and 02h�i,then 0isalsosubmodular.Usingresultsfrom[8]and[24],itcanbeshownthatanysubmodularcostfunction canbeexpressedasthesumofanite-valuedsubmodularcostfunc-tion fin,andasubmodularrelation crisp,thatis, = fin+ crisp.More-over,itisknownthatallsubmodularrelationsarebinarydecomposable[15],4 Foranyd2Dandc2 Q+,wedenetheunarycostfunctioncdasfollows:cd=(cifx6=d,0ifx=d.Itisstraightforwardtocheckthatall wandcdaresubmodular.Wedenetheconstraintlanguage�cuttobethesetofallcostfunctions wandcdoveraBooleandomain,forc;w2 Q+andd2f0;1g.Theorem5.Theproblems(s;t)-Min-CutandVCSP(�cut)arelinear-timeequivalent.Proof.Consideranyinstanceof(s;t)-Min-Cutwith(directed)graphG=hV;Eiandweightfunctionw:E! Q+.DeneacorrespondinginstanceIofVCSP(�cut)asfollows:I=hV;f0;1g;fhhi;ji; w(i;j)ijhi;ji2Eg[fhs;10i;ht;11igi:NotethatinanysolutiontoIthesourceandtargetnodes,sandt,musttakethevalues0and1,respectively.Moreover,theweightofanycutcontainingsandnotcontainingtisequaltothecostofthecorrespondingassignmenttoI.Hencewehaveshownthat(s;t)-Min-CutcanbereducedtoVCSP(�cut)inlineartime.Ontheotherhand,givenaninstanceI=hV;D;CiofVCSP(�cut),constructagraphonV[fs;tgasfollows:anyunaryconstraintonvariablevwithcostfunctionc0(respectivelyc1)isrepresentedbyanedgeofweightcfromthesourcenodestonodev(respectively,fromnodevtothetargetnodet).Anybinaryconstraintonvariableshv1;v2iwithcostfunction wisrepresentedbyanedgeofweightwfromnodev1tov2.ItisstraightforwardtocheckthatasolutiontoIcorrespondstoaminimum(s;t)-cutofthisgraph.utCorollary6.VCSP(�cut)canbesolvedincubictime.Proof.ByTheorem5,VCSP(�cut)hasthesametimecomplexityas(s;t)-Min-Cut,whichisknowntobesolvableincubictime[13].utUsingastandardreduction(see,forexample,[3]),wenowshowthatallbinarysubmodularcostfunctionsoveraBooleandomaincanbeexpressedover�cut.Theorem7.�sub;2h�cuti.Proof.ByCorollary4,anycostfunctionfrom�sub;2canberepresentedbyaquadraticBooleanpolynomialp(x1;x2)=a0+a1x1+a2x2+a12x1x2wherea120.Thiscanthenbere-writtenasp(x1;x2)=a00+Xi2Pa0ixi+Xj2Na0j(1�xj)+a012(1�x1)x2;whereP\N=;,P[N=f1;2g,a012=�a12,anda0i;a0j;a0120.(Thisisknownasaposiform[3].)6 Givenanypolynomialp,wecanuseasimilarconstructiontoreplaceeachtermofdegree3inturn,introducingadistinctnewvariableyeachtime.Proceedinginthisway,wecanexpressanypolynomialprepresentingacostfunctionin�negasaquadraticpolynomialwithnon-positivequadraticcoe-cients,introducingknewvariables,wherekisthetotalnumberoftermsofdegree3.Suchaquadraticpolynomialcanbeexpressedover�sub;2,byCorol-lary4.utCorollary10.Foranyxedk,VCSP(�neg;k)canbesolvedincubictime.Proof.ByTheorem9,anyinstanceofVCSP(�neg;k)canbereducedtoVCSP(�sub;2)inlineartimebyreplacingeachconstraintwithasuitablegadget.Foranyxedk,thenumberofnewvariablesintroducedinanyofthesegadgetsisboundedbyaconstant.TheresultthenfollowsfromCorollary8.NextweconsidertheclassofsubmodularconstraintsoveraBooleandomainwhichtakeonlythecostvalues0and1.(SuchconstraintscanbeusedtomodeloptimisationproblemssuchasMax-CSP,see[7].)Dene�f0;1g;ktobethesetofallf0;1g-valuedsubmodularcostfunctionsoveraBooleandomain,ofarityatmostk,andset�f0;1g=[k�f0;1g;k.Theminimisationofsubmodularcostfunctionsfrom�f0;1gwasstudiedin[11],wheretheywerecalled2-monotonefunctions.Theequivalenceof2-monotoneandsubmodularcostfunctionsandageneralisationof2-monotonefunctionstonon-Booleandomainswasshownin[7].Denition11.Acostfunction iscalled2-monotoneifthereexisttwosetsA;Bf1;:::;ngsuchthat (x)=0ifAxorxBand (x)=1otherwise(whereAxmeans8i2A;xi=1andxBmeans8i62B;xi=0).Theorem12.�f0;1gh�sub;2i.Proof.Any2-monotonecostfunction canbeexpressedover�sub;2using2extravariables,y1;y2: (x)=miny1;y22f0;1gf(1�y1)y2+y1Xi2A(1�xi)+(1�y2)Xi62Bxig:utCorollary13.Foranyxedk,VCSP(�f0;1g;k)canbesolvedincubictime.Finally,weconsidertheclass�sub;3ofternarysubmodularcostfunctionsoveraBooleandomain.Thisclasswasstudiedin[1],fromwhereweobtainthefollowingusefulcharacterisationofcubicsubmodularpolynomials.Lemma14([1]).Acubicpolynomialp(x1;:::;xn)overBooleanvariablesrep-resentsasubmodularcostfunctionifandonlyifitcanbewrittenasp(x1;:::;xn)=a0+Xfig2C+1aixi�Xfig2C�1aixi�Xfi;jg2C2aijxixj+Xfi;j;kg2C+3aijkxixjxk�Xfi;j;kg2C�3aijkxixjxk;8 Lemma17.Ifaquarticpolynomialp(x1;:::;xn)overBooleanvariablesrep-resentsasubmodularcostfunction,thenitcanbewrittensuchthat,forallfi;jg2C2:1.aij0,and2.aij+Pkjfi;j;kg2C+3aijk+Pk;ljfi;j;k;lg2C+4aijkl+Fij0whereFi;jisanon-positivevaluewhichisequaltothesumofthecoecientsofcertainnon-positivecubicandquarticterms,C2denotesthesetofquadraticterms,andC+idenotesthesetoftermsofdegreeiwithpositivecoecients,fori=3;4.Proof.Letpbeaquarticsubmodularpolynomialandletiandjbegiven,thenpcanalwaysbeputinaformsothatthesecondorderderivativeis:i;j=ai;j+Xkjfi;j;kg2C+3aijkxk+Xk;ljfi;j;k;lg2C+4aijklxkxl�Xkjfi;j;kg2C�3aijkxk�Xk;ljfi;j;k;lg2C�4aijklxkxl:Consideranassignmentwhichsetsxi=xj=1andxk=08k6=i6=j.ByProposition3,aij0,whichprovestherstcondition.Bysettingxk=1forallksuchthatfi;j;kg2C+3andxk=xl=1forallk;lsuchthatfi;j;k;lg2C+4,wegetthesecondcondition.Wesetto1allvariableswhichoccurinsomepositivecubicorquarticterm.Thesecondconditionthensaysthatthesumofallthesepositivecoecientsminusthosewhichareforced,byoursettingofvariables,tobe1(Fij),isatmost0.(NotethatthisalsoprovesLemma14.)utNextweshowausefulexampleofa4-arysubmodularcostfunctionwhichcanbeexpressedoverthebinarysubmodularcostfunctionsusingoneextravariable.Example18.Let bethe4-arycostfunctiondenedasfollows: (x)=minf2k;5g,wherekisthenumberof0sinx2f0;1g4.Thecorrespondingquarticpolynomialrepresenting isp(x1;x2;x3;x4)=5+x1x2x3x4�x1x2�x1x3�x1x4�x2x3�x2x4�x3x4:Isiseasytocheckthatpissubmodular.Itcanbeshownbysimplecaseanalysisthatpcannotbeexpressedasaquadraticpolynomialwithnon-positivequadraticcoecients(fromthedenitionofp,thepolynomialwouldhavetobe5�x1x2�x1x3�x1x4�x2x3�x2x4�x3x4whichisnotequaltoponx1=x2=x3=x4=1).However,pcanbeexpressedover�sub;2usingjustoneextravariable,viathefollowinggadget:p(x1;x2;x3;x4)=miny2f0;1gf5+(3�2x1�2x2�2x3�2x4)yg:10 5.2ThegeneralcaseWenowgeneralisetheresultfromtheprevioussectiontosubclassesofsub-modularconstraintsofarbitraryarities.Wedene�new;ktobethesetofallk-arysubmodularcostfunctionsoveraBooleandomainwhosecorrespondingpolynomialssatisfy,forevery1ijk,aij+k�2Xs=1Xfi;j;i1;:::;isg2C+s+2ai;j;i1;:::;is0:Inotherwords,forany1ijk,thesumofaijandallpositivecoecientsofcubicandhigherdegreetermswhichincludexiandxjisnon-positive.Theorem22.Foreveryk4,�new;kh�sub;2i.Proof.Notethatthecasek=4isprovedbyTheorem19.Firstweshowthatinordertoprovethestatementofthetheorem,itissucienttohaveauniformwayofgeneratinggadgetsover�sub;2forpolynomialsofthefollowingtype:pk(x1;:::;xk)=kYi=1xi�X1ikxixj:Notethatpk(x)=��m2,wheremisthenumberof1sinx,and�02=�12=0,unlessm=k(xconsistsof1sonly),inwhichcasepk(x)=��m2+1.Assumethatforanyk5,wecanconstructagadgetPkforpkover�sub;2.Givenacostfunction 2�new;k,letpbethecorrespondingpolynomialwhichrepresents .BytheconstructioninTheorem9,wecanreplaceallnegativetermsofdegree3.BytheconstructionsinTheorem15andTheorem19,wecanreplaceallpositivecubicandquarticterms.Nowforanypositivetermofdegreed,5dk,wereplaceitwiththegadgetPd.Thisconstructionworksifallquadraticcoecientsoftheresultingpolynomialarenon-positive.However,thisisensuredbythedenitionof�new;kandbythechoiceofthegadgets.Itremainstoshowhowtouniformlygenerategadgetsforpk,wherek5.Weclaim,thatforanyk4,thefollowing,denotedbyPk,isagadgetforpk:pk(x1;:::;xk)=miny0;:::;yk�42f0;1gfy0(3�2kXi=1xi)+k�4Xj=1yj(2+j�kXi=1xi)g:Noticethatinthecaseofk=4,thegadgetcorrespondstothegadgetusedintheproofofTheorem19,andthereforethebasecaseisproved.Weproceedbyinductionink.AssumethatPiisagadgetforpiforeveryik.WeprovethatPk+1isagadgetforpk+1.Firstly,takethegadgetPkforpk,andreplaceeverysumPki=1xiwithPk+1i=1xi.WedenotethenewgadgetP0.ItisnotdiculttoseethatP0isavalidgadgetforpk+1onallassignmentswithatmostk�11s.Also,onany12 thisimpliesthatoneofthetwoisredundant,asallconstraintsinvolvingthatvariablecanreplaceitwiththeothervariablewithoutchangingtheoverallcost.HencewerequireatmostjDjjDjkdistincthiddenvariablestoexpress .ut6ConclusionInthispaperwerstconsideredbinarysubmodularconstraintsoveraBooleandomain,andshowedthattheycanbeminimisedincubictimeviaareductiontotheminimumcutproblemforgraphs.Wetheninvestigatedwhichothersub-modularconstraintsareexpressibleusingbinarysubmodularconstraintsoveraBooleandomain,andhencecanalsobeminimisedecientlyusingminimumcuts.Usingknownresultsfromcombinatorialoptimisation,weidentiedseveralsuchclassesofconstraints,includingallternarysubmodularconstraints,andallf0;1g-valuedsubmodularconstraintsofanyarity.Byconstructingsuitablegadgets,weidentiedcertainnewclassesofk-arysubmodularconstraints,wherek4,whichcanalsobeexpressedbybinarysubmodularconstraints.Themainopenproblemraisedbythispaperiswhetherallbounded-aritysubmodularconstraintsoveraBooleandomaincanbeexpressedbybinarysub-modularconstraints,andhencesolvedincubictime.Intermsofpolynomials,thisisequivalenttothefollowingproblem:cananyBooleanpolynomialwithnon-positivesecondorderderivativesbeexpressedastheprojectionofaquadraticpolynomialwithnon-positivequadraticcoecients?Theresultspresentedinthispaperprovideapartialanswertothisquestionusingconstructivemethodswhichcanbeusedtoobtainconcretereductionstoproblemssuchas(s;t)-Min-Cut.Wenotethatanalternativegeneralap-proachtotheproblemofdeterminingtheexpressivepowerofvaluedconstraintswasdevelopedin[5].Itwasshowntherethattheexpressivepowerofanyval-uedconstraintlanguageischaracterisedbyacollectionofalgebraicpropertiescalledfractionalpolymorphisms[5].Inordertoshowthat�sub;kh�sub;2iitwouldthereforebesucienttoshowthat�sub;2and�sub;khavethesamefrac-tionalpolymorphisms.However,thisalgebraicapproachisnon-constructive,andhencehascertainlimitations:evenifitcouldbeestablishedinthiswaythat�sub;kh�sub;2i,thiswouldnotdirectlyprovideuswithagadgetforanygivenproblem(andhenceanecientalgorithm).Conversely,ifitcouldbeestablishedusingthealgebraicapproachthat�sub;k6h�sub;2i,thatwouldstillleaveopenthequestionofidentifyingwhichsubclassesof�sub;kcanbeexpressedover�sub;2,andhencesolvedeciently.Thispaperprovidesarststepinansweringthatquestionusingconstructivetechniquesthatcouldbeimplementedinvaluedcon-straintsolvers.AcknowledgementsTheauthorswouldliketothankDavidCohenandMar-tinCooperforfruitfuldiscussionsonsubmodularconstraintsandChrisJe ersonforhelpwithusingtheconstraint-solverMINION,whichhelpedustondandsimplifysomeofthegadgetspresentedinthispaper.StanislavZivnygratefullyacknowledgesthesupportofEPSRCgrantEP/F01161X/1.14