Arti - PDF document

ArtiÞcialIntelligence129(2001)5Ð33 PlanningasheuristicsearchBlaiBonet,HŽctorGeffnerDepto.deComputación,UniversidadSimónBolívar,Aptdo.89000,Caracas1080-A,Venezuela AbstractIntheAIPS98PlanningContest,theplannershowedthatheuristicsearchplannerscanbecompetitivewithstate-of-the-artGraphplanandSATplanners.Heuristicsearchplannerslike 1.IntroductionThelastfewyearshaveseenanumberofpromisingnewapproachesinPlanning.MostprominentamongtheseareGraphplan[3]andSatplan[24].Bothworkinstages Correspondingauthor.E-mailaddresses:bonet@usb.ve(B.Bonet),hector@usb.ve(H.Geffner). B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 bybuildingsuitablestructuresandthensearchingthemforsolutions.InGraphplan,thestructureisagraph,whileinSatplan,itisasetofclauses.Bothplannershaveshownimpressiveperformanceandhaveattractedagooddealofattention.RecentimplementationsandsigniÞcantextensionshavebeenreportedin[2,19,25,27].IntherecentAIPS98PlanningCompetition[34],threeoutofthefourplannersintheStripstrack,track,STAN[27],andBLACKBOX[25],werebasedontheseideas.Thefourthplanner,[4],wasbasedontheideasofheuristicsearch[35,39].In,thesearchisassumedtobesimilartothesearchinproblemslikethe8-Puzzle,themaindifferencebeingintheheuristic:whileinproblemslikethe8-Puzzletheheuristicisnormallygiven(e.g.,asthesumofManhattandistances),inplanningithastobeextractedautomaticallyfromthedeclarativerepresentationoftheproblem.thusappealstoasimpleschemeforcomputingtheheuristicfromStripsencodingsandusestheheuristictoguidethesearchforthegoal.TheideaofextractingheuristicsfromdeclarativeproblemrepresentationsforguidingthesearchinplanninghasbeenadvancedrecentlybyDrewMcDermott[31,33],andbyBonet,LoerincsandGeffner[6].Inthispaper,weextendtheseideas,testthemoveralargenumberofproblemsanddomains,andintroduceafamilyofplannersthatarecompetitivewithsomeofthebestcurrentplanners.Plannersbasedontheideasofheuristicsearcharerelatedtospecializedsolverssuchasthosedevelopedfordomainslikethe24-Puzzle[26],RubikÕsCube[23],andSokoban[17]butdifferfromthemmainlyintheuseofagenerallanguageforstatingproblemsandageneralmechanismforextractingheuristics.Heuristicsearchplanners,likeallplanners,generalproblemsolversinwhichthesamecodemustbeabletoprocessproblemsfromdifferentdomains[37].Thisgeneralitycomesnormallyataprice:asnotedin[17],theperformanceofthebestcurrentplannersisstillwellbehindtheperformanceofspecializedsolvers.Closingthisgap,however,isthemainchallengeinplanningresearchwheretheultimategoalistohavesystemsthatcombineßexibilityandefÞciency:ßexibilityformodelingawiderangeofproblems,andefÞciencyforobtaininggoodsolutionsfast.Inheuristicsearchplanning,thischallengecanonlybemetbytheformulation,analysis,andevaluationofsuitabledomain-independentheuristicsandoptimizations.Inthispaperweaimtopresentthebasicideasandresultswehaveobtained,anddiscussmorerecentideasthatweÞndpromising.Moreprecisely,wewillpresentafamilyofheuristicsearchplannersbasedonasimpleandgeneralheuristicthatassumesthatactionpreconditionsareindependent.Thisheuristicisthenusedinthecontextofbest-Þrstandhill-climbingsearchalgorithms,andistestedoveralargeclassofdomains.Wealsoconsidervariationsandextensions,suchasreversingthedirectionofthesearchforspeedingnodeevaluation,andextractinginformationaboutpropositionalinvariantsforavoidingdead-ends.Wecomparetheresultingplannerswith Thisideaalsoappears,inadifferentform,intheplannerIxTex;see[29].Interestingly,theareaofconstraintprogramminghassimilargoalsalthoughitisfocusedonadifferentclassofproblems[16].Yet,see[45]forarecentattempttoapplytheideasofconstraintprogramminginplanning.Anotherwaytoreducethegapbetweenplannersandspecializedsolversisbymakingroominplanninglanguagesforexpressingdomain-dependentcontrolknowledge(e.g.,[5]).Inthispaper,however,wedonÕtconsiderthisoptionwhichisperfectlycompatibleandcomplementarywiththeideasthatwediscuss. B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 someofthebestcurrentplannersandshowthatthesimplestplannerbasedonapurebest-Þrstsearch,yieldsthemostsolidperformanceoveralargesetofproblems.Wealsodiscussthestrengthsandlimitationsofthisapproach,establishacorrespondencebetweenheuristicsearchplanningandGraphplan,andbrießysurveyrecentideasthatcanhelpreducetheperformancegapbetweengeneralheuristicsearchplannersandspecializedsolvers.Ourfocusisonnon-optimalsequentialplanning.ThisiscontrastwiththerecentemphasisonoptimalparallelplanningfollowingGraphplanandSAT-basedplanners[24].Algorithmsareevaluatedintermsoftheproblemsthattheysolve(givenlimitedtimeandmemoryresources),andthequalityofthesolutionsfound(measuredbythesolutiontimeandlength).Theuseofheuristicsforoptimalsequentialandparallelplanningisconsideredin[13]andisbrießydiscussedinSection8.Inthispaperwereviewandextendtheideasandresultsreportedin[4].However,ratherthanfocusingonthetwospeciÞcplannersr,weconsiderandanalyzeabroaderspaceofalternativesandperformamoreexhaustiveempiricalevaluation.Thismoresystematicstudyledustorevisesomeoftheconjecturesin[4]andtounderstandbetterthestrengthsandlimitationsinvolvedinthechoiceoftheheuristics,thesearchalgorithms,andthedirectionofthesearch.Therestofthepaperisorganizedasfollows.WecoverÞrstgeneralstatemodels(Section2)andthestatemodelsunderlyingproblemsexpressedinStrips(Section3).Wethenpresentadomain-independentheuristicthatcanbeobtainedfromStripsencodings(Section4),andusethisheuristicinthecontextofforwardandbackwardstateplanners(Sections5and6).Wethenconsiderrelatedwork(Section7),summarizethemainideasandresults,anddiscusscurrentopenproblems(Section8).2.StatemodelsStatespacesprovidethebasicactionmodelforproblemsinvolvingdeterministicactionsandcompleteinformation.AstatespaceconsistsofaÞnitesetofstates,aÞnitesetofactions,astatetransitionfunctionthatdescribeshowactionsmaponestateintoanother,andacostfunction 0thatmeasuresthecostofdoingactioninstate[35,38].Astatespaceextendedwithagiveninitialstateandasetofgoalstateswillbecalledastatemodel.Statemodelsarethusthemodelsunderlyingmostoftheproblemsconsideredinheuristicsearch[39]aswellastheproblemsthatfallintothescopeofclassicalplanning[35].Formally,astatemodelisatuple
whereisaÞniteandnon-emptysetofstatesistheinitialstate,isanon-emptysetofgoalstates, denotestheactionsapplicableineachstate denotesastatetransitionfunctionforall ,and standsforthecostofdoingactioninstatesolutionofastatemodelisasequenceofactions,...,thatgeneratesastate,...,suchthateachactionisapplicableinisagoalstate,i.e.,.Thesolutionisoptimalwhenthetotalcostisminimized. B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 Inproblemsolving,itiscommontobuildstatemodelsadaptedtothetargetdomainbyexplicitlydeÞningthestatespaceandexplicitlycodingthestatetransitionfunction andtheactionapplicabilityconditions inasuitableprogramminglanguage.Inplanning,statemodelsaredeÞnedimplicitlyinageneraldeclarativelanguagethatcaneasilyaccommodaterepresentationsofdifferentproblems.WeconsidernextthestatemodelsunderlyingproblemsexpressedintheStripslanguage[9].3.TheStripsstatemodelAplanningprobleminStripsisrepresentedbyatuplewhereasetofatoms,isasetofoperators,andencodetheinitialandgoalsituations.Theoperatorsareallassumedgrounded(i.e.,withthevariablesreplacedbyconstants).Eachoperatorhasaprecondition,add,anddeletelistsdenotedasPrec,andrespectively.Theyareallgivenbysetsofatomsfrom.AStripsproblemdeÞnesastatespace
 where(S1)thestatesarecollectionsofatomsfrom(S2)theinitialstate(S3)thegoalstatesaresuchthat(S4)theactions aretheoperatorssuchthatPrec(S5)thetransitionfunctionmapsstatesintostates   (S6)allactioncosts are1.The(optimal)solutionsoftheproblemarethe(optimal)solutionsofthestatemodel.ApossiblewaytoÞndsuchsolutionsisbyperformingasearchinsuchspace.Thisapproach,however,hasnotbeenpopularinplanningwhereapproachesbasedondivide-and-conquerideasandsearchinthespaceofplanshavebeenmorecommon[35,46].Thissituationhoweverhaschangedinthelastfewyears,afterGraphplan[3]andSATapproaches[24]achievedordersofmagnitudespeedupsoverpreviousapproaches.Morerecently,theideaofplanningasstatespacesearchhasbeenadvancedin[31]and[6].Inbothcases,thekeyingredientistheheuristicusedtoguidethesearchthatisextractedautomaticallyfromtheproblemrepresentation.Herewefollowtheformulationin[6].4.HeuristicsTheheuristicfunctionforsolvingaproblemin[6]isobtainedbyconsideringaÔrelaxedÕprobleminwhichalldeletelistsareignored.Fromanystate,theoptimalcost forsolvingtherelaxedproblemcanbeshowntobealowerboundontheoptimalcost forsolvingtheoriginalproblem.Asaresult,thefunction couldbeusedasanadmissibleheuristicforsolvingtheoriginalproblem.However,solvingtheÔrelaxedÕproblemandobtainingthefunctionareNP-hardproblems.Wethususean Asitiscommon,weusethecurrentversionoftheStripslanguageasdeÞnedbytheStripssubsetofPDDL[32]ratherthanoriginalversionin[9].Thiscanbeshownbyreducingset-coveringtoStripswithnodeletes. B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 approximationandset toanestimateoftheoptimalvaluefunction oftherelaxedproblem.Inthisapproximation,weestimatethecostofachievingthegoalatomsfromandthenset toasuitablecombinationofthoseestimates.Thecostofindividualatomsiscomputedbyaprocedurewhichissimilartotheonesusedforcomputingshortestpathsingraphs[1].Indeed,theinitialstateandtheactionscanbeunderstoodasdeÞningagraphinatomspaceinwhichforeveryactionthereisadirectedlinkfromthepreconditionstoitspositiveeffects.Thecostofachievinganatomisthenreßectedinthelengthofthepathsthatleadtofromtheinitialstate.Thisintuitionisformalizedbelow.Wewilldenotethecostofachievinganatomfromthestate .TheseestimatescanbedeÞnedrecursivelyas 0ifmin Precotherwise,where standsfortheactionsthatadd,i.e.,with,andPrectobedeÞnedbelow,standsfortheestimatedcostofachievingthepreconditionsofactionWhiletherearemanyalgorithmsforobtainingthefunctiondeÞnedby(1),weuseasimpleforwardchainingprocedureinwhichthemeasures areinitializedto0ifandtootherwise.Then,everytimeanoperatorisapplicablein,eachatomisaddedto isupdatedto min PrecTheseupdatescontinueuntilthemeasures donotchange.ItÕssimpletoshowthattheresultingmeasuressatisfyEq.(1).Theprocedureispolynomialinthenumberofatomsandactions,andcorrespondsessentiallytoaversionoftheBellmanÐFordalgorithmforÞndingshortestpathsingraphs[1].TheexpressionPrecinboth(1)and(2)standsfortheestimatedcostofthesetofatomsgivenbythepreconditionsofaction.Inplannerssuchas,thecost asetofatomsisdeÞnedintermsofthecostoftheatomsintheset.Aswewillseebelow,thiscanbedoneindifferentways.Inanycase,theresultingheuristic thatestimatesthecostofachievingthegoalfromastateisdeÞnedas Thecost setsofatomscanbedeÞnedastheweightedsumofcostsofindividualatoms,theminimumofthecosts,themaximumofthecosts,etc.Weconsidertwoways.TheÞrstisasthesumofthecostsoftheindividualatomsin  (additivecosts).(4)Wecalltheheuristicthatresultsfromsettingthecosts  ,theadditiveheuristicanddenoteitby.Theheuristicassumesthatsubgoalsareindependent.Thisisnottrueingeneralastheachievementofsomesubgoalscanmaketheachievement WheretheminofanemptysetisdeÞnedtobeinÞnite. B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 oftheothersubgoalsmoreorlessdifÞcult.Forthisreason,theadditiveheuristicisnotadmissible(i.e.,itmayoverestimatethetruecosts).Still,wewillseethatitisquiteusefulinplanning.Second,anadmissibleheuristiccanbedeÞnedbycombiningthecostofatomsbythemaxoperationas:  (maxcosts).(5)Wecalltheheuristicthatresultsfromsettingthecosts  ,themaxheuristicanddenoteitby.Themaxheuristicunliketheadditiveheuristicisadmissibleasthecostofachievingasetofatomscannotbelowerthanthecostofachievingeachoftheatomsintheset.Ontheotherhand,themaxheuristicisoftenlessinformative.Infact,whiletheadditiveheuristiccombinesthecostsofallsubgoals,themaxheuristicfocusesonlyonthemostdifÞcultsubgoalsignoringallothers.InSection7,however,wewillseethatareÞnedversionofthemaxheuristicisusedinGraphplan.5.ForwardstateplanningAhill-climbingplannerTheplanner[4]thatwasenteredintotheAIPS98PlanningContest,usestheadditiveheuristictoguideahill-climbingsearchfromtheinitialstatetothegoal.Thehill-climbingsearchisverysimple:ateverystep,oneofthebestchildrenisselectedforexpansionandthesameprocessisrepeateduntilthegoalisreached.Tiesarebrokenrandomly.Thebestchildrenofanodearetheonesthatminimizetheheuristic.Thus,ineverystep,theestimatedatomcosts andtheheuristic arecomputedforthestatesthataregenerated.In,thehillclimbingsearchisextendedinseveralways;inparticular,thenumberofconsecutiveplateaumovesinwhichthevalueoftheheuristicisnotdecrementediscountedandthesearchisterminatedandrestartedwhenthisnumberexceedsagiventhreshold.Inaddition,allstatesthathavebeengeneratedarestoredinafastmemory(ahashtable)sothatstatesthathavebeenalreadyvisitedareavoidedinthesearchandtheirheuristicvaluesdonothavetoberecomputed.Also,asimpleschemeforrestartingthesearchfromdifferentstatesisusedforavoidinggettingtrappedintothesameplateaus.Manyofthedesignchoicesinaread-hoc.TheyweremotivatedbythegoalofgettingabetterperformanceforthePlanningContestandbyourearlierworkonareal-timeplannerbasedonthesameheuristic[6].didwellintheContestcompetingintheÒStripstrackÓagainstthreestate-of-the-artGraphplanandSATplanners::STAN[27]andBLACKBOX[25].Table1from[34]showsasummaryoftheresults.Inthecontestthereweretworoundswith140and15problemseach,inbothcasesdrawnfromseveraldomains.Thetableshowsforeachplannerineachround,thenumberofproblemssolved,theaveragetimetakenovertheproblemsthatweresolved,andthenumberofproblemsinwhicheachplannerwasfastestorproducedshortestsolutions.STANBLACKBOXoptimalparallelplannersthatminimizethenumberoftimesteps(inwhichseveralactionscanbeperformedconcurrently)butnotthenumberofactions. B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 Table1ResultsoftheAIPS98Contest(Stripstrack).Columnsshowthenumberofproblemssolvedbyeachplanner,theaveragetimeovertheproblemssolved(inseconds)andthenumberofproblemsinwhicheachplannerwasfastestorproducedshortestplans(from[34]) RoundPlannerAvg.timeSolvedFastestShortest Round1BLACKBOX1.49631655HSP35.488219617.40632949STAN55.41642447Round2BLACKBOX2.46836HSP25.8791517.371138STAN1.33754 Asitcanbeseenfromthetable,solvedmoreproblemsthantheotherplannersbutitoftentookmoretimeorproducedlongerplans.Moredetailsaboutthesettingandresultsofthecompetitioncanbefoundin[34]andinanarticletoappearintheAIMagazineHSP2Abest-rstsearchplannerTheresultsaboveandanumberofadditionalexperimentssuggestthatcompetitivewiththebestcurrentplannersovermanydomains.However,isnotanoptimalplanner,andwhatÕsworseÑconsideringthatoptimalityhasnotbeenatraditionalconcerninplanningÑthesearchalgorithminisnotcomplete.Inthissection,weshowthatthislastproblemcanbeovercomebyswitchingfromhill-climbingtoabest-rstsearchBFS)[39].Moreover,theresultingBFSplannerissuperiorinperformancetoanditappearstobesuperiortosomeofthebestplannersoveralargeclassofproblems.Byperformancewemean:thenumberofproblemssolved,thetimetogetthosesolutions,andthelengthofthosesolutionsmeasuredbythenumberofactions.Wewillrefertotheplannerthatresultsfromtheuseoftheadditiveheuristicinabest-Þrstsearchfromtheinitialstatetothegoal,asHSP2.Thisbest-ÞrstsearchkeepsanOpenandaClosedlistofnodesas[35,39]butweightsnodesbyanevaluationfunction  ,where istheaccumulatedcost,istheestimatedcosttothegoal,and1isaconstant.For1,thealgorithmisandfor1itcorrespondstotheso-calledalgorithm[39].Highervaluesofusuallyleadtothegoalfasterbutwithsolutionsoflowerquality[22].Indeed,iftheheuristicisadmissible,thesolutionsfoundbyareguaranteednottoexceedtheoptimalcostsbymorethanafactorofHSP2usesthealgorithmwiththenon-admissibleheuristicwhichisevaluatedfromscratchineverynewstategenerated.ThevalueoftheparameterisÞxedat5,eventhoughvaluesintherangedonotmakeasigniÞcantdifference. B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 5.3.ExperimentsIntheexperimentsbelow,weassesstheperformanceofthetwoheuristicsearchplanners,HSP2,incomparisonwithtwostate-of-the-artplanners,STAN3.0[27],BLACKBOX3.6[25].HSP2bothperformaforwardstatespacesearchguidedbytheadditiveheuristic.TheÞrstperformsanextendedhill-climbingsearchandthelaterperformsabest-ÞrstsearchwithanevaluationfunctioninwhichtheheuristicismultipliedbytheconstantSTANBLACKBOXarebothbasedonGraphplan[3],butthelattermapstheplangraphintoasetofclausesthatarecheckedforsatisÞability.Theversionofusedintheexperiments,1.2,improvestheversionusedintheAIPS98Contestandtheoneusedin[4].Foreachplannerandeachplanningproblemweevaluatewhethertheplannersolvestheproblem,andifso,thetimetakenbytheplannerandthenumberofactionsinthesolution.STANBLACKBOXareoptimalparallelplannersthatminimizethenumberoftimestepsbutnotnecessarilythenumberofactions.Bothplannerswererunwiththeirdefaultoptions.TheexperimentswereperformedonanUltra-5with128MbRAMand2MbofCacherunningSolaris7at333MHz.TheexceptionaretheresultsforBLACKBOXontheLogisticsproblemsthatweretakenfromtheBLACKBOXdistributionasthedefaultoptionsproducedmuchpoorerresults.Thedomainsconsideredare:Blocks,Logistics,Gripper,8-Puzzle,Hanoi,andTire-World.TheyconstitutearepresentativesampleofdifÞcultbutsolvableinstancesforcurrentplanners.Fiveofthe10Blocks-Worldinstancesareofourown[4],therestoftheproblemsaretakenfromothersasnoted.Aplannerissaidnottosolveaproblemwheniteitherrunsoutofmemoryorrunsoutoftime(10minutes).FailuretoÞndsolutionsdueeithertomemoryortimeconstraintsaredisplayedasplanswithlengthAlltheplannersareimplementedinCandacceptproblemsinthelanguage(thestandardlanguageusedintheAIPS98Contest;[32]).Moreover,theplannersinthefamilyconverteveryprobleminstanceinintoaprograminC.Generating,compiling,linking,andloadingsuchprogramtakesintheorderof2seconds.Thistimeisroughlyconstantforallinstancesanddomains,andisnotincludedintheÞguresbelow.5.3.1.Blocks-WorldTheÞrstexperimentsdealwiththeblocksworld.Theblocksworldischallengingfordomain-independentplannersduetotheinteractionsamongsubgoalsandthesizeofthestatespace.Theteninstancesconsideredinvolvefrom7to19blocks.FiveoftheseinstancesaretakenfromtheBLACKBOXdistributionandÞvearefrom[4].TheresultsforthisdomainareshowninFig.1thatdisplaysforeachplannerthelengthofthesolutionsontheleft,andthetogetthosesolutionsontheright.ThelengthsproducedbySTANBLACKBOXarenotnecessarilyoptimalinthisdomainasthereissomeparallelism(e.g.,movingblocksamongdisjointpairsoftowers).Thisisthereasonthelengthstheyreportdonotalwayscoincide.Inanycase,thesolutionsreportedbythefourplannersareroughlyequivalentoverinstances1Ð5,withSTAN ThoseresultswereobtainedonaSPARCUltra2witha296MHzclockand256MofRAM[5]. B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 Fig.1.Solutionlength(left)andtime(right)over10Blocks-Worldinstances.producingslightlylongersolutionsforinstances4and5.OverthemoredifÞcultinstances6Ð10,thesituationchangesandonlyHSP2reportsolutions,withtheplansfoundHSP2beingshorter.Regardingsolutiontimes,thetimesforHSP2areroughlyeven,andslightlyshorterthanthoseforSTANBLACKBOXovertheÞrstÞveinstances.OverthelastÞveinstancesSTANBLACKBOXrunoutofmemory.5.3.2.LogisticsThesecondsetofexperimentsdealswiththelogisticsdomain,adomainthatinvolvesthetransportationofpackagesbyeithertrucksorairplanes.Truckscanmoveamonglocationsinthesamecity,andairplanescanmovebetweenairportsinonecitytoairportsinanothercity.Packagescanbeloadedandunloadedintrucksandairplanes,andthetaskistotransportthemfromtheiroriginallocationstosometargetlocations.Thisisahighlyparalleldomain,wheremanyoperationscanbedoneinparallel.Asaresult,plansinvolvemanyactionsbutthenumberoftimestepsisusuallymuchsmaller.ThedomainisfromKautzandSelmanfromanearlierversionduetoManuelaVeloso.The30instancesweconsiderarefromtheBLACKBOXdistribution.TheresultsforthisdomainareshowninFig.2.Thenumberofactionsintheplansarereportedontheupperplot,timesarereportedonthelowerplot,andfailurestoÞndaplanarereportedwithlength1.BothHSP2BLACKBOXsolveall30instances,HSP2beingroughlytwoordersofmagnitudefaster.STAN,ontheotherhand,failon13and10instancesrespectively.Interestingly,thetimesreportedbySTANontheinstancesitsolvestendtobeclosetothosereportedbyHSP2.OntheotherinstancesSTANrunsoutofmemory(instances328)ortime(instances221).5.3.3.GripperThethirdsetofexperimentsdealswiththeGripperdomainusedintheAIPS98PlanningContestandduetoJ.Koehler.Thisisadomainthatconcernsarobotwithtwogrippersthatmusttransportasetofballsfromoneroomtoanother.ItisverysimpleforhumanstosolvebutinthePlanningContestproveddifÞculttomostoftheplanners.Indeed,thedomainisnotchallengingforspecializedsolvers,butischallengingforcertaintypesofdomain-independentplanners. B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 Fig.2.Solutionlength(upper)andtime(lower)over30LogisticsinstancesfromKautzandSelman. Fig.3.Solutionlength(left)andtime(right)over10GripperinstancesfromAIPS98Contest.Theresultsover10GripperinstancesfromtheAIPS98ContestareshowninFig.3.TheplannersHSP2havenodifÞcultiesandcomputeplanswithsimilarlengths.Ontheotherhand,BLACKBOXsolvestheÞrsttwoinstancesonly,andSTANtheÞrstfourinstances.Asshownontheright,thetimerequiredbybothplannersgrowsexponentiallyandtheyrunoutoftimeoverthelargerinstances.Ontheotherhand,HSP2scaleupsmoothlywithHSP2beingslightlyfasterthan B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 OneofthereasonsforthefailureofbothSTANBLACKBOXinGripperisthattheheuristicimplicitlyrepresentedbytheplangraphisaverypoorestimatorinthisdomain.Asaresult,Graphplan-basedplanners,suchasSTANBLACKBOXthatperformaformIDAsearchmustdomanyiterationsbeforeÞndingasolution.Actually,thesameexponentialgrowthinGripperoccursalsoinplannerswhentheheuristicisusedinplaceoftheadditiveheuristic.Asbefore,theproblemisthattheheuristicisalmostuselessinthisdomainwheresubgoalsaremostlyindependent.TheheuristicimplicitintheplangraphisareÞnementoftheheuristic;therelationbetweenGraphplanandheuristicsearchplanningwillbeanalyzedfurtherinSection7.5.3.4.PuzzleThenextproblemsarefourinstancesofthefamiliar8-Puzzleandtwoinstancesfromthelarger15-Puzzle.Threeofthefour8-Puzzleinstancesarehardastheiroptimalsolutionsinvolves31steps,themaximumplanlengthinsuchdomain.The15-PuzzleinstancesareofmediumdifÞculty.AsshowninFig.4,STANsolvetheÞrstfourinstances,andHSP2solvestheÞrstÞve.ThesolutionscomputedbySTANareoptimalinthisdomainwhichispurelyserial.ThesolutionscomputedbyHSP2,ontheotherhand,arepoorer,andareoftentwiceaslong.Ontheotherhand,asshownontheleftpartoftheÞgure,HSP2istwoordersofmagnitudefasterthanSTANoverthedifÞcult8-Puzzleinstances(2Ð4)andcanalsosolveinstance5.Thetimesforareworseanddoesnotsolveinstance5.BLACKBOXnotsolveanyoftheinstances.5.3.5.HanoiFig.5showstheresultsforHanoi.Instance2disks,thusproblemsrangefrom3disksupto8disks.OveralltheseproblemsHSP2STANgenerateplansofthesamequality,HSP2beingslightlyfasterthanSTANalsosolvesallinstancesbutthesolutionsarelonger.BLACKBOXsolvestheÞrsttwoinstances. Fig.4.Solutionlength(left)andtime(right)overfourinstancesofthe8-Puzzle(1Ð4)andtwoinstancesofthe15-Puzzle(5Ð6). B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 Fig.5.Solutionlength(left)andtime(right)oversixHanoiinstances.Instance2disks. Fig.6.Solutionlength(left)andtime(right)overthreeinstancesofTire-World.5.3.6.Tire-WorldTheTire-WorlddomainisduetoS.RussellandinvolvesoperationsforÞxingßattires:openingandclosingthetrunkofacar,fetchingandputtingawaytools,looseningandtighteningnuts,etc.Fig.6showstheresults.HerebothSTANBLACKBOXsolveallthreeinstancesproducingoptimalplans.HSP2alsosolvetheseinstancesbutinsomecasestheyproduceinferiorsolutions.Onthetimescale,HSP2isslightlyfasterthanSTAN,andbotharefasterthanBLACKBOXinonecasebytwoordersofmagnitude.AsisslowerthanHSP2andproduceslongersolutions.5.4.Summary:ForwardstateplanningTheexperimentsabove,basedonarepresentativesampleofproblems,showthatthetwoforwardheuristicsearchplannersHSP2arecapableofsolvingtheproblemssolvedbytwostate-of-the-artplanners.Inaddition,insomedomains,andinparticularHSP2solveproblemsthattheotherplannerswiththeirdefaultsettingsdonotcurrentlysolve.TheplannerHSP2,basedonastandardbestÞrstsearch,tendstobefasterandmorerobustthanthehill-climbingplanner.Thus,theargumentsin[4]insupportofahill-climbingstrategybasedontheslownodegenerationratethatresultsfromthecomputationofthe B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 heuristicineverystatedonotappeartoholdingeneral.Indeed,thecombinationoftheadditiveheuristicandthemultiplyingconstant1oftendrivethebest-Þrstplannertothegoalwithasfewnodeevaluationsasthehill-climbingplanner,alreadyprovidingthenecessaryÔgreedyÕbias.Ansearchwithanadmissibleandconsistentheuristic,ontheotherhand,isboundtoexpandallnodeswhosecost isbelowtheoptimalcost.ThishoweverdoesnoapplytothestrategyusedinHSP2IntheexperimentstheparameterinHSP2wassettotheconstantvalue5.YetHSP2notparticularlysensitivetotheexactvalueofthisconstant.Indeed,inmostofthedomains,valuesintheintervalproducesimilarresults.Thisislikelyduetothefactthattheheuristicisnotadmissibleandbyitselftendstooverestimatethetruecostswithouttheneedofamultiplyingfactor.Ontheotherhand,insomedomainslikeLogisticsandGripper,thevalue1doesnotleadtosolutions.Thisispreciselybecauseinthesedomainsthatinvolvesubgoalsthataremostlyindependent,theadditiveheuristicisnotÔsufÞcientlyÕoverestimating.Finally,inproblemsliketheslidingtilepuzzles,valuesofcloserto1producebettersolutionsinmoretime,incorrespondencewiththenormalpatternobservedincasesinwhichtheheuristicisadmissible[22].Fig.7showstheeffectsofthreedifferentvaluesofonthequalityandtimesofthesolutions,andthenumberofnodesgenerated.Thevaluesconsideredare5.ThetopthreecurvesthatcorrespondtoHanoi,aretypicalformostoftheother Fig.7.InßuenceofvalueofintheHSPplanneronthelengthofthesolutions(left),thetimerequiredtoÞndsolutions(center),andnumberofnodesgenerated(right).ThedomainsfromtoptobottomareHanoi,Gripper,andPuzzle. B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 domainsandshowlittleeffect.ThesecondsetofcurvescorrespondstoGripperwhereHSP2failstosolvethelastsixinstancesfor1.Indeed,thetworightmostcurvesshowanexponentialgrowthintimeandthenumberofgeneratednodes.InLogistics,HSP21alsofailstosolvemostoftheinstances.Asnotedabove,thesearetwodomainswheresubgoalsaremostlyindependentandwheretheadditiveheuristicisnotsufÞcientlyoverestimatingandhencefailstoprovidetheÔgreedybiasÕnecessarytoÞndthesolutions.Indeed,thestatespaceinLogisticsisverylarge,whileinGripperitÕsthebranchingfactorthatislargeduetothe(undetected)symmetriesintheproblem.ThebottomsetofcurvesinFig.7correspondtothePuzzledomain.Inthisdomain,HSP21and2producebettersolutionsandinsomecasestakemoretime.ThisprobablyhappensinPuzzlebecause,asinGripperandLogistics,thereisadegreeofdecomposabilityinthedomain(thatÕswhythesumoftheManhattandistanceworks),thatmakestheadditiveheuristicbehaveasanadmissibleheuristicin.UnlikeGripperandLogistic,howeverthebranchingfactoroftheproblemandthesizeofthestatespaceallowtheresultingBFSalgorithmtosolvetheinstancesevenwith1.Actually,withHSP2solvesthesixthinstanceofPuzzlewhichisnotsolvedwithHSPHeuristicregressionplanningAmainbottleneckinbothHSP2isthecomputationoftheheuristicfromscratchineverynewstate.Thistakesmorethan80%ofthetotaltimeinbothplannersandmakesthenodegenerationrateverylow.Indeed,inaproblemlikethe15-Puzzle,bothplannersgeneratelessthanathousandnodespersecond,whileaspecializedsolversuchas[26]generatesseveralhundredthousandnodespersecondforthemorecomplex24-Puzzle.Thereasonforthelownodegenerationrateisthecomputationoftheheuristicinwhichtheestimatedcosts forallatomsarecomputedfromscratchineverynewstateIn[4],wenotedthatthisproblemcouldbysolvedbyperformingthesearchbackwardfromthegoalratherthanforwardfromtheinitialstate.Inthatcase,theestimatedcosts derivedforallatomsfromtheinitialstatecouldbeusedwithoutrecomputationdeÞningtheheuristicofanystatearisinginthebackwardsearch.Indeed,theestimateddistancefromisequaltothedistancefrom,andthisdistancecanbeestimatedsimplyasthesum(ormax)ofthecosts fortheatoms.Thistrickforsimplifyingthecomputationoftheheuristicandspeedingupnodegenerationresultsfromcomputingtheestimatedatomcostsfromastatewhichthenbecomesthetargetofthesearch.Analternativeistoestimatetheatomcostsfromthegoalandthenperformaforwardsearchtowardthegoal.Thisisactuallytheideain[43].Theproblemwiththislatterschemeisthatthegoalinplanningisnotastatebutasetofstates;namely,thestateswherethegoalatomshold.AndcomputingtheheuristicfromasetofstatesinaprincipledmannerisboundtobemoredifÞcultthancomputingtheheuristicfromagivenstate(thustheneedtoÔcompleteÕthegoaldescriptionin[43]).Wethuspresentbelowaschemeforperformingplanningasheuristicsearchthatavoidstherecomputationoftheatomcostsineverynewstatebycomputingthesecosts ThesameappliesalsotoMcDermottÕsUNPOP B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 theinitialstate.ThesecostsarethenusedwithoutrecomputationtodeÞneanheuristicthatisusedtoguidearegressionsearchfromthegoal.ThebeneÞtofthesearchschemeisthatnodegenerationwillbe6Ð7timesfaster.ThiswillshowinthesolutionofsomeoftheproblemsconsideredabovesuchasLogisticsandGripper.However,aswewillalsosee,inmanyproblemsthenewsearchschemedoesnothelp,andinseveralcases,itactuallyhurts.Wediscusssuchissuesbelow.6.1.RegressionstatespaceWerefertotheplannerthatsearchesbackwardfromthegoalratherthanforwardfromtheinitialstateasr.Backwardsearchisanoldideainplanningthatisknownregressionsearch[35,46].Inregressionsearch,thestatescanbethoughtassetsofsubgoals;i.e.,theÔapplicationÕofanactioninagoalyieldsasituationinwhichtheexecutionoftheactionachievesthegoal.Moreover,whileasetofatomsintheforwardsearchrepresentstheuniquestateinwhichtheatoms,andaretrueandallotheratomsarefalse,thesamesetofatomsintheregressionsearchrepresentsthecollectionofstatesinwhichtheatoms,andaretrue.Inparticular,thesetofgoals,whichdeterminestherootnodeoftheregressionsearch,standsforthecollectionofgoalstates,thatis,thestatessuchthatFormakingprecisethenatureofthebackwardsearch,wewillthusdeÞneexplicitlythestatespacebeingsearched.WewillcallittheregressionspaceanddeÞneitinanalogytotheprogressionspacedeÞnedby(S1)Ð(S5)above.TheregressionspaceassociatedwithaStripsproblemisgivenbythetuplewhere(R1)thestatesaresetsofatomsfrom(R2)theinitialstateisthegoal(R3)thegoalstatesarethestatesforwhich(R4)thesetofactionsA(s)applicableinaretheoperatorsthatarerelevantconsistent;namely,forwhich(R5)thestatethatfollowstheapplicationofA(s)issuchthat Prec (R6)theactioncostsareall1.Thesolutionofthisstatespaceis,likethesolutionofanystatemodelaÞnitesequenceofactions,...,suchthatforasequenceofstates,...,,for,and.Thesolutionoftheprogressionandregressionspacesarerelatedintheobviousway;oneistheinverseoftheother.Weusedifferentfontsforreferringtostatesintheprogressionspaceandstatesintheregressionspace.Whiletheyarebothrepresentedbysetsofatoms,theyhaveadifferentmeaning.Aswesaidabove,thestateintheregressionspacestandsforthesetofstatesintheprogressionspace.Forthisreason,forwardandbackwardsearchinplanningaresymmetric,unlikeforwardandbackwardsearchinproblemslikethe15-PuzzleorRubikÕsCube. B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 6.2.HeuristicTheplannerrsearchestheregressionspace(R1)Ð(R5)usinganheuristicbasedontheadditivecostestimates describedinSection4.Theseestimatesarecomputedonlyoncefromtheinitialstate.TheheuristicassociatedwithstatethendeÞnedas Whilein,theheuristic combinesthecostestimates ofaÞxedsetofgoalatomscomputedfromeachstate,inr,theheuristiccombinesthecostestimatesofthesetofsubgoalsfromaÞxedstate.TheheuristiccanbedeÞnedinananalogouswaybyreplacingsumsbymaximizations.6.3.MutexesTheregressionsearchoftenleadstostatesthatarenotreachablefromtheinitialstate.Forexample,intheBlocks-World,theregressionofthestate  throughtheactionmove leadstothestate    Thisstaterepresentsasituationinwhichtwoblocks,areonthesameblockItissimpletoshowthatsuchsituationsareunreachableintheBlock-WorldsgivenaÔnormalÕinitialstate.Suchunreachablesituationsarecommoninregressionplanning,andifundetected,causealotofuselesssearch.AgoodheuristicwouldassignaninÞnitecosttosuchsituationsbutourheuristicsarenotasgood.Indeed,thebasicassumptionunderlyingboththeadditiveandthemaxheuristicsÑthattheestimatedcostofasetofatomsisafunctionoftheestimatedcostoftheatomsinthesetÑisviolatedinsuchsituations.Indeed,whilethecostofeachoftheatoms  isÞnite,thecostofthepair  isinÞnite.Betterheuristicsthatdonotmakethisassumptionandcorrectlyreßectthecostofsuchpairsofatomshavebeenrecentlydescribedin[13].Herewefollow[4]anddevelopasimplemechanismfordetectingsomepairsofatomssuchthatanystatecontainingthosepairscanbeproventobeunreachablefromtheinitialstate,andthuscanbegivenaninÞniteheuristicvalueandpruned.TheideaisadaptedfromasimilarideausedinGraphplan[3]andthuswecallsuchpairsofunreachableatomsmutuallyexclusivepairsormutexpairs.AsinGraphplan,thedeÞnitionbelowisnotguaranteedtoidentifyallmutexpairs,andfurthermore,itsaysnothingaboutlargersetsofatomsthatareachievablefrombutwhosepropersubsetsare.AtentativedeÞnitionistoidentifyapairofatomsasamutexwhenisnottrueintheinitialstateandeveryactionthatassertsanatomindeletestheother.ThisdeÞnitionissound(itonlyrecognizespairsofatomsthatarenotachievablejointly)butistooweak.Inparticular,itdoesnotrecognizeasetofatomslike  asamutex,sinceactionslikemove addtheÞrstatombutdonotdeletethesecond. B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 WethususeadifferentdeÞnitioninwhichapairofatomsisrecognizedasmutexwhentheactionsthataddoneoftheatomsinanddonotdeletetheotheratom,canguaranteethroughtheirpreconditionsthatsuchatomwillnotbetrueaftertheaction.Toformalizethis,weconsidersetsofmutexesratherthatindividualpairs.DeÞnition1.Asetofatompairsisamutexsetgivenasetofoperatorsandaninitialstateiffforallatomspairsisnottruein(2)foreverythatadds,eitherdeletes,ordoesnotaddandforsomepreconditionisapairinItissimpletoverifythatifapairofatomsbelongstoamutexset,thentheatomsarereallymutuallyexclusive,i.e.,notachievablefromtheinitialstategiventheavailableoperators.Alsoifaretwomutexsets,willbeamutexsetaswell,andhenceaccordingtothisdeÞnition,thereisasinglelargestmutexset.Ratherthancomputingthisset,however,thatisdifÞcult,wecomputeanapproximationasfollows.WesaythatapairisaÔbadpairÕinwhendoesnotcomplywithoneoftheconditions(1)Ð(2)above.Theprocedureforconstructingamutexsetstartswithasetofpairsanditerativelyremovesallbadpairsfromuntilnobadpairremains.TheinitialsetofÔpotentialÕmutexescanbechoseninanumberofways.Inallcases,theresultofthisprocedureisamutexsetsuchthat.Onepossibilityistosetthesetofallpairsofatoms.In[4],toavoidtheoverheadinvolvedindealingwiththepairsofatomsandmanyuselessmutexes,wechoseasmallersetofpotentialmutexesthatturnsouttobeadequateformanydomains.SuchsetwasdeÞnedastheunionofthesetswhereisthesetofpairssuchthatsomeactionaddsanddeletesisthesetofpairssuchthatforsomepairandsomeactionPrec  ThestructureofthisdeÞnitionmirrorsthestructureofthedeÞnitionofmutexsets.AmutexinrreferstoapairinthesetobtainedfromthesetbysequentiallyremovingallÔbadÕpairs.LiketheanalogousdeÞnitioninGraphplan,thesetdoesnotcaptureallactualmutexes,yetitcanbecomputedfast,andinmanyofthedomainswehaveconsideredappearstoprunetheobviousunreachablestates.AdifferencewithGraphplanisthatthisdeÞnitionidentiÞesstructuralmutexeswhileGraphplanidentiÞestime-dependentmutexes.Thesetwosetsoverlap,buteachcontainspairstheotherdoesnot.TheyareusedindifferentwaysinGraphplanandr.Forexample,inthecompleteTSPdomain[27],pairslikewouldberecognizedasamutexbythisdeÞnitionbutnotbyGraphplan,astheactionsofgoingtodifferentcitiesarenotmutuallyexclusiveforGraphplan. Yetsee[28]forusingGraphplantoidentifysomestructuralmutexes. B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 6.4.AlgorithmTheplannerrusestheadditiveheuristicandthemutexsettoguidearegressionsearchfromthegoal.Theadditiveheuristicisobtainedfromtheestimatedcosts computedonceforallatomsfromtheinitialstate.ThemutexsetisusedtoÔpatchÕtheheuristic:statesarisinginthesearchthatcontainapairingetaninÞnitecostandarepruned.ThealgorithmusedforsearchingtheregressionspaceisthesameastheonesusedinHSP2algorithmwiththeconstantsetto5.Herewedepartfromthedescriptionofrin[4]wherethealgorithmwasgivenaÔgreedyÕbias.Asabove,westicktoapureBFSalgorithm.Thesetofexperimentsbelowcovermoredomainsthanthosein[4]andwillhelpustoassessbetterthestrengthsandlimitationsofregressionheuristicplanninginrelationtoforwardheuristicplanning.6.5.ExperimentsIntheexperiments,wecomparetheregressionplannerrwiththeforwardplannerHSP2.Botharebasedonasearchandbothusethesameadditiveheuristic(inthecaser,patchedwiththemutexinformation).ravoidstherecomputationoftheatomcostsineverystate,andthuscomputestheheuristicfasterandcanexploremorenodesinthesametime.Aswewillsee,thishelpsinsomedomains.However,inotherdomains,risnotmorepowerfulthanHSP2,andinsomedomainsrisactuallyweaker.Thisisduetotworeasons:Þrst,theadditionalinformationobtainedbytherecomputationoftheatomcostsineverystatesometimespaysoff,andsecond,theregressionsearchoftengeneratesspuriousstatesthatarenotrecognizedassuchbythemutexmechanismandcausealotofuselesssearch.TheseproblemsarenotsigniÞcantinthetwodomainsconsideredin[4]butaresigniÞcantinotherdomains.6.5.1.LogisticsFig.8showstheresultsofthetwoplannersrandHSP2overtheLogisticsinstances16Ð30.Thecurvesshowthelengthofthesolutions(left),thetimerequiredtoÞndthesolutions(center),andthenumberofgeneratednodes(right).ItisinterestingtoseethatrgeneratesmorenodesthanHSP2andyetittakesroughlyfourtimeslesstimethanHSP2tosolvetheproblems.Thisfollowsfromthefasterevaluationoftheheuristic.Ontheotherhand,theplansfoundbyrareoftenlongerthanthosefoundbyHSP2.SimilarresultsobtainforthelogisticsinstancesthatarenotshownintheÞgure. Fig.8.ComparisonbetweenHSPrversusHSPoverLogisticsinstances16Ð30.Curvesshowsolutionlength(left),time(center),andnumberofnodesgenerated(right). B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 Fig.9.ComparisonbetweenHSPrversusHSPoverGripperinstances.Curvesshowsolutionlength(left),solution(center),andnumberofnodesgenerated(right). Fig.10.ComparisonbetweenHSPrversusHSPoverHanoi.Curvesshowsolutionlength(left),solution(center),andnumberofnodesgenerated(right).Instance2disks.6.5.2.GripperAsshowninFig.9,asimilarpatternarisesinGripper.HerergeneratesslightlylessnodesthanHSP2,butsinceitgeneratesnodesfaster,thetimegapbetweenthetwoplannergetslargerasthesizeoftheproblemsgrows.Inthiscase,thesolutionsfoundbyrareuniformlybetterthanthesolutionsfoundbyHSP2,andthisdifferencegrowswiththesizeoftheproblems.risalsostrongerthanHSP2inPuzzlewhere,unlikeHSP2(with5),itsolvesthelastinstanceintheset(a15-Puzzleinstance).However,fortheotherthreedomainsdoesnotimproveonHSP2,andindeed,intwoofthesedomains(HanoiandTire-World)itdoessigniÞcativelyworse.6.5.3.HanoiandTire-WorldTheresultsforHanoiareshowninFig.10.rsolvestheÞrstthreeinstances(upto5disks),butitdoesnotsolvetheotherthree.Indeed,asitcanbeseen,intheÞrstthreeinstancesthetimetoÞndthesolutionsandthenumberofnodesgeneratedgrowmuchfasterinrthaninHSP2.ThesamesituationarisesintheTire-WorldwhereHSP2solvesallthreeinstancesandrsolvesonlytheÞrstone.Theproblems,aswementionedabove,aretwo:spuriousstatesgeneratedintheregressionsearchthatarenotdetectedbythemutexmechanisms,andthelackoftheÔfeedbackÕprovidedbytherecomputationoftheatomcostsineverystate.Indeed,errorsintheestimatedcostsofatomsinHSP2canbecorrectedwhentheyarerecomputed;inr,ontheotherhand,theyareneverrecomputed.Sotherecomputationofthesecostshastwoeffects,onethatisbad(timeoverhead)andonethatisgood(additionalinformation).Indomainswheresubgoalsinteractincomplexways,theideaofaforwardsearchinwhichatomcostsarerecomputedineverystateas B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 implementedinHSP2willprobablymakesense;ontheotherhand,indomainswheretheadditiveheuristicisadequate,thebackwardsearchwithnorecomputationsasimplementedrcanbemoreefÞcient.TheresultsfortherandHSP2plannersintheTire-WorldshowthesamepatternasHanoi.Indeed,rsolvesjusttheÞrstinstance,whileHSP2solvesthethreeinstances.Asweshowbelow,however,partoftheprobleminthisdomainhastodowiththespuriousstatesgeneratedintheregressionsearch.6.6.ImprovedmutexcomputationTheprocedureusedinrtoidentifymutexesstartswithasetofpotentialmutexesandthenremovestheÔbadÕpairsfromuntilnoÔbadÕpairremains.AproblemwehavedetectedwiththedeÞnitionin[4],whichwehaveusedhere,isthatthesetofpotentialmutexessometimesisnotlargeenoughandhenceusefulmutexesarelost.Indeed,weperformedexperimentsinwhichissettothecollectionofallatompairs,andthesameprocedureisappliedtothissetuntilnoÔbadÕpairsremains.Inmostofthedomains,thischangedidnÕtyieldadifferentbehavior.However,thereweretwoexceptions.WhilesolvedonlytheÞrstinstanceoftheTire-World,rusingtheextendedsetofpotentialmutexessolvedthethreeinstances.Thisshowsthatinthiscaserwasaffectedbytheproblemofspuriousstates.Ontheotherhand,inproblemslikelogistics,thenewsetleadstoamuchlargersetofmutexesthatarenotasusefulandyethavetobecheckedinallthestatesgenerated.Thisslowsdownnodegenerationwithnocompensatinggainthusmakingrseveraltimesslower.ThecorrespondingcurvesareshowninFig.11,whereÔmutex-1ÕandÔmutex-2ÕcorrespondtotheoriginalandextendeddeÞnitionofthesetofpotentialmutexes.Since,thebeneÞtsappeartobemoreimportantthantheloses,theextendeddeÞnitionseemsworthwhileandwewillmakeitthedefaultoptioninthenextversionofr.However,sinceforthereasonsabove,thenewmechanismisnotcomplete Fig.11.ImpactoforiginalversusextendeddeÞnitionofthesetofpotentialmutexesinTire-WorldandLogistics. B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 either,theproblemcausedbythepresenceofspuriousstatesinregressionplanningremains6.7.Additionalissuesinheuristicregressionplanning6.7.1.AdditiveversusmaxheuristicWhilewedeÞnedtwoheuristics,theadditiveheuristicandthemaxheuristicwehaveusedonlytheadditiveheuristic.Theintuitionunderlyingthischoiceisthattheadditiveheuristicismoreinformedasittakesintoaccountallsubgoals,whilethemaxheuristiconlyfocusesonthesubgoalsthatareperceivedasmostcostly.Inordertotestthisintuitionweranroverallthedomainswiththeadditiveheuristicandthemaxheuristic.InproblemsthatinvolvemanyindependentsubgoalssuchasGripperandLogistics,themaxheuristicisalmostuselessandveryfewinstancesaresolved.Ontheotherhand,inproblemthatinvolvemorecomplexinteractionsamonggoalssuchasHanoiandTire-World,theheuristicdoesslightlybetterthan,andindeed,inTire-Worlditsolvesthesecondinstancethatdoesnotsolve(withinr).FinallyinBlocks-WorldandPuzzlewherethereisacertaindegreeofdecomposability,theheuristicisworsethantheheuristicbutstillmanagestosolveroughlythesamesetofinstancestakingmoretime.Insummary,theadditiveheuristicyieldsabetterbehaviorinrthanthemaxheuristic,butthisdoesnotmeanthatthemaxheuristicisuseless.TheheuristicusedimplicitlyinGraphplanisasareÞnementoftheheuristic,asisthefamilyofhigher-orderheuristicformulatedin[13].Wewillsaymoreaboutthoseheuristicsbelow.6.7.2.Greedybest-rstsearchThealgorithmusedintheversionoftherplannerpresentedin[4]usesthesamealgorithmbutwiththefollowingvariation:whensomeofthechildrenofthelastexpandednodeimprovetheheuristicvalueoftheparent,thebestsuchchildisselectedforexpansionevenifsuchnodeisnotaleastcostnodeintheOpenlist.Theideaistoprovideanadditionalgreedybiasinthesearch.ThismodiÞcationhelpsinsomeinstancesandingeneraldoesnotappeartohurt,yettheboostinperformanceacrossalargesetofdomainsissmall.Forthisreason,wehavedroppedthisfeaturefromrwhichisnowapureBFSregressionplanner.6.7.3.BranchingfactorAcommonargumentforperformingregressionsearchratherthanforwardsearchinplanninghasbeenbasedonconsiderationsrelatedtothebranchingfactorsofthetwospaces[35,46].Wehavemeasuredtheforwardandbackwardbranchingfactorinallthedomainsandfoundthattheyvaryalotfrominstancetoinstance.Forexample,theforwardbranchingfactorintheBlocks-Worldinstancesrangesfrom1683to8462,whilethebackwardbranchingfactorrangesfrom473to1215.InLogistics,theforwardbranchingfactorrangesfrom789to3791,whilethebackwardfactorrangesfrom968to2580.In Asnotedin[3],theproblemofdetectingmutexes,andevenonlyallmutexpairs,isashardastheplanexistenceproblem.Forotherworkaddressingthederivationofinvariantsfromplanningtheories;see[11,42]. B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 problemslikePuzzleandHanoi,theaveragebranchingfactorsareroughlyconstantoverthedifferentinstances,andaresimilarinbothdirections.Wehavefound,however,thattheperformanceofthetwoplanners,HSP2onthesameproblemisnotindirectcorrespondencewiththesizeoftheforwardandbackwardbranchingfactors.Forexample,whileforeachBlocks-WorldinstancetheaveragebranchingfactorinrislessthanhalftheoneinHSP2,andmoreovertheÞrstplannergeneratesnodes6Ð7timesfasteronaveragethanthesecond,risnotbetterthanHSP2inblocksworld.Ontheotherhand,inlogistics,wheretheaveragebranchingfactorHSP2isoftensmallerthantheoneinrdoesbetter.Thuswhileconsiderationsrelatedtothebranchingfactoroftheforwardandbackwardspacesarerelevanttotheperformanceofplanners,theyarenottheonlyormostimportantconsideration.Aswementioned,twoconsiderationsthatarerelevantforexplainingtheperformanceofinrelationtoHSP2arethequalityoftheheuristic(whichinHSP2isrecomputedineverystate),andthepresenceofspuriousstatesintheregressionsearch(thatdonotariseintheforwardsearch).Thislastproblem,however,couldbesolvedbytheformulationofbetterplanningheuristicsinwhichthecostofasetofatomsisdeÞnedintermsofthecostsoftheindividualatomsinthesetasintheheuristics.Suchheuristicsareconsideredin[13]andarebrießydiscussedbelow.7.Relatedwork7.1.HeuristicsearchplanningTheideaofextractingheuristicsfromdeclarativeproblemrepresentationsinplanninghasbeenproposedrecentlybyMcDermott[31]andbyBonet,Loerincs,andGeffner[6].In[6],theheuristicisusedtoguideareal-timeplannerbasedontheLRTAalgorithm[21],whilein[31],theheuristicisusedtoguidealimiteddiscrepancysearch[12].Theheuristicsinbothcasesaresimilar,eventhoughtheformulationandthealgorithmsusedforcomputingthemaredifferent.TheperformanceofMcDermottÕs,however,doesnotappeartobecompetitivewiththetypeofplannersdiscussedinthispaper.ThismaybeduetothefactthatitiswritteninLispanddealswithvariablesandmatchingoperationsatrun-time.Mostcurrentplanners,includingthosereportedinthispaper,arewritteninCanddealwithgroundedoperatorsonly.Ontheotherhand,whilemostoftheseplannersarerestrictedtosmallvariationsoftheStripslanguage,dealswiththemoreexpressiveeTheideaofperformingaregressionsearchfromthegoalforavoidingtherecomputationoftheatomcostswaspresentedin[4]wheretherplannerwasintroduced.Theversionofrconsideredhere,unliketheversionreportedin[4],isbasedonapureBFSalgorithm.Likewise,thepureBFSforwardplannerthatwehavecalledHSP2,hasnÕtbeendiscussedelsewhere.HSP2isthesimplest,andastheexperimentshaveillustrated,itisalsothemostsolidplannerinthefamily.TwoadvantagesofforwardplannersoverregressionplannersisthattheformerdonotgeneratespuriousstatesandtheyoftenbeneÞtfromtheadditionalinformationobtainedbytherecomputationoftheatomcostsineverystate.Mutexmechanismssuchasthose B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 usedbyrandGraphplancanprunesomeofspuriousstatesinsomeproblems,buttheycannotbecomplete.Theideaofcombiningaforwardpropagationfromtocomputeallatomcostsandabackwardsearchfromthegoalforavoidingtherecomputationofthesecostsappearsinreverseformin[43].RefanidisandVlahavascomputecostestimatesbyabackwardpropagationfromthegoalandthenusethoseestimatestoperformaforwardstatespacesearchfromtheinitialstate.Inaddition,theycomputetheheuristicinadifferentwaysotheygetmoreaccurateestimates.7.2.DerivationofheuristicsThenon-admissibleheuristicusedinisderivedasanapproximationoftheoptimalcostfunctionofarelaxedproblemwheredeleteslistsareignored.Thisformulationhastwoobviousproblems.First,theapproximationisnotverygoodasitignoresthepositiveinteractionsamongsubgoalsthatcanmakeonegoalsimplerafterasecondonehasbeenachieved(thisresultsintheheuristicbeingnon-admissible).Second,therelaxationisnotgoodasitignoresthenegativeinteractionsamongsubgoalsthatarelostwhendeletelistsarediscarded.ThesetwoproblemsareaddressedintheheuristicproposedbyRefanidisandVlahavas[43]buttheirheuristicisstillnon-admissibleanddoesnothaveclearjustiÞcation.Adifferentapproachforaddressingtheselimitationshasbeenreportedrecentlyin[13].WhiletheideaoftheheuristicpresentedinSection4istoapproximatethecostofasetofatomsbythecostofthemostcostlyatomintheset,theideain[13]istoapproximatethecostofasetofatomsbythecostofthethemostcostlyatompairintheset.Theresultingheuristic,called,isadmissibleandmoreinformativethantheheuristic,andcanbecomputedreasonablyfast.Indeed,in[13]theheuristicisusedinthecontextofanIDAsearchtocomputeoptimalplans.Higher-orderheuristicsinwhichthecostofsetsofatomsisapproximatedbythemostcostlysubsetofsizearealsodiscussed.Suchhigher-orderheuristicsmayproveusefulinproblemsinwhichsubgoalsinteractincomplexways.ThederivationofadmissibleheuristicsbytheconsiderationofrelaxedmodelshasalonghistoryinAI.Indeed,theManhattandistanceheuristicinslidingtilepuzzlesisnormallyexplainedintermsofthesolutionofarelaxedprobleminwhichtilescanmovetoanyneighboringposition[39].AsimilarrelaxationisusedtoexplaintheMinimumSpanningTreeheuristicusedforsolvingtheTravelingSalesmanProblem.Moreover,in[39],theserelaxationsareshowntofollowfromsimpliÞcationsinsuitableStripsencodings,andinparticulartheManhattanheuristicisderivedbyignoringsomeactionpreconditions.Theideaofderivingheuristicsfromsuitablerelaxationsisapowerfulidea.However,itisoftentoogeneraltoprovidepracticalguidanceintheformulationofconcreteheuristicsforspeciÞcproblems.Indeed,theideaofdroppingactionpreconditionsfromStripsencodingsisguaranteedtoleadtoadmissibleheuristicsbutcomputingsuchheuristicscanbeashardassolvingtheoriginalproblem.Indeed,unlessweremoveallpreconditionstheclassofÔrelaxedÕplanningproblemsisstillintractable.Inthispaper,wehaveusedadifferentrelaxationinwhichdeletelistsareremoved.While,theresultingproblemisstillintractable,itsoptimalcostcanbeapproximatedbythemethodsdiscussedinSection4. B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 Itwouldbeinterestingtoseeifusefulheuristicsforplanningcouldbeobtainedbypolynomialapproximationsthatsimplifythepreconditionsratherthantheactiondeletelists.Theschemeforderivingadmissibleheuristicsfrom[13]canactuallybeseenfromthisperspective.TheautomaticderivationsofusefuladmissibleheuristicshasalsobeentackledbyPrieditis[41].PrieditisÕschemeisbasedonasetoftransformationsthatgeneratealargespaceofrelaxationsgivenproblemsexpressedinaversionofStrips.Thisspaceisthensearchedforrelaxationsthatproduceheuristicsthatspeedupthesearchintheoriginalproblem.HeshowsthatanumberofinterestingheuristicscanbeidentiÞedinthisway.Ourworkdepartsfromthisinthatwesticktooneparticulartypeofrelaxationforallproblems.However,anschemelikePrieditisÕcouldbeusedasanoff-linelearningcomponentofheuristicsearchplannersthatcouldtunethetypeofheuristicforthegivendomain.AmorerecentschemeforderivingheuristicsisbasedonthenotionofpatterndatabasesdevelopedbyCulbersonandSchaeffer[8]andusedbyKorfforÞndingoptimalsolutionstoRubikÕsCube[23].Inaproblemlikethe15-Puzzle,apatterndatabasecanbeunderstoodasatablethatcontainstheoptimalcostsassociatedwitharelaxed(abstracted)statemodelinwhichthelocationofacertainsetoftilesareignored.Sincetherelaxedstatemodelcanhaveamuchsmallersizethantheoriginalstatemodel,itcanbesolvedoptimallybyblindsearch(e.g.,breadth-Þrstsearch).Thentheheuristic ofastatecanbeobtainedbytakingthedistancefromtheprojectionoftotheprojectionofthegoalintherelaxedstatemodel.Ifthereareseveralpatterndatabases,themaximumofthesedistancesistakeninstead.Theideaofpatterndatabasesispowerfulbutisnotcompletelygeneral.Indeed,thesizeoftherelaxedstatemodelthatarisesinplanningproblemswhenthevaluesofcertainstate-variablesareignored,isnotnecessarilysmallerthanthesizeoftheoriginalproblem([14]mentionsthecaseoftheBlocks-World).However,theideaappliesverywelltopermutationproblemssuchasslidingtilepuzzlesandRubikÕsCube,andmayhaveapplicationinmanyplanningdomains.KorfandTaylor[26]alsosketchatheoryofheuristicsthatmayhaveapplicationindomain-independentplanning.Intheslidingtilepuzzles,theirideaistosolveanumberofÔrelaxedÕproblemsinwhichweonlycareaboutdisjointsubsetsoftilesandineachcaseweonlycountthemovesofthetilesselected.Thentheadditionofsuchcountsprovidesanadmissibleheuristicfortheoriginalproblem.Asinthecaseofpatterndatabases,eachoftherelaxedproblemsinvolvesasmallerstatemodelthatcanbesolvedbybrute-forcemethods.Alsoasforpatterndatabases,theapproachseemsapplicabletopermutationproblemsbutnottoarbitraryplanningproblems.Inparticular,itÕsnotclearhowtoapplytheseideastoaproblemlikeBlocks-World.7.3.HeuristicregressionplanningandGraphplanTheoperationoftheregressionplannerrconsistsoftwophases.IntheÞrst,aforwardpropagationisusedtoestimatethecostsofallatomsfromtheinitialstate,andinthesecond,aregressionsearchisperformedusingthosemeasures.ThesetwophasesareincorrespondencewiththetwooperationphasesinGraphplan[3]whereaplangraphisbuiltforwardinaÞrstphase,andissearchedbackwardforplansinthesecond.Thetwoplannersarealsorelatedintheuseofmutexes,andideathatrborrowsfromGraphplan.Forthe B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 rest,randGraphplanlookquitedifferent.However,Graphplancanalsobeunderstoodasanheuristicsearchplannerwithapreciseheuristicfunctionandsearchalgorithm.Fromthispointofview,themaininnovationinGraphplanistheimplementationofthesearchthattakesadvantageoftheplangraphandisquiteefÞcient,andthederivationoftheheuristicthatmakesuseofthemutexinformation.Moreprecisely,fromtheperspectiveofheuristicsearchplanning,themainfeaturesofGraphplancanbeunderstoodasfollows:Plangraph:TheplangraphencodesanadmissibleheuristicwhereiffistheindexoftheÞrstlevelinthegraphthatincludeswithoutamutexandinwhichisnotmemoized(memoizationsareupdatesontheheuristicfunction;see(4).TheheuristicisareÞnedversionoftheheuristicdiscussedinSection4,andiscloselyrelatedtothefamilyofadmissibleheuristicsformulatedin[13].Mutex:Mutexesareusedtoprunestatesintheregressionsearch(asinr)andtoreÞnetheheuristic.Inparticular,thecostofasetofatomsisnolongergivenbythecostofthemostcostlyatominthesetwhenintheÞrstlayerthatcontainsoccurswithamutex.Algorithm:ThesearchalgorithmisaversionofIterativeDeepeningIDA)[20],wherethesumoftheaccumulatedcost andtheestimatedcostisusedtoprunenodeswhosecostexceedthecurrentthreshold.ActuallyGraphplannevergeneratessuchnodes.ThealgorithmtakesadvantageoftheinformationstoredintheplangraphandconvertsthesearchinaÔsolutionextractionÕprocedure.Memoization:Memoizationsareupdatesontheheuristicfunction(see(1)).Theresultingalgorithmisamemory-extendedversionofIDAthatcloselycorrespondstotheMRECalgorithm[44].InMREC,theheuristicofanodeisupdatedandstoredinahash-tableafterthesearchbelowthechildrenofcompleteswithoutasolution(giventhecurrentthreshold).Parallelism:Graphplan,unliker,searchesaparallelregressionspace.Whilethebranchingfactorinthissearchcanbeveryhigh,GraphplanmakessmartuseoftheinformationinthegraphtogenerateonlythechildrenthatareÔrelevantÕandwhosecostdoesnotexceedthecurrentthreshold.ThebranchingruleusedinGraphplanismadeexplicitin[13].In[13]GraphplaniscomparedwithapureIDAplannerbasedonanadmissibleheuristicequivalenttoGraphplanÕs.Insequentialdomainstheplannershaveasimilarperformance,butonparalleldomains,GraphplanismorethananorderofmagnitudefasterduetothemoreefÞcientIDAsearchaffordedbytheplangraph.Theplangraph,however,restrictsGraphplantoIDAsearches,anditcannotbeeasilyadaptedtobest-ÞrstsearchesWIDAsearches[22]unlessoneabandonstheideaofsearchassolutionextraction;seee8.ConclusionsWehavepresentedaformulationofplanningasheuristicsearchandhaveshownthatsimplestatespacesearchalgorithmsguidedbyageneraldomain-independentheuristic Theheuristicisextractedfromtheplangraphandusedtoguideanexplicitsearchin[36]. B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 produceafamilyofplannersthatarecompetitivewithsomeofthebestcurrentplanners.Wehavealsoexploredanumberofvariations,suchasreversingthedirectionofthesearchforacceleratingnodeevaluation,andextractinginformationaboutpropositionalinvariantsforavoidingdead-ends.Theplannerthatshowedthemostsolidperformance,however,wasthesimplestplanner,HSP2,basedonabest-Þrstforwardsearch,inwhichatomcostsarerecomputedfromscratchineverystate.Heuristicsearchplannersarerelatedtospecializedproblemsolversbutdifferfromthemintheuseofageneraldeclarativelanguageforstatingproblemsandageneralmechanismforextractingheuristics.Plannersmustoffergoodmodelinglanguageforexpressingproblemsinaconvenientway,andgeneralsolversforoperatingonthoserepresentationsandproducingefÞcientsolutions.Aconcretechallengeforthefutureistoreducethegapinperformancebetweenheuristicsearchplannersandspecializedproblemsolversindomainslikethe24-Puzzle[26],RubikÕsCube[23],andSokoban[17].Inaddition,forplannerstobemoreapplicabletorealproblems,itisnecessarythattheyhandleaspectssuchasnon-Booleanvariables,actiondurations,andparallelactions.Plannersshouldbeabletoaccommodatearichclassofschedulingproblems,yetveryfewplannerscurrentlyhavesuchcapabilities,andevenfewerifanycancompetewithspecializedsolvers.Threeissuesthatwebelievemustbeaddressedinordertomakeheuristicsearchplannersmoregeneralandmorepowerfularetheonesdiscussedbelow.Heuristics:Theheuristicsconsideredinthispaperarepoorestimators,andcannotcompetewithspecializedheuristics.TheheuristicusedinGraphplanisbetterthanbutitisnotgoodenoughforproblemslikeRubikÕsCubeorthe24-Puzzlewheresubgoalsinteractincomplexways.In[13],aclassofadmissibleheuristicsareformulatedinwhichthecostofasetofatomsisapproximatedbythecostofthemostlycostlysubsetofsize.Forreducestotheheuristic,andforreducestotheGraphplanheuristic.Higher-orderheuristics,for2,mayproveeffectiveincomplexproblemssuchasthe24-PuzzleandRubikÕsCube,andtheymayactuallybecompetitivewiththespecializedheuristicsusedforthoseproblems[23,26].Asmentionedin[13],thechallengeistocomputesuchheuristicsreasonablyfast,andtousethemwithlittleoverheadatruntime.Suchhigher-orderheuristicsarerelatedtopatterndatabases[8],buttheyareapplicabletoallplanningproblemsandnotonlytopermutationproblems.Branchingrules:InhighlyparalleldomainslikeRocketsandLogistics,SATapproachesappeartoperformbestamongoptimalparallelplanners.Thismaybeduetothebranchingschemeused.InSATformulations,thespaceisexploredbysettingthevalueofanyvariableatanytimepoint,andthenconsideringeachoftheresultingstatepartitionsseparately.InGraphplanandinheuristicsearchapproaches,thesplittingisdonebyapplyingallpossibleactions.Yetalternativebranchingschemes,arecommoninheuristicbranch-and-boundsearchprocedures[30],inparticular,inschedulingapplications[7].Workonparallelplanning,inparticularinvolvingactionsofdifferentdurations,wouldmostlikelyrequiresuchalternativebranchingschemes.Modelinglanguages:AlltheplannersdiscussedinthispaperareStripsplanners.YetfewrealproblemscanactuallybeencodedefÞcientlyinStrips.Thishasmotivatedthedevelopmentofextensionssuchas[40]andFunctionalStrips[10].Fromthe B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 pointofviewofheuristicsearchplanning,theissuebecomesthederivationofgoodheuristicsfromsuchricherlanguages.Theideasconsideredinthispaperdonotcarrydirectlytosuchlanguagesbutitseemsthatitshouldbepossibletoexploitthericherrepresentationsforextractingbetterheuristics.AcknowledgementsWethankDanielLeBerre,RinaDechter,PatrikHaslum,JšrgHoffmann,RaoKambhampati,RichardKorf,andDrewMcDermottfordiscussionsrelatedtothiswork.ThisworkhasbeenpartiallysupportedbygrantS1-96001365fromConicit,VenezuelaandbytheWallenbergFoundation,Sweden.BlaiBonetiscurrentlyatUCLAwithaUSB-Conicitfellowship.AppendixA.PostscriptAfewmonthsafterÞnishingthispaper,theSecondAIPSPlanningCompetitiontookplace.ThechairwasFahiemBacchus,whowillreporttheresultsinaforthcomingissueoftheAIMagazine.Thistimethereweretwotracks,onefordomain-independentplanners,theotherforhand-tailoredsystems.HSP2participatedintheÞrsttrack,alongwithelevenotherplanners,includingthethreeotherplannersthatparticipatedtheÞrsttime:STAN,andBLACKBOX.Inthissecondcompetition,halfoftheplanners,i.e.,sixplanners,extractedandusedheuristicestimatorstoguidethesearchforplans.Thetopperformingplannerwas[15],anheuristicsearchplannerbasedonaforwardhill-climbingsearch.HSP2endedsecond,alongwiththreeotherplanners.Theperformanceofwasquiteimpressivesolvingalmostallproblems,reallyfast,producinginmostcasesverygoodsolutions.differsfrominthreeways:(1)theheuristic,whichprovidesabetterapproximationofthecostoftherelaxedplanningproblemwithoutdeletes,(2)thesearchalgorithm,whichisahill-climbingsearchwithalookaheadmechanism,(3)theadditionofafastpruningcriterionthatallowssomestatestobediscardedwithoutevencomputingtheirheuristicvalues.ThesethreemodiÞcations,together,seemtocombineverywell,asJšrgHoffmannshowsinaforthcomingpaper.References[1]R.Ahuja,T.Magnanti,J.Orlin,NetworkFlows:Theory,Algorithms,andApplications,Prentice-Hall,EnglewoodCliffs,NJ,1993.[2]C.Anderson,D.Smith,D.Weld,ConditionaleffectsinGraphplan,in:Proc.4thInternationalConferenceonAIPlanningSystems,AAAIPress,MenloPark,CA,1998,pp.44Ð53.[3]A.Blum,M.Furst,Fastplanningthroughplanninggraphanalysis,ArtiÞcialIntelligence90(1Ð2)(1997) B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 [4]B.Bonet,H.Geffner,Planningasheuristicsearch:Newresults,in:RecentAdvancesinAIPlanning:Proc.5thEuropeanConferenceonPlanning,LectureNotesinArtiÞcialIntelligence,Vol.1809,Springer,Berlin,1999,pp.359Ð371.[5]F.Bacchus,F.Kabanza,Usingtemporallogicstoexpresssearchcontrolknowledgeforplanning,ArtiÞcialIntelligence116(1Ð2)(2000)123Ð191.[6]B.Bonet,G.Loerincs,H.Geffner,Arobustfastactionselectionmechanismforplanning,in:Proc.AAAI-97,Providence,RI,AAAIPress,MenloPark,CA,1997,pp.714Ð719.[7]J.Carlier,E.Pinson,Analgorithmforsolvingthejobshopproblem,ManagementSci.35(2)(1989)164Ð[8]J.Culberson,J.Schaeffer,Patterndatabases,Comput.Intelligence14(3)(1998)318Ð334.[9]R.Fikes,N.Nilsson,STRIPS:Anewapproachtotheapplicationoftheoremprovingtoproblemsolving,ArtiÞcialIntelligence2(1971)189Ð208.[10]H.Geffner,FunctionalStrips:Amoregenerallanguageforplanningandproblemsolving,Logic-basedAIWorkshop,Washington,DC,1999.[11]A.Gerevini,L.Schubert,Inferringstateconstraintsfordomain-independentplanning,in:Proc.AAAI-98,Madison,WI,AAAIPress,MenloPark,CA,1998,pp.905Ð912.[12]W.Harvey,M.Ginsberg,Limiteddiscrepancysearch,in:Proc.IJCAI-95,Montreal,Quebec,MorganKaufmann,SanFrancisco,CA,1995,pp.607Ð613.[13]P.Haslum,H.Geffner,Admissibleheuristicsforoptimalplanning,in:Proc.5thInternationalConferenceonAIPlanningSystems,AAAIPress,MenloPark,CA,2000,pp.70Ð82.[14]R.Holte,I.Hernadvolgyi,Aspace-timetradeoffformemory-basedheuristics,in:Proc.AAAI-99,Orlando,FL,AAAIPress,MenloPark,CA,1999,pp.704Ð709.[15]J.Hoffmann,Aheuristicfordomainindependentplanning,anditsuseinanenforcedhill-climbingalgorithm,in:Proc.12thInternationalSymposiumonMethodologiesforIntelligentSystems(ISMIS-00),Springer,Berlin,2000,pp.216Ð227.[16]P.VanHentenryck,H.Simonis,M.Dincbas,Constraintsatisfactionusingconstraintlogicprogramming,ArtiÞcialIntelligence58(1Ð3)(1992)113Ð159.[17]A.Junghanns,J.Schaeffer,Domain-dependentsingle-agentsearchenhancements,in:Proc.IJCAI-99,Stockholm,Sweden,MorganKaufmann,SanFrancisco,CA,1999,pp.570Ð575.[18]S.Kambhampati,R.SanchezNigenda,DistancebasedgoalorderingheuristicsforGraphplan,in:Proc.5thInternationalConferenceonArtiÞcialIntelligencePlanningandScheduling,AAAIPress,MenloPark,CA,2000,pp.315Ð322.[19]J.Koehler,B.Nebel,J.Hoffman,Y.Dimopoulos,ExtendingplanninggraphstoanADLsubset,in:RecentAdvancesinAIPlanning:Proc.4thEuropeanConferenceonPlanning,LectureNotesinArtiÞcialIntelligence,Vol.1348,Springer,Berlin,1997,pp.273Ð285.[20]R.Korf,Depth-Þrstiterative-deepening:Anoptimaladmissibletreesearch,ArtiÞcialIntelligence27(1)(1985)97Ð109.[21]R.Korf,Real-timeheuristicsearch,ArtiÞcialIntelligence42(2Ð3)(1990)189Ð211.[22]R.Korf,Linear-spacebest-Þrstsearch,ArtiÞcialIntelligence62(1)(1993)41Ð78.[23]R.Korf,FindingoptimalsolutionstoRubikÕsCubeusingpatterndatabases,in:Proc.AAAI-98,Madison,WI,AAAIPress,MenloPark,CA,1998,pp.700Ð705.[24]H.Kautz,B.Selman,Pushingtheenvelope:Planning,propositionallogic,andstochasticsearch,in:Proc.AAAI-96,Portland,OR,AAAIPress,MenloPark,CA,1996,pp.1194Ð1201.[25]H.Kautz,B.Selman,UnifyingSAT-basedandGraph-basedplanning,in:Proc.IJCAI-99,Stockholm,Sweden,MorganKaufmann,SanFrancisco,CA,1999,pp.318Ð327.[26]R.Korf,L.Taylor,FindingoptimalsolutionstotheTwenty-FourPuzzle,in:Proc.AAAI-96,Portland,OR,AAAIPress,MenloPark,CA,1996,pp.1202Ð1207.[27]D.Long,M.Fox,TheefÞcientimplementationoftheplan-graphinSTAN,J.ArtiÞcialIntelligenceRes.10(1999)85Ð115.[28]D.Long,M.Fox,Utilizingautomaticallyinferredinvariantsingraphconstructionandsearch,in:Proc.5thInternationalConferenceonAIPlanningSystems,AAAIPress,MenloPark,CA,2000,pp.102Ð111.[29]P.Laborie,M.Ghallab,Planningwithsharableresourcesconstraints,in:Proc.IJCAI-95,Montreal,Quebec,MorganKaufmann,SanFrancisco,CA,1995,pp.1643Ð1649. B.Bonet,H.Geffner/ArticialIntelligence129(2001)5–33 [30]E.Lawler,A.RinnooyKan(Eds.),TheTravelingSalesmanProblem:AGuidedTourofCombinatorialOptimization,Wiley,NewYork,1985.[31]D.McDermott,Aheuristicestimatorformeans-endsanalysisinplanning,in:Proc.3rdInternationalConferenceonAIPlanningSystems,AAAIPress,MenloPark,CA,1996,pp.142Ð149.[32]D.McDermott,PDDLÑThePlanningDomainDeÞnitionLanguage,1998,Availableathttp://www.cs.yale.edu/dvm.[33]D.McDermott,Usingregression-matchgraphstocontrolsearchinplanning,ArtiÞcialIntelligence109(1Ð2)(1999)111Ð159.[34]D.McDermott,The1998AIPlanningSystemsCompetition,AIMagazine21(2)(2000)35Ð56.[35]N.Nilsson,PrinciplesofArtiÞcialIntelligence,Tioga,PaloAlto,CA,1980.[36]X.Nguyen,S.Kambhampati,Extractingeffectiveandadmissibleheuristicsfromtheplanninggraph,in:Proc.AAAI-2000,Austin,TX,AAAIPress,MenloPark,CA,2000.[37]A.Newell,H.Simon,GPS:Aprogramthatsimulateshumanthought,in:E.Feigenbaum,J.Feldman(Eds.),ComputersandThought,McGrawHill,NewYork,1963,pp.279Ð293.[38]A.Newell,H.Simon,HumanProblemSolving,Prentice-Hall,EnglewoodCliffs,NJ,1972.[39]J.Pearl,Heuristics,MorganKaufmann,SanFrancisco,CA,1983.[40]E.Pednault,ADL:ExploringthemiddlegroundbetweenStripsandthesituationcalculus,in:Proc.1stInternationalConferenceonPrinciplesofKnowledgeRepresentationandReasoning(KR-89),Toronto,Ontario,MorganKaufmann,SanFrancisco,CA,1989,pp.324Ð332.[41]A.Prieditis,Machinediscoveryofeffectiveadmissibleheuristics,MachineLearning12(1993)117Ð141.[42]J.Rintanen,Aplanningalgorithmnotbasedondirectionalsearch,in:Proc.6thInternationalConferenceonPrinciplesofKnowledgeRepresentationReasoning(KR-98),Trento,Italy,MorganKaufmann,SanFrancisco,CA,1998,pp.617Ð624.[43]I.Refanidis,I.Vlahavas,GRT:AdomainindependentheuristicforStripsWorldsbasedongreedyregressiontables,in:RecentAdvancesinAIPlanning:Proc.5thEuropeanConferenceonPlanning,LectureNotesinArtiÞcialIntelligence,Vol.1809,Springer,Berlin,1999,pp.346Ð358.[44]A.Sen,A.Bagchi,FastrecursiveformulationsforBFSthatallowcontrolledusedofmemory,in:Proc.IJCAI-89,Detroit,MI,MorganKaufmann,SanFrancisco,CA,1989,pp.297Ð302.[45]P.vanBeek,X.Chen,CPlan:Aconstraintprogrammingapproachtoplanning,in:Proc.AAAI-99,Orlando,FL,AAAIPress,MenloPark,CA,1999,pp.585Ð590.[46]D.Weld,Anintroductiontoleastcommitmentplanning,AIMagazine15(4)(1994)27Ð61.