Is Robotics Going Statistics The Field of Probabilistic Robotics Sebastian Thrun School - PDF document

IsRoboticsGoingStatistics?TheFieldofProbabilisticRoboticsSebastianThrunSchoolofComputerScienceCarnegieMellonUniversityhttp://www.cs.cmu.edu/thrundraft,pleasedonotcirculateAbstractInthe1970s,mostresearchinroboticspresupposedtheavailabilityofexactmodels,ofrobotsandtheirenvironments.Littleemphasiswasplacedonsensingandtheintrinsiclimitationsofmodelingcomplexphysicalphenomena.Thischangedinthemid-1980s,whentheparadigmshiftedtowardsreactivetechniques.Reactivecontrollersrelyoncapablesensorstogeneraterobotcontrol.Rejectionsofmodelsweretypicalforresearchersinthiseld.Sincethemid-1990s,anewapproachhasbeguntoemerge:probabilisticrobotics.Thisapproachreliesonstatisticaltechniquestoseamlesslyintegrateimperfectmodelsandimperfectsensing.Thepresentarticledescribesthebasicsofprobabilisticroboticsandhighlightssomeofitsrecentsuccesses.1IntroductionInrecentyears,theeldofroboticshasmadesubstantialprogress.Inthepast,robotsweremostlyconnedtofactoryoorsandassemblylines,boundtoperformthesamenarrowtasksoverandoveragain.Arecentseriesofsuccessfulrobotsystems,however,hasdemonstratedthatroboticshasadvancedtoalevelwhereitisreadytoconquermanynewelds,suchasspace,medicaldomains,personalservices,entertainment,andmilitaryapplications.Manyofthesenewdomainsarehighlydynamicanduncertain.Uncertaintyarisesformanydifferentreasons:theinherentlimitationstomodeltheworld,noiseandperceptuallimitationsinarobot'ssensormeasurements,andtheapproximatenatureofmanyalgorithmicsolutions.Inthisuncertainlyliesoneoftheprimarychallengesfacedbyroboticsresearchtoday.ThreeexamplesofsuccessfulrobotsystemsthatoperateinuncertainenvironmentsareshowninFig-ure1:acommerciallydeployedautonomousstraddlecarrier[3],aninteractivemuseumtourguiderobot[7,11],andaprototyperoboticassistantfortheelderly.Thestraddlecarrieriscapableoftransportingcontain-ersfasterthantrainedhumanoperators.Thetourguiderobot—oneinaseriesofmany—cansafelyguidevisitorsthroughdenselycrowdedmuseums.TheNursebotrobotispresentlybeingdevelopedtointeractwithelderlypeopleandassisttheminvariousdailytasks.Alloftheserobotshavetocopewithuncertainty.Thestraddlecarrierfacesintrinsiclimitationswhensensingitsownlocationandthatofthecontainers.Asimilarproblemisfacedbythemuseumtourguiderobot,butheretheproblemisaggravatedbythepresenceofpeople.Theelderlycompanionrobotfacestheadditionaluncertaintyofhavingtounderstandspokenlanguagebyelderlypeople,andcopingwiththeirinabilitytoexpresstheirexactwishes.Inalltheseappli-cationdomains,theenvironmentsarehighlyunpredictable,andsensorsarecomparativelypoorwithregardtotheperformancetasksathand.1 (a)StraddleCarrier(b)Roboticmuseumtourguide(c)PersonalRoboticAssistantfortheElderlyFigure1:Threerobotscontrolledbyprobabilisticsoftware:Aroboticstraddlecarrier,amuseumtourguiderobot,andtheNursebot,aroboticassistantfornursesandtheelderly.Astheseexamplessuggest,theabilitytoaccommodateuncertaintyisakeyrequirementforcontempo-raryroboticsystems.Thisraisesthequestionastoappropriatemechanismsforcopingwithuncertainty.Whattypeofinternalworldmodelsshouldrobotsemploy?Andhowshouldsensormeasurementsbeinte-gratedintotheirinternalstatesofinformation?Howshouldrobotsmakedecisionseveniftheyareuncertainabouteventhemostbasicstatevariablesintheworld?Theprobabilisticapproachtoroboticsaddressesthesequestionsthroughasinglekeyidea:representinginformationprobabilistically.Inparticular,worldmodelsintheprobabilisticapproachareconditionalprob-abilitydistributions,whichdescribethedependenceofcertainvariablesonothersinprobabilisticterms.Arobot'sstateofknowledgeisalsorepresentedbyprobabilitydistributions,whicharederivedbyintegratingsensormeasurementsintotheprobabilisticworldmodelsgiventotherobot.Probabilisticrobotcontrolan-ticipatesvariouscontingenciesthatmightariseinuncertainworlds,therebyseamlesslyblendinginformationgathering(exploration)withrobustperformance-orientedcontrol(exploitation).Themovetoprobabilistictechniquesinroboticsisparalleledinmanyothersubeldsofarticialintelli-gence,suchascomputervision,language,andspeech.Probabilisticroboticsleveragesdecadesofresearchinprobabilitytheory,statistics,engineeringandoperationsresearch.Inrecentyears,probabilistictech-niqueshavesolvedmanyoutstandingroboticsproblems,andtheyhaveledtonewtheoreticalinsightsintothestructureofroboticsproblemsandtheirsolutions.2Models,Sensors,andThePhysicalWorldClassicalroboticstextbooksoftendescribeatlengththekinematicsanddynamicsofroboticdevices.Thesetopicsaddressthequestionofhowcontrolsaffectthestateoftherobotand,morebroadly,theworld.However,textbooksoftensuggestadeterministicrelationship:Theeffectofapplyingcontrolactionutotherobotatstatexisgovernedbythefunctionalrelationshipx0=f(u;x),forsome(deterministic)functionf.Forexample,xmightbethecongurationandvelocityofaroboticarm,andumightbethemotorcurrentsassertedinaxedtimeinterval.Suchanapproachcharacterizesidealizedrobotsonly—freeofwearandtear,inaccuracies,controlnoise,andthealike.Inreality,theoutcomesofcontrolactionsareuncertain.Forexample,arobotthatexecutesacontrolleveragingitspositionbyonemeterforwardmightexpecttobeexactlyonemeterawayfromwhereitstarted,butinrealitywilllikelynditselfinanunpredictablelocationnearby.Theprobabilisticapproachaccountsforthisuncertaintybyusingconditionalprobabilitydistributionstomodelrobots.Suchmodels,commonlydenotedp(x0ju;x),specifytheposteriorprobabilityoverstatesx0thatmightresultwhenapplyingcontrolutoarobotwhosestateisx.Putdifferently,insteadofmakingadeterministicprediction,probabilistictechniquesmodelthefactthattheoutcomeofrobotcontrols2 AlgorithmparticleFilters(X;u;z)letX0=X0aux=;//forwardprojectionstepfori=1toNdoretrievei-thparticlexifromparticlesetXdrawx0p(x0ju;xi)usingthemotionmodelp(x0ju;x)addx0toX0auxendfor//resamplingstepforj=1toNdodrawrandomx0fromX0auxwithprobabilityproportionaltop(zjx0j)addxjtoX0endforreturnX0endalgorithmTable1:Basicparticlelteralgorithm,whichimplementsBayesltersusingapproximateparticlerepre-sentation.TheposteriorisrepresentedbyasetofNparticlesX0,whichisroughlydistributedaccordingtotheposteriordistributionofallstatesxgiventhedatathatiscommonlycalculatedbyBayeslters.isuncertain,byassigningaprobabilitydistributionoverthespaceofallpossibleoutcomes.Assuch,theygeneralizeclassicalkinematicsanddynamicstoreal-worldrobotics.Inthesamevein,manytraditionaltextbookspresupposethatthestateoftherobotxbeknownatalltimes.Usually,thestatexcomprisesallnecessaryquantitiesrelevanttorobotpredictionandcontrol,suchastherobot'sconguration,itsposeandvelocity,thelocationofsurroundingitems(obstacles,people,etc.).Inidealizedworlds,therobotmightpossesssensorsthatcanmeasure,withouterror,thestatex.Suchsensorsmaybecharacterizedbyadeterministicfunctiong,capableofrecoveringthefullstatefromsensormeasurementsz,thatis,x=g(z).Realsensorsarecharacterizedbynoiseand,moreimportantly,byrangelimitations.Forexample,camerascannotseethroughwalls.Theprobabilisticapproachgeneralizesthisidealizedviewbymodelingrobotsensorsbyconditionalprobabilitydistributions.Sensorsmaybecharacterizedbyforwardmodelsp(zjx),whichreasonfromstatetosensormeasurements,ortheirinversep(xjz)—dependingonalgorithmicdetailsbeyondthescopeofthisarticle.Asthisdiscussionsuggests,probabilisticmodelsareindeedgeneralizationsoftheirclassicalcounter-parts.Theexplicitmodelingofuncertainty,however,raisesfundamentalquestionsastowhatcanbedonewiththeseworldmodels.Canwerecoverthestateoftheworld?Canwestillcontrolrobotssoastoachievesetgoals?3ProbabilisticStateEstimationArstanswertothesequestionscanbefoundintherichliteratureonprobabilisticstateestimation.Thisliteratureaddressestheproblemofrecoveringthestatevariablesxfromsensordata.Commonstatevariablesincludeparametersregardingtherobot'sconguration,suchasitslocationrelativetoanexternalcoordinate3 (a)Robot particlesPerson particles(b)Robot particlesPerson particles(c)Figure2:Evolutionoftheconditionalparticlelterfromglobaluncertaintytosuccessfullocalizationandtracking.frame.Theproblemofestimatingsuchparametersisoftenreferredtoaslocalization,parametersspecifyingthelocationofitemsintheenvironment,suchasthelocationofwalls,doors,andobjectsofinterest.Thisproblem,knownasmapping,isregardedoneofthemostdifcultstateestimationduetothehighdimensionalityofsuchparameterspaces[1],andparametersofobjectswhosepositionchangesovertime,suchaspeople,doors,andotherrobots.Thisproblemissimilartothemappingproblem,withtheaddeddifcultychanginglocationsovertime.ThepredominantapproachforstateestimationinprobabilisticroboticsisknownasBayeslters.Bayesltersofferamethodologyforestimatingaprobabilitydistributionoverthestatex,conditionedonallavailabledata(controlsandsensormeasurements).Theydosorecursively,basedonthemostrecentcontroluandmeasurementz,thepreviousprobabilisticestimateofthestate,andtheprobabilisticmodelsp(x0jx;u)andp(zjx)discussedintheprevioussection.Thus,Bayeslterdonotjust“guess”thestatex.Rather,theycalculatetheprobabilitythatanystatexiscorrect.PopularexamplesofBayesltersarehiddenMarkovmodels,Kalmanlters,dynamicBayesnetworksandpartiallyobservableMarkovdecisionprocesses[5,10].Forlow-dimensionalstatespaces,researchinroboticsandappliedstatisticshasproducedawealthofliteratureonefcientprobabilisticestimation.Remarkablypopularisanalgorithmknownasparticlelters,whichincomputervisionisknownascondensationalgorithmandinroboticsasMonteCarlolocaliza-tion[2].Thisalgorithmapproximatesthedesiredposteriordistributionthroughasetofparticles.ParticlesaresamplesofstatesxwhicharedistributedroughlyaccordingtotheveryposteriorprobabilitydistributionspeciedbyBayeslters.Table1statesthebasicparticlelteringalgorithm.InanalogytoBayeslters,thealgorithmgeneratesaparticlesetX0recursively,fromthemostrecentcontrolu,themostrecentmea-surementz,andtheparticlesetXthatrepresentstheprobabilisticestimatebeforeincorporatinguandz.Itdoessointwophases:First,it“guesses”statesx0basedonparticlesdrawnfromXandtheprobabilisticmotionmodelp(x0ju;x).Subsequently,theseguessesareresampledinproportiontotheperceptuallike-lihood,p(zjx0i).TheresultingsamplesetisapproximatelydistributedaccordingtotheBayesianposterior,takinguandzintoaccount.Figure2illustratesparticleltersviaanexample.Amobilerobot,equippedwithalaserrangender,simultaneouslyestimatesitslocationrelativetoatwo-dimensionalmapofacorridorenvironmentandthenumberandlocationsofnearbypeople.Inthebeginning(Panel2(a)),therobotisgloballyuncertainasto4 (a)-20-15-10-505101520-10010203040X (m)Y (m)Estimated Path of the VehicleFeature Returns (b)Figure3:(a)3Dvolumetricmap,acquiredbyamobilerobotinreal-time.Thelowerpartofthemapisbelowtherobot'ssensors,henceisnotmodeled.(b)Mapofunderwaterlandmarks,acquiredbythesubmersiblevehicleOberonattheUniversityofSydney.CourtesyofStefanWilliamsandHughDurrant-Whyte.whereitis.Consequently,theparticlesrepresentingitslocationandthatofthepersonarespreadthroughoutthefreespaceinthemap.Astherobotmoves(Panel2(b)),theparticlesrepresentingtherobot'slocationquicklyconvergetotwodistinctlocationsinthecorridor,asdotheparticlesrepresentingtheperson'slo-cation.Afewtimestepslater,theambiguityisresolvedandbothsetsofparticlesfocusonthecorrectpositionsinthemap,asshowninPanel2(c).Localizationalgorithmsbasedonparticleltersarearguablethemostpowerfulalgorithmsinexistence.Asthisexampleillustrates,particlelterscanrepresentawiderangeofmulti-modaldistributions.Theyareeasilyimplementedasresource-adaptivealgorithm,capableofadaptingthenumberofparticlestotheavailablecomputationalresources.Andnally,theyconvergeforalargerangeofdistributions,fromgloballyuncertaintonear-deterministiccases.4TowardsMillionsofDimensionsInhigh-dimensionalstatespaces,computationalconsiderationsmayposeseriousobstacleswhenestimatingstate.Robotmapping,tonameapopularexampleofahigh-dimensionalproblem,ofteninvolvesthousandsofdimensions,ifnotmillions!Forexample,thevolumetricmapshowninFigure3(a)iscomprisedofseveralmillionsoftexturevalues,inadditiontothousandsofstructuralparameters.Thisraisesthequestionastowhetherprobabilistictechniquesareequippedtoperformstateestimationinsuchhigh-dimensionalspaces.Theanswerisquiteintriguing.Todate,virtuallyallstate-of-the-artalgorithmsinareassuchaslocalization,mapping,andpeopletrackingareprobabilistic.Manyprobabilisticapproachesestimatethemodeoftheposterior,whichissimplythemostlikelystatex(theremightbemorethanone).Sometechniques,suchasKalmanlters,alsocomputeacovariancematrix,whichmeasuresthecurvatureoftheposterioratthemode.Thespecictechniquesforestimatingthemodeandthecovariancevarywidely,dependingonthenatureofthestateestimationproblem.Intheroboticmappingproblem,twoofthemostwidelyusedalgorithmsareextendedKalmanlters(EKFs)[5]andtheexpectationmaximization(EM)algorithm[6].ExtendedKalmanltersareapplicablewhentheposteriorcanreasonablyassumedtobeGaussian.Thisisusuallythecasewhenmappingthelocationsoflandmarksthatcanbeuniquelyidentied.Kalmanltertechniqueshaveproventobecapableofmapping5 large-scaleoutdoorandunderwaterenvironmentswhilesimultaneouslyestimatingthelocationoftherobotrelativetothemap[1].Figure3(b)showsanexamplemapoflandmarksinanunderwaterenvironment,obtainedbyresearchersattheUniversityofSydney[12].Inthegeneralmappingproblem,thedesiredposteriormayhaveexponentiallymanymodes—notjustone.Differentmodescommonlyarisefromuncertaintyincalculatingthecorrespondencebetweenmapitemssensedatdifferentpointsintime—aproblemcommonlyknownasdataassociationproblem.Manyoftoday'sbestalgorithmsforstateestimationwithunknowndataassociationarebasedontheEMalgo-rithm[6].Thisalgorithmperformsalocalhill-climbingsearchinthespaceofallstatesx(e.g.,maps),withtheaimofcalculatingthemode.The“trick”oftheEMalgorithmistosearchiteratively,byalternatingastepthatcalculatesexpectationsoverthedataassociationandrelatedlatentvariables,followedbyastepthatcomputesanewmodeunderthesexedexpectations.Thisleadstoasequenceofstateestimates(e.g.,maps)ofincreasinglikelihood.Incaseswhereboththesestepscanbecalculatedinclosedform,EMcanbeahighlyeffectivealgorithmforestimatingthemodeofcomplexposteriors.Forexample,themapshowninFigure3(a)hasbeengeneratedthroughanon-linevariantoftheEMalgorithm,accommodatingerrorsintherobotodometryandexploitingaBayesianpriorthatbiasestheresultingmapstowardsplanarsurfaces[4].Inalltheseapplications,probabilisticmodelselectiontechniquesareemployedforndingmodelsofthe“right”complexity.5ProbabilisticPlanningandControlStateestimationisonlyhalfthestory.Clearly,theultimategoalofanyroboticssoftwaresystemistocontrolroboticdevices.Itshouldcomeatnosurprisethatprobabilistictechniquesspecicallytakeuncertaintyintoconsiderationwhendevisingrobotcontrol.Bydoingso,theyarerobusttosensornoiseandincompleteinformation.Probabilitytheoryprovidesasoundframeworkforactiveinformationgathering,smoothlyblendingexplorationandexploitationasmostbenecialforthecontrolgoalsathand.Existingprobabilisticcontrolalgorithmscanmainlybegroupedintotwocategories:greedyandnon-greedy.Bothfamiliesassumetheavailabilityofapayofffunction,whichspeciesthecostsandbenetsassociatedwiththevariouscontrolchoices.Whereasgreedyalgorithmsmaximizethepayofffortheimme-diatenexttimestep,non-greedyalgorithmconsiderentiresequencesofcontrols,therebymaximizingthe(moreappropriate)cumulativepayoffoftherobot.Clearly,non-greedymethodsaremoredesirablefromaperformancepointofview.Thecomputationalcomplexityofplanningunderuncertainty,however,makegreedyalgorithmswelcomealternativesthathavefoundwidespreadapplicationsinpractice.Theimmediatenextpayoffiseasilycalculatedbymaximizingtheconditionalexpectationofthepayoffundertheposteriorprobabilityoverthestatespace.Thus,greedytechniquesmaximizeaconditionalexpec-tation.Inthemuseumtourguideproject,suchanapproachwassuccessfullyemployedtopreventtherobotfromfallingdownstaircases.Similartechniqueshavebeensuccessfullybroughttobearforactiveenviron-mentexplorationwithteamsofrobots[9],usingpayofffunctionsthatmeasuretheresidualuncertaintyinthemap.Non-greedilyoptimizingrobotcontrol—overmultipletimesteps—remainsachallengingcomputationalproblem.Thisisbecausetherobothastoconsidermultiplecontingenciesduringplanning,payingtributetotheuncertaintyintheworld.Worseso,thenumberofcontingenciesmayincreaseexponentiallywiththeplanninghorizon,whichmakesforamostchallengingplanningproblem[10].Nevertheless,recentresearchhasledtoaurryofapproximatealgorithmsthatarecomputationallyefcient.Thecoastalnavigationalgorithmdescribedin[8]condensestheposteriorbelieftotwoquantities:themostlikelystate,andtheentropyoftheposterior.Thisstatespacerepresentationisexponentiallymorecompactthanthespaceofallposteriordistributions.Itcaptures,however,stillthedegreeofuncertaintyin6 therobot'sposterior.Planningwiththiscondensedstatespacehasledtoscalableroboticplanningsystemsthatcancopewithuncertainty.Forexample,inamobilerobotimplementationreportedin[8],thistechniquehasbeenfoundtonavigaterobotsclosedtoknownlandmarks,inordertominimizethedangerofgettinglost—eventhoughthismightincreasetheoverallpathlength.Experimentally,coastalnavigationwasshowntobesuperiortomotionplannersthatdonotregarduncertaintyintheplanningprocess,indenselypopulatedenvironments.Thisandmanyotherexamplesintheliteratureillustratehowacarefulconsiderationofuncertaintyoftenleadstosuperiorcontrolalgorithms,whichexplicitlyconsideruncertaintyinplanningandcontrol.ConclusionThisarticleprovidedabriefintroductionintothevibranteldofprobabilisticrobotics.Thekeyideaofprobabilisticapproachesisacommitmenttoprobabilitydistributionasthebasicrepresentationofinforma-tion.Theyprovidesoundsolutionsfortheintegrationofinaccuratemodelinformationandnoisysensordata.Todate,probabilisticroboticsisoneofthemostrapidlygrowingsubeldofrobotics.Whilemanyresearchchallengesremain,theapproachhasalreadyledtofundamentallymorescalablesolutionstomanyhardroboticsproblems,specicallyintheareaofmobilerobotics.Theyhaveledtodeepmathematicalinsightsintothestructureofroboticsproblemsandsolutions,Andnally,probabilistictechniqueshaveproventheirvalueinpractice.Theyareatthecoreofdozensofsuccessfulroboticsystemstodate.Thisarticlewasnecessarilybrief,andtheinterestedreaderisinvitedtoconsulttherichliteratureonthistopic.Additionalintroductorymaterialcanbefoundattheauthor'sWebsitehttp://www.cs.cmu.edu/thrun.AcknowledgmentTheauthoracknowledgestheinvaluablecontributionsbyvariousthemembersofCMU'sRobotlearninglab.GenerousnancialsupportbyDARPA(TMR,MARS,CoABSandMICAprograms)andNSF(ITR,Robotics,andCAREERprograms)isalsogratefullyacknowledged.References[1]G.Dissanayake,P.Newman,S.Clark,H.F.Durrant-Whyte,andM.Csorba.Anexperimentalandtheoreticalinvestigationintosimultaneouslocalisationandmapbuilding(SLAM).InP.CorkeandJ.Trevelyan,editors,LectureNotesinControlandInformationSciences:ExperimentalRoboticsVI,pages265–274,London,2000.SpringerVerlag.[2]A.Doucet,J.F.G.deFreitas,andN.J.Gordon,editors.SequentialMonteCarloMethodsInPractice.SpringerVerlag,NewYork,2001.[3]H.F.Durrant-Whyte.Autonomousguidedvehicleforcargohandlingapplications.InternationalJour-nalofRoboticsResearch,15(5),1996.[4]Y.Liu,R.Emery,D.Chakrabarti,W.Burgard,andS.Thrun.UsingEMtolearn3Dmodelswithmobilerobots.InProceedingsoftheInternationalConferenceonMachineLearning(ICML),2001.[5]P.Maybeck.StochasticModels,Estimation,andControl,Volume1.AcademicPress,Inc,1979.7 [6]G.J.McLachlanandT.Krishnan.TheEMAlgorithmandExtensions.WileySeriesinProbabilityandStatistics,NewYork,1997.[7]I.Nourbakhsh,J.Bobenage,S.Grange,R.Lutz,R.Meyer,andA.Soto.Anaffectivemobilerobotwithafull-timejob.ArticialIntelligence,114(1–2):95–124,1999.[8]N.RoyandS.Thrun.Coastalnavigationwithmobilerobot.InProceedingsofConferenceonNeuralInformationProcessingSystems(NIPS),1999.toappear.[9]R.Simmons,D.Apfelbaum,W.Burgard,M.Fox,D.anMoors,S.Thrun,andH.Younes.Coordinationformulti-robotexplorationandmapping.InProceedingsoftheAAAINationalConferenceonArticialIntelligence,Austin,TX,2000.AAAI.[10]E.Sondik.TheOptimalControlofPartiallyObservableMarkovProcesses.PhDthesis,StanfordUniversity,1971.[11]S.Thrun,M.Beetz,M.Bennewitz,W.Burgard,A.B.Cremers,F.Dellaert,D.Fox,D.H¨ahnel,C.Rosenberg,N.Roy,J.Schulte,andD.Schulz.Probabilisticalgorithmsandtheinteractivemu-seumtour-guiderobotminerva.InternationalJournalofRoboticsResearch,19(11):972–999,2000.[12]S.Williams,G.Dissanayake,andH.F.Durrant-Whyte.Towardsterrain-aidednavigationforunderwa-terrobotics.AdvancedRobotics,15(5),2001.8