Walter Matthew Antone Andrew Correa Randall Davis Luke Fletcher Emilio Frazzoli Jim Glass Jonathan P How Albert S Huang Jeong hwan Jeon Sertac Karaman Brandon Luders Nicholas Roy Tara Sainath Abstract One longstanding challenge in robotics is the r ID: 20682
Download Pdf The PPT/PDF document "A VoiceCommandable Robotic Forklift Work..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
limitedpallettypesandenvironmentclasses.Incontrast,ourvehicleisdesignedtooperateinthedynamic,unstructured,andhuman-occupiedfacilitiesthataretypicalofthemilitarysupplychain,andtohandlecargopalletswithdifferinggeometry,appearance,andloads.Moregenerally,substantialattentionhasfocusedondevel-opingmobilemanipulatorscapableofoperatingindynamicenvironments.Muchofthisworkhasfocusedontheprob-lemsofplanningandcontrol[8]–[10],whicharenon-trivialforarobotwithmanydegreesoffreedomandactuatorsexertingconsiderableforceandtorque.Othershavestudiedsensinginthecontextofobjectmanipulationusingtactilefeedback[11]orcomputervision[12]tolearngrasps[13]andtomanipulatearticulatedobjects[14].Researchershavedevelopedremotely-controlledmobilemanipulators[15]andgroundrobots[16],[17],requiringthattheuserteleoperatethevehicle,afundamentaldifferencefromourwork,whicheschewsteleoperationinfavorofatask-levelhuman-robotinterface[18].II.DESIGNCONSIDERATIONSAnumberofelementsofoursystem'sdesignaredictatedbytheperformancerequirementsofourtask.Theforkliftmustoperateoutdoorsongravelandpackedearth.Thus,wechosetoadoptanon-planarterrainrep-resentationandafull6-DOFmodelofchassisdynamics.WeusedanIMUtocharacterizetheresponseoftheforklifttoacceleration,braking,andturningalongpathsofvaryingcurvaturewhenunloadedandloadedwithvariousmasses.Theforkliftrequiresfull-surroundsensingforobstacleavoidance.Wechosetobasetheforklift'sperceptiononlidarsensors,duetotheirrobustnessandhighrefreshrate.Weaddedcamerastoprovidesituationalawarenesstoa(possiblyremote)humansupervisor,andtosupportfuturevision-basedobjectrecognition.Wedevelopedanautomaticmulti-sensorcalibrationmethodtobringalllidarandcameradataintoacommoncoordinateframe.Theforkliftrequiresaneffectivecommandmechanismusablebymilitarypersonnelafterminimaltraining.Wechosetodevelopaninterfacebasedonspokencommandsandstylusgesturesmadeonahandheldtabletcomputer.Commandsinclude:summoningtheforklifttoaspeciedarea;pickingupapalletbycirclingitsimageonthetablet;andplacingapalletatalocationindicatedbycircling.Toenablethesystemtoaccomplishcomplexpallet-handlingtasks,wecurrentlyrequirethehumansupervisortobreakdowncomplexcommandsintohigh-levelsubtasks(i.e.,notteleoperation).Forexample,tounloadatruck,thesuper-visormustsummontheforklifttothetruck,indicateapallettopickup,summontheforklifttothepallet'sdestination,andindicatetotheforkliftwhereonthegroundthepalletmustbeplaced.Thisproceduremustberepeatedforeachpalletonthattruck.Wecallthistaskbreakdown“hierarchicaltask-levelautonomy.”Ourultimategoalistoreducethesupervisorburdenbymakingtherobotcapableofcarryingouthigher-leveldirectives(e.g.,completelyunloadingatruckpursuanttoasingledirective).Werecognizethatanearlydeploymentoftherobotwouldnotmatchthecapabilityofanexperthumanoperator.Ourmentalmodelfortherobotisa“rookieoperator,”whichbehavescautiouslyandasksforhelpwithdifcultmaneu-vers.Thus,whenevertheplannercannotidentifyasafeactiontowardthedesiredgoal,therobotcansignalthatitis“stuck”andrequestsupervisorassistance.Whentherobotisstuck,thehumansupervisorcaneitherusetheremoteinterfacetoabandonthecurrenttask,oranynearbyhumancanclimbintotherobot'scabandguideitthroughthedifcultyviaordinarymannedoperation.Thetechnicalchallengeshereincludedesigningthedrive-by-wiresystemtoseamlesslytransitionbetweenunmannedandmannedoperation,anddesigningtheplannertohandlemixed-initiativeoperation.Humansinmilitarywarehousesettingsexpecthumanforkliftoperatorstostopwheneverawarningisshouted.Wehaveincorporatedacontinuously-running“shoutedwarningdetector”intotheforklift,whichpausesoperationwheneverashoutedstopcommandisdetected,andstayspauseduntilgivenanexplicitgo-aheadtocontinue.Humanshavealifetimeofpriorexperiencewithoneanother,andhavebuiltuppowerfulpredictivemodelsofhowotherhumanswillbehaveinalmostanyordinarysituation[19].Wehavenosuchpriormodelsforrobots,whichinourviewispartofthereasonwhyhumansareuncomfortablearoundrobots:wedonothaveagoodideaofwhattheywilldonext.Asignicantdesignpriorityisthusthedevelopmentofsubsystemstosupportsocialacceptanceoftherobot.Weaddedan“annunciationsubsystem”thatusesvisibleandaudiblecuestoannouncethenear-termintentionoftherobottoanyhumanbystanders.Therobotalsousesthissystemtoconveyitsowninternalstate,suchastheperceivednumberandlocationofanybystanders.III.MOBILEMANIPULATIONPLATFORMOurrobotisbaseduponaToyota8FGU-15mannedforklift(Fig.1),arearwheel-steered,liquid-propanefueledlifttruckwithagrossvehicleweightof2700kgandaliftcapacityof1350kg.WechosetheToyotavehicleforitsrelativelysmallsizeandthepresenceofelectroniccontrolofsomeofthevehicle'smobilityandmastdegreesoffreedom,whichfacilitatedourdrive-by-wiremodications.Wedevisedasetofelectrically-actuatedmechanismsinvolvingservomotorstobringthesteeringcolumn,brakepedal,andparkingbrakeundercomputercontrol.Asolenoidservestoactivatethereleaselatchtodisengagetheparkingbrake.(Puttingtheparkingbrakeundercomputercontrolisessential,sinceOSHAregulations[20]dictatethatthepark-ingbrakebeengagedwhenevertheoperatorexitsthecabin;inoursetting,therobotsetstheparkingbrakewheneveritrelinquishescontroltoahumanoperator.)Theinterpositionofcircuitryintotheoriginalforkliftwiringpermitscontrolofthethrottle,mast,carriage,andtinedegreesoffreedom,andenablesdetectionofanycontrolactionsmadebyahumanoperator.Thisdetectioncapabilityisessentialbothforsafetyandforseamlesshuman-robothandoff. Fig.3.Anotionalmilitarywarehouselayout.anddiagnosticmodules(Fig.2).An“operator-in-the-cabin”detector,buttonsonthesupervisortablet,andaradio-controlledkillswitch(E-stop)providelocalandremotesystem-pauseandsystem-stopcapabilities.Thetabletalsomaintainsa10Hz“heartbeat”connectionwiththeforklift,whichpausesafterseveralmissedheartbeats.F.RobotSystemIntegrityThearchitectureoftheforkliftisbasedonahierarchyofincreasinglycomplexandcapablelayers.Atthelowestlevel,kill-switchwiringdisablesignitiononcommand.Next,aprogrammablelogiccontroller(PLC)usesasimplerelayladderprogramtoenablethedrive-by-wirecircuitryandtheactuatormotorcontrollersfromtheirdefault(braking)state.ThePLCrequiresaregularheartbeatsignalfromthehigher-levelsoftwareandmatchingsignalsfromtheactuatormodulestoenabledrive-by-wirecontrol.Higherstill,thesoftwarearchitectureisdesignedwithredundantsafetychecksdistributedacrossseveralnetworkedcomputersthat,upondetectingafault,causethebottoentera“paused”state.Thesesafetychecksincludeanumberofinter-processheartbeatmessages,suchasa50Hzautonomystatemessagewithoutwhichallactuationprocessesdefaulttoastopped(braking)state.Additionalprocessesmonitorsensorandinter-processcommunicationtimingand,upondetectinganyfault,bringtherobottoasafestoppedstate.IV.MINIMALLY-PREPAREDENVIRONMENTSTheforkliftoperatesinoutdoorenvironmentswithmini-malphysicalpreparation.Specically,weassumeonlythatthewarehouseconsistsofadjoiningregions.WecapturetheapproximateGPSperimeterofeachregionanditsmilitarydesignation(e.g.,“receiving,”“storage,”and“issuing”),aswellasapairof“summoningpoints”thatspecifyaroughlocationandorientationforpointsofinterestwithineachregionandneareachpalletbayinstorage(Fig.3).WealsospecifyGPSwaypointsalongasimpleroadnetworkconnectingtheregions.ThisdataisprovidedstaticallytotheforkliftaspartofanASCIIcongurationle.ThespeciedGPSlocationsneednotbeprecise;theirpurposeisonlytoprovideroughgoallocationsfortherobottoadoptinresponsetosummoningcommands.Ournaviga-tionmethodology[21]emphasizeslocalsensinganddead-reckoning.Subsequentmanipulationcommandsareexecutedusingonlylocalsensing,andthushavenorelianceonGPS. (a)Apalletpickupgestureappearsinred. (b)Lidarreturns(red)withintheresultingvolumeofinterest.Fig.4.(a)Thepalletindicationgestureand(b)thelidarreturnsinthevolumeofinterest.Successfulengagementdoesnotrequirethatthegestureenclosetheentirepalletandload.A.SummoningandManipulationCommandsThehumansupervisordirectstheforkliftusingaNokiaN810internettabletthatrecognizesspokencommandsandsketchedgestures[18].OurSUMMITlibrary[24]handlesspeechrecognitionforsummoning.Spokencommandsarecurrentlylimitedtoasmallsetofutterancesdirectingmovement,suchas“Cometoreceiving.”Thesupervisorindicatesatargetpalletformanipulationusingaroughcirclinggesture(Fig.4(a)).Theinterfaceechoeseachgestureasacleaned-upclosedshape,andpublishesa“volumeofinterest”correspondingtotheinterioroftheconeemanatingfromthecameraandhavingthecapturedgestureasitsplanarcrosssection(Fig.4(b)).Thevolumeofinterestneednotcontaintheentirepalletforengagementtosucceed.Asimilargesture,madeonatruckbedoronemptyground,indicatesthelocationofadesiredpalletplacement.Gestureinterpretationisthuscontextdependent.B.ObstacleDetectionObstacledetectionisimplementedusingtheskirtlidars,withanadaptationoftheobstacledetectionalgorithmusedontheDARPAUrbanChallengevehicle[22].Returnsfromalllidarsarecollectedinasmoothly-varyinglocalcoordinateframe[21],clusteredbasedonspatiotemporalconsistency,andpublished(Fig.2).Thelidarsareintentionallytilteddownby5degrees,sothattheywillgeneraterangereturnsfromthegroundwhennoobjectispresent.Theexistenceof“innite”rangedatathenenablesthedetectortoinfer Fig.5.Anapproachingpedestriancausestherobottopause.Lightsskirtingtherobotindicatedistancetoobstacles(green:fartored:close).Verbalannunciatorsandsignageindicatetheinducedpause.environmentalpropertiesfromfailedreturns(e.g.,fromab-sorptivematerial).Theconsequenceofthedownwardorien-tationisashortermaximumrange,around15meters.Sincethevehicle'sspeeddoesnotexceed2m/s,thisstillprovides7-8secondsofsensinghorizonforcollisionavoidance.Torejectfalsepositivesfromtheground(atdistancesgreaterthantheworstcasegroundslope),werequirethatconsistentreturnsbeobservedfrommorethanonelidar.Missinglidarreturnsarelledinatareducedrangetosatisfytheconservativeassumptionthattheyarisefromahuman(assumedtobe30cmwide).Pedestriansafetyiscentraltoourdesignchoices.Thoughlidar-basedpeopledetectorsexist[25]–[27],weoptedtoavoidtheriskofmisclassicationbytreatingallobjectsofsuitablesizeaspotentialhumans.Therobotproceedsslowlyaroundstationaryobjects.Pedestrianswhoapproachtoocloselycausetherobottopause(Fig.5),indicatingassuchtothepedestrian.C.Lidar-BasedServoingPickingupapalletrequiresthattheforkliftaccuratelyinsertitstinesintothepalletslots,achallengefora2700kgforkliftwhenthepallet'sposeandinsertlocationsarenotknownaprioriandwhenpalletstructureandgeometryvary.Additionally,whenthepalletistobepickedupfromorplacedonatruckbed,theforkliftmustaccountfortheunknownposeofthetruck(distancefromtheforklift,orientation,andheight),onwhichthepalletmayberecessed.Complicatingtheserequirementsisthefactthatwehaveonlycoarseextrinsiccalibrationforthemastlidarsduetotheunobservablecomplianceofthemast,carriage,andtines.Weaddressthesechallengeswithaclosed-loopperceptionandcontrolstrategythatregulatesthepositionandorientationofthetinesbaseddirectlyonlidarobservationsofthepalletandtruckbed.V.OPERATIONINCLOSEPROXIMITYTOPEOPLETherobotemploysanumberofmechanismsintendedtoincreaseoverallsafety.Bydesign,allpotentialrobottrajectoriesconcludewiththerobotcomingtoacompletestop(eventhoughthislegofthetrajectorymaynotalwaysbeexecuted,particularlyifanothertrajectoryischosen).Consequentlytherobotmovesmoreslowlywhenclosetoobstacles(conservativelyassumedtobepeople).Therobotalsosignalsitsinternalstateandintentions,inanattempttomakepeoplemoreacceptingofitspresenceandmoreeasilyabletopredictitsbehavior[18].A.AnnunciationofIntentTheLEDsignagedisplaysshorttextmessagesdescribingcurrentstate(e.g.,“paused”or“fault”)andanyimminentactions(e.g.,forwardmotionormastlifting).Themarqueelightsencodeforkliftstateascolors,andimminentmotionasmovingpatterns.Open-sourcesoftwareconvertsthetextmessagestospokenEnglishforbroadcastthroughtheaudiospeakers.Textannouncementsarealsoexportedtothetabletfordisplaytothesupervisor.B.AwarenessDisplayTheforkliftalsousesitsannunciatorstoinformbystandersthatitisawareoftheirpresence.Wheneverahumanisdetectedinthevicinity,themarqueelights,consistingofstringsofindividuallyaddressableLEDs,displayabrightregionorientedinthedirectionofthedetection(Fig.5).Iftheestimatedmotiontrackisconvergingwiththeforklift,theLEDsignageandspeakersannounce“Humanapproaching.”C.AutonomyHandoffWhenahumancloselyapproachestherobot,itpausesforsafety.(Aspeechrecognizerrunsontheforklifttoenabledetectionofshoutedphrasessuchas“Forkliftstopmoving,”whichalsocausetherobottopause.)Whenahuman(presumablyahumanoperator)entersthecabinandsitsdown,therobotdetectshis/herpresenceinthecabinthroughthereportofaseat-occupancysensor,oranyuncommandedpressofthebrakepedal,turnofthesteeringwheel,ortouchofthemastortransmissionlevers.Inthisevent,therobotrevertstobehavingasamannedforklift,cedingautonomy.VI.DEPLOYMENTANDRESULTSWedeployedoursystemintwotestenvironmentscon-guredasmilitarySupplySupportActivities(SSAs),inthegeneralformshowninFig.3.Theseoutdoorwarehousesincludedreceiving,bulkyard,andissuingareasconnectedbyasimpleroadnetwork.Thebulkyardscontainedanumberofalphanumerically-labeledpalletstoragebays.AnArmystaffsergeant,knowledgeableinmilitarylo-gisticsandanexpertforkliftoperator,actedastherobotsupervisor.Inabrieftrainingsession,shelearnedhowtoprovidespeechandgestureinputtothetabletcomputer,anduseitsPAUSEandRUNbuttons. Fig.6.(top)Duringatestingsession,therobotnavigatesfromastationarypositionaroundrowsofconesandpalletizedcargo.(bottom)Therobotroundstherstrowofcones,identifyingatreeoffeasiblepathsandexecutinganobstacle-freetrajectory(magenta)throughtheperceivedobstacleeld(red,withblackpenaltyregions)toatargetpose(green).A.PathPlanningandObstacleAvoidanceThemostbasicmobilityrequirementfortherobotistomovesafelyfromastartingposetoitsdestinationpose.Thepathplanningsubsystem(Fig.2)adaptsthenavigationframeworkdevelopedatMITfortheDARPAUrbanChal-lengevehicle[22],[28].Thenavigatoridentiesawaypointpaththroughthewarehouseroutenetwork.Aclosed-looppredictionmodelincorporatespurepursuitsteeringcon-trol[29]andPIspeedcontrol.Thispredictionmodelmayrepresentgeneralclassesofautonomousvehicles;inthiscase,wedevelopedaspecicmodelforthedynamicsofourforkliftplatform.Themotionplannerusesthepredic-tionmodeltogrowrapidly-exploringrandomtrees(RRT)ofdynamicallyfeasibleandsafetrajectoriestowardthesewaypoints[28].Thecontrollerexecutesaselectedtrajectoryprogressingtowardthedestinationwaypoint(Fig.6).Thesetrajectoriesareselectedinreal-timetominimizeanappro-priateobjectivefunction,andaresafebyconstruction.Theclosed-loopnatureofthealgorithm[30]andtheoccasionaluseofre-planningmitigateanydisturbancesormodelingerrorsthatmaybepresent.Akeyperformancemetricforthenavigationsubsystemistheabilitytocloselymatchthepredictedtrajectorywiththeactualpath,assignicantdeviationsmaycausetheactualpathtobecomeinfeasible(e.g.,duetoobstacles).Duringnormaloperationinseveraloutdoorexperiments,werecorded97differentcomplexpathsofvaryinglengths(6mto90m)andcurvatures.Foreach,wemeasuredtheaverageandmaximumerrorbetweenthepredictedandactualvehicleposeoverthelengthofthepath.Inallcases,theaveragepredictionerrordidnotexceed12cm,whilethemaximum Fig.7.Outputofthepalletestimationalgorithmduringengagementofapalletonatruckbed.Thegureshowsapositivedetectionandthecorrespondingestimateforthepallet'sposeandslotgeometrybaseduponthelidarreturnsfortheregionofinterest(inpink).Insetsatlowerrightshowscanswithintheinterestvolumethatthesystemcorrectlyclassiedasnotarisingfromapalletface;thesescanswereofthetruckbedandundercarriage.predictionerrordidnotexceed35cm.Wealsotestedtherobot'sabilitytoaccomplishcom-mandedmotiontoavarietyofdestinationposesinthevicinityofobstaclesofvaryingsizes.Whentheroutewasfeasible,theforkliftidentiedandexecutedacollision-freeroutetothegoal.Forexample,Fig.6showsanobstacle-freetrajectorythroughaworkingshuttleparkinglot,includingpallets,trafccones,pedestrians,andvehicles.Someactuallyfeasiblepathswereerroneouslyclassiedasinfeasible,duetoa25cmsafetybuffersurroundingeachdetectedobstacle.Wealsotestedtherobot'sbehaviorwhenobstructedbyapedestrian(amannequin),inwhichcasetherobotstopsandwaitsforthepedestriantomoveoutoftheway.B.PalletEngagement:EstimationandManipulationAfundamentalcapabilityofoursystemisitsabilitytoengagepallets,bothfromthegroundandfromtruckbeds.Withuneventerrainsupportingthepalletandvehicle,unknowntruckgeometry,variableunknownpalletgeometryandstructure,andvariationinload,successfullylocalizingandengagingthepalletisachallengingproblem.Giventhevolumeofinterestarisingfromthesupervisor'sgesture(Fig.4(b)),therobotmustdetecttheindicatedpalletandlocatetheinsertionslotsonthepalletface.Theestimationphaseproceedsastherobotscansthevolumeofinterestwiththetine-mountedlidarsbyvaryingmasttiltandheight.Theresultisasetofplanarscans(Fig.7).Thesystemthensearcheswithinindividualscanstoidentifycandidatereturnsfromthepalletface.Weuseafastedgedetectionstrategythatsegmentsascanintoreturnsthatformedgesegments.Thedetectionalgorithmthenclassiessetsoftheseweak“features”astowhethertheycorrespondtoapallet,baseduponaroughpriorongeneralpalletstructure.Whenapalletisdetected,themoduleestimatesitspose,width,depth,andslotgeometry.Asimilarmoduleusesscansfromtheverticallidarstodetectthetruckbedandestimate itsposerelativetotherobot.Afterdetectingthetargetpalletandestimatingitspositionandorientation,thevehicleproceedswiththemanipula-tionphaseofpalletengagement.Inordertoaccountforunavoidabledriftinthevehicle'spositionrelativetothepallet,thesystemreacquiresthepalletseveraltimesduringitsapproach.Finally,thevehiclestopsabout2mfromthepallet,reacquirestheslots,andservosthetinesintotheslotsusingthelteredlidarscans.Wetestedpalletengagementinagravellotwithpalletsofdifferenttypesandwithdifferentloads.Usingthetabletinterface,wecommandedtheforklifttopickuppalletizedcargooffofthegroundaswellasatruckbedfromavarietyofinitialdistancesandorientations.Detectiontypicallysuc-ceedswhentheforkliftstartsnomorethan7.5mfromthepallet,andtheangleofthepalletfacenormalisnomorethan30offoftheforklift'sinitialheading.In69trialsinwhichdetectionsucceeded,engagingpalletsofvarioustypesfromthegroundandatruckbedsucceeded64times;the5engagementfailuresoccurredwhentheforklift'sinitiallateraloffsetfromthepalletwasmorethan3meters.C.ShoutedWarningDetectionPreliminarytestingoftheshoutedwarningdetectorwasperformedwithvemalesubjectsinanoutdoorgravellotonafairlywindyday(6m/saveragewindspeed),withwindgustsclearlyaudibleinthearraymicrophones.Sub-jectswereinstructedtoshouteither“Forkliftstopmoving”or“Forkliftstop”undersixdifferentoperatingconditions:idling(reverberantnoise);beeping;revvingengine;movingforward;backingup(andbeeping);andmovingwithanothertrucknearbybackingup(andbeeping).Eachsubjectshoutedcommandsundereachcondition(typicallyatincreasingvolume)untilsuccessfuldetectionoccurred.Allsubjectswereultimatelysuccessfulundereachcondition;theworstcaserequiredfourattemptsfromonesubjectduringtheinitialidlingcondition.Includingrepetitions,atotalof36shoutedcommandsweremade,ofwhich26weredetectedsuccessfullyonthersttry.Themostdifcultoperatingconditionoccurredwhentheenginewasbeingrevved(lowSNR),resultinginvemisseddetectionsandtheonlytwofalsepositives.Theothertwomisseddetectionsoccurredwhenthesecondarytruckwasactive.D.End-to-EndOperationTherobotwassuccessfullydemonstratedoutdoorsovertwodaysinJune2009atFortBelvoirinVirginia.UndervoiceandgesturecommandofaU.S.ArmyStaffSergeant,theforkliftunloadedpalletsfromaatbedtruckinthereceivingarea,drovetoabulkyardlocationspeciedver-ballybythesupervisor,andplacedthepalletontheground.Therobot,commandedbythesupervisor'sstylusgestureandverbally-specieddestination,retrievedanotherindicatedpalletfromthegroundandplaceditonaatbedtruckintheissuingarea.Duringoperation,therobotwasinterruptedbyshouted“Stop”commands,pedestrians(mannequins)wereplacedinitspath,andobserversstoodandwalkednearby.Wealsodirectedtherobottoperformimpossibletasks,suchasliftingapalletwhoseinsertswerephysicallyandvisuallyobscuredbyfallencargo.Inthiscase,theforkliftpausedandrequestedsupervisorassistance.Ingeneral,suchassistancecancomeinthreeforms:thesupervisorcancommandtherobottoabandonthetask;ahumancanmodifytheworldtomaketherobot'staskfeasible;orahumancanclimbintotheforkliftcabinandoperateitthroughthechallengingtask.(Inthiscase,wemanuallymovedtheobstructionandresumedoperation.)E.LessonsLearnedandFutureWorkWhileourdemonstrationswerejudgedsuccessfulbymil-itaryobservers,theprototypecapabilityiscrude.Inoper-ationalsettings,therequirementthatthesupervisorbreakdowneachcomplextaskintoexplicitsubtasks,andexplicitlyissueacommandforeachsubtask,wouldlikelybecomeburdensome.Weareworkingonincreasingtherobot'sauton-omylevel,forexample,byenablingittoreasonabouthigher-leveltasks.Moreover,ourrobotisnotyetcapableofthesortofmanipulationsexhibitedbyexperthumanoperators(e.g.,liftingtheedgeofapalletwithonetinetorotateorrepositionit,gentlybouncingaloadtosettleitonthetines,shovingoneloadwithanother,etc.).Welearnedanumberofvaluablelessonsfromtestingwithrealmilitaryusers.First,palletindicationgesturesvariedwidelyinshapeandsize.Theresultingconicalregionsometimesincludedextraneousobjects,causingthepalletdetectortofailtolockontothecorrectpallet.Second,peoplewerespontaneouslyaccommodatingoftherobot'slimitations.Forexample,ifaspeechcommandorgesturewasmisunderstood,thesupervisorwouldcancelexecutionandrepeatthecommand;ifashoutwasn'theard,theshouterwouldrepeatitmoreloudly.Thisbehaviorisconsistentwiththewayahumanworkermightinteractwitharelativelyinexperiencednewcomer.Recognitionofshoutedspeechinnoisyenvironmentshasreceivedlittleattentioninthespeechcommunity,andpresentsasignicantchallengetocurrentspeechrecognitiontechnology.Fromauserperspective,itislikelythatausermaynotbeabletorememberspecic“stop”commands,andthattheshouterwillbestressed,especiallyiftheforkliftdoesnotrespondtoaninitialshout.Fromasafetyperspective,itmaybeappropriatefortheforklifttopauseifithearsanyoneshoutinitsgeneralvicinity.Thus,wearecollectingamuchlargercorpusofgeneralshoutedspeech,andaimtodevelopacapabilitytoidentifygeneralshoutedspeech,asaprecursortoidentifyinganyparticularcommand.Inaddition,wearealsoexploringmethodsthatallowthedetectionmoduletoadapttonewaudioenvironmentsthroughfeedbackfromusers.RatherthanrequireaGPS-delineatedregionmaptobesuppliedpriortooperation,wearedevelopingtherobot'sabilitytounderstandanarrated“guidedtour”oftheworkspaceasaninitializationstep.Duringthetour,ahumanwoulddrivetheforkliftthroughtheworkspaceandspeakthename,type,orpurposeofeachenvironmentalregionasit