Collaborative Teleoperation Using Networked Spatial Dynamic Voting KEN GOLDBERG SENIOR - PDF document

CollaborativeTeleoperationUsingNetworkedSpatialDynamicVoting KENGOLDBERG,SENIORMEMBER,IEEE,DEZHENSONG,STUDENTMEMBER,IEEEANTHONYLEVANDOWSKI,STUDENTMEMBER,IEEEInvitedPaperWedescribeanetworkedteleoperationsystemthatallowsgroupsofparticipantstocollaborativelyexploreliveremoteenvironments. Fig.1.TheSDVinterfaceasviewedbyeachuser.Intheremoteenvironment,thetele-actortakesimageswithadigitalcamerawhicharetransmittedoverthenetworkanddisplayedtoallparticipantswitharelevantquestion.Withamouseclick,eachuserplacesacolor-codedmarker(a“votel”orvotingelement)ontheimage.Usersviewthepositionofallvotelsandcanchangetheirvotelpositionsbasedonthegroup’sresponse.Votelpositionsarethenprocessedtoidentifya“consensusregion”inthevotingimagethatissentbacktothetele-actor.Inthismanner,thegroupcollaboratestoguidetheactionsofthetele-actor.withvideocameras.Prosthetichandswerealsoappliedtoteleoperation[62].Morerecently,teleoperationisbeingcon-sideredformedicaldiagnosis[5],manufacturing[22],andmicromanipulation[59].Sheridan[60]providesanexcellentreviewoftheextensiveliteratureonteleoperationandtele-Networkedrobots,controllableovernetworkssuchastheInternet,areanactiveresearcharea.Inadditiontothechal-lengesassociatedwithtimedelay,supervisorycontrol,andstability,on-linerobotsmustbedesignedtobeoperatedbynonspecialiststhroughintuitiveuserinterfacesandtobeac-cessible24hoursaday.TheMercuryProjectwasthefirstpubliclyaccessiblenetworkedrobot[26],[27];itwenton-lineinAugust1994.Workingindependently,ateamledbyK.TaylorandJ.TrevelyanattheUniversityofWesternAustralia,Crawley,Australia,demonstratedaremotelycontrolledsix-axistele-robotinSeptember1994[15],[33].TherearenowdozensofInternetrobotson-line,abookfromMITPress[29],andanIEEETechnicalCommitteeonInternetandOnlineRobots.See[32],[34]–[36],[38],[40],[48],[50],[58]forexamplesofrecentprojects.Tanieetal.[12]proposedausefultaxonomyfortele-operationsystems:SOSR,single-operatormultiple-robot(SOMR),andmultiple-operatormultiple-robot(MOMR).MostnetworkedrobotsareSOSR,wherecontrolislimitedtoonehumanoperatoratatime.Tanieetal.analyzedanMOMRsystemwhereeachoperatorcontrolsonerobotarmandtherobotarmshaveoverlappingworkspaces.Theyshowthatpredictivedisplaysandscaledratecontrolareeffectiveinreducingpick-and-placetaskcompletiontimesthatrequirecooperationfrommultiplearms[12].InanMOMRprojectbyElhajjetal.[17],[18],twore-motehumanoperatorscollaboratetoachieveasharedgoalsuchasmaintainingagivenforceonanobjectheldatoneendbyamobilerobotandbyamultijointedrobotattheother.Theoperators,distantfromtherobotsandfromeachother,eachcontroladifferentrobotviaforcefeedbackde-vicesconnectedtotheInternet.Theauthorsshowboththeo-reticallyandexperimentallythatevent-basedcontrolallowsthesystemtomaintainstablesynchronizationbetweenoper-atorsdespitevariabletimelagontheInternet.MOMRmodelsarealsorelevanttoon-linecollaborativegamessuchasQuakeTheSimsOnline,whereplayersremotelycontrolindividualavatarsinasharedvirtualenvironment.InSOMRsystems,onehumanoperatorcontrolsmultiplerobots.Avariantisno-operatormultiple-robot(NOMR)sys-tems,sometimescalledcollaborativeorcooperativerobotics,wheregroupsofautonomousrobotsinteracttosolveanob-jective[2].Recentresultsarereportedin[9],[16],[54],andandAnumberofSOSRsystemshavebeendesignedtofacili-tateremoteinteraction.PaulosandCanny’sPersonalRovingPresence(ProP)telerobots,builtonblimporwheeledplat-forms,weredesignedtofacilitateremotesocialinteractionwithasingleremoteoperator[51],[52].Hamel[31]studiedhownetworkedrobotscanbeusefulinhazardousenviron-ments.Fongetal.studySOSRsystemswherecollaborationoccursbetweenasingleoperatorandamobilerobotthatistreatedasapeertothehumanandmodeledasanoisyinfor-mationsource[20].RelatedexamplesofSOSR“cobots”areanalyzedin[1],[7],[41],[44],[45],and[61].Oneprecedentforanon-lineMOSRsystemisdescribedinMcDonaldetal.[46].Forwastecleanup,severalusersas-sistremotelyusingpoint-and-direct(PAD)commands[10].Userspointtocleanuplocationsinasharedimageandarobotexcavateseachlocationinturn.InthisInternet-basedMOSRsystem,collaborationisserialbutpipelined,withoverlap-pingplanandexecutionphases.Theauthorsdemonstratethatsuchcollaborationimprovesoverallexecutiontime,butdonotaddressconflictresolutionbetweenusers.Pirjanianstudieshowreliablerobotbehaviorcanbeproducedfromanensembleofindependentprocessors[53].Drawingonresearchinfault-tolerantsoftware[39],Pirjanianconsiderssystemswithanumberofhomogenousprocessorssharingacommonobjective.Heconsidersavarietyofvotingschemesandshowsthatfault-tolerantbehaviorfusioncanbeoptimizedusingpluralityvoting[8],butdoesnotconsiderspatialvotingmodelssuchasours.In[24],wepresentanInternet-basedMOSRsystemthataveragedmultiplevectorinputstocontrolthepositionofanindustrialrobotarm.Wereportexperimentswithmazefol-lowingthatsuggestedthatgroupsofhumansmayperformbetterthanindividualsinthepresenceofnoiseduetocentrallimiteffects.In[25],weusedfiniteautomatatomodelcollaboratingusersinaMOSRsystemsuchasCinematrix,acommercialetal.:COLLABORATIVETELEOPERATIONUSINGNETWORKEDSPATIALDYNAMICVOTING431 Fig.2.Systemarchitecture.ParticipantsontheInternetviewandvotingonaseriesofvotingimages.Thehumantele-actor,withhead-mountedwirelessaudio/videolink,movesthroughtheremoteenvironmentinresponse.The“localdirector”facilitatesinteractionbypostingtextualqueries.system[11]thatallowslargeaudiencestointeractinathe-aterusingplasticpaddles.Tomodelsuchsystems,weaver-agedanensembleoffiniteautomataoutputstocomputeasinglestreamofincrementalstepstocontrolthemotionofapointrobotmovingintheplane.Weanalyzedsystemperfor-mancewithauniformensembleofwell-behaveddetermin-isticsourcesandthenmodeledmalfunctioningsourcesthatgosilentorgenerateinvertedcontrolsignals.Wefoundthatperformanceissurprisinglyrobustevenwhenasizablefrac-tionofsourcesmalfunction.Outsideofrobotics,thenotionofMOSRisrelatedtoaverybroadrangeofgroupactivitiesincludingsocialpsy-chology,voting,economics,marketpricing,trafficflows,etc.TheAssociationforComputingMachineryorganizesannualconferencesoncomputer-supportedcollaborativelearningandcomputer-supportedcooperativework.Surveysofre-searchinthisbroadercontextcanbefoundin[3],[19],[21],[28],[37],[47],and[55].Wenotethattheconceptofhuman-mountedcameraswithnetworkconnectionsisnotnovel:thereisextensiveliteratureon“wearablecomputer”systems[42],[43].Thefocusofourresearchisoncollaborativecontrol.Apreliminaryreportonthetele-actorsystemappearedin[30].III.SThetele-actorsystemarchitectureisillustratedinFig.2.Asthetele-actor(seeFig.3)movesthroughtheenvironment,cameraimagesaresenttothetele-actorserverfordistribu-tiontousers,whorespondfromtheirInternetbrowsers.Uservotingresponsesarecollectedatthetele-actorserver,whichupdatesJavaappletsforallusersandforthetele-actorinthefield.Thetele-actorcarriesalaptopwhichcommunicatestotheInternetusingthe2.4-GHz802.11bwirelessprotocol.Acamerapersonwithasecondcameraprovidesthird-personperspectivesasneeded.Usingthisarchitecture,theusers,thetele-actorserver,thelocaldirector,thecameraperson,andthetele-actorcommunicateviatheInternet. Fig.3.Thehumantele-actortransmitsimagesfromtheremoteenvironmentusingthehelmet-mountedvideocameraandrespondstouservotes.HelmetdesignbyE.Paulos,C.Myers,andM.Fogarty.(PhotobyB.Nagel.)IV.SDVUNTERFACEWehavedevelopedanewgraphicalinterfacetofacilitateinteractionandcollaborationamongremoteusers.Fig.1il-lustratestheSDVinterfacethatisdisplayedonthebrowserPROCEEDINGSOFTHEIEEE,VOL.91,NO.3,MARCH2003 Fig.4.Navigationquery.Participantsindicatewheretheywantthetele-actortogo. Fig.5.Pointquery.Participantspointoutaregionofinterestinthevotingimage.ofallactivevoters.Usersregisteron-linetoparticipatebyselectingavotelcolorandsubmittingtheire-mailaddresstothetele-actorserver,whichstoresthisinformationinourdatabaseandsendsbackapasswordviae-mail.TheserveralsomaintainsatutorialandaFAQsectiontofamiliarizenewuserswithhowthesystemworks.UsingtheSDVinterface,votersparticipateinaseriesof30-to60-svotingimages.Eachvotingimageisasingleimagewithatextualquestion.IntheexamplefromFig.1,thetele-actorisvisitingabiologylab.Votersclickontheirscreenstopositiontheirvotels.UsingtheHTTPprotocol,thesepositionsaresentbacktothetele-actorserverandappearinanupdatedvotingimagesenttoallvotersevery3–5s.Inthisway,voterscanchangetheirvotes.Whenthevotingcycleiscompleted,SDVanalysisalgorithmsanalyzethevotingpatterntodetermineaconsensuscommandthatissenttothetele-actor.TheSDVinterfacediffersfrommultiple-choicepollingbecauseitallowsspatiallyandtemporallycontinuousinputs.Tofacilitateusertrainingandasynchronoustesting,thetele-actorsystemhastwomodes.Intheoff-linemode,votingimagesaredrawnfromaprestoredlibrary.Intheon-linemode,votingimagesarecapturedlivebythetele-actor.Bothoff-lineandon-linemodeshavepotentialforcollaborativeeducation,testing,andtraining.Inthispaper,wefocusontheon-linemode.Figs.4–7illustratefourtypesofSDVqueriesandtheirassociatedbranchingstructures.Ineachcase,thepositionofthemajorityofvotelsdecidestheoutcome.Wetriedincludingalivevideobroadcastingstreambutfoundthatduetobandwidthlimitations,theresolutionandframerateisunacceptableforlow-latencyapplications.Standardvideobroadcastingsoftwarerequiresabout20sofbufferedvideodataforcompression,whichintroducesunacceptabledelaysforlivevisits.WearehopingthiscanbereducedinthefuturewithfasternetworkssuchasInternet2.V.HARDWAREANDOFTWAREThetele-actorWebserverisanAMDK7950-MHzPCwith1.2-GBmemoryconnectedtoa100-Mb/sT3line.TheVideoserverisalsoanAMDK7950-MHzPCwith512-MBmemoryconnectedtoa100-Mb/sT3line.ThelocalbasestationcouldbeanymachineontheInternetequippedwithJava-enabledWebbrowsers.Theprimarytele-actoriscar-ryinga600-MHzSonypicture-booklaptopwith128-MBmemoryconnectedtoa11-Mb/s802.11bwirelessLANattheremotesite.IthasaUSBvideocard,whichcapturesvideoat320 240resolution.ThecamerapersonhasaPentiumIII750-MHzSonyVaiolaptopwith256-MBmemorywithsim-ilarUSBvideocapturedevice.Thelaptopsdirecttheirvideodisplaystohand-mountedTVstoprovideupdatesonvotingpatterns.Fig.2showsthattheprimarytele-actorhasaCanoncameramountedonherhelmet.Fig.8showsthatthecamer-apersonhasaSonycamcorderwithnightvisioncapability,whichprovidesveryhigh-qualityimageandvideostream.Bothofthemareequippedwithashutterlikedevicetoallowthemtocapturethepreciousmomentintheliveevent.Customsoftwareincludes:1)theclient-sideSDVbrowserinterfacebasedonJava;2)thetele-actorimagecaptureandcommunicationsoftware;3)thelocalbasestationvotingquestionformulationinterface;and4)thetele-actorserver.Duringtheon-linemode,thelocaldirectorusesaJavaapplettoaddtextualquestionstovotingimages.Duringbothon-lineandoff-linemodes,thetele-actorserverusescustomCandC++applicationstomaintainthedatabaseandcommunicatewiththelocalbasestationandwithallactivevoters.Thetele-actorserverrunsRedhatLinux7.1andtheApacheWebserver1.3.20.TheResinetal.:COLLABORATIVETELEOPERATIONUSINGNETWORKEDSPATIALDYNAMICVOTING433 Fig.6.Opinionquery.Votelpositioncanbeanywherebetweenextremevaluestoindicatedegreeofopinion. Fig.7.Multiple-choicequery.Avariationonthepointquerywithasmallnumberofexplicitoptions.2.0.1Apacheplug-inandSunJDK1.3.1withMysqldata-base3.23.36provideJavaserverpagestohandletheuserregistrationanddatalogging.VI.PROBLEMEFINITIONANDAsillustratedinFig.9,usersexpressresponsesbyclickingonthevotingimagetospatiallyindicateapreferredobjectordirectioninthefieldofview.Asanalternativetosemanticanalysisofthevotingimage,weconsidervotelsasspatialdistributionsandidentifypreferred“consensus”regionsintheimage.Wethenusetheseregionstodefinetwometrics[Online]Available:http://www.caucho.com Fig.8.Hardwareconfigurationforthecameraperson.Thehardwareconfigurationofthetele-actorissimilarbuthasahelmet-mountedcamera. Fig.9.Evolutionofvotingimageasvotelsarrive.forindividualandgroupperformanceintermsofleadershipandcollaboration.A.ProblemDefinition1)VoterInterestFunctions:Considerthe thvotingimage.Theserverreceivesaresponsefromuser intheformofan( )mouseclickonimage attime .Wedefinethecorresponding Eachvotelrepresentsauser’sresponsetothevotingimage.Wemodelsuchresponseswithavoterinterest,adensityfunctionbasedonthebivariatenormaldistribution where isthemeanvectorand isa2 2variancematrix,suchthat where istheareaofthevotingimage.Since isaboundedtwo-dimensionalregion,thevoterinterestfunctionisatrun-catedbivariatenormaldensityfunctionwithmeanat asillustratedinFig.10.PROCEEDINGSOFTHEIEEE,VOL.91,NO.3,MARCH2003 Fig.10.Interestfunctionsandconsensusregions.(a)and(b)showtheinterestfunctionforasinglevoter.(c)and(d)showanensembleinterestdensityfunctionforfivevoters.(e)and(f)illustrateresultingconsensusregions.2)EnsembleInterestFunction:Whenvotingonimage endsatstoppingtime ,thelastvotelreceivedfromeachof activevotersdetermines ,asetof votels.Wedefineensembleinterestfunctionforvotingimage asthenor-malizedsumofthesevoterinterestfunctions 3)ConsensusRegions:Wecanextractspatialregionsfromtheensembleinterestfunctionasfollows.Let bethemaximumoftheensembleinterestdensityfunction,andlet besomevaluebetween0and1.Ahorizontalplaneatheight definesanisodensitycontourintheensembleinterestfunctionthatdefinesasetofofoneormoreclosedsubsetsofthevotingimage etal.:COLLABORATIVETELEOPERATIONUSINGNETWORKEDSPATIALDYNAMICVOTING435 Table1SDVAnalysisofVotingImageFromFig.11 Intervalsandwidthsareinpixels.AsillustratedinFig.10,werefertothesesubsetsassensusregions Sincethereare voters, Given , ,ratio ,andvariancematrixfunction ,wecancomputetheconsensusre- B.EnsembleConsensusRegionGiven , ,theensembleconsensusregionisaregionwiththemostvotels.Let if otherwise Thecount isthenumberofvotelsinsideconsensusregion ofvoting .Breakingtiesarbitrarily,let ,theensemblecon-sensusregion,beany withmax Aconsensusregioncanbeprojectedontoalineinthevotingimageplanetoobtainaconsensus.Table1summarizesvotelanalysisforthevotelsshowninFig.11,whereconsensusregionsareprojectedontothe axistoob-tainthreeconsensusintervals.Consensusintervalthree,withthemostvotels,istheensembleconsensusinterval.C.LeadershipMetricOnewaytoscoreindividualperformanceistodefineameasureof“leadership.”Bydefinition,a“leader”isanindividualwhoisfollowedbythegroup.Incollaborativeteleoperation,aleaderisanindividualwhoanticipatestheconsensus,byvotingearlyinapositionthatcorrespondstowhatemergeslaterastheensembleconsensusregion.Wecanformalizethisbasedonamovingaverageofvotelarrivaltimesandensembleconsensusregionsasfollows. where istheleadershipmetricofvoter for voting isthevoter ’svotelarrivaltimeforthe voting Fig.11.Votingimageofanindustrialrobotarmwith27votels. isthetotalvotingtimeforvotingimage ,and isanoutcomeindex if otherwise Recallthat istheconsensusregionofvotingimage and[ ]isthepositionofvoter ’svotelat ; isthemovingaverageof randomvariables .Ifweassumethateachvotingimageisin-dependent,andthatthevariablesareidenticallydistributed, willconvergetoitstruemeanasthe and Wecandetermineaconfidenceintervalusingthecentrallimittheoremforafinite .Itisimportanttochooseafinite becauseeachvotingimageisnotindependent.Leadershipcanalsobecomputedincrementally Fig.12illustratestheleadershipmeasureforfourparticipantsasitevolvesoveraseriesofvotingimages.D.CollaborationMetricTowhatdegreearevoterscollaborating?Wedefineamea-sureofcollaborationbasedonthedensityofvotelsineachconsensusregion.Forconsensusregion invotingimage definethevoteldensityratioas where isthevoteldensity(votesperunitarea)forcon-sensusregion , istheoverallaveragevoteldensityforthevotingimage , isnumberofvotelsinconsensusre- , istheareaorwidthoftheconsensusregion , PROCEEDINGSOFTHEIEEE,VOL.91,NO.3,MARCH2003 Fig.12.Movingaverageoftheleadershipmeasureforfourparticipantsover14votingcycles.Table2SDVAnalysisforAnotherVotingImage isthetotalnumberofvotes,and istheareaofthevotingimage.Thismetricisproportionaltotheratio andin-verselyproportionaltotheareaoftheconsensusregion.Themetricishighwhenmanyvotesareconcentratedinasmallconsensusregionandlowwhenvotesareuniformlyspreadamongmultipleconsensusregions.Wecanalsocomputeanoverallcollaborationlevelforvotingimage whichcanmeasurehowfocusedthevotelsare.Table2givesresultsforanothervotingimage.Thecol-laborationmeasureforeachconsensusregionisgiveninthelastcolumnofTables1and2.InTable2,thedatasuggeststhatusersarecollaboratinginafocusedmannertovoteforconsensusintervaltwoeventhoughithasfewervotesthanconsensusintervalthree.VII.FThispaperdescribesanetworkedteleoperationsystemthatallowsgroupsofparticipantstocollaborativelyexploreremoteenvironments.Weproposetwoinnovations:theSDV,anetworkedinterfaceforcollectingspatialinputsfrommanysimulataneoususers,andthe“tele-actor,”askilledhumanwithcamerasandmicrophoneswhonavigatesandperformsactionsintheremoteenvironmentbasedonthisinput.Wepresentedproblemformulation,systemarchitectureandinterface,andalgorithmsforprocessingvotingdata.Collaborativeteleoperationsystemswillbenefitfromad-vancesinbroadbandInternet,localwirelessdigitalstandards(802.11x),videoteleconferencingstandards,andgigahertzprocessingcapabilitiesatbothclientandserver.Weareworkingonefficientalgorithmsforconsensusidentificationand“scoring”tomotivateuserinteraction.Wewillperformaseriesoffieldtestswithdifferentusergroupsanddifferentremoteenvironments.Inrelatedresearch,wearedevelopingcollaborativetele-operationsystemswherethesharedresourceisamachinesuchasaroboticpan-tilt-zoomcamera.Ourgoalissystemsthatareviableforverylargegroups(1000personandup),al-lowingcollectiveexplorationovernetworkssuchasInternet2andinteractivetelevision.Thelatestversionofoursystemcanbefoundathttp://www.tele-actor.net.CKNOWLEDGMENTTheauthorswouldliketothankE.PaulosandD.Pescovitzforvaluableinputoninitialexperiments;J.DonathandherstudentsatMITMediaLab;E.Paulos,C.Myers,andM.Fogartyforhelmetdesign;theotherstudentswhohaveparticipatedintheproject—A.Ho,M.McKelvin,I.Song,B.Chen,R.Aust,M.Metz,M.Faldu,V.Colburn,Y.Khor,J.Himmelstein,J.Shih,K.“Gopal”Gopalakrishnan,F.Hsu,J.McGonigalandM.Last;oursponsorsattheNationalScienceFoundation,Intel,andMicrosoft;andresearchcolleaguesR.Bajcsy,J.Canny,P.Wright,G.Niemeyer,A.Pashkevich,R.Luo,R.Siegwart,S.Montgomery,B.Laurel,V.Lumelsky,N.Johnson,R.Arkin,L.Leifer,P.Pirjanian,D.Greenbaum,K.Pister,C.Cox,D.Plautz,andT.Shlain.[1]H.Arai,T.Takubo,Y.Hayashibara,andK.Tanie,“Human-robotcooperativemanipulationusingavirtualnonholonomicconstraint,”Proc.IEEEInt.Conf.RoboticsandAutomation,vol.4,2000,pp.[2]R.Arkin,“Cooperationwithoutcommunication:Multiagentschema-basedrobotnavigation,”J.Robot.Syst.,vol.9,no.3,1992.[3]R.Baecker,ReadingsinGroupwareandComputerSupportedCooperativeWork:AssistingHuman-HumanCollaboration.SanMateo,CA:MorganKaufmann,1992.[4]R.D.Ballard,“AlastlonglookatTitanic,”Natl.Geograph.,vol.170,no.6,Dec.1986.[5]A.Bejczy,G.Bekey,R.Taylor,andS.Rovetta,“Aresearchmethodologyfortelesurgerywithtimedelays,”inProc.1stInt.Symp.MedicalRoboticsandComputerAssistedSurgery,vol.2,1994,pp.142–144.[6]A.K.Bejczy,“Sensors,controls,andman-machineinterfaceforad-vancedteleoperation,”,vol.208,no.4450,pp.1327–1335,June20,1980.[7]C.Bernard,H.Kang,S.K.Sigh,andJ.T.Wen,“Roboticsystemforcollaborativecontrolinminimallyinvasivesurgery,”Ind.Robotvol.26,no.6,pp.476–484,1999.[8]D.M.BloughandG.F.Sullivan,“Acomparisonofvotingstrategiesforfault-tolerantdistributedsystems,”inProc.9thSymp.ReliableDistributedSystems,1990,pp.136–145.[9]Z.Butler,A.Rizzi,andR.Hollis,“Cooperativecoverageofrecti-linearenvironments,”inProc.IEEEInt.Conf.RoboticsandAutoma-,vol.3,2000,pp.2722–2727.[10]D.J.Cannon,“Point-and-directtelerobotics:objectlevelstrategicsupervisorycontrolinunstructuredinteractivehuman-machinesystemenvironments,”Ph.D.dissertation,Mech.Eng.Dept.,StanfordUniv.,1992.etal.:COLLABORATIVETELEOPERATIONUSINGNETWORKEDSPATIALDYNAMICVOTING437 [11]Cinematrix,R.CarpenterandL.Carpenter.(1991).[Online].Avail-able:www.cinematrix.com[12]N.Chong,T.Kotoku,K.Ohba,K.Komoriya,N.Matsuhira,andK.Tanie,“Remotecoordinatedcontrolsinmultipletelerobotcooper-ation,”inProc.IEEEInt.Conf.RoboticsandAutomation,vol.4,2000,pp.3138–3143.[13]P.Coppinetal.,“Eventscope:Amplifyinghumanknowledgeandexperienceviaintelligentroboticsystemsandinformationinterac-tion,”inProc.9thIEEEInt.WorkshopRobotandHumanInteractiveCommunication(RO-MAN),2000,pp.292–296.[14]C.H.CrouchandE.Mazur,“Peerinstruction:Tenyearsofexperi-enceandresults,”Am.J.Phys.,vol.69,no.9,pp.970–977,2001.[15]BCB.DaltonandK.Taylor,“AframeworkforInternetrobotics,”presentedattheIEEEInt.Conf.IntelligentRobotsandSystems(IROS):WorkshoponWebRobots,Victoria,Canada,1998.[16]B.Donald,L.Gariepy,andD.Rus,“Distributedmanipulationofmultipleobjectsusingropes,”inProc.IEEEInt.Conf.RoboticsandAutomation,vol.1,2000,pp.450–457.[17]I.Elhaji,J.Tan,N.Xi,W.Fung,Y.Liu,T.Kaga,Y.Hasegawa,andT.Fukuda,“Multi-siteInternet-basedcooperativecontrolofroboticoperations,”inIEEE/RSJInt.Conf.IntelligentRobotsandSystems(IROS),vol.2,2000,pp.826–831.[18]I.Elhajj,J.Tan,N.Xi,W.K.Fung,Y.H.Liu,T.Kaga,Y.Hasegawa,andT.Fukuda,“Multi-siteInternet-basedtele-cooperation,”Integr.Comput.-AidedEng.,vol.9,no.2,2002.[19]H.Kuzuokaetal.,“Gestureman:Amobilerobotthatembodiesare-moteinstructor’sactions,”inProc.ACMConf.ComputerSupportedCollaborativeLearning,2000,pp.155–162.[20]T.Fong,C.Thorpe,andC.Baur,“Collaboration,dialogue,andhuman-robotinteration,”presentedatthe10thInt.Symp.RoboticsResearch,Lorne,Australia,2001.[21]L.Gasser,“Multi-agentsystemsinfrastructuredefinitions,needs,andprospects,”presentedattheWorkshoponScalableMASInfra-structure,Barcelona,Spain,2000.[22]M.Gertz,D.Stewart,andP.Khosla,“Ahuman-machineinterfacefordistributedvirtuallaboratories,”IEEERobot.Automat.Mag.vol.1,Dec.1994.[23]R.GoertzandR.Thompson,“Electronicallycontrolledmanipu-lator,”,vol.12,no.11,pp.46–47,Nov.1954.[24]K.Goldberg,B.Chen,R.Solomon,S.Bui,B.Farzin,J.Heitler,D.Poon,andG.Smith,“CollaborativeteleoperationviatheInternet,”Proc.IEEEInt.Conf.RoboticsandAutomation,vol.2,2000,pp.[25]K.GoldbergandB.Chen,“Collaborativecontrolofrobotmotion:Robustnesstoerror,”inProc.Int.Conf.IntelligentRobotsandSys-,vol.2,2001,pp.655–660.[26]K.Goldberg,M.Mascha,S.Gentner,N.Rothenberg,C.Sutter,andJ.Wiegley,“BeyondtheWeb:ManipulatingthephysicalworldviatheWWW,”Comput.Netw.ISDNSyst.J.,vol.28,no.1,Dec.1995. ,“DesktopteleoperationviatheWorldWideWeb,”inProc.IEEEInt.Conf.RoboticsandAutomation,vol.1,1995,pp.654–659.[28]K.Goldberg,T.Roeder,D.Gupta,andC.Perkins,“Eigentaste:Aconstanttimecollaborativefilteringalgorithm,”Inf.Retriev.,vol.4,no.2,July2001.[29]K.GoldbergandR.Siegwart,Eds.,BeyondWebcams:AnIntroduc-tiontoOnlineRobots.Cambridge,MA:MITPress,2002.[30]K.Goldberg,D.Song,Y.Khor,D.Pescovitz,A.Levandowski,J.Himmelstein,J.Shih,A.Ho,E.Paulos,andJ.Donath,“Collabo-rativeonlineteleoperationwithspatialdynamicvotingandahuman‘tele-actor’,”inProc.IEEEInt.Conf.RoboticsandAutomation,vol.2,2002,pp.1179–1184.[31]W.R.Hamel,P.Murray,andR.L.Kress,“Internet-basedroboticsandremotesystemsinhazardousenvironments:Reviewandprojec-Adv.Robot.,vol.16,no.5,pp.399–413,2002.[32]K.Han,Y.Kim,J.Kim,andS.Hsia,“InternetcontrolofpersonalrobotbetweenKAISTandUCDavis,”inProc.IEEEInt.Conf.RoboticsandAutomation,vol.2,2002,pp.2184–2189.[33]Australia’stelerobotontheWeb(1994).[Online].Available:http://telerobot.mech.uwa.edu.au/[34]H.Hu,L.Yu,P.Tsui,andQ.Zhou,“Internet-basedroboticsystemsforteleoperation,”AssemblyAutomat.J.,vol.21,no.2,pp.143–151,2001.[35]S.Jia,Y.Hada,G.Ye,andK.Takase,“DistributedtelecareroboticsystemsusingCORBAasacommunicationarchitecture,”inInt.Conf.RoboticsandAutomation,vol.2,2002,pp.2202–2207.[36]S.JiaandK.Takase,“ACORBA-basedInternetroboticsystem,”Adv.Robot.,vol.15,no.6,pp.663–673,2001.[37]N.L.Johnson,S.Rasmussen,andM.Kantor,“Newfrontiersincollectiveproblemsolving,”LosAlamosNationalLaboratory,LosAlamos,NM,NMLA-UR-98–1150,1998.[38]J.Kim,B.Choi,S.Park,K.Kim,andS.Ko,“Remotecontrolsystemusingreal-timeMPEG-4streamingtechnologyformobilerobot,”inDig.Tech.PapersIEEEInt.Conf.ConsumerElectronics,2002,pp.[39]Y.W.Leung,“Maximumlikelihoodvotingforfault-tolerantsoftwarewithfiniteoutputspace,”IEEETrans.Rel.,vol.44,pp.419–427,Sept.1995.[40]R.C.LuoandT.M.Chen,“Developmentofamultibehavior-basedmobilerobotforremotesupervisorycontrolthroughtheInternet,”IEEE/ASMETrans.Mechatron.,vol.5,pp.376–385,Dec.2000.[41]K.LynchandC.Liu,“Designingmotionguidesforergonomiccol-laborativemanipulation,”inProc.IEEEInt.Conf.RoboticsandAu-,vol.3,2000,pp.2709–2715.[42]S.Mann,“Wearablecomputing:AfirststeptowardpersonalIEEEComputer,vol.30,pp.25–31,Feb.1997. ,“Humanisticintelligence:Wearcompasanewframeworkforintelligentsignalprocessing,”Proc.IEEE,vol.86,pp.2123–2151,Nov.1998.[44]R.Marin,P.J.Sanz,andJ.S.Sanchez,“Objectrecognitionandin-crementallearningalgorithmsforaWeb-basedteleroboticsystem,”Proc.IEEEInt.Conf.RoboticsandAutomation,vol.3,2002,pp.2719–2724. ,“AveryhighlevelinterfacetoteleoperatearobotviaWebincludingaugmentedreality,”inProc.IEEEInt.Conf.RoboticsandAutomation,vol.3,2002,pp.2725–2730.[46]M.McDonald,D.Small,C.Graves,andD.Cannon,“Virtualcollab-orativecontroltoimproveintelligentroboticsystemefficiencyandquality,”inProc.IEEEInt.Conf.RoboticsandAutomation,vol.1,1997,pp.418–424.[47]D.G.Meyers,Ed.,SocialPsychology.NewYork:McGraw-Hill,[48]T.MirfakhraiandS.Payandeh,“AdelaypredictionapproachforteleoperationovertheInternet,”inProc.IEEEInt.Conf.RoboticsandAutomation,vol.2,2002,pp.2178–2183.[49]R.S.Mosher,“Industrialmanipulators,”Sci.Amer.,vol.211,no.4,pp.88–96,1964.[50]L.Ngai,W.S.Newman,andV.Liberatore,“AnexperimentinInternet-based,human-assistedrobotics,”inProc.IEEEInt.Conf.RoboticsandAutomation,vol.2,2002,pp.2190–2195.[51]E.PaulosandJ.Canny,“Ubiquitoustele-embodiment:Applicationsandimplications,”Int.J.Human-Comput.Stud.,vol.46,no.6,pp.861–877,June1997. ,“Designingpersonaltele-embodiment,”inProc.IEEEInt.Conf.RoboticsandAutomation,vol.4,1998,pp.3173–3178.[53]P.Pirjanian,“Multipleobjectiveactionselectionandbehaviorfusionusingvoting,”Ph.D.dissertation,AalborgUniv.,Aalborg,Denmark,[54]P.PirjanianandM.Mataric,“Multi-robottargetacquisitionusingmultipleobjectivebehaviorcoordination,”inProc.IEEEInt.Conf.RoboticsandAutomation,vol.3,2000,pp.2696–2702.[55]S.RasmussenandN.L.Johnson,“Self-organizationinandaroundtheInternet,”presentedatthe6thSantaFeChaosinManufacturingConf.,SantaFe,NM,1998.[56]I.Rekleitis,G.Dudek,andE.Milios,“Multi-robotcollaborationforrobustexploration,”inProc.IEEEInt.Conf.RoboticsandAutoma-,vol.4,2000,pp.3164–3169.[57]B.Rogoff,E.Matusov,andC.White,ModelsofTeachingandLearning:ParticipationinaCommunityofLearners.Oxford,U.K.:Blackwell,1996.[58]R.Safaric,M.Debevc,R.Parkin,andS.Uran,“TeleroboticsexperimentsviaInternet,”IEEETrans.Ind.Electron.,vol.48,pp.424–431,Apr.2001.[59]T.Sato,J.Ichikawa,M.Mitsuishi,andY.Hatamura,“Anewmicro-teleoperationsystememployingahand-heldforcefeedbackpencil,”inProc.Int.Conf.RoboticsandAutomation,vol.2,1994,pp.1728–1733.[60]B.S.Thomas,Telerobotics,Automation,andHumanSupervisoryControl.Cambridge,MA:MITPress,1992.[61]R.SiegwartandP.Saucy,“InteractingmobilerobotsontheWeb,”presentedattheIEEEInt.Conf.RoboticsandAutomation,Detroit,MI,1999.[62]R.Tomovic,“Onman-machinecontrol,”Automatica,vol.5,no.4,pp.401–404,July1969.PROCEEDINGSOFTHEIEEE,VOL.91,NO.3,MARCH2003 KenGoldberg(SeniorMember,IEEE)receivedthePh.D.degreefromtheSchoolofComputerScience,CarnegieMellonUniversity,Pittsburgh,PA,in1990.HeisProfessorofIndustrialEngineeringattheUniversityofCalifornia,Berkeley,withajointappointmentinElectricalEngineeringandCom-puterScience.Goldberg’sprimaryresearchareaisgeometricalgorithmsforfeeding,sorting,andfixturingindustrialparts.Dr.GoldbergreceivedtheNationalScienceFoundationYoungInvestigatorAwardin1994,theNSFPresidentialFacultyFellowshipin1995,theJosephEngelbergerAwardforRoboticsEducationin2000,andtheIEEEMajorEducationalInnovationAwardin2001.HeservesontheAdministrativeCommitteeoftheIEEESocietyofRoboticsandAutomation. DezhenSong(StudentMember,IEEE)receivedtheB.S.degreeinprocesscontrolandtheM.S.degreeinautomationfromZhejiangUniversity,Hangzhou,China,in1995and1998,respectively.HeisworkingtowardthePh.D.degreeintheDe-partmentofIndustrialEngineeringandOperationResearch,UniversityofCalifornia,Berkeley.HisresearchinterestsareInternetrobotics,geometricalgorithmsforrobotics,optimization,distributedcomputing,andstochasticmodeling. AnthonyLevandowski(StudentMember,IEEE)receivedtheB.S.degreeinindustrialengineeringfromtheUniversityofCalifornia,Berkeley,in2002.HeisworkingtowardthePh.D.degreeinindustrialengineeringattheUniversityofCalifornia,Berkeley.Hisprimaryresearchinterestisgeometrical-gorithmsforpartfeedingandorienting.Mr.LevandowskireceivedtheJamesGoslingAwardin2000andtheUniversityofCalifornia,Berkeley,CollegeofEngineeringUndergraduateResearchAwardin2001.etal.:COLLABORATIVETELEOPERATIONUSINGNETWORKEDSPATIALDYNAMICVOTING439