Collaborative Teleoperation Using Networked Spatial Dynamic Voting KEN GOLDBERG SENIOR MEMBER IEEE DEZHEN SONG STUDENT MEMBER IEEE AND ANTHONY LEVANDOWSKI STUDENT MEMBER IEEE Invited Paper We des - Description

Participants collaborate using a spatial dynamic voting SDV interface that allows them to vote on a sequence of images via a network such as the Internet The SDV interface runs on each client computer and communicates with a central server that coll ID: 27656 Download Pdf

Similar presentations

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II BlockInterlaced LDPC Decoders with Reduced Interconnect Complexity Ahmad Darabiha Student Member IEEE Anthony Chan Carusone Member IEEE and Frank R
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II BlockInterlaced LDPC Decoders with Reduced Interconnect Complexity Ahmad Darabiha Student Member IEEE Anthony Chan Carusone Member IEEE and Frank R

Kschischang Fellow IEEE Abstract Two design techniques are proposed for high throughput lowdensity paritycheck LDPC decoders A broad casting technique mitigates routing congestion by reducing the total global wirelength An interlacing technique inc

Nonparametric Scene Parsing via Label Transfer Ce Liu Member IEEE Jenny Yuen Student Member IEEE and Antonio Torralba Member IEEE Abstract While there has been a lot of recent work on object recogn
Nonparametric Scene Parsing via Label Transfer Ce Liu Member IEEE Jenny Yuen Student Member IEEE and Antonio Torralba Member IEEE Abstract While there has been a lot of recent work on object recogn

In this paper we propose a novel nonparametric approach for object recognition and scene parsing using a new technology we name labeltransfer For an input image our system first retrieves its nearest neighbors from a large database containing fully

Distributed Control Applications Within Sensor Networks BRUNO SINOPOLI STUDENT MEMBER IEEE COURTNEY SHARP LUCA SCHENATO STUDENT MEMBER IEEE SHAWN SCHAFFERT STUDENT MEMBER IEEE AND S
Distributed Control Applications Within Sensor Networks BRUNO SINOPOLI STUDENT MEMBER IEEE COURTNEY SHARP LUCA SCHENATO STUDENT MEMBER IEEE SHAWN SCHAFFERT STUDENT MEMBER IEEE AND S

SHANKAR SASTRY FELLOW IEEE Invited Paper Sensor networks are gaining a central role in the research community This paper addresses some of the issues arising from the use of sensor networks in control applications Classical control theory proves to

Contour Detection and Hierarchical Image Segmentation Pablo Arbel aez Member IEEE Michael Maire Member IEEE Charless Fowlkes Member IEEE and Jitendra Malik Fellow IEEE
Contour Detection and Hierarchical Image Segmentation Pablo Arbel aez Member IEEE Michael Maire Member IEEE Charless Fowlkes Member IEEE and Jitendra Malik Fellow IEEE

Abstract This paper investigates two fundamental problems in computer vision contour detection and image segmentation We present stateoftheart algorithms for both of these tasks Our contour detector combines multiple local cues into a globalization

The Golden Code A FullRate SpaceTime Code with NonVanishing Determinants JeanClaude Belore Member IEEE Ghaya Rekaya Student Member IEEE and Emanuele Viterbo Member IEEE Abstract In this paper we pr
The Golden Code A FullRate SpaceTime Code with NonVanishing Determinants JeanClaude Belore Member IEEE Ghaya Rekaya Student Member IEEE and Emanuele Viterbo Member IEEE Abstract In this paper we pr

This is a fullrate linear dispersion algebraic spacetime code with unprecedented performance based on the Golden number 1 Index Terms Number 64257elds Cyclic Division Algebras Space Time Lattices I I NTRODUCTION Ull rate and full diversity codes fo

Accurately Measuring Denial of Service in Simulation and Testbed Experiments Jelena Mirkovic Member IEEE Aleya Hussain Sonia Fahmy Senior Member IEEE Peter Reiher Member IEEE Roshan K
Accurately Measuring Denial of Service in Simulation and Testbed Experiments Jelena Mirkovic Member IEEE Aleya Hussain Sonia Fahmy Senior Member IEEE Peter Reiher Member IEEE Roshan K

Thomas Abstract Researchers in the denial of service DoS 64257eld lack accurate quantitative and versatile metrics to measure service denial in simulation and testbed experiments Without such metrics it is impossible to measure severity of various a

Actions as SpaceTime Shapes Lena Gorelick Moshe Blank Eli Shechtman Student Member IEEE Michal Irani Member IEEE and Ronen Basri Member IEEE Abstract Human action in video sequences can be seen as si
Actions as SpaceTime Shapes Lena Gorelick Moshe Blank Eli Shechtman Student Member IEEE Michal Irani Member IEEE and Ronen Basri Member IEEE Abstract Human action in video sequences can be seen as si

We regard human actions as threedimensional shapes induced by the silhouettes in the spacetimevolumeWeadoptarecentapproach14foranalyzing2Dshapesand generalizeittodealwithvolumetricspacetimeactionshapesOurmethodutilizes properties of the solution to

Face Description with Local Binary Patterns Application to Face Recognition Timo Ahonen Student Member IEEE Abdenour Hadid and Matti Pietik ainen Senior Member IEEE Abstract This paper presents a nov
Face Description with Local Binary Patterns Application to Face Recognition Timo Ahonen Student Member IEEE Abdenour Hadid and Matti Pietik ainen Senior Member IEEE Abstract This paper presents a nov

The face image is divided into several regions from which the LBP feature distributions are extrac ted and concatenated into an enhanced feature vector to be used as a face descriptor Th e performance of the proposed method is assessed in the face r

Efficient Shape Matching Using Shape Contexts Greg Mori Member IEEE Serge Belongie Member IEEE and Jitendra Malik Senior Member IEEE Abstract We demonstrate that shape contexts can be used to quickly
Efficient Shape Matching Using Shape Contexts Greg Mori Member IEEE Serge Belongie Member IEEE and Jitendra Malik Senior Member IEEE Abstract We demonstrate that shape contexts can be used to quickly

We present two algorithms for rapid shape retrieval representative shape contexts performing comparisons based on a small number of shape contexts and shapemes using vector quantization in the space of shape contexts to obtain prototypical shape p

Fast Approximate Energy Minimization via Graph Cuts Yuri Boykov Member IEEE Olga Veksler Member IEEE and Ramin Zabih Member IEEE Abstract Many tasks in computer vision involve assigning a label suc
Fast Approximate Energy Minimization via Graph Cuts Yuri Boykov Member IEEE Olga Veksler Member IEEE and Ramin Zabih Member IEEE Abstract Many tasks in computer vision involve assigning a label suc

A common constraint is that the labels should vary smoothly almost everywhere while preserving sharp discontinuities that may exist eg at object boundaries These tasks are naturally stated in terms of energy minimization In this paper we consider a

218K - views

Collaborative Teleoperation Using Networked Spatial Dynamic Voting KEN GOLDBERG SENIOR MEMBER IEEE DEZHEN SONG STUDENT MEMBER IEEE AND ANTHONY LEVANDOWSKI STUDENT MEMBER IEEE Invited Paper We des

Published bynatalia-silvester

Participants collaborate using a spatial dynamic voting SDV interface that allows them to vote on a sequence of images via a network such as the Internet The SDV interface runs on each client computer and communicates with a central server that coll

Tags : Participants collaborate using
Embed :
Pdf Download Link

Download Pdf - The PPT/PDF document "Collaborative Teleoperation Using Networ..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Collaborative Teleoperation Using Networked Spatial Dynamic Voting KEN GOLDBERG SENIOR MEMBER IEEE DEZHEN SONG STUDENT MEMBER IEEE AND ANTHONY LEVANDOWSKI STUDENT MEMBER IEEE Invited Paper We des






Presentation on theme: "Collaborative Teleoperation Using Networked Spatial Dynamic Voting KEN GOLDBERG SENIOR MEMBER IEEE DEZHEN SONG STUDENT MEMBER IEEE AND ANTHONY LEVANDOWSKI STUDENT MEMBER IEEE Invited Paper We des"— Presentation transcript:

CollaborativeTeleoperationUsingNetworkedSpatialDynamicVoting KENGOLDBERG,SENIORMEMBER,IEEE,DEZHENSONG,STUDENTMEMBER,IEEEANTHONYLEVANDOWSKI,STUDENTMEMBER,IEEEInvitedPaperWedescribeanetworkedteleoperationsystemthatallowsgroupsofparticipantstocollaborativelyexploreliveremoteenvironments. Fig.1.TheSDVinterfaceasviewedbyeachuser.Intheremoteenvironment,thetele-actortakesimageswithadigitalcamerawhicharetransmittedoverthenetworkanddisplayedtoallparticipantswitharelevantquestion.Withamouseclick,eachuserplacesacolor-codedmarker(a“votel”orvotingelement)ontheimage.Usersviewthepositionofallvotelsandcanchangetheirvotelpositionsbasedonthegroup’sresponse.Votelpositionsarethenprocessedtoidentifya“consensusregion”inthevotingimagethatissentbacktothetele-actor.Inthismanner,thegroupcollaboratestoguidetheactionsofthetele-actor.withvideocameras.Prosthetichandswerealsoappliedtoteleoperation[62].Morerecently,teleoperationisbeingcon-sideredformedicaldiagnosis[5],manufacturing[22],andmicromanipulation[59].Sheridan[60]providesanexcellentreviewoftheextensiveliteratureonteleoperationandtele-Networkedrobots,controllableovernetworkssuchastheInternet,areanactiveresearcharea.Inadditiontothechal-lengesassociatedwithtimedelay,supervisorycontrol,andstability,on-linerobotsmustbedesignedtobeoperatedbynonspecialiststhroughintuitiveuserinterfacesandtobeac-cessible24hoursaday.TheMercuryProjectwasthefirstpubliclyaccessiblenetworkedrobot[26],[27];itwenton-lineinAugust1994.Workingindependently,ateamledbyK.TaylorandJ.TrevelyanattheUniversityofWesternAustralia,Crawley,Australia,demonstratedaremotelycontrolledsix-axistele-robotinSeptember1994[15],[33].TherearenowdozensofInternetrobotson-line,abookfromMITPress[29],andanIEEETechnicalCommitteeonInternetandOnlineRobots.See[32],[34]–[36],[38],[40],[48],[50],[58]forexamplesofrecentprojects.Tanieetal.[12]proposedausefultaxonomyfortele-operationsystems:SOSR,single-operatormultiple-robot(SOMR),andmultiple-operatormultiple-robot(MOMR).MostnetworkedrobotsareSOSR,wherecontrolislimitedtoonehumanoperatoratatime.Tanieetal.analyzedanMOMRsystemwhereeachoperatorcontrolsonerobotarmandtherobotarmshaveoverlappingworkspaces.Theyshowthatpredictivedisplaysandscaledratecontrolareeffectiveinreducingpick-and-placetaskcompletiontimesthatrequirecooperationfrommultiplearms[12].InanMOMRprojectbyElhajjetal.[17],[18],twore-motehumanoperatorscollaboratetoachieveasharedgoalsuchasmaintainingagivenforceonanobjectheldatoneendbyamobilerobotandbyamultijointedrobotattheother.Theoperators,distantfromtherobotsandfromeachother,eachcontroladifferentrobotviaforcefeedbackde-vicesconnectedtotheInternet.Theauthorsshowboththeo-reticallyandexperimentallythatevent-basedcontrolallowsthesystemtomaintainstablesynchronizationbetweenoper-atorsdespitevariabletimelagontheInternet.MOMRmodelsarealsorelevanttoon-linecollaborativegamessuchasQuakeTheSimsOnline,whereplayersremotelycontrolindividualavatarsinasharedvirtualenvironment.InSOMRsystems,onehumanoperatorcontrolsmultiplerobots.Avariantisno-operatormultiple-robot(NOMR)sys-tems,sometimescalledcollaborativeorcooperativerobotics,wheregroupsofautonomousrobotsinteracttosolveanob-jective[2].Recentresultsarereportedin[9],[16],[54],andandAnumberofSOSRsystemshavebeendesignedtofacili-tateremoteinteraction.PaulosandCanny’sPersonalRovingPresence(ProP)telerobots,builtonblimporwheeledplat-forms,weredesignedtofacilitateremotesocialinteractionwithasingleremoteoperator[51],[52].Hamel[31]studiedhownetworkedrobotscanbeusefulinhazardousenviron-ments.Fongetal.studySOSRsystemswherecollaborationoccursbetweenasingleoperatorandamobilerobotthatistreatedasapeertothehumanandmodeledasanoisyinfor-mationsource[20].RelatedexamplesofSOSR“cobots”areanalyzedin[1],[7],[41],[44],[45],and[61].Oneprecedentforanon-lineMOSRsystemisdescribedinMcDonaldetal.[46].Forwastecleanup,severalusersas-sistremotelyusingpoint-and-direct(PAD)commands[10].Userspointtocleanuplocationsinasharedimageandarobotexcavateseachlocationinturn.InthisInternet-basedMOSRsystem,collaborationisserialbutpipelined,withoverlap-pingplanandexecutionphases.Theauthorsdemonstratethatsuchcollaborationimprovesoverallexecutiontime,butdonotaddressconflictresolutionbetweenusers.Pirjanianstudieshowreliablerobotbehaviorcanbeproducedfromanensembleofindependentprocessors[53].Drawingonresearchinfault-tolerantsoftware[39],Pirjanianconsiderssystemswithanumberofhomogenousprocessorssharingacommonobjective.Heconsidersavarietyofvotingschemesandshowsthatfault-tolerantbehaviorfusioncanbeoptimizedusingpluralityvoting[8],butdoesnotconsiderspatialvotingmodelssuchasours.In[24],wepresentanInternet-basedMOSRsystemthataveragedmultiplevectorinputstocontrolthepositionofanindustrialrobotarm.Wereportexperimentswithmazefol-lowingthatsuggestedthatgroupsofhumansmayperformbetterthanindividualsinthepresenceofnoiseduetocentrallimiteffects.In[25],weusedfiniteautomatatomodelcollaboratingusersinaMOSRsystemsuchasCinematrix,acommercialetal.:COLLABORATIVETELEOPERATIONUSINGNETWORKEDSPATIALDYNAMICVOTING431 Fig.2.Systemarchitecture.ParticipantsontheInternetviewandvotingonaseriesofvotingimages.Thehumantele-actor,withhead-mountedwirelessaudio/videolink,movesthroughtheremoteenvironmentinresponse.The“localdirector”facilitatesinteractionbypostingtextualqueries.system[11]thatallowslargeaudiencestointeractinathe-aterusingplasticpaddles.Tomodelsuchsystems,weaver-agedanensembleoffiniteautomataoutputstocomputeasinglestreamofincrementalstepstocontrolthemotionofapointrobotmovingintheplane.Weanalyzedsystemperfor-mancewithauniformensembleofwell-behaveddetermin-isticsourcesandthenmodeledmalfunctioningsourcesthatgosilentorgenerateinvertedcontrolsignals.Wefoundthatperformanceissurprisinglyrobustevenwhenasizablefrac-tionofsourcesmalfunction.Outsideofrobotics,thenotionofMOSRisrelatedtoaverybroadrangeofgroupactivitiesincludingsocialpsy-chology,voting,economics,marketpricing,trafficflows,etc.TheAssociationforComputingMachineryorganizesannualconferencesoncomputer-supportedcollaborativelearningandcomputer-supportedcooperativework.Surveysofre-searchinthisbroadercontextcanbefoundin[3],[19],[21],[28],[37],[47],and[55].Wenotethattheconceptofhuman-mountedcameraswithnetworkconnectionsisnotnovel:thereisextensiveliteratureon“wearablecomputer”systems[42],[43].Thefocusofourresearchisoncollaborativecontrol.Apreliminaryreportonthetele-actorsystemappearedin[30].III.SThetele-actorsystemarchitectureisillustratedinFig.2.Asthetele-actor(seeFig.3)movesthroughtheenvironment,cameraimagesaresenttothetele-actorserverfordistribu-tiontousers,whorespondfromtheirInternetbrowsers.Uservotingresponsesarecollectedatthetele-actorserver,whichupdatesJavaappletsforallusersandforthetele-actorinthefield.Thetele-actorcarriesalaptopwhichcommunicatestotheInternetusingthe2.4-GHz802.11bwirelessprotocol.Acamerapersonwithasecondcameraprovidesthird-personperspectivesasneeded.Usingthisarchitecture,theusers,thetele-actorserver,thelocaldirector,thecameraperson,andthetele-actorcommunicateviatheInternet. Fig.3.Thehumantele-actortransmitsimagesfromtheremoteenvironmentusingthehelmet-mountedvideocameraandrespondstouservotes.HelmetdesignbyE.Paulos,C.Myers,andM.Fogarty.(PhotobyB.Nagel.)IV.SDVUNTERFACEWehavedevelopedanewgraphicalinterfacetofacilitateinteractionandcollaborationamongremoteusers.Fig.1il-lustratestheSDVinterfacethatisdisplayedonthebrowserPROCEEDINGSOFTHEIEEE,VOL.91,NO.3,MARCH2003 Fig.4.Navigationquery.Participantsindicatewheretheywantthetele-actortogo. Fig.5.Pointquery.Participantspointoutaregionofinterestinthevotingimage.ofallactivevoters.Usersregisteron-linetoparticipatebyselectingavotelcolorandsubmittingtheire-mailaddresstothetele-actorserver,whichstoresthisinformationinourdatabaseandsendsbackapasswordviae-mail.TheserveralsomaintainsatutorialandaFAQsectiontofamiliarizenewuserswithhowthesystemworks.UsingtheSDVinterface,votersparticipateinaseriesof30-to60-svotingimages.Eachvotingimageisasingleimagewithatextualquestion.IntheexamplefromFig.1,thetele-actorisvisitingabiologylab.Votersclickontheirscreenstopositiontheirvotels.UsingtheHTTPprotocol,thesepositionsaresentbacktothetele-actorserverandappearinanupdatedvotingimagesenttoallvotersevery3–5s.Inthisway,voterscanchangetheirvotes.Whenthevotingcycleiscompleted,SDVanalysisalgorithmsanalyzethevotingpatterntodetermineaconsensuscommandthatissenttothetele-actor.TheSDVinterfacediffersfrommultiple-choicepollingbecauseitallowsspatiallyandtemporallycontinuousinputs.Tofacilitateusertrainingandasynchronoustesting,thetele-actorsystemhastwomodes.Intheoff-linemode,votingimagesaredrawnfromaprestoredlibrary.Intheon-linemode,votingimagesarecapturedlivebythetele-actor.Bothoff-lineandon-linemodeshavepotentialforcollaborativeeducation,testing,andtraining.Inthispaper,wefocusontheon-linemode.Figs.4–7illustratefourtypesofSDVqueriesandtheirassociatedbranchingstructures.Ineachcase,thepositionofthemajorityofvotelsdecidestheoutcome.Wetriedincludingalivevideobroadcastingstreambutfoundthatduetobandwidthlimitations,theresolutionandframerateisunacceptableforlow-latencyapplications.Standardvideobroadcastingsoftwarerequiresabout20sofbufferedvideodataforcompression,whichintroducesunacceptabledelaysforlivevisits.WearehopingthiscanbereducedinthefuturewithfasternetworkssuchasInternet2.V.HARDWAREANDOFTWAREThetele-actorWebserverisanAMDK7950-MHzPCwith1.2-GBmemoryconnectedtoa100-Mb/sT3line.TheVideoserverisalsoanAMDK7950-MHzPCwith512-MBmemoryconnectedtoa100-Mb/sT3line.ThelocalbasestationcouldbeanymachineontheInternetequippedwithJava-enabledWebbrowsers.Theprimarytele-actoriscar-ryinga600-MHzSonypicture-booklaptopwith128-MBmemoryconnectedtoa11-Mb/s802.11bwirelessLANattheremotesite.IthasaUSBvideocard,whichcapturesvideoat320 240resolution.ThecamerapersonhasaPentiumIII750-MHzSonyVaiolaptopwith256-MBmemorywithsim-ilarUSBvideocapturedevice.Thelaptopsdirecttheirvideodisplaystohand-mountedTVstoprovideupdatesonvotingpatterns.Fig.2showsthattheprimarytele-actorhasaCanoncameramountedonherhelmet.Fig.8showsthatthecamer-apersonhasaSonycamcorderwithnightvisioncapability,whichprovidesveryhigh-qualityimageandvideostream.Bothofthemareequippedwithashutterlikedevicetoallowthemtocapturethepreciousmomentintheliveevent.Customsoftwareincludes:1)theclient-sideSDVbrowserinterfacebasedonJava;2)thetele-actorimagecaptureandcommunicationsoftware;3)thelocalbasestationvotingquestionformulationinterface;and4)thetele-actorserver.Duringtheon-linemode,thelocaldirectorusesaJavaapplettoaddtextualquestionstovotingimages.Duringbothon-lineandoff-linemodes,thetele-actorserverusescustomCandC++applicationstomaintainthedatabaseandcommunicatewiththelocalbasestationandwithallactivevoters.Thetele-actorserverrunsRedhatLinux7.1andtheApacheWebserver1.3.20.TheResinetal.:COLLABORATIVETELEOPERATIONUSINGNETWORKEDSPATIALDYNAMICVOTING433 Fig.6.Opinionquery.Votelpositioncanbeanywherebetweenextremevaluestoindicatedegreeofopinion. Fig.7.Multiple-choicequery.Avariationonthepointquerywithasmallnumberofexplicitoptions.2.0.1Apacheplug-inandSunJDK1.3.1withMysqldata-base3.23.36provideJavaserverpagestohandletheuserregistrationanddatalogging.VI.PROBLEMEFINITIONANDAsillustratedinFig.9,usersexpressresponsesbyclickingonthevotingimagetospatiallyindicateapreferredobjectordirectioninthefieldofview.Asanalternativetosemanticanalysisofthevotingimage,weconsidervotelsasspatialdistributionsandidentifypreferred“consensus”regionsintheimage.Wethenusetheseregionstodefinetwometrics[Online]Available:http://www.caucho.com Fig.8.Hardwareconfigurationforthecameraperson.Thehardwareconfigurationofthetele-actorissimilarbuthasahelmet-mountedcamera. Fig.9.Evolutionofvotingimageasvotelsarrive.forindividualandgroupperformanceintermsofleadershipandcollaboration.A.ProblemDefinition1)VoterInterestFunctions:Considerthe thvotingimage.Theserverreceivesaresponsefromuser intheformofan( )mouseclickonimage attime .Wedefinethecorresponding Eachvotelrepresentsauser’sresponsetothevotingimage.Wemodelsuchresponseswithavoterinterest,adensityfunctionbasedonthebivariatenormaldistribution where isthemeanvectorand isa2 2variancematrix,suchthat where istheareaofthevotingimage.Since isaboundedtwo-dimensionalregion,thevoterinterestfunctionisatrun-catedbivariatenormaldensityfunctionwithmeanat asillustratedinFig.10.PROCEEDINGSOFTHEIEEE,VOL.91,NO.3,MARCH2003 Fig.10.Interestfunctionsandconsensusregions.(a)and(b)showtheinterestfunctionforasinglevoter.(c)and(d)showanensembleinterestdensityfunctionforfivevoters.(e)and(f)illustrateresultingconsensusregions.2)EnsembleInterestFunction:Whenvotingonimage endsatstoppingtime ,thelastvotelreceivedfromeachof activevotersdetermines ,asetof votels.Wedefineensembleinterestfunctionforvotingimage asthenor-malizedsumofthesevoterinterestfunctions 3)ConsensusRegions:Wecanextractspatialregionsfromtheensembleinterestfunctionasfollows.Let bethemaximumoftheensembleinterestdensityfunction,andlet besomevaluebetween0and1.Ahorizontalplaneatheight definesanisodensitycontourintheensembleinterestfunctionthatdefinesasetofofoneormoreclosedsubsetsofthevotingimage etal.:COLLABORATIVETELEOPERATIONUSINGNETWORKEDSPATIALDYNAMICVOTING435 Table1SDVAnalysisofVotingImageFromFig.11 Intervalsandwidthsareinpixels.AsillustratedinFig.10,werefertothesesubsetsassensusregions Sincethereare voters, Given , ,ratio ,andvariancematrixfunction ,wecancomputetheconsensusre- B.EnsembleConsensusRegionGiven , ,theensembleconsensusregionisaregionwiththemostvotels.Let if otherwise Thecount isthenumberofvotelsinsideconsensusregion ofvoting .Breakingtiesarbitrarily,let ,theensemblecon-sensusregion,beany withmax Aconsensusregioncanbeprojectedontoalineinthevotingimageplanetoobtainaconsensus.Table1summarizesvotelanalysisforthevotelsshowninFig.11,whereconsensusregionsareprojectedontothe axistoob-tainthreeconsensusintervals.Consensusintervalthree,withthemostvotels,istheensembleconsensusinterval.C.LeadershipMetricOnewaytoscoreindividualperformanceistodefineameasureof“leadership.”Bydefinition,a“leader”isanindividualwhoisfollowedbythegroup.Incollaborativeteleoperation,aleaderisanindividualwhoanticipatestheconsensus,byvotingearlyinapositionthatcorrespondstowhatemergeslaterastheensembleconsensusregion.Wecanformalizethisbasedonamovingaverageofvotelarrivaltimesandensembleconsensusregionsasfollows. where istheleadershipmetricofvoter for voting isthevoter ’svotelarrivaltimeforthe voting Fig.11.Votingimageofanindustrialrobotarmwith27votels. isthetotalvotingtimeforvotingimage ,and isanoutcomeindex if otherwise Recallthat istheconsensusregionofvotingimage and[ ]isthepositionofvoter ’svotelat ; isthemovingaverageof randomvariables .Ifweassumethateachvotingimageisin-dependent,andthatthevariablesareidenticallydistributed, willconvergetoitstruemeanasthe and Wecandetermineaconfidenceintervalusingthecentrallimittheoremforafinite .Itisimportanttochooseafinite becauseeachvotingimageisnotindependent.Leadershipcanalsobecomputedincrementally Fig.12illustratestheleadershipmeasureforfourparticipantsasitevolvesoveraseriesofvotingimages.D.CollaborationMetricTowhatdegreearevoterscollaborating?Wedefineamea-sureofcollaborationbasedonthedensityofvotelsineachconsensusregion.Forconsensusregion invotingimage definethevoteldensityratioas where isthevoteldensity(votesperunitarea)forcon-sensusregion , istheoverallaveragevoteldensityforthevotingimage , isnumberofvotelsinconsensusre- , istheareaorwidthoftheconsensusregion , PROCEEDINGSOFTHEIEEE,VOL.91,NO.3,MARCH2003 Fig.12.Movingaverageoftheleadershipmeasureforfourparticipantsover14votingcycles.Table2SDVAnalysisforAnotherVotingImage isthetotalnumberofvotes,and istheareaofthevotingimage.Thismetricisproportionaltotheratio andin-verselyproportionaltotheareaoftheconsensusregion.Themetricishighwhenmanyvotesareconcentratedinasmallconsensusregionandlowwhenvotesareuniformlyspreadamongmultipleconsensusregions.Wecanalsocomputeanoverallcollaborationlevelforvotingimage whichcanmeasurehowfocusedthevotelsare.Table2givesresultsforanothervotingimage.Thecol-laborationmeasureforeachconsensusregionisgiveninthelastcolumnofTables1and2.InTable2,thedatasuggeststhatusersarecollaboratinginafocusedmannertovoteforconsensusintervaltwoeventhoughithasfewervotesthanconsensusintervalthree.VII.FThispaperdescribesanetworkedteleoperationsystemthatallowsgroupsofparticipantstocollaborativelyexploreremoteenvironments.Weproposetwoinnovations:theSDV,anetworkedinterfaceforcollectingspatialinputsfrommanysimulataneoususers,andthe“tele-actor,”askilledhumanwithcamerasandmicrophoneswhonavigatesandperformsactionsintheremoteenvironmentbasedonthisinput.Wepresentedproblemformulation,systemarchitectureandinterface,andalgorithmsforprocessingvotingdata.Collaborativeteleoperationsystemswillbenefitfromad-vancesinbroadbandInternet,localwirelessdigitalstandards(802.11x),videoteleconferencingstandards,andgigahertzprocessingcapabilitiesatbothclientandserver.Weareworkingonefficientalgorithmsforconsensusidentificationand“scoring”tomotivateuserinteraction.Wewillperformaseriesoffieldtestswithdifferentusergroupsanddifferentremoteenvironments.Inrelatedresearch,wearedevelopingcollaborativetele-operationsystemswherethesharedresourceisamachinesuchasaroboticpan-tilt-zoomcamera.Ourgoalissystemsthatareviableforverylargegroups(1000personandup),al-lowingcollectiveexplorationovernetworkssuchasInternet2andinteractivetelevision.Thelatestversionofoursystemcanbefoundathttp://www.tele-actor.net.CKNOWLEDGMENTTheauthorswouldliketothankE.PaulosandD.Pescovitzforvaluableinputoninitialexperiments;J.DonathandherstudentsatMITMediaLab;E.Paulos,C.Myers,andM.Fogartyforhelmetdesign;theotherstudentswhohaveparticipatedintheproject—A.Ho,M.McKelvin,I.Song,B.Chen,R.Aust,M.Metz,M.Faldu,V.Colburn,Y.Khor,J.Himmelstein,J.Shih,K.“Gopal”Gopalakrishnan,F.Hsu,J.McGonigalandM.Last;oursponsorsattheNationalScienceFoundation,Intel,andMicrosoft;andresearchcolleaguesR.Bajcsy,J.Canny,P.Wright,G.Niemeyer,A.Pashkevich,R.Luo,R.Siegwart,S.Montgomery,B.Laurel,V.Lumelsky,N.Johnson,R.Arkin,L.Leifer,P.Pirjanian,D.Greenbaum,K.Pister,C.Cox,D.Plautz,andT.Shlain.[1]H.Arai,T.Takubo,Y.Hayashibara,andK.Tanie,“Human-robotcooperativemanipulationusingavirtualnonholonomicconstraint,”Proc.IEEEInt.Conf.RoboticsandAutomation,vol.4,2000,pp.[2]R.Arkin,“Cooperationwithoutcommunication:Multiagentschema-basedrobotnavigation,”J.Robot.Syst.,vol.9,no.3,1992.[3]R.Baecker,ReadingsinGroupwareandComputerSupportedCooperativeWork:AssistingHuman-HumanCollaboration.SanMateo,CA:MorganKaufmann,1992.[4]R.D.Ballard,“AlastlonglookatTitanic,”Natl.Geograph.,vol.170,no.6,Dec.1986.[5]A.Bejczy,G.Bekey,R.Taylor,andS.Rovetta,“Aresearchmethodologyfortelesurgerywithtimedelays,”inProc.1stInt.Symp.MedicalRoboticsandComputerAssistedSurgery,vol.2,1994,pp.142–144.[6]A.K.Bejczy,“Sensors,controls,andman-machineinterfaceforad-vancedteleoperation,”,vol.208,no.4450,pp.1327–1335,June20,1980.[7]C.Bernard,H.Kang,S.K.Sigh,andJ.T.Wen,“Roboticsystemforcollaborativecontrolinminimallyinvasivesurgery,”Ind.Robotvol.26,no.6,pp.476–484,1999.[8]D.M.BloughandG.F.Sullivan,“Acomparisonofvotingstrategiesforfault-tolerantdistributedsystems,”inProc.9thSymp.ReliableDistributedSystems,1990,pp.136–145.[9]Z.Butler,A.Rizzi,andR.Hollis,“Cooperativecoverageofrecti-linearenvironments,”inProc.IEEEInt.Conf.RoboticsandAutoma-,vol.3,2000,pp.2722–2727.[10]D.J.Cannon,“Point-and-directtelerobotics:objectlevelstrategicsupervisorycontrolinunstructuredinteractivehuman-machinesystemenvironments,”Ph.D.dissertation,Mech.Eng.Dept.,StanfordUniv.,1992.etal.:COLLABORATIVETELEOPERATIONUSINGNETWORKEDSPATIALDYNAMICVOTING437 [11]Cinematrix,R.CarpenterandL.Carpenter.(1991).[Online].Avail-able:www.cinematrix.com[12]N.Chong,T.Kotoku,K.Ohba,K.Komoriya,N.Matsuhira,andK.Tanie,“Remotecoordinatedcontrolsinmultipletelerobotcooper-ation,”inProc.IEEEInt.Conf.RoboticsandAutomation,vol.4,2000,pp.3138–3143.[13]P.Coppinetal.,“Eventscope:Amplifyinghumanknowledgeandexperienceviaintelligentroboticsystemsandinformationinterac-tion,”inProc.9thIEEEInt.WorkshopRobotandHumanInteractiveCommunication(RO-MAN),2000,pp.292–296.[14]C.H.CrouchandE.Mazur,“Peerinstruction:Tenyearsofexperi-enceandresults,”Am.J.Phys.,vol.69,no.9,pp.970–977,2001.[15]BCB.DaltonandK.Taylor,“AframeworkforInternetrobotics,”presentedattheIEEEInt.Conf.IntelligentRobotsandSystems(IROS):WorkshoponWebRobots,Victoria,Canada,1998.[16]B.Donald,L.Gariepy,andD.Rus,“Distributedmanipulationofmultipleobjectsusingropes,”inProc.IEEEInt.Conf.RoboticsandAutomation,vol.1,2000,pp.450–457.[17]I.Elhaji,J.Tan,N.Xi,W.Fung,Y.Liu,T.Kaga,Y.Hasegawa,andT.Fukuda,“Multi-siteInternet-basedcooperativecontrolofroboticoperations,”inIEEE/RSJInt.Conf.IntelligentRobotsandSystems(IROS),vol.2,2000,pp.826–831.[18]I.Elhajj,J.Tan,N.Xi,W.K.Fung,Y.H.Liu,T.Kaga,Y.Hasegawa,andT.Fukuda,“Multi-siteInternet-basedtele-cooperation,”Integr.Comput.-AidedEng.,vol.9,no.2,2002.[19]H.Kuzuokaetal.,“Gestureman:Amobilerobotthatembodiesare-moteinstructor’sactions,”inProc.ACMConf.ComputerSupportedCollaborativeLearning,2000,pp.155–162.[20]T.Fong,C.Thorpe,andC.Baur,“Collaboration,dialogue,andhuman-robotinteration,”presentedatthe10thInt.Symp.RoboticsResearch,Lorne,Australia,2001.[21]L.Gasser,“Multi-agentsystemsinfrastructuredefinitions,needs,andprospects,”presentedattheWorkshoponScalableMASInfra-structure,Barcelona,Spain,2000.[22]M.Gertz,D.Stewart,andP.Khosla,“Ahuman-machineinterfacefordistributedvirtuallaboratories,”IEEERobot.Automat.Mag.vol.1,Dec.1994.[23]R.GoertzandR.Thompson,“Electronicallycontrolledmanipu-lator,”,vol.12,no.11,pp.46–47,Nov.1954.[24]K.Goldberg,B.Chen,R.Solomon,S.Bui,B.Farzin,J.Heitler,D.Poon,andG.Smith,“CollaborativeteleoperationviatheInternet,”Proc.IEEEInt.Conf.RoboticsandAutomation,vol.2,2000,pp.[25]K.GoldbergandB.Chen,“Collaborativecontrolofrobotmotion:Robustnesstoerror,”inProc.Int.Conf.IntelligentRobotsandSys-,vol.2,2001,pp.655–660.[26]K.Goldberg,M.Mascha,S.Gentner,N.Rothenberg,C.Sutter,andJ.Wiegley,“BeyondtheWeb:ManipulatingthephysicalworldviatheWWW,”Comput.Netw.ISDNSyst.J.,vol.28,no.1,Dec.1995. ,“DesktopteleoperationviatheWorldWideWeb,”inProc.IEEEInt.Conf.RoboticsandAutomation,vol.1,1995,pp.654–659.[28]K.Goldberg,T.Roeder,D.Gupta,andC.Perkins,“Eigentaste:Aconstanttimecollaborativefilteringalgorithm,”Inf.Retriev.,vol.4,no.2,July2001.[29]K.GoldbergandR.Siegwart,Eds.,BeyondWebcams:AnIntroduc-tiontoOnlineRobots.Cambridge,MA:MITPress,2002.[30]K.Goldberg,D.Song,Y.Khor,D.Pescovitz,A.Levandowski,J.Himmelstein,J.Shih,A.Ho,E.Paulos,andJ.Donath,“Collabo-rativeonlineteleoperationwithspatialdynamicvotingandahuman‘tele-actor’,”inProc.IEEEInt.Conf.RoboticsandAutomation,vol.2,2002,pp.1179–1184.[31]W.R.Hamel,P.Murray,andR.L.Kress,“Internet-basedroboticsandremotesystemsinhazardousenvironments:Reviewandprojec-Adv.Robot.,vol.16,no.5,pp.399–413,2002.[32]K.Han,Y.Kim,J.Kim,andS.Hsia,“InternetcontrolofpersonalrobotbetweenKAISTandUCDavis,”inProc.IEEEInt.Conf.RoboticsandAutomation,vol.2,2002,pp.2184–2189.[33]Australia’stelerobotontheWeb(1994).[Online].Available:http://telerobot.mech.uwa.edu.au/[34]H.Hu,L.Yu,P.Tsui,andQ.Zhou,“Internet-basedroboticsystemsforteleoperation,”AssemblyAutomat.J.,vol.21,no.2,pp.143–151,2001.[35]S.Jia,Y.Hada,G.Ye,andK.Takase,“DistributedtelecareroboticsystemsusingCORBAasacommunicationarchitecture,”inInt.Conf.RoboticsandAutomation,vol.2,2002,pp.2202–2207.[36]S.JiaandK.Takase,“ACORBA-basedInternetroboticsystem,”Adv.Robot.,vol.15,no.6,pp.663–673,2001.[37]N.L.Johnson,S.Rasmussen,andM.Kantor,“Newfrontiersincollectiveproblemsolving,”LosAlamosNationalLaboratory,LosAlamos,NM,NMLA-UR-98–1150,1998.[38]J.Kim,B.Choi,S.Park,K.Kim,andS.Ko,“Remotecontrolsystemusingreal-timeMPEG-4streamingtechnologyformobilerobot,”inDig.Tech.PapersIEEEInt.Conf.ConsumerElectronics,2002,pp.[39]Y.W.Leung,“Maximumlikelihoodvotingforfault-tolerantsoftwarewithfiniteoutputspace,”IEEETrans.Rel.,vol.44,pp.419–427,Sept.1995.[40]R.C.LuoandT.M.Chen,“Developmentofamultibehavior-basedmobilerobotforremotesupervisorycontrolthroughtheInternet,”IEEE/ASMETrans.Mechatron.,vol.5,pp.376–385,Dec.2000.[41]K.LynchandC.Liu,“Designingmotionguidesforergonomiccol-laborativemanipulation,”inProc.IEEEInt.Conf.RoboticsandAu-,vol.3,2000,pp.2709–2715.[42]S.Mann,“Wearablecomputing:AfirststeptowardpersonalIEEEComputer,vol.30,pp.25–31,Feb.1997. ,“Humanisticintelligence:Wearcompasanewframeworkforintelligentsignalprocessing,”Proc.IEEE,vol.86,pp.2123–2151,Nov.1998.[44]R.Marin,P.J.Sanz,andJ.S.Sanchez,“Objectrecognitionandin-crementallearningalgorithmsforaWeb-basedteleroboticsystem,”Proc.IEEEInt.Conf.RoboticsandAutomation,vol.3,2002,pp.2719–2724. ,“AveryhighlevelinterfacetoteleoperatearobotviaWebincludingaugmentedreality,”inProc.IEEEInt.Conf.RoboticsandAutomation,vol.3,2002,pp.2725–2730.[46]M.McDonald,D.Small,C.Graves,andD.Cannon,“Virtualcollab-orativecontroltoimproveintelligentroboticsystemefficiencyandquality,”inProc.IEEEInt.Conf.RoboticsandAutomation,vol.1,1997,pp.418–424.[47]D.G.Meyers,Ed.,SocialPsychology.NewYork:McGraw-Hill,[48]T.MirfakhraiandS.Payandeh,“AdelaypredictionapproachforteleoperationovertheInternet,”inProc.IEEEInt.Conf.RoboticsandAutomation,vol.2,2002,pp.2178–2183.[49]R.S.Mosher,“Industrialmanipulators,”Sci.Amer.,vol.211,no.4,pp.88–96,1964.[50]L.Ngai,W.S.Newman,andV.Liberatore,“AnexperimentinInternet-based,human-assistedrobotics,”inProc.IEEEInt.Conf.RoboticsandAutomation,vol.2,2002,pp.2190–2195.[51]E.PaulosandJ.Canny,“Ubiquitoustele-embodiment:Applicationsandimplications,”Int.J.Human-Comput.Stud.,vol.46,no.6,pp.861–877,June1997. ,“Designingpersonaltele-embodiment,”inProc.IEEEInt.Conf.RoboticsandAutomation,vol.4,1998,pp.3173–3178.[53]P.Pirjanian,“Multipleobjectiveactionselectionandbehaviorfusionusingvoting,”Ph.D.dissertation,AalborgUniv.,Aalborg,Denmark,[54]P.PirjanianandM.Mataric,“Multi-robottargetacquisitionusingmultipleobjectivebehaviorcoordination,”inProc.IEEEInt.Conf.RoboticsandAutomation,vol.3,2000,pp.2696–2702.[55]S.RasmussenandN.L.Johnson,“Self-organizationinandaroundtheInternet,”presentedatthe6thSantaFeChaosinManufacturingConf.,SantaFe,NM,1998.[56]I.Rekleitis,G.Dudek,andE.Milios,“Multi-robotcollaborationforrobustexploration,”inProc.IEEEInt.Conf.RoboticsandAutoma-,vol.4,2000,pp.3164–3169.[57]B.Rogoff,E.Matusov,andC.White,ModelsofTeachingandLearning:ParticipationinaCommunityofLearners.Oxford,U.K.:Blackwell,1996.[58]R.Safaric,M.Debevc,R.Parkin,andS.Uran,“TeleroboticsexperimentsviaInternet,”IEEETrans.Ind.Electron.,vol.48,pp.424–431,Apr.2001.[59]T.Sato,J.Ichikawa,M.Mitsuishi,andY.Hatamura,“Anewmicro-teleoperationsystememployingahand-heldforcefeedbackpencil,”inProc.Int.Conf.RoboticsandAutomation,vol.2,1994,pp.1728–1733.[60]B.S.Thomas,Telerobotics,Automation,andHumanSupervisoryControl.Cambridge,MA:MITPress,1992.[61]R.SiegwartandP.Saucy,“InteractingmobilerobotsontheWeb,”presentedattheIEEEInt.Conf.RoboticsandAutomation,Detroit,MI,1999.[62]R.Tomovic,“Onman-machinecontrol,”Automatica,vol.5,no.4,pp.401–404,July1969.PROCEEDINGSOFTHEIEEE,VOL.91,NO.3,MARCH2003 KenGoldberg(SeniorMember,IEEE)receivedthePh.D.degreefromtheSchoolofComputerScience,CarnegieMellonUniversity,Pittsburgh,PA,in1990.HeisProfessorofIndustrialEngineeringattheUniversityofCalifornia,Berkeley,withajointappointmentinElectricalEngineeringandCom-puterScience.Goldberg’sprimaryresearchareaisgeometricalgorithmsforfeeding,sorting,andfixturingindustrialparts.Dr.GoldbergreceivedtheNationalScienceFoundationYoungInvestigatorAwardin1994,theNSFPresidentialFacultyFellowshipin1995,theJosephEngelbergerAwardforRoboticsEducationin2000,andtheIEEEMajorEducationalInnovationAwardin2001.HeservesontheAdministrativeCommitteeoftheIEEESocietyofRoboticsandAutomation. DezhenSong(StudentMember,IEEE)receivedtheB.S.degreeinprocesscontrolandtheM.S.degreeinautomationfromZhejiangUniversity,Hangzhou,China,in1995and1998,respectively.HeisworkingtowardthePh.D.degreeintheDe-partmentofIndustrialEngineeringandOperationResearch,UniversityofCalifornia,Berkeley.HisresearchinterestsareInternetrobotics,geometricalgorithmsforrobotics,optimization,distributedcomputing,andstochasticmodeling. AnthonyLevandowski(StudentMember,IEEE)receivedtheB.S.degreeinindustrialengineeringfromtheUniversityofCalifornia,Berkeley,in2002.HeisworkingtowardthePh.D.degreeinindustrialengineeringattheUniversityofCalifornia,Berkeley.Hisprimaryresearchinterestisgeometrical-gorithmsforpartfeedingandorienting.Mr.LevandowskireceivedtheJamesGoslingAwardin2000andtheUniversityofCalifornia,Berkeley,CollegeofEngineeringUndergraduateResearchAwardin2001.etal.:COLLABORATIVETELEOPERATIONUSINGNETWORKEDSPATIALDYNAMICVOTING439