IEEE TH INTERNATIONAL CONFERENCE ON DEVELOPMENT AND LEARNING Learning motor dependent - PDF document

2009IEEE8THINTERNATIONALCONFERENCEONDEVELOPMENTANDLEARNING7 Fig.8.EvolutionofmeansquarederrorsinpredictingCrutcheld'smetricsbetweenthetwosensorsofbodies1,2,3and4whenvariousnumberoflearningexamplarsareavailable(eachlearningexamplarisanassociationbetweenarandommotorcongurationandtheassociatedmeasuredCrutch-eldinformationaldistance)andwithatestdatabasecomposedof100motorcongurationsuniformlyspreadinthespaceofcongurations. Fig.9.ComparisonbetweentheCrutchelddistancesinthetestdatabaseofbody2withthecorrespondingpredictedinformationaldistancesafter1024learningexamplarshavebeenprovided:oneobservesthatthesystempredictsaccuratelytheCrutcheldinformationdistanceamongthetwosensorsformotorcongurationsthatwerenotexperimentedbeforehand.Thesequenceofexamplarsisgeneratedbyxingthepositionofthersteffector,thenrotatingprogressivelythesecondeffectoraroundtheclock,beforerotatingthersteffectorbyaddingit2 10androtatingagainthesecondeffectoraroundtheclock,whichisrepeateduntilthersteffectorhasitselfrotatedby2.ThepeakswheretheCrutchelddistanceiszerocorrespondstomotorcongurationsinwhichsensorssuperposeandperceiveexactlythesamething. Fig.10.ComparisonbetweentheCrutchelddistancesinthetestdatabaseofbody3withthecorrespondingpredictedinformationaldistancesafter1024learningexamplarshavebeenprovided:oneobservesthatthesystempredictsaccuratelytheCrutcheldinformationdistanceamongthetwosensorsformotorcongurationsthatwerenotexperimentedbeforehand.Thesequenceofexamplarsisgeneratedinthesamewaythaninthepreviousgure.Theshapeofthecurveisnotequivalentsincerotationsofthesecondeffectorsarenotsymmetricinrelationshiptorotationsofthersteffector.Thepeakswithlowvaluescorrespondtomotorcongurationforwhichsensorsperceiveoverlappingregions,andthepeakswithvaluescloseto1correspondtomotorcongurationsforwhichsensorsperceivenon-overlappingregions.REFERENCES[1]T.M.CoverandThomasJ.A.ElementsofInformationTheory.JohnWileyandSons,Inc.,N.Y.,1991.[2]J.P.Crutcheld.Informationanditsmetric.NonlinearStructuresinPhysicalSystems-Patternformation,ChaosandWaves,pages119–130,1990.[3]J.J.Gibson.Thetheoryofaffordances.NJ:LaurenceErlbaumAssociates.[4]G.Goodhill,S.Finch,andT.Sejnowski.Quantifyingneighbourhoodpreservationintopographicmappings.InstituteforNeuralComputationTechnicalReportSeries,No.INC-9505,1995.[5]E.R.Kandel,J.H.Schwartz,andT.M.Jessel.PrinciplesofNeuralScience,4thedition.McGraw-Hill,N.Y.,2000.[6]F.KaplanandV.V.Hafner.Information-theoreticframeworkforun-supervisedactivityclassication.AdvancedRobotics,20:1087–1103,2006.[7]T.M.Mitchell.MachineLearning.McGraw-Hill,N.Y.,1997.[8]L.Olsson,C.L.Nehaniv,andD.Polani.Sensorychannelgroupingandstructurefromuninterpretedsensordata.In2004NASA/DoDConferenceonEvolvableHardwareJune24-26,2004Seattle,Washington,USA,2004.[9]L.Olsson,C.L.Nehaniv,andD.Polani.Fromunknownsensorsandactuatorstoactionsgroundedinsensorimotorperceptions.ConnectionScience,18:121–144,2006.[10]D.PierceandB.Kuipers.Maplearningwithuninterpretedsensorsandeffectors.ArticialIntelligence,92:169–229,1997.[11]P.Rochat.Self-perceptionandactionininfancy.ExperimentalBrainResearch,123:102–109,1998.[12]S.Vijayakumar,A.DSouza,andS.Schaal.Incrementalonlinelearninginhighdimensions.NeuralComputation,17(12):119–130.[13]J.Weng,J.McClelland,A.Pentland,andO.Sporns.Autonomousmentaldevelopmentbyrobotsandanimals.Science,291:599–600,2001.[14]I.H.WittenandF.Eibe.DataMining.2000. 2009IEEE8THINTERNATIONALCONFERENCEONDEVELOPMENTANDLEARNING6 Fig.4.Apparentanglesensor.Thelocationandorientationofthesensorarerepresentedbythebluearrow.Perceivetheangleunderwhichisseenfromthesensortheclosestobjectoftheenvironmentwithinitsperceptiveeld. Fig.5.Apparentdiametersensor.Thelocationandorientationofthesensorarerepresentedbythebluearrow.Theperceivedlengthdcorrespondtothesizeoftheprojectionofthesensedobjectonaplanorthogonaltothesensordirectionandsituatedatafocaldistancefbehindthesensor. Fig.6.Pathofacoloreddisk.Ateverymoment,ithasacertainprobabilityofmovingtowardanotheruniformlyrandomlychosendirection.Itsspeedremainsconstant. Fig.7.Body1(withtworigidsegments,twosuperposedjointandtwosensors),Body2(withtwochainedrigidsegments,twojointsandtwosensors),Body3(withtwoindependantrigidsegments,twojointsandtwosensors),Body4(withtwoindependantchainsofsegments,fourjointsandtwosensors)databasesareindependant.Aswecanobserveongure8,weseethatthesystemisabletolearnveryefcientlytopredicttheCrutchelddistancebetweentwosensorsinallbodiesandformotorcongurationsthatwerenotencounteredinthetrainingdatabase.Figures9and10comparesallCrutchelddistancesinthetestdatabasewiththecorrespondingdistancespredictedbythelearntmodelafter1024learningexamplarshavebeenprovided.ThisallowsustoseethatrelativelytotheglobalamplitudeoftheCrutchelddistancevariationswhenmotorcongurationsarevaried,thepredictionerrorsarelow.ThisshowsthatamodelofCrutchelddistancesoverthewholespaceofmotorcongurationhasbeenefcientlylearnt,andthus,reusingthebodymapreconstructiontechniquespresentedin[6],[8],couldbeusedtodynamicallyanticipateinformationalsensorybodymapswhenmotorcongurationsarechanging.E.Discussion:ComputationalcomplexityandscalabilityIntheexperimentspresentedabove,100000measureswereusedforestimatingtheinformationaltopologyofagivenmo-torconguration.Thisisalotandmaypreventthesystemtobeapplicableinrealtimeroboticexperiments.Yet,ontheonehandthesemeasurecanbedoneatahigh-frequencywithoutaffectingtheresults,e.g.100hz,ifthesensoriapparatusper-mitsit,andwebelievethataccuratemeasuresofinformationaldistancecanberealizedwithmuchlesssamples.Thiswillbethetopicoffutureexperiments.Furthermore,itcanbenotedthatincreasingthenumberofsensorsinthesystemwouldincreasethecomplexityquadratically,sinceCrutchelddistancesneedonlybemeasured,learntandpredictedpairwisetoreconstructthewholeinformationaltopology.Asfarasthemotorspaceisconcerned,increasingthenumberofdegrees-of-freedomtoseveraldozensshouldnotbeproblematicsincemathematically,theregressionperformedhereisquitesim-ilartohigh-dimensionalregressionproblemsofrobotmotorlearningadressedbytechniquesrecentlydevelopped,suchasLWPR[12]. 2009IEEE8THINTERNATIONALCONFERENCEONDEVELOPMENTANDLEARNING5sensorswasxed.ThisisindeedthecaseinexamplessuchasthedistancesensorsofKheperarobotsorthepixelsensorsofacamera.In[6]'swork,themotorcommandschangedaccordingtooneofseveralpre-denedactivity:theyidentiedthefactthatagivenmotoractivityorconguration,aswellasagivendynamicsoftheenvironmentitself,couldmodifytheinformationalstructureofasetofsensors.Yet,intheirexperiment,onlyanitesetofmotorcongurationswasusedandthepairwiseCrutcheldmetricswascomputedforeachcongurationaftermanymeasurementsofthesensorvalues.Inmostrealrobotbodies,achangeinmotorcongurationwillprovokeachangeintheinformationalstructureofthesetofsensors,andbecausewithcontinuousmotorsthereisaninnite(orverylarge)numberofmotorcongurations,itisimpossibletocomputetheCrutcheldmetricforeachofthem.Rather,wearguethatitisnecessarythatamodelofCrutchelddistances,correspondingtoallparticularmotorcongurations,belearntfromasmallnitesetofreallymeasuredmetrics,suchthatmetricsfornovelmotorcongurationscanbeaccu-ratelypredicted.Inthefollowing,wewillshowthatitisindeedpossibletolearnCrutchelddistancesinmotordependantdynamicallyrecongurableinformationalbodymaps.B.LearningalgorithmAmemory-basedlearningalgorithmischoseninthisarticle,basedontheuseofagaussiankernelonthek-nearestneighboursofaquerypoint.Letasystemwithmmotorchannelsbe.FromadatabasecontainingthedistanceobtainedwithCrutcheld'smetricsbetweentwosensorss1ands2ofthesystemforngivenmotorcommand,apredictionofthedistancebetweenthemforanycommandxisgivenby:ds1;s2(x)=Pki=1ds1;s2(xi)e�jjx�xijj22 2 Pki=1e�jjx�xijj22 2;where(x1;x2;::;xk)arethekclosestcommandtoxinthedatabaseinthesenseoftheeuclideanmetricsonRm(withkn).Thealgorithmisparameterizedbyandk.Adecreaseinacceleratethedecreasingoftheinuenceoftheknearestneighboursonthepredictionwiththedistancetothepredictionpoint.See[7]formoredetailsonthisalgorithm.Inthefollowingexperiments,optimalvaluesofkandwerechosentomaximizeperformancesingeneralizations(dependingontheexperiment,kisbetween1and10).C.PresentationofthesimulationToevaluatetheperformanceofCrutcheld'smetrics'au-tomatedlearning,measuresweremadeondifferentsimulatedsystems.Thesimulatedspaceisaplan.Thebodiesaremadeofrigidsegmentsassembledtogetherbypivotjointswithamotorineachjoint(cf.gure2).Severaltypesofsensors(distance,color,apparentangleandapparentdiameter)canbeplacedonthesegments.Figures3,4and5showtheprinciplesofdistance,apparentangleandapparentdiametersensorsrespectively.ThecolorsensortakesthevalueoftheRGBencodingon24bitsofthecolorofthenearestobjectpresentinitsperceptiveeld.Theenvironmentismadeofcolored Fig.2.Exampleofabodymadeoftwosegments.Thesegmentinthebottomislinkedtothegroundbyapivotjointandtheuppersegmentislinkedtothesegmentinthebottombyanotherpivotjoint.Onthisbody,sensorscanbeattachedatanyplacealongthesegments.Therearefourtypesofsensors,threeofwhicharedescribedinthefollowinggures(thefourthoneisacolorsensor).Thesebodyareplacedinanenvironmentwherecoloreddisksmoveonarandompath(seegure6). Fig.3.Distancesensor.Thelocationandorientationofthesensorarerepresentedbythebluearrow.Theperceiveddistancedcorrespondtothedistancebetweentheoriginofthesensorandthenearestobjectoftheenvironmentwithintheperceptiveeldofthesensor(yellow-greenarea).disksofvariousradiusmovingintheplan.Themovementofadiskisdeterminedbytwoparameters,itsspeedanditsinstantaneousprobabilityofchangingitsdirection,thepathofthediskbeingconsequentlyabrokenline(cf.gure10).Topreservethesimplicityofthemodel,collisionsbetweenthedifferentelementsinpresencearen'ttakenintoaccount.Atanytimeitispossibletoaccessthevaluesofthesensorsandtoxatethepositionofthejoints.D.ExperimentsandresultsWeranexperimentsonfourdifferentsimulatedbodiesillustratedongure7.Thetypesofsensorsonthesebodiesarerandomlychosen(butxedoncechosen).Figure8summarizestheevolutionofperformancesinpredictingCrutcheld'sin-formationdistancebetweenthetwosensorsofeachofthefourbodiesforvariousnumbersoflearningexamplars.AlearningexamplarconsistsinanassociationbetweenarandommotorcongurationandtheassociatedCrutcheldmetricsbetweenthetwosensorsofthebodycomputeddirectlywith100000samples.FortestingthelearntmodelofCrutchelddistances,weuseatestdatabasecomposedof100uniformlydistributedmotorcongurationsassociatedtothecorrespondingrecom-putedCrutchelddistances.Ofcourse,thetrainingandtest 2009IEEE8THINTERNATIONALCONFERENCEONDEVELOPMENTANDLEARNING4(sfreq2(1);sfreq2(2):::;sfreq2(N))be,wheresfreqi(f)standsfortheratioofvaluestakenbysiinthefthinterval.Then:dN;T(s1;s2)=1 2NXf=1jsfreq1(f)�sfreq2(f)j:However,thesemetricssuffersseveralproblems,notablytheycan'taccountfornon-linearcorrelationsbetweensensors,whicharenonethelessveryusualinthecaseoftwosensorsphysicallyclose,butofdifferentmodalities(e.g.conesandrodcellsintheretina).Evenanafnecorrelationwon'tbedetected.Olssonandal.introducedanothermetrics,originatingfrominformationtheory:Crutcheld'smetrics[2].Asareminder,beinggivenarandomvariableXofprobabilitylawpwithvaluesinE,X'sShannonentropyormeaninformationis:H(X)=Xe2E�p(fX=eg)log2(p(fX=eg)):Beinggivenarandomvector(X;Y)ofprobabilitylawpwithvaluesinEF,theconditionalentropyofXknowingYis:H(XjY)=X(e;f)2EF�p(f(X;Y)=(e;f)g)log2(p(f(X;Y)=(e;f)g) p(fY=fg))(See[1]foranin-depthpresentationofinformationtheory).Lets1ands2betwosensors,and(s1(1);s1(2):::;s1(T)),(s2(1);s2(2):::;s2(T))thevaluestakenbythesesensorsatTconsecutivetimes.WeconsiderthiscoupleoftimeseriesasaT-sampleofarandomvectorofdimension2(S1;S2),S1andS2takingtheirvaluesinnitesets.Fromtheempiricprobabilitylawassociatedtothecoupleofsamples,wethencompute:dT(s1;s2)=HT(S1jS2)+HT(S2jS1) HT(S1)+HT(S2):Theonlyinterestofthefraction'sdenominatoristonormalizethemetricsin[0;1].HT(SijSj)canbeinterpretedastheinformationbroughtinproperbySi,i.e.theinformationcontainedbySi,whichisn'talreadyinSj.SoCrutcheld'smetricsgivesameasureofthequantityofinformationwhichisn'tcommontoSiandSj,whichisindeedaformofdistance.Besidesit'sapseudo-metricsinthemathematicalsense,i.e:it'ssymetric:dT(S1;S2)=dT(S2;S1),it'spositive:dT(S1;S2)2R+,itcompliestotriangularinequality:dT(S1;S3)dT(S1;S2)+dT(S2;S3):Itlacksthepropertyofbeingdenedtobearealmetrics.IndeeditispossibletondS1;S2suchasdT(S1;S2)=0andS16=S2.However,itcanbeshownthatCrutcheld'smetricsbetweentworandomvariablesiszeroif,andonlyif,theyaretwoequivalentencodingofasamesource.See[2]forproofs.Oncethemetricsoverthesetofsensorsismeasured,[10]proposetodetermineawell-tteddimensionforthesensoritopicmapwiththemetricscalingmethod:thedistance Fig.1.SensoritopicmapoftheAIBOrobotwithCrutcheld'smetrics.Eachpixelofthevisualsensorisdepictedbyanumberbetween1and100.TheinfrareddistancesensorisdepictedbyIr,theaccelerometersbyBa,LaandDa,thetemperaturesensorbyTe,theremainingpowerinthebatterybyP,thebinarycontactsensorsofthepawsbyBLP,BRP,FLPandFRP,whereFLmeansfrontright,BRbackright,etc.Eachpawhasthreedegreesoffreedom:E(elevation),R(rotation)etK(knee),foreachdegreeofeachpawthereisapositionsensor(forinstanceFLKforFrontLeftKnee)andatorquesensor,depictedinlower-caseletters(forinstancek).Theneckhasthreedegreesoffreedomtoonamedtilt,panandrollandsothreepositionsensorsNT,NPandNRandthreetorquesensorsnt,np,nr.It'sthesameforthetail:TT,TP,TR,tt,tp,tr.Atlastthejawhasonlyonedegreeoffreedom,andsoonlytwosensorsJAWandjaw.[After[8]]betweenthepairsofsensorsarerepresentedbyasquarematrix(mi;j)suchasmi;j=dT(si;sj).Thismatrixissymetricreal,soit'sdiagonalizableandit'seigenvaluesarereals,let'scallthem1;2;:::;n,with12:::n.Weshallthenbeabletoselectinformationinanumberofdimensionmorerestrictedthannwhilelosingaminimumofinformationbyretainingonlythemaineigenvalues.Theselecteddimensionmissuchthat2m+1�2mbemaximal.Intuitivelytheselecteddimensionissuchthattherebethebiggestpossibledifferencebetweenthequantityofinformationbroughtinbythisdimensionandtheonebroughtinbyasupplementarydimension.Foradetailedpresentationofprincipalcomponentanalysis,see[14].Eventually,eachsensorisdepictedbyapointinaneu-clideanspaceofdimensionm,insuchawaythattheeuclideandistancebetweentwopointscoincideatbestwiththemeasureddistancebetweenthetwocorrespondingsensors.Therearenumerousmethodsforthisclassicoptimizationproblem,see[4].Oneexampleofsensoritopicmapsobtainedby[8]isgivenongure1.II.AUTOMATEDLEARNINGOFCRUTCHFIELD'SMETRICSA.IntroductionIntheworkof[10]and[8]theinformationalCrutchelddistanceoverallpairsofsensorswascomputedbasedonaseriesofsensorvaluesmeasurementscorrespondingtorandomlychangingmotorcommands:thisimplicitlyassumedthattherewasonlyonepossibledistanceperpairofsensor,i.e.itassumedthattheinformationalstructureofthesetof