AndrewC.Connolly,1andJ.SwaroopGuntupalli11DepartmentofPsychologicaland - PDF document

AndrewC.Connolly,1andJ.SwaroopGuntupalli11DepartmentofPsychologicalandBrainSciences,CenterforCognitiveNeuroscience,DartmouthCollege,Hanover,NewHampshire03755;email:james.v.haxby@dartmouth.edu,andrew.c.connolly@dartmouth.edu,swaroopgj@gmail.com2CenterforMind/BrainSciences(CIMeC),UniversityofTrento,Rovereto,Trentino38068,ItalyAnnu.Rev.Neurosci.2014.37:435Ð56FirstpublishedonlineasaReviewinAdvanceonJune25,2014TheAnnualReviewofNeuroscienceisonlineatneuro.annualreviews.orgThisarticleÕsdoi: expressedastheresponsestrengthsforfeaturesofthatpattern,e.g.,voxels,singleneurons,ormodeldimensionsHyperalignment:transformationofindividualrepresentationalspacesintoamodelrepresentationalspaceinwhicheachdimensionhasacommontuningfunctionREPRESENTATIONALSPACERepresentationalspaceisahigh-dimensionalspaceinwhicheachneuralresponseorstimulusisexpressedasavectorwithdifferentvaluesforeachdimension.Inaneuralrepresentationalspace,eachpatternfeatureisameasureoflocalactivity,suchasavoxelorasingleneuron.Inastimulusrepresentationalspace,eachfeatureisastimulus components,ormeasuresofsynchronybetweensources.Thecomputationaladvantagesofrepresentationalvectorspacesextendbeyondneuralrepre-sentationalspacestorepresentationalspacesforstimuliorcognitivestates.Forexample,avisualstimuluscanbemodeledasasetoffeaturesbasedonresponsepropertiesofneuronsinV1,ashigher-ordervisualfeatures,orasasetofsemanticlabels.SoundsÑvoicesandmusicÑcanbemod-eledassetsofacousticfeatures,wordscanbemodeledassetsofsemanticfeatures,actionsassetsofmovementandgoalfeatures,etc.Oncethedescriptionofthestimulusisinarepresentationalspace,variouscomputationalmanipulationscanbeappliedforrelatingstimulusrepresentationalspacestoneuralrepresentationalspaces.Allthemajorvarietiesofneuraldecodingandencodinganalysesfollowfromthisconversionofpatternsofbrainactivityorstimulitosinglepointsinhigh-dimensionalrepresentationalvectorspaces.MVPclassiÞcationusesmachinelearningmethodstodeÞnedecisionboundariesinaneu-ralrepresentationalspacethatbestdistinguishasetofresponsevectorsforonebrainstatefromothers.RSAanalyzesthesimilaritybetweenresponsevectorsasdistancesintherepresentationalspace.Stimulus-model-basedencodingpredictsthelocationoftheneuralresponsevectorforanewstimulusonthebasisofthecoordinatesofthatstimulusinastimulusfeaturespace.Stimulus-model-baseddecodingtestswhethertheencoding-basedpredictionallowscorrectclassiÞcationofneuralresponsevectorstonewstimuli.Buildingamodelofaneuralrepresentationalspacethatiscommonacrossbrainsrequireshyperalignmenttorotatethecoordinateaxesofindividualrepresentationalspacestominimizethedifferenceinthelocationsofresponsevectorsforthesamestimuli.Thus,stimuliandothercognitiveeventsarerepresentedasvectorsinneuralrep-resentationalspacesaswellasinstimulusrepresentationalspaces,andthecomputationaltaskforunderstandingrepresentationbecomesoneofcharacterizingthegeometrieswithinspacesandrelatingthegeometriesofthesespacestoeachother.Numerically,asetofresponsevectorsinarepresentationalspaceisamatrixinwhicheachcol-umnisalocalpatternfeature(e.g.,voxel)andeachrowisaresponsevector(Figure1).Thevaluesineachcolumnreßectthedifferentialresponsesofthatpatternfeaturetoconditionsorstimuli. vectorsAnMVPclassiÞcationanalysisbeginswithdividingthedataintoindependenttrainingandtestdatasets(Figure2).ThedecisionrulesthatdeterminetheconÞnesofeachclassofneuralresponsevectorsaredevelopedontrainingdata.Theborderbetweensectorsfordifferentconditionsiscalledadecisionsurface.ThevalidityoftheclassiÞeristhentestedontheindependenttestdata.Forvalidgeneralizationtesting,thetestdatamustplaynoroleinthedevelopmentoftheclassiÞer,includingdatapreprocessing(Kriegeskorteetal.2009).EachtestdataresponsevectoristhenclassiÞedasanotherexemplaroftheconditionassociatedwiththesectorinwhichitislocated.ClassiÞeraccuracyisthepercentageoftestvectorsthatarecorrectlyclassiÞed.AmorerevealingassessmentofclassiÞerperformanceisaffordedbyexaminingtheconfusionmatrix.AconfusionmatrixpresentsthefrequenciesforallclassiÞcationsofeachexperimentalcondition,includingthedetailsaboutmisclassiÞcations.ExaminationofmisclassiÞcationsaddsinformationaboutwhichconditionsaremostdistinctandwhicharemoresimilar.ThisinformationisanalyzedusingadditionalmethodsinRSA(seenextsection).ExaminationofclassiÞcationaccuracyforeachconditionseparatelycanalerttheinvestigatortowhetheraverageaccuracyisreallydependentonasmallnumberofconditions,ratherthananaccuratereßectionofperformanceacrossallormostconditions.Thus,averageclassiÞcationaccuracyisausefulmetricbutdiscardsinformationthatcanbediscoveredbyexaminingtheclassiÞcationconfusionmatrix.Confusionmatricesareshownin Percent classi! Figure5Threeexamplesofrepresentationalsimilarityanalysis(RSA).(a)Dendrogramderivedfrommultiplesingle-unitrecordingsinmacaqueinferiortemporal(IT)cortex(fromKianietal.2007)showshierarchicalcategorystructurewithremarkabledetailtothelevelofdifferentclassesofanimalbodytype.( Figure6).CorticalÞeldscanalsobeidentiÞedbyvirtueofhavingsimilaritystructuresthat00.85Mean correlationa MVP classi!cationb Between-subject correlation of DSMs112Number of subjects112Number of subjectsc DSM cluster 1 - LOCd DSM cluster 2 - early visualFigure6MVPAsearchlightanalyses(Kriegeskorteetal.2006)foridentifyingcorticalÞeldsofinterest(fromConnollyetal.2012a).MVPclassiÞcationaccuracies(a)andconsistencyinlocalsimilaritystructuresacrosssubjects(b)identifysimilarlylargeswathsofthevisuallyresponsivecortex.ClusteringofvoxelsbasedonsimilaritiesbetweenlocallydeÞnedsearchlightdissimilaritymatrix(DSMs)providesameanstoidentifycorticalÞeldswithuniquesharedstructuresuchasthelateraloccipitalcomplex(LOC)(c)andtheearlyvisual howRSAmaybeusedtotestawell-controlledmodel.OneofthegreatadvantagesofRSAisthatitstripsaclusterofresponsevectorsoutofafeature-basedrepresentationalspaceintoarepresentationalspacebasedonrelativedistancesamongvec-tors.Thisformatallowscomparisonofrepresentationalgeometriesacrosssubjects,acrossbrainregions,acrossmeasurementmodalities,andevenacrossspecies.Thesecond-orderisomorphismacrossthesespacesisaffordedbythefeature-independentformatofDSMs.Forexample,between-subjectsimilarityofDSMshasbeenexploitedtoaffordbetween-subjectMVPclassiÞcation(Abdietal.2012b,Raizada&Connolly2012).Thefeature-independentsecond-orderisomorphism,however,doeshavesomecost.Strippingrepresentationalspacesoffeaturesmakesitimpossibletocomparepopulationcodesintermsoftheconstituenttuningfunctionsofthosefeatures.Thus,onecannotinvestigatewhetherthespacesindifferentsubjectssharethesamefeaturetuningfunctionsorhowthesetuningfunctioncodesdifferfordifferentbrainregions.Onecannotpredicttheresponsetoanewstimulusinasubjectonthebasisoftheresponsestothatstimulusinothersubjects.Onecannotpredictthetuningfunctionforindividualneuralfeaturesintermsofstimulusfeatures,precludinginvestigatorsfrompredictingtheresponsepatternvectorforanewstimulusonthebasisofitsfeatures.Thenexttwosections sumofthesepatternsofweights(Figure8a).ModelsofVTcortexbasedonsingledimensions,suchascontraststhatdeÞnecategory-selectiveregions,aremodeledwellinthe35-dimensionalmodelspace(Figure8b),butthesesingledimensionsaccountforonlyasmallportionofthevarianceinresponsestoadynamicandvariednaturalstimulussuchasthemovie.Forexample,thecontrastbetweenresponsestofacesandresponsestoobjects,whichdeÞnesthefusiformfacearea(FFA)(Kanwisheretal.1997),accountsforonly12%ofthevariancethatisaccountedforbythe35-dimensionalmodel.Thisresultindicatesthatmodelsbasedonsimple,univariatecontrastsareinsufÞcientasmodelsofneuralrepresentationalspaces.Theuseofacomplex,dynamicstimulusisessentialforderivingtransformationmatrixparam-etersthataffordgeneralvalidityacrossawiderangeofstimuli.Transformationmatricescanalsobecalculatedonthebasisofresponsestomorecontrolledexperiments,suchasthecategoryper-ceptionexperiments.Thesetransformationmatricesarevalidformodelingtheresponsevectorsforstimuliinthatexperimentbut,whenappliedtodatafromotherexperiments,donotaffordBSCofnewstimuli(Haxbyetal.2011).Thisresultindicatesthatdatafromalimitedsamplingofbrainstates,suchasthosesampledinastandardcategoryperceptionexperiment,donotprovideasufÞcientbasisforbuildingacommonmodelofaneuralrepresentationalspace.STIMULUS-MODEL-BASEDENCODINGANDDECODINGForMVPclassiÞcation,RSA,andhyperalignment,aresponsevectortobedecodediscomparedwithresponsevectorsforthatsamestimulusmeasuredinthesamesubjectorinothersubjects.Thesemethodscannotpredicttheresponsepatternforanovelstimulusorexperimentalcondition.Stimulus-model-basedmethodsextendneuraldecodingtonovelstimulibypredictingtheresponsetostimulusfeaturesratherthantowholestimuli.Thestimuliusedtoproducetrainingdataforstimulus-model-baseddecodingareanalyzedintoconstituentfeatures.FeaturesetsusedforthistypeofanalysisincludemodelsofV1neuron ChenY,NamburiP,ElliottLT,HeinzleJ,SoonCS,etal.2011.Corticalsurface-basedsearchlightdecoding.NeuroImage56:582Ð92ConnollyAC,GobbiniMI,HaxbyJV.2012a.Threevirtuesofsimilarity-basedmulti-voxelpatternanalysis:anexamplefromthehumanobjectvisionpathway.InUnderstandingVisualPopulationCodes(UVPC):TowardACommonMultivariateFrameworkforCellRecordingandFunctionalImaging NeuroImage 54:2418Ð25 AlfonsoCaramazza,StefanoAnzellotti,LukasStrnad,andAngelikaLingnau!!!!!!!!!!!1TranslationalControlinSynapticPlasticityandCognitiveDysfunctionShellyA.BufÞngton,WeiHuang,andMauroCosta-Mattioli!!!!!!!!!!!!!!!!!!!!!!!!!!!!17ThePerirhinalCortexWendyA.SuzukiandYujiNaya!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!39AutophagyandItsNormalandPathogenicStatesintheBrainAiYamamotoandZhenyuYueApolipoproteinEinAlzheimerÕsDisease:AnUpdateJin-TaiYu,LanTan,andJohnHardy!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!79FunctionandDysfunctionofHypocretin/Orexin:AnEnergeticsPointofViewXiao-BingGaoandTamasHorvath !!!!!!!289Motion-DetectingCircuitsinFlies:ComingintoViewMarionSilies,DarylM.Gohl,andThomasR.Clandinin!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!307NeuromodulationofCircuitswithVariableParameters:SingleNeuronsandSmallCircuitsRevealPrinciplesofState-DependentandRobustNeuromodulationEveMarder,TimothyOÕLeary,andSonalShruti!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!329 CAN) | Tel: 650.493.4400 | Fax: 650.424.0910 | Email: service@annualreviews.orgNew From Annual Reviews:Annual Review of Statistics and Its Application=VS\TL‹6USPUL1HU\HY`‹O[[W!Z[H[PZ[PJZHUU\HSYL]PL^ZVYNEditor: Stephen E. Fienberg!"Carnegie Mellon UniversityAssociate Editors: Nancy Reid!" Jonathan Rougier, Michael Goldstein‹Probabilistic Forecasting, Tilmann Gneiting, Matthias Katzfuss Martin J. Wainwright‹High-Dimensional Statistics with a View Toward Applications in Biology, Peter BŸhlmann, Markus Kalisch, Lukas Meier‹Next-Generation Statistical Genetics: Modeling, Penalization, and Optimization in High-Dimensional Data, Kenneth Lange, Jeanette C. Papp, Janet S. Sinsheimer, Eric M. Sobel‹Breaking Bad: Two Decades of Life-Course Data Analysis in Criminology, Developmental Psychology, and Beyond, Elena A. Erosheva, Ross L. Matsueda, Donatello Telesca‹Event History Analysis, Niels Keiding‹:[H[PZ[PJHS,]HS\H[PVUVM-VYLUZPJ+5(7YVÄSL,]PKLUJLChristopher D. Steele, David J. Balding‹Using League Table Rankings in Public Policy Formation: Statistical Issues, Harvey Goldstein‹Statistical Ecology, Ruth King‹Estimating the Number of Species in Microbial Diversity Studies, John Bunge, Amy Willis, Fiona Walsh‹Dynamic Treatment Regimes, Bibhas Chakraborty,