Why neurons mix high for highercognition - PDF document

1 Why neurons mix: high for highercognitio
Why neurons mix: high for highercognition MillerMattiaRigotti Neurobiologywww.sciencedirect.com informationperceptionthere.However,circuitsoutoverlydeterminemixedselectivityrepresentationsmakingaccessiblefurtherprocessingyardstickdeterminerepresentationscircuitsreasonablyinterpret.this,neuralnetworks.SimplyartiÞ-simplifyingbiologicalprinciplesoutrelevantinformation,conservativeeasilyimplementedweight-operationindividualadvantageinformationpopulationneuronswithmixedcombinationsfactorsratherthanhighlyspecializedselec-tivityÕconsideraxesrepresentsratelinearfactorlinearcombina-twofactors.otherwords,withoutnonlinearselectivity.ratesuchthatitsactivityincreaseslinearlysuchthatactivityvisualcon-trast;activityneuronrelatedeitherthosefactorslinearcombinationfactorspointsrepresentthreevectors)fourdifferentcombina-factors).taskrespondcombinations(shownred)anothertwocombinations(yellow).neuronsÕlinearrelationshipfactors,arereadoutthatseparatestask(redyellow).withlinearneuralreadoutthatcanseparateyellowfromred.readoutthatseparateslargerfactorfromsuchcontrast,notpossibleseparatecombinationssignals.Neurobiology Figure 1 (a)(b)(c)(d)f1f2f3f1f2f3Low-dimensional low separability High-dimensional high separability 1f1f2f2High-dimensionalLow-dimensional high generalization Current Opinion in Neurobiology high-dimensionalneuralneuronalpopulationrepresentrepresentpopulationresponsessensorydimensionalityrepresentationsspecifyrepresentation:panellongerdimensionality.yellowtheyspaces)yellowclearlyprototypicalwell-knowntetrahedron.arrangementdimensionality,Dimensionalityimprovegeneralization.distributionspecificcorrespondingthecloudsdistributeddimensions.directionsuboptimalreadout.classifier dimensionality.betweenhyperplane www.sciencedirect.comOpinionNeurobiology Considerone(Neuronmixedselectivityneuronreßectsnon-linearcombinationotherfactors.neuronrepre-sentationstri

2 dimensionalpointsnowverticestetrahedron.
dimensionalpointsnowverticestetrahedron.oneseparateanyarbitrarycombinationdifferentterminologymachinelearning,ÔshatteredÕredÕ20].Or alternatively,possiblecoloringsclassifyingpointsyellow)implemen-generally,classiÞcationsperformedlinearexponentiallydimensionalityrepresentationsinputs.consequence,linearendowedhigh-dimensionalnumbertask-relatedresponses.general,high-dimensionalrepresentationsrequirenon-linearselectivityillustratedpreviousexample.Neuralrepresentationsspecializedselectivityneuronsselectiveindividualvariable),combinationstask-relevantselectivity)dimensionalexempliÞedimportanthighlyspecializedencodinginformationsegregatedpopulations.However,representationsdimensional,thereforeuseupperbounddimensions).non-linearselectivity,necessarydimensionalitydiversity:differentselectivityshouldresponseproperties.words,shoulddifferentcombinationsfactors.neuralactivationshouldhetero-geneousguaranteeresponsegeneratingtechnicallyknownmeasurediversityresponsesproposedd12], wherethe authorsquantitativelyneuronsposteriorparietalhighlyheterogeneousrecordedperformsmulti-sensoryDiversitynon-linearselectivityessentialingredients,mightnotsufÞcientfactoreddeÞnedcomponentneuronalresponsereproducible,example,ßuctuationsneuralactivityacrossdifferentcorrespondrepetitionsexperimentalcondition.Becausenoise,realisticsituationspointswhererepresentstrial-to-trialdistributionactivityspeciÞccondition.Noiseactivitycompo-dimensionality.However,non-repeatability,componentsdiscriminateresponseacrossconditions.example,pointsseparatingpointsperformedNeurobiologybehavior dimensionalitynumberattainablevectorscorrespondingdimensionalityspecializeddimensionalitythispopulationsdifferentdimensionalitypopulationstogethermaximaldimensionalityClearly,smallervariablestherepresentationsselectivityneuralrepresentationsspecializedselectivitythosethatdimensionalitythedimensionalnumberrecordedd2

3 1]).There are situations in whichunambig
1]).There are situations in whichunambiguouslyexample,ple,22],one of the experiments analyzed in [21]). The animal is required totouch a target on a screenmovementsseveraldirections.reachingmightdimensionalityHowever,theconsideredanglesdistinguishable,anglesdimensionalitypatterns..21]the authors formalized this problem and they essentially proposedthat two conditions correspondingshoulddeclaredcorrespond-neuronalvectorscriterionsensemaximaldimensionality.approximatelyresponse,dimensionalityactualothercorrelationspresent Neurobiologywww.sciencedirect.com Whenpointsdisplacedandrealiza-becomeseparatingHowever,separatingworkspeciÞcwhich(over)Þt,notseparabilityproblemrealizations.noisecomponentsreducediscriminabilitydirection,evenrepresentationshigh-dimensionalIndeed,imaginenoiseßuctuationsdistinctconditionsdifÞcultdiscriminateßuctuationsdistancesectionÔHowmeasuredimensionalityÕdiscusspossibledetermin-dimensionalitypresencenoise.dimensionalrepresentationsnotdesir-Somesituationsactuallyrequiredimensionalityreduction.exampleclassiÞcationwhichbeneÞtrepresentationsclusivelyinformationdiscrimi-distinctclasses,variationsunimportantdimen-illustrateexampledimensionalityreductionwhereconsidertwoneurons,respondingdifferentsourcesoundsrightsoundslouder.soundsgeneratedaccord-distribution,reßecteddistributionsrepre-distributionsdifÞcult,twointensitiesdiscriminate.numbersamplesoundsrepresentedpossibleresultingseparatingoptimalseparatesoundsmisclassiÞed.betterperformanceachievedprojectingpointspriatelydimensionalandreadout.wouldunitesdistributions,reducingdimensionalitydimensionalitycorrespondsfeature,whichdifferenceintensitybetweentwosounds.short,brainneedsreducedimensionalitysamerecastremaininghigh-dimensionalprocessedgeneratecomplexbehavior.Dimensionalityminimalnumbercoordinateneededspecifypositionspointsdimensionalitycontaineddimension-pointsverticestetrahedron,ob

4 ject.formally,dimensionalitydeÞnedmatrix
ject.formally,dimensionalitydeÞnedmatrixwhoserepresentratevectors.rowsequalconditions.deÞnitionsamplingnoise,estimatingtrial-to-trialßuctuationsinßatesdimensionalityestimates.example,wouldpointsmaximalconditions,assumingnumberdiscrim-geometry(high-dimensional).estimatingdimensionalitythattakesnoiseaccountrecentlyrelatedPrincipalComponentent23]. PCA identiÞes the directions alongÞringratespacedistributions(variance).areprincipalcomponentsarecharacterizedvariancethatcancapturedexample,componentswillaxesthatdeÞneplane,orthogonalThevariancelargeÞrstcomponents,zeroforthirdone.dimensionality,simplycomponentsnon-zerovariance.smallÞringrates,thirdfromcouldrightdimensionalitymoderatecuttingsmallestcomponents.formally,theeigenvectorscovariancema-areprincipalcomponents,andcorre-spondingeigenvaluesdeterminevariancethatcomponent.noiselesscase,rankcase,thedimensionalitycanthatarethresholdcanestimatedfromtrial-to-trialneu-ronall23].Notice that this methoddimensionsvariation,whetherexample,principalcomponentactuallyorthogonallinewhichpointsMoreover,inaccurateprovidesbounddimensionality,thresholdestimatedbasiscomponent.Neurobiology www.sciencedirect.comOpinionNeurobiology alternativemethodrelatedcomputationaldimensionalityemployedoyed6]. The fact that apparentincreasesdimensionalitycomponentsgeneralizedifferentnoisesuggestsdimensionalityestimatedcountinglinearlyseparablecoloringsthroughcross-validation.Neurobiologybehavior Figure 2 Variance f1 f3f2 f1f2f3(b)(c)1 2 3 45 678(d)... 3 separable y representations:noiseestimationmethods.responsesconditionscoordinatesthecorrespondstrial-(representedtransparentthatdirectionconditions.thethat,appropriatelychosendirections,signal-to-noise(1,1,1)-directionresponsesincedirectionsactivitypatterns..58–60] for more general consideration regarding when and how noise correlationsdecoding.)separatelyexamplecorrelatedinformation,n,

5 23]) shows that thenoise component is mu
23]) shows that thenoise component is much higher than the signaldirections.componentdirections,populationDespiteappropriateboundarydiscriminatedimensionalitytrial-averagedpatterns:within-conditiondimensionalityimplementedresponseswherethenumberaveragelowerpartitionsclassifier,theplot. Neurobiologywww.sciencedirect.com separationsexploitnon-repeatableponentsrevealedartifactualdifferent(correspondingdifferentestimatedimensionalitycountingseparablecoloringsmaintaincross-validationperformancenoiseadvantagemethodactuallyorientationdistribu-relationdirectionspoints.techniquee23] assumes a worst-casewherelargestcomponentdimensionsdimensionalitywouldanalysis).Quantifyingdimensionalityneuralrepresenta-offersinformativeglimpsehowsensorycognitivebeingprocessedobservationderivestationalunderstandingsensoryhigher-ordercorticalareasthroughmodernmachinemethodsnetworkstworks24]. One of the fundamentalgainedinterdisciplinarysolutionrequireprocessingnaturalstimuli(naturalimages,language)decompositionmultiplehierarchicalelaborationstagesabstraction,pioneeringWieselsel25].The ultimate goal of such stagewise processingrelationhigh-dimensionallow-dimensionaldecisionsvariables.centralstrategynetworksbetweendimensionalityexpansiondimensionalityreductionprocessingprogressesthroughouthierarchy.Dimensionalityreductioncompetingguaranteesbettereralizationenforcinginvarianceiance26–30]. This is impor-tant in object recognition,desirabledifferentviewsobject(say,pose,position,sameclassiÞcationresponse.Geometricallytualizedtransformationcollapseshigh-dimensionalcorrespondingferentobjecthigh-levelrepresentationc,d).resultingrepresen-allowspotentiallyreductionsamplecomplexity,sinceprovidedsamplegeneralizedcorrespond-sameobjectt26,28].Invariancecomputationproposedprinciplerecognitionandhallmarksystemem19,27consistentselectivityhighlyvisualreportedinferotemporalal31].WhatÕs more, dee

6 poptimizedperformrecognitionbeenshownspo
poptimizedperformrecognitionbeenshownspontaneouslygenerateactivationshighlycortexresponseses32].Dimensionality expansion for input separability andoutput ßexibilityAs we pointedourexample,oppositeoperationdimensionalityreduction,dimensionalityimportantcognitionperforminvolveovervariablesDimensionalityexpansiononefundamentaloperationsperformedmodernalgorithms(Supportrt33]) andby deep artiÞcial neural networkstworks24]. It has the functionof guaranteeinghigh-margindiscrimina-generatedistinctinct6,34]). Moreover,high-dimensionalrepresentationscomponentdynamicalmodelsneurallikemachinesines35–40]. Thesebehaviorcorticalrecordingssolvenon-linearlymixingnetworkexternalgeneratinghigh-dimensionalrepresentationscontinuouslyupdatedinformationstimuli.Dimensionalityexpandedprocedureintermediateneuronsback-propagationopagation41]). It can also be achievedengineeringdisplaymixedselectivitysynapticknown).example,selectivityretinalvisualpositiongeneratedÞeldsrepresentpositiondeterminemovementscoordinatesates42–44]. This kind of mixed selectivitystimulusidentitycontextsignalvisuomotorremappingapping45–48].An alternativesurprisinglygeneralapproachhigh-dimensionalrepresentationsnon-linearRandomprojectionsextremelygeneratingmixedselectivityexpandmensionalityty35–39,49]), withoutcompromisinggeneralizelize50,51]. The analysisconnectivityKenyonprojectionsNeurobiology www.sciencedirect.comOpinionNeurobiology strategyexpanddimensionalitysystemdrosophilaila52]. Interestinglydimensionalitysensoryimagingstudiespreservedacrossindividuals.electrophysiologicalstudiesdissimilaritymatrix,whichessentiallycontainspointsratedeterminesdimensionalityinvariant(hyperalignment)ment)53], and acrossspecieses54].Measures of dimensionality and theircorrelation with behaviorRecentexperimentalrecordingsduringexecutioncognitivedimensionalityrepresentationspre-frontalcortexreachesaccommo-stimu

7 lus-responsee6]. Crucially,rodentsprim
lus-responsee6]. Crucially,rodentsprimatesobserveddimensionalityfrontalcollapseserrorsuggestingdimensionalityrecordedrepresentationsimportantcorrectect6,55]. Interestingly,pharmacologicalinterventionalsoshownmensionality.SpeciÞcally,moderateamphet-enhancedseparationhigh-dimensionalcomponentshoweverconcentrationsions56].The collapse of dimensionalityprecedingpossibleexplanations:animalsmistakeperceptionmemorysensoryseen,forgotten)subtleproblemformingmixedcombinationsinformation.ation.6] that the second explanationseemscorrect,identityaccuracyThissuggestsselectivityimportantrepresentinginformationvisualspeciÞcdimensionalutilizeddownstreamcircuitsgeneratebehavior.hypothe-predictscollapsedimensionalityimpairsabilitydownstreamresponse,explainingcorrelationdimensionalitybehavioralshowncollapsedimensionalitynon-linearselectivitycomponentresponseses6].ConclusionsThe cortex is not just a patchworkspecializedneuronprettyuniquepropertiesgoodcomputationalreasonrepresentationspurelectivitygreatlylimitresponsesimplementeddownstreamreadouts.However,stressinghighlyspecializedbeneÞcial.exampledependsvariable,representationsvariable.diversityimportanttheiractivitypopulationparticular,searchconsidernottask-relevantinformationrepresented,represented.whichdependvariables,brainlinkeddimensionality.experimentstestinghypothesissufÞcientlycomplexdimensionalitywidechanges.essentiallydeterminedmaximaldimensionality,depends(essentiallyconditionsexperiment,details)correlationsneuralrepresentations.advisablepossible,whichdifferentconditions,compatibleconditionsufÞcientstatisticsrecordedDimensionalityglobalpropertygeometryrepresentations,studiedmultipatternanalysiss53]). Studyinghigh-dimensionalrepresentationsorganization,whichmixedselectivitymightsuppressiondetermineaccuratelydimension-neuralrepresentationstions57].Conßict of interest statementNothing declared.published

8 periodhighlightedoutstandingattentional1
periodhighlightedoutstandingattentional10.1016/j.neuron.2013.09.015MatsumotoPrefrontalactivitiessortingNeurosciTemporalNeurophysiol:4235-4257.Neurobiologybehavior Neurobiologywww.sciencedirect.com representationsSussilloContext-computationrecurrentprefrontalhttp://dx.doi.org/10.1038/integrateneuronsresponsesobservedhigh-dimensionalBarakimportance:585-590http://dx.doi.org/10.1038/nature12160single-cellrecordingsmemorypopulationanimalHeterogenouspopulationshort-term:916-929dx.doi.org/10.1523/JNEUROSCI.2062-09.2010Neuronal:9424-9430http://dx.doi.org/10.1523/JNEUROSCI.1875-10.2010prefrontal:364-375http://dx.doi.org/10.1016/j.neuron.2013.01.039inferotemporalperirhinaluntanglinginformationNeurosci:1132-1139.themonkeyperformingrepre-amountsinformation.theequivalently,multiplexingsingle-computationsdecision-makingintraparietaldirection-discriminationmanipulatingdecision-making.signalsdisplayheterogeneousdynamicsAK:Neurosciexperimentalprotocolsanalysismethodsmultisensorydecisionsdifferentvelocityfreely-movingMcNaughtonInteractionsthedirectionalhippocampalhttp://markus.lab.uconn.edu/wp-content/uploads/sites/1005/2015/01/Markus1995.pdf.RobitsekHippocampalmemoryoccurringhttp://hsinnamon.web.wesleyan.edu/wescourses/NSB-Psyc255/Readings/17.Hippocampalrepresentationhierarchicallydx.doi.org/10.1016/j.neuron.2014.05.019relevantposition,neuronscombinationorganization.OstojicAbstractrepresentationsprefrontal:869-881http://dx.doi.org/10.1016/networksbrainrecognition?statisticalStatisticsforscienceSpringer-Verlag;braveJD,NeuralreachingseparationprefrontalNeuroscihttp://dx.doi.org/10.1523/JNEUROSCI.3276-09.2010Bengiohttp://dx.doi.org/10.1038/nature14539ReceptivecortexNeuroscirecognitioninvariance2015arXiv:1503.05938.Carreira-PerpinadimensionalityNationalConferenceIntelligencePintobenchmark2013arXiv:1301.3530.Rocha-MirandaBenderinferotemporalPerformance-optimizedresponse

9 soptimizedclassicationactivationshigher
soptimizedclassicationactivationshigherarticialpredictivetheresponsesinteriorSupport-vectorhttp://dx.doi.org/10.1007/BF00994018derivationhttp://dx.doi.org/10.1162/Neurobiology www.sciencedirect.comOpinionNeurobiology http://dx.doi.org/10.1126/science.1091277perturbationshttp://dx.doi.org/10.1162/089976602760407955BuonomanoState-dependentspatiotemporalcorticalnetworkshttp://dx.doi.org/10.1038/nrn2558coherentactivitychaoticNeurondx.doi.org/10.1016/j.neuron.2009.07.018Benrepresentationimportancediversityneuralresponseshttp://dx.doi.org/10.3389/fncom.2010.00024BuonomanonetworksNeurosciback-propagationpropertiesposteriortransformationsSejnowskitransformationsusingfunctionsNeurosciCoordinatethecontextualresponsesBiolhttp://dx.doi.org/10.1371/journal.pcbi.1001091MillernetworksperformancediscriminationsState-dependentcoupledHuangG-B,ZhuQ-Y,SiewC-K:Extremelearningmachine:theoryandapplicationsNeurocomputing2006,:489-501.Rigottisparsenessselectivitygeneralization-discriminationhttp://dx.doi.org/10.1523/JNEUROSCI.2753-12.2013Sompolinskyrepresentations:1213-1226.CaronRutaAbbottAxelconvergenceinputsthemushroom2013,497:113-117Usinganatomicaltracingtechniques,reconstructedconnectivitybetweenglomeruliKenyonDrosophila.Theythateachcellintegratesfromapparentlyglomeruli.TheserandomconnectionscouldexpanddimensionalityglomerularneuralrepresentationsorderfacilitatelearningolfactoryassociationsGuntupalliHankehigh-humancortexDimensionalityobjectinferotemporalcortexDurstewitzdecision-making:e1002057http://dx.doi.org/10.1371/journal.pcbi.1002057Balaguer-BallesterAmphetaminedose-dependentprefrontalcortexdynamicshttp://dx.doi.org/10.1523/JNEUROSCI.2421-14.2015Estimatingdimensionalityworkshop:91-101.correlations,codingNeurosci:358-366http://dx.doi.org/10.1038/nrn1888KanitscheiderInformation-limitinghttp://dx.doi.org/10.1038/nn.3807Neurobiologybehavior Neurobiologywww.sciencedi