UnsupervisedSalienceLearningforPersonRe-identi - PDF document

UnsupervisedSalienceLearningforPersonRe-identi“cationRuiZhaoWanliOuyangXiaogangWangDepartmentofElectronicEngineering,TheChineseUniversityofHongKongrzhao,wlouyang,xgwangHumaneyescanrecognizepersonidentitiesbasedonsomesmallsalientregions.However,suchvaluablesalientinformationisoftenhiddenwhencomputingsimilaritiesofimageswithexistingapproaches.Moreover,manyexist-ingapproacheslearndiscriminativefeaturesandhandledrasticviewpointchangeinasupervisedwayandrequirelabelingnewtrainingdataforadifferentpairofcameraviews.Inthispaper,weproposeanovelperspectiveforper-sonre-identi“cationbasedonunsupervisedsaliencelearn-ing.Distinctivefeaturesareextractedwithoutrequiringidentitylabelsinthetrainingprocedure.First,weapplyadjacencyconstrainedpatchmatchingtobuilddensecor-respondencebetweenimagepairs,whichshowseffective-nessinhandlingmisalignmentcausedbylargeviewpointandposevariations.Second,welearnhumansalienceinanunsupervisedmanner.Toimprovetheperformanceofpersonre-identi“cation,humansalienceisincorporatedinpatchmatchingto“ndreliableanddiscriminativematchedpatches.TheeffectivenessofourapproachisvalidatedonthewidelyusedVIPeRdatasetandETHZdataset.1.IntroductionPersonre-identi“cationhandlespedestrianmatchingandrankingacrossnon-overlappingcameraviews.Ithasmanyimportantapplicationsinvideosurveillancebysavingalotofhumaneffortsonexhaustivelysearchingforapersonfromlargeamountsofvideosequences.However,thisisalsoaverychallengingtask.Asurveillancecameramayobservehundredsofpedestriansinapublicareawithinoneday,andsomeofthemhavesimilarappearance.Thesamepersonobservedindifferentcameraviewsoftenunder-goessigni“cantvariationinviewpoints,poses,cameraset-tings,illumination,occlusionsandbackground,whichusu-allymakeintra-personalvariationsevenlargerthaninter- a2 a3 a5 ( a4 b4 b3 b1 )( b2 ) Figure1.ExamplesofhumanimagematchingandsalienceImagesontheleftoftheverticaldashedblacklinearefromcameraviewandthoseontherightarefromcameraviewUpperpartofthe“gureshowsanexampleofmatchingbasedondensecorrespondenceandweightingwithsaliencevalues,andthelowerpartshowssomepairsofimageswiththeirsaliencemaps.byemployingsupervisedmodels,whichrequiretrainingdatawithidentitylabels.Also,mostofthemrequirelabel-ingnewtrainingdatawhencamerasettingschange,sincethecross-viewtransformsaredifferentfordifferentpairsofcameraviews.Thisisimpracticalinmanyapplicationses-peciallyforlarge-scalecameranetworks.Inthispaper,weproposeanewapproachoflearningdiscriminativeandreli-abledescriptionsofpedestriansthroughunsupervisedlearn-ing.Therefore,ithasmuchbetteradaptabilitytogenerelcameraviewsettings.Inpersonre-identi“cation,viewpointchangeandposevariationcauseuncontrolledmisalignmentbetweenimages.ForexampleinFigure1,thecentralregionofimageisabackpackincameraview,whileitbecomesanarm inimageincameraview.Thusspatiallymisalignedfeaturevectorscannotbedirectlycompared.Inourmethod,patchmatchingisappliedtotacklethemisalignmentprob-lem.Inaddition,basedonpriorknowledgeonpedestrianstructures,someconstraintsareaddedinpatchmatchinginordertoenhancethematchingaccuracy.Withpatchmatch-ing,weareabletoalignthebluetiltedstripeonthehandbagoftheladyinthedashedblackboxesinFigure1.Salientregionsinpedestrianimagesprovidevaluablein-formationinidenti“cation.However,iftheyaresmallinsize,salienceinformationisoftenhiddenwhencomputingsimilaritiesofimages.Inthispaper,meansdistinctfeaturesthat1)areinmakingapersonstand-ingoutfromtheircompanions,and2)arereliablein“ndingthesamepersonacrossdifferentviews.Forexample,inFig-ure1,ifmostpersonsinthedatasetwearsimilarclothesandtrousers,itishardtoidentifythem.However,humaneyesareeasytoidentifythematchingpairsbecausetheyhavedistinctfeatures,e.g.personhasabackpackwithtiltedbluestripes,personhasaredfolderunderherarms,andpersonhasaredbottleinhishand.Thesedistinctfeaturesarediscriminativeindistinguishingonefromothersandrobustinmatchingthemselvesacrossdifferentcameraviews.Intuitively,ifabodypartissalientinonecameraview,itisusuallyalsosalientinanothercam-eraview.Moreover,ourcomputationofsalienceisbasedonthecomparisonwithimagesfromalargescalereferencedatasetratherthanasmallgroupofpersons.Therefore,itisquitestableinmostcircumstances.However,thesedistinctfeaturesmaybeconsideredbyexistingapproachesasout-lierstoberemoved,sincesomeofthey(suchasbaggagesorfolders)donotbelongtobodyparts.Clothesandtrousersaregenerallyconsideredasthemostimportantregionsforpersonre-identi“cation.Aidedbypatchmatching,thesediscriminativeandreliablefeaturesareemployedinthispa-perforpersonre-identi“cation.Thecontributionsofthispapercanbesummarizedinthree-folds.First,anunsupervisedframeworkisproposedtoextractdistinctivefeaturesforpersonre-identi“cationwithoutrequiringmanuallylabeledpersonidentitiesinthetrainingprocedure.Second,patchmatchingisutilizedwithadjacencyconstraintforhandlingthemisalignmentprob-lemcausedbyviewpointchange,posevariationandar-ticulation.Weshowthattheconstrainedpatchmatchinggreatlyimprovespersonre-identi“cationaccuracybecauseofits”exibilityinhandlinglargeviewpointchange.Third,humansalienceislearnedinanunsupervisedway.Differ-entfromgeneralimagesaliencedetectionmethods[4],oursalienceisespeciallydesignedforhumanmatching,andhasthefollowingproperties.1)Itisrobusttoviewpointchange,posevariationandarticulation.2)Distinctpatchesarecon-sideredassalientonlywhentheyarematchedanddistinctinbothcameraviews.3)Humansalienceitselfisausefuldescriptorforpedestrianmatching.Forexample,apersononlywithsalientupperbodyandapersononlywithsalientlowerbodymusthavedifferentidentities.2.RelatedWorkDiscriminativemodelslikeSVMandboosting[25,13,15]arewidelyusedforfeaturelearning.Prosseretal.[25]formulatedpersonre-identi“cationasarankingproblem,andusedensembledRankSVMstolearnpairwisesimilar-ity.Grayetal.[13]combinedspatialandcolorinforma-tioninanensmebleoflocalfeaturesbyboosting.Schwartzetal.[26]extractedhigh-dimensionalfeaturesincludingcolor,gradient,andtexture,andthenutilizedthepartialleastsquare(PLS)fordimensionreduction.Anotherdirec-tionistolearntask-speci“cdistancefunctionswithmetriclearningalgorithms[29,8,24,16].LiandWang[17]parti-tionedtheimagespacesoftwocameraviewsintodifferentcon“gurationsandlearneddifferentmetricsfordifferentlo-callyalignedcommonfeaturespaces.Lietal.[18]pro-posedatransferredmetriclearningframeworkforlearningspeci“cmetricforindividualquery-candidatesettings.Inallthesesupervisedmethods,trainingsampleswithidentitylabelsarerequired.Someunsupervisedmethodshavealsobeendevelopedforpersonre-identi“cation[10,21,22,19].Farenzenaetal[10]proposedtheSymmetry-DrivenAccumulationofLocalFeatures(SDALF).Theyexploitedthepropertyofsymme-tryinpedestrianimagesandobtainedgoodviewinvariance.etal.[21]developedtheBiCovdescriptor,whichcom-binedtheGabor“ltersandthecovariancedescriptortohan-dleilluminationchangeandbackgroundvariations.Mal-etal.[22]employedFisherVectortoencodehigherorderstatisticsoflocalfeatures.Allthesemethodsfocusedonfeaturedesign,butrichinformationfromthedistribu-tionofsamplesinthedatasethasnotbeenfullyexploited.Ourapproachexploitthesalienceinformationamongper-sonimages,anditcanbegeneralizedtotakeuseoftheseSeveralappoachesweredevelopedtohandleposevari-ations[27,11,1,7].Wangetal.[27]proposedshapeandappearancecontexttomodelthespatialdistributionsofap-pearancerelativetobodypartsinordertoextractdiscrimi-nativefeaturesrobusttomisalignment.Gheissarietal.[11]“tatriangluargraphmodel.Baketal.[1]andCheng.[7]adoptedpart-basedmodelstohandleposevariation.However,theseappoachesarenot”exibleenoughandonlyapplicablewhentheposeestimatorsworkaccurately.Ourapproachdiffersfromtheminthatpatchmatchingisem-ployedtohandlespatialmisalignment.Contextualvisualknowledgecomingfromsurroundingpeoplewasusedtoenrichhumansignature[28].Liu.[19]usedanattribute-basedweightingscheme,whichsharedsimilarspiritwithoursaliencein“ndingtheunique andinherentappearanceproperty.Theyclusteredproto-typesinanunsupervisedmanner,andlearnedattribute-basedfeatureimportanceforfeatureweighting.Theirap-proachwasbasedonglobalfeatures.Theyweighteddif-ferenttypesoffeaturesinsteadoflocalpatches.ThereforetheycouldnotpickupsalientregionsasshowninFigure1.Experimentalresultsshowthatourde“nedsalienceismuchmoreeffective.3.DenseCorrespondenceDensecorrepondencehasbeenappliedtofaceandscenealignment[23,20].Inheritingthecharacteristicsofpart-basedandregion-basedapproaches,“ne-grainedmeth-odsincludingoptical”owinpixel-level,keypointfeaturematchingandlocalpatchmatchingareoftenbetterchoicesformorerobustalignment.Inourapproach,consideringmoderateresolutionofhumanimagescapturedbyfar-“eldsurveillancecameras,weadoptthemid-levellocalpatchesformatchingpersons.Toensuretherobustnessinmatching,localpatchesaredenselysampledineachimage.Differentthangeneralpatchmatchingapproaches,asimplebuteffec-tivehorizontalconstraintisimposedonsearchingmatchedpatches,whichmakespatchmatchingmoreadaptiveinper-sonre-identi“cation.3.1.FeatureExtractionDenseColorHistogram.Eachhumanimageisdenselysegmentedintoagridoflocalpatches.ALABcolorhis-togramisextractedfromeachpatch.Torobustlycapturecolorinformation,LABcolorhistogramsarealsocomputedondownsampledscales.Forthepurposeofcombinationwithotherfeatures,allthehistogramsareL2normalized.DenseSIFT.Tohandleviewpointandilluminationchange,SIFTdescriptorisusedasacomplementaryfeaturetocolorhistograms.Thesameasthesettingofextractingdensecolorhistograms,adensegridofpatchesaresampledoneachhumanimage.Wedivideeachpatchintoquantizetheorientationsoflocalgradientsintobins,andobtaina8=128dimentionalSIFTfeature.SIFTfeaturesarealsoL2normalized.DensecolorhistogramsanddenseSIFTfeaturesarecon-catenatedasthe“nalmulti-dimensionaldescriptorvectorforeachpatch.Inourexperiment,theparametersoffea-tureextractionareasfollows:patchesofsizepixelsaresampledonadensegridwithagridstepsizebincolorhistogramsarecomputedinL,A,Bchannelsre-spectively,andineachchannel,levelsofdownsamplingareusedwithscalingfactors;SIFTfeaturesarealsoextractedincolorchannelsandthusproducesafeaturevectorforeachpatch.Inasummary,eachpatchis“nallyrepresentedbyadiscriminativedescriptorvectorwithlength3+1283=672.WedenotethecombinedfeaturevectorasdColorSIFT.3.2.AdjacencyConstrainedSearchInordertodealwithmisalignment,weconductadja-cencyconstrainedsearch.dColorSIFTfeaturesinhumanimagearerepresentedas,whereA,pdenotesthe-thimageincamera,andm,ndenotesthepatchcen-teredatthe-throwandthe-thcolumnofimage.The-throwofimagefromcameraarerepresentedas:,...,NAllpatchesinhavethesamesearchsetforpatchmatchinginimagefromcameraB,qB,qB,qrepresentthecollectionofallpatchfeaturesinfromcamera.Therestrictsthesearchsetinwithinthe-throw.However,boundingboxesproducedbyahumandetectorarenotalwayswellaligned,andalsouncontrolledhumanposevariationsexistinsomeconditions.Tocopewiththespatialvariations,werelaxthestricthorizontalconstrainttohavealargersearchrange.B,qB,ql,...,m,...mde“nesthesizeoftherelaxedadjacentverticalspace.Ifisverysmall,apatchmaynot“ndcorrectmatchduetoverticalmisalignment.Whenissettobeverylarge,apatchintheupperbodywould“ndamatchedpatchonthelegs.Thuslessrelaxedsearchspacecannotwelltoleratethespatialvariationwhilemorerelaxedsearchspaceincreasesthechanceofmatchingdifferentbodyparts.ischoseninoursetting.AdjacencySearching.Generalizedpatchmatchingisaverymaturetechniqueincomputervision.Manyoff-the-shelfmethods[2,3]areavailabletoboosttheperformanceandef“ciency.Inthiswork,wesimplydoaneighborsearchforeachinsearchsetB,qofeveryimageinthereferenceset.ThesearchreturnsthenearestneighborforeachimageaccordingtotheEu-clideandistance.Assuggestedin[23],aggregaingsimi-larityscoresismuchmoreeffectivethanminimizingaccu-mulateddistances,especiallyforthosemisalignedorback-groundpatcheswhichcouldgenerateverylargedistancesduringmatching.Byconvertingtosimilarity,theireffectcouldbereduced.WeconvertdistancevaluetosimilarityscorewiththeGaussianfunction:x,yx,y x,yistheEuclideandistancebe-tweenpatchfeatures,andisthebandwidthof (a) Figure2.Examplesofadjacencysearch.(a)AtestimagefromtheVIPeRdataset.Localpatchesaredenselysampled,and“veexemplarpatchesondifferentbodypartsareshowninredboxes.(b)Onenearestneighborfromeachreferenceimageisreturnedbyadjacencysearchforeachpatchontheleft,andthenneighborsfromreferenceimagesaresorted.Thetoptennearestneighborpatchesareshown.Notethatthetennearestneighborsarefromtendifferentimages.theGaussianfunction.Figure2showssomevisuallysim-ilarpatchesreturnedbythediscriminativeadjacencycon-strainedsearch.4.UnsupervisedSalienceLearningWithdensecorrepondence,welearnhumansaliencewithunsupervisedmethods.Inthispaper,weproposetwomethodsforlearninghumansalience:theNeighbor(KNN)andOne-ClassSVM(OCSVM).4.1.K-NearestNeighborSalienceetal.[5]foundtheKNNdistancescanbeusedforclutterremoval.ToapplytheKNNdistancetopersonre-identi“cation,wesearchfortheK-nearestneighborsofatestpatchintheoutputsetofthedensecorrespon-dence.Withthisstrategy,salienceisbetteradaptedtore-identi“cationproblem.Followingthesharedgoalofabnormalitydetectionandsaliencedetection,werede“nethesalientpatchinourtaskasfollows:Salienceforpersonre-identi“cationsalientpatchesarethosepossessuniquenesspropertyamongaspeci“cset.DenotethenumberofimagesinthereferencesetbyAfterbuildingthedensecorrespondecesbetweenatestim-ageandimagesinreferenceset,themostsimilarpatchineveryimageofthereferencesetisreturnedforeachtest.,eachtestpatchhaveneighborsinset,...,NB,qisthesearchsetinEq.(3),isthesimilarityscorefunctioninEq.(4). Figure3.Illustrationofsalientpatchdistribution.patchesaredistributedfarwayfromotherpathes.Weapplyasimilarschemein[5]toofeachtestpatch,andtheKNNdistanceisutilizedtode“nethesaliencescore:knndenotesthedistanceofthe-thnearestneighbor.Ifthedistributionofthereferencesetwellrelectsthetestscenario,thesalientpatchescanonly“ndlimitednumber)ofvisuallysimilarneighbors,asshowninFig-ure3(a),andthenknnisexpectedtobelarge.isaproportionparameterrelectingourexpecta-tiononthestatisticaldistributionofsalientpatches.Sincedependsonthesizeofthereferenceset,thede“nedsaliencescoreworkswellevenifthereferencesizeisverylarge.ChoosingtheValueofThegoalofsaliencedetec-tionforpersonre-identi“catioinistoidentifypersonswithuniqueappearance.Weassumethatifapersonhassuchuniqueappearance,morethanhalfofthepeopleintheref-erencesetaredissimilarwithhim/her.Withthisassump-isusedinourexperiment.Forseekingamoreprincipledmethodtocomputehumansalience,one-classSVMsalienceisdiscussioninSection4.2.Toqualitativelycomparewithsophiscatedsupervisedlearningmethods,Figure4(a)showsthefeatureweightingmapestimatedbypartialleastsquare(PLS)[26].PLSisusedtoreducethedimensionalityandtheweightsofthe“rstprojectionvectorareshownastheaverageofthefea-tureweightsineachblock.OurresultsofunsupervisedKNNsalienceareshowinFigure4(b)ontheETHZdatasetand4(c)ontheVIPeRdataset.Saliencescoresareassignedtothecenterofpatches,andthesaliencemapisupsampledforbettervisualization.Ourunsupervisedlearningmethodbettercapturesthesalientregions.4.2.One-classSVMSalienceOne-classSVM[14]hasbeenwidelyusedforoutlierdetection.Onlypositivesamplesareusedintraining.Thebasicideaofone-classSVMistouseahyperspheretode-scribedatainthefeaturespaceandputmostofthedatainto thehypersphere.Theproblemisformulatedintoanobjec-tivefunctionasfollows: ,...listhemulti-dimensionalfeaturevectoroftrainingsampleisthenumberoftrainingsamples,aretheradiusandcenterofthehypersphere,andand,1]isatrade-offparameter.Thegoalofoptimizingtheobjectivefunctionistokeepthehypersphereassmallaspossibleandincludemostofthetrainingdata.Theopti-mizationproblemcanbesolvedinadualformbyQPopti-mizationmethods[6],andthedecisionfunctionis:X,Xaretheparametersforeachconstraintinthedualproblem.Inourtask,weusetheradiusbasisfunc-tion(RBF)X,Y)=expaskernelinone-classSVMtodealwithhigh-dimensional,non-linear,multi-modedistributions.Asshownin[6],thedecisionfunctionofkernelone-classSVMcanwellcapturetheden-sityandmodalityoffeaturedistribution.ToapproximatetheKNNsaliencealgorithm(Section4.1)inanonparamet-ricform,thesailencescoreisre-de“nedintermsofkernelone-classSVMdecisionfunction:ocsvm=argmaxistheEuclideandistancebetweenpatchfeatures.Ourexperimentsshowverysimilarresultsinpersonre-identi“cationwiththetwosaliencedetectionmethods.ocsvmperformsslightlybetterthanknninsome5.Matchingforre-identi“cationDensecorrespondenceandsaliencedescribedinSection3and4areusedforpersonre-identi“cation.5.1.Bi-directionalWeightedMatchingAbi-directionalweightedmatchingmechanismisde-signedtoincorporatesalienceinformationintodensecorre-spondencematching.First,weconsidermatchingbetweenapairofimages.AsmentionedinSection4.1,patch Figure4.Qualitativecomparisononsalience.(a)showsthefea-tureweightingmapsestimatedbypartialleastsquare[26].(b)showsourKNNsalienceestimation.Redindicateslargeweights.matchedtoB,qwithinsearchrangeB,qDenotethenearestneighborproducedbydensecorrespon-dencealgorithmasB,q=argmaxThensearchingforthebestmatchedimageinthegallerycanbeformulatedas“ndingthemaximalsimilarityscore.=argmaxB,qB,qarecollectionofpatchfeaturesintwoimages,,andB,qB,q,andthesimilaritybetweentwoimageiscomputedwithabi-directionalweightingmechanismil-lustratedinFigure5.Intuitively,imagesofthesameper-sonwouldbemorelikelytohavesimilarsaliencedistri-butionsthanthoseofdifferentpersons.Thus,thediffer-enceinsaliencescorecanbeusedasapenaltytothesim-ilarityscore.Inanotheraspect,largesaliencescoresareusedtoenhancethesimilarityscoreofmatchedpatches.Fi-nally,weformulatethebi-directionalweightingmechanismasfollows:B,qknnB,qknnB,q knnknnB,q Figure5.Illustrationofbi-directionalweightingforpatchmatching.Patchesinredboxesarematchedindensecorrespon-dencewiththeguidenceofcorrespondingsaliencescoresindarkblueboxes.isaparametercontrollingthepenaltyofsaliencedifference.Onecanalsochangethesaliencescoretoocsvminamoreprincipledframeworkwithoutchoos-ingtheparameterinEq.(5).5.2.CombinationwithexistingapproachesOurapproachiscomplementarytoexistingapproaches.Inordertocombinethesimilarityscoresofexistingap-proacheswiththesimilarityscoreinEq.(11),thedistancebetweentwoimagescanbecomputedasfollows:eSDCSDCB,qistheweightforthethdistancemeasureSDCtheweightforourapproach.correspondtothedistancemeasuresandfeatures(wHSVandMSCR)in[10].Intheexperiment,arechosenthesameasin[10].SDCis“xedas6.ExperimentsWeevaluatedourapproachontwopubliclyavailabledatasets,theVIPeRdataset[12],andtheETHZdataset[26].Thesetwodatasetsarethemostwidelyusedforevaluationandre”ectmostofthechallengesinreal-worldpersonre-identi“cationapplications,e.g.,viewpoint,pose,andilluminationvariation,lowresolution,backgroundclutter,andocclusions.TheresultsareshowinstandardCumulatedMatchingCharacteristics(CMC)curve[27].Comparisonstothestate-of-the-artfeaturebasedmethodsareprovided,andwealsoshowthecomparisonwithsomeclassicalmetriclearningalgorithms.VIPeRDataset[12].TheVIPeRdatasetiscapturedbytwocamerasinoutdooracademicenvironmentwithtwoimagesforeachpersonsseenfromdifferentviewpoints. TheVIPeRdatasetisavailabletodownloadatthewebsiteItisoneofthemostchallengingpersonre-identi“cationdatasets,whichsuffersfromsigni“cantviewpointchange,posevariation,andilluminationdifferencebetweentwocameraviews.Itcontains632pedestrianpairs,eachpaircontainstwoimagesofthesameindividualseenfromdif-ferentviewpoints,onefromCAMAandanotherfromCAMB.Allimagesarenormalizedtoforex-periments.CAMAcapturedimagesmainlyfrom0degreeto90degreewhileCAMBmostlyfrom90degreeto180degree,andmostoftheimagepairsshowviewpointchangelargerthan90degree.Followingtheevaluationprotocolin[13],werandomlysamplehalfofthedataset,.,316imagepairs,fortraining(however,theidentityinformationisnotused),andthere-mainingfortest.Inthe“rstround,imagesfromCAMAareusedasprobeandthosefromCAMBasgallery.Eachprobeimageismatchedwitheverygalleryimage,andthecor-rectlymatchedrankisobtained.Rank-recognitionrateistheexpectationofthematchesatrank,andtheCMCcurveisthecumulatedvaluesofrecognitionrateatallranks.Af-terthisround,theprobeandgalleryareswitched.WetaketheaverageofthetworoundsofCMCcurvesastheresultofonetrial.10trialsofevaluationarerepeatedtoachievestablestatistics,andtheaverageresultisreported.SinceELF[13],SDALF[10],andLDFV[22]havepublishedtheirresultsontheVIPeRdataset,theyareusedforcomparison.Thesplittingassignmentsintheseapproachesareusedinourexperiments.Figure6reportthecomparisonresults.Itisobservedthatourtwosaliencedetectionbasedmethods(SDC knnandSDC outperformallthethreebenchmarkingapproaches.Inpar-ticular,rank1matchingrateisaround24%forSDC and25%forSDC ocsvm,versus20%forSDALF,15%forLDFV,and12%forELF.Thematchingrateatrank10isaround52%forSDC knn,and56%forSDC versus49%forSDALF,48%forLDFV,and44%forELF.Theimprovementisduetotwoaspectsofourapproach.First,thedensecorrespondecematchingcantoleratelargerextentofposeandappearancevariations.Second,weincorporatehumansalienceinformationtoguidedensecorrespondence.Bycombiningwithotherdescriptors,therank1matchingrateofeSDC knngoesto26.31%and ocsvmgoesto26.74%.Thisshowsthecomple-mentarityofourSDCapproachtootherfeatures.MorecomparisonresultsareshowinTable1.Thecomparedmethodsincludestheclassicalmetriclearningapproaches,suchasLMNN[29],andITML[29],andtheirvariantsmodi“edforpersonre-identi“cation,suchasPRDC[29],attributePRDC(denotedasaPRDC)[19],andPCCA[24]. ThesplittingassignmentofSDALFcanbefoundintheircodeathttp://www.lorisbazzani.info/code-datasets/sdalf-descriptor/ Method r=1r=5r=10r=20 LMNN[29] 6.2319.6532.6352.25 ITML[29] 11.6131.3945.7663.86 PRDC[29] 15.6638.4253.8670.09 aPRDC[19] 16.1437.7250.9865.95 PCCA[24] 19.2748.8964.9180.28 ELF[13] 12.0031.0041.0058.00 SDALF[10] 19.8738.8949.3765.73 CPS[7] 21.8444.0057.2171.00 eBiCov[21] 20.6642.0056.1868.00 eLDFV[22] 22.3447.0060.0471.00 eSDC knn 26.3146.6158.8672.77 eSDC ocsvm 26.7450.7062.3776.36 Table1.VIPeRdataset:toprankedmatchingratesin[%]with316persons. 5 10 15 20 25 0 10 20 30 40 50 60 70 80 Cumulative Matching Characteristic (CMC)RankMatching Rate (%) SDALF SDC_knn SDC_ocsvm eSDC_knn eSDC_ocsvm Figure6.PerformanceontheVIPeRdataset.Ourapproach: knnandSDC ocsvm.OurapproachcombinedwithwHSVandMSCR[10]:eSDC knnandeSDC ETHZDataset[9].Thisdatasetcontainsthreevideose-quencescapturedfrommovingcameras.Itcontainsalargenumberofdifferentpeopleinuncontrolledconditions.Withthesevideossequences,Schwartz,etal.[26]extractedasetofimagesforeachpeopletotesttheirPartialLeastSquaremethod.Sincetheoriginalvideosequencesarecapturedfrommovingcameras,imageshavearangeofvariationsinhumanappearanceandillumination,andsomeevensufferfromheavyocclusions.Followingthesettingsin[26],allimagesamplesarenormalizedtopixels,andthedatasetisstructuredasfollows:SEQ.#1contains83per-sons(4,857images);SEQ.#2contains35persons(1,936images);SEQ.#3contains28persons(1,762images).Thesamesettingsofexperimentsin[10,26]arerepro-ducedtomakefaircomparisons.Similartothem,weuseasingle-shotevaluationstrategy.Foreachperson,oneim- TheETHZdatasetisavailabletodownloadatthewebsiteageisrandomlyselectedtobuildgallerysetwhiletherestimagesformtheprobeset.Eachimageinprobeismatchedtoeverygalleryimageandthecorrectmatchedrankisob-tained.Thewholeprocedureisrepeatedfor10times,andtheaverageCMCcurvesareplottedinFigure7.AsshowninFigure7,ourapproachoutperformsthethreebenchmarkingmethods,PLS,SDALFandeBiCov[21]onallthreesequences.Comparisonswithsu-pervisedlearningmethodsPLSandRPLMarereportedinTable2.OnSEQ.#2andSEQ.#3,oureSDC knnand ocsvmoutperformsallothermethods.OnSEQ.#1,ourSDCapproachhasbetterresultsthansupervisedmeth-ods,PLSandRPLM,andhascomparableperformancewiththerecentlyproposedeLDFV[22].7.ConclusionInthiswork,weproposeanunsupervisedframeworkwithsaliencedetectionforpersonre-identi“cation.Patchmatchingisutilizedwithadjacencyconstraintforhandlingtheviewpointandposevariation.Itshowsgreat”exibilityinmatchingacrosslargeviewpointchange.Humansalienceisunsupervisedlylearnedtoseekfordiscriminativeandre-liablepatchmatching.Experimentsshowthatourunsuper-visedsaliencelearningapproachgreatlyimprovetheper-formanceofpersonre-identi“cation.8.AcknowledgementThisworkissupportedbytheGeneralResearchFundsponsoredbytheResearchGrantsCouncilofHongKong(ProjectNo.CUHK417110andCUHK417011)andNa-tionalNaturalScienceFoundationofChina(ProjectNo.References[1]S.Bak,E.Corvee,F.Bremond,M.Thonnat,etal.Personre-identi“cationusingspatialcovarianceregionsofhumanbodyparts.InAVSS,2010.[2]C.Barnes,E.Shechtman,A.Finkelstein,andD.Goldman.Patchmatch:arandomizedcorrespondencealgorithmforstructuralimageediting.TOG,2009.[3]C.Barnes,E.Shechtman,D.Goldman,andA.Finkelstein.Thegeneralizedpatchmatchcorrespondencealgorithm.In,2010.[4]A.BorjiandL.Itti.Exploitinglocalandglobalpatchraritiesforsaliencydetection.In,2012.[5]S.ByersandA.Raftery.Nearest-neighborclutterremovalforestimatingfeaturesinspatialpointprocesses.JournaloftheAmericanStatisticalAssociation,1998.[6]Y.Chen,X.Zhou,andT.Huang.One-classsvmforlearninginimageretrieval.In,2001.[7]D.Cheng,M.Cristani,M.Stoppa,L.Bazzani,andV.Murino.Custompictorialstructuresforre-identi“cation.,2011. 1 2 3 4 5 6 7 60 70 80 90 100 RankMatching Rate (%)Cumulated Matching Characteristics (CMC) SDALF eBiCov eSDC_knn eSDC_ocsvm SEQ.#1 2 3 4 5 6 7 60 65 70 75 80 85 90 95 RankMatching Rate (%)Cumulated Matching Characteristics (CMC) SDALF eBiCov eSDC_knn eSDC_ocsvm SEQ.#2 2 3 4 5 6 7 70 75 80 85 90 95 100 RankMatching Rate (%)Cumulated Matching Characteristics (CMC) SDALF eBiCov eSDC_knn eSDC_ocsvm Figure7.PerformancescomparisonusingCMCcurvesonSEQ.#1,SEQ.#2,andSEQ.#3oftheETHZdataset.Accordingto[10],onlythe“rst7ranksareshown.Allthecomparedmethodsarereportedundersingle-shotsetting. Method SEQ.#1 SEQ.#2 SEQ.#3 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 PLS[26] 79 85 86 87 88 89 90 74 79 81 83 84 85 87 77 81 82 84 85 87 89 RPLM[16] 77 83 87 90 91 92 92 65 77 81 82 86 89 90 83 90 92 94 96 96 97 SDALF[10] 65 73 77 79 81 82 84 64 74 79 83 85 87 89 76 83 86 88 90 92 93 eBiCov[21] 74 80 83 85 87 88 89 71 79 83 86 88 90 91 82 87 90 92 93 94 95 eLDFV[22] 83 87 90 91 92 93 94 79 85 88 90 92 93 94 91 94 96 97 97 97 97 eSDC knn 81 86 89 90 92 93 94 79 84 87 90 91 92 93 90 95 96 97 98 98 99 eSDC ocsvm 80 85 88 90 91 92 93 80 86 89 91 93 94 95 89 94 96 97 98 98 99 Table2.Matchingratesin[%]onthedataset.Ourapproach(eSDC knnandeSDC ocsvm)iscomparedwithsupervisedlearningmethodsPLSandRPLM,andunsupervisedmethodsSDALF,eBiCov,andeLDFV.Inaccordancewithwhatreportedinothermethods,onlythematchingratesatthe“rst7ranksareshown.[8]M.Dikmen,E.Akbas,T.Huang,andN.Ahuja.Pedestrianrecognitionwithalearnedmetric.ACCV,2011.[9]A.Ess,B.Leibe,andL.VanGool.Depthandappearanceformobilesceneanalysis.In,2007.[10]M.Farenzena,L.Bazzani,A.Perina,V.Murino,andM.Cristani.Personre-identi“cationbysymmetry-drivenac-cumulationoflocalfeatures.In,2010.[11]N.Gheissari,T.Sebastian,andR.Hartley.Personreidenti“-cationusingspatiotemporalappearance.In,2006.[12]D.Gray,S.Brennan,andH.Tao.Evaluatingappearancemodelsforrecognition,reacquisition,andtracking.In[13]D.GrayandH.Tao.Viewpointinvariantpedestrianrecogni-tionwithanensembleoflocalizedfeatures.,2008.[14]K.Heller,K.Svore,A.Keromytis,andS.Stolfo.Oneclasssupportvectormachinesfordetectinganomalouswindowsregistryaccesses.InWorkshoponDataMiningforComputerSecurity(DMSEC),2003.[15]M.Hirzer,C.Beleznai,P.Roth,andH.Bischof.Personre-identi“cationbydescriptiveanddiscriminativeclassi“ca-ImageAnalysis,2011.[16]M.Hirzer,P.Roth,M.Kostinger,andH.Bischof.Relaxedpairwiselearnedmetricforpersonre-identi“cation.[17]W.LiandX.Wang.Locallyalignedfeaturetransformsacrossviews.In,2013.[18]W.Li,R.Zhao,andX.Wang.Humanreidenti“cationwithtransferredmetriclearning.InACCV,2012.[19]C.Liu,S.Gong,C.Loy,andX.Lin.Personre-identi“cation:Whatfeaturesareimportant?In,2012.[20]C.Liu,J.Yuen,andA.Torralba.Sift”ow:Densecorrespon-denceacrossscenesanditsapplications.TPAMI,2011.[21]B.Ma,Y.Su,andF.Jurie.Bicov:anovelimagerepresenta-tionforpersonre-identi“cationandfaceveri“cation.2012.[22]B.Ma,Y.Su,andF.Jurie.Localdescriptorsencodedby“shervectorsforpersonre-identi“cation.2012.[23]K.MaandJ.Ben-Arie.Vectorarraybasedmulti-viewfacedetectionwithcompoundexemplars.In,2012.[24]A.MignonandF.Jurie.Pcca:Anewapproachfordistancelearningfromsparsepairwiseconstraints.In,2012.[25]B.Prosser,W.Zheng,S.Gong,T.Xiang,andQ.Mary.Per-sonre-identi“cationbysupportvectorranking.In[26]W.SchwartzandL.Davis.Learningdiscriminativeappearance-basedmodelsusingpartialleastsquares.InBrazilianSymposiumonComputerGraphicsandImagePro-cessing(SIBGRAPI),2009.[27]X.Wang,G.Doretto,T.Sebastian,J.Rittscher,andP.Tu.Shapeandappearancecontextmodeling.In,2007.[28]W.Zheng,S.Gong,andT.Xiang.Associatinggroupsofpeople.In,2009.[29]W.Zheng,S.Gong,andT.Xiang.Personre-identi“cationbyprobabilisticrelativedistancecomparison.In,2011.