/
peoplevaryintheir peoplevaryintheir

peoplevaryintheir - PDF document

stefany-barnette
stefany-barnette . @stefany-barnette
Follow
353 views
Uploaded On 2015-11-22

peoplevaryintheir - PPT Presentation

000204060810 False accept rateVerification rate 00010010110 Bad 005 016 047 000204060810 False accept rateVerification rate 00010010110 005 017 abFig9ROCfortheLRPCAocul ID: 201511

0.00.20.40.60.81.0 False accept rateVerification rate 0.0010.010.11.0 Bad 0.05 0.16 0.47 0.00.20.40.60.81.0 False accept

Share:

Link:

Embed:

Download Presentation from below link

Download Pdf The PPT/PDF document "peoplevaryintheir" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

peoplevaryintheirrecognizability.ŽTocontrolforthissourceofvariability,thefaceimagesofthesamepeopleareineachpartition.Inaddition,eachpartitionhasthesamenumberofimagesofeachperson.Becausethepartitiondesigncontrolsforvariationintherecognizabilityoffaces,thedifferencesinperformanceamongthethreepartitionsarearesultofhowafaceispresentedineachimage,andwiththepairsoffacesthatarematched.II.GENERATIONOFTHEGLYARTITIONSTheGBUpartitionswereconstructedfromtheNotreDamemulti-biometricdatasetusedintheFRVT2006[4].Theimagesforthepartitionswereselectedfromasupersetof9,307imagesof570subjects.Alltheimagesinthesupersetarefrontalstillfaceimagescollectedeitheroutsideorwithambientlightinginhallways.Theimageswereacquiredwitha6Mega-pixelNikonD70camera.Allphotosweretakeninthe2004-2005academicyear(Aug2004throughMayEachpartitionintheGBUisspeci“edbytwosetsofimages,atargetsetandaqueryset.Foreachpartition,analgorithmcomputesasimilarityscorebetweenallpairsofimagesinthatpartitionstargetandquerysets.Asimilarityscoreisameasureofthesimilaritybetweentwofaces.Ahighersimilarityscoresimpliesgreaterlikelihoodthatthefaceimagesareofthesameperson.Ifanalgorithmreportsadistancemeasure,thenasmallerdistancemeasureimpliesgreaterlikelihoodthatthefaceimagesareofthesameperson.Adistancemeasureisconvertedtoasimilarityscorebymultiplyingbyminusone.Thesetofallsimilarityscoresbetweenatargetandaquerysetiscalledasimilaritymatrix.Apairoffaceimagesofthesamepersoniscalledamatchpair;andapairoffaceimagesofdifferentpeopleiscalledanon-matchpair.Fromthesimilaritymatrix,receiveroperatingcharacteristics(ROC)andothermeasuresofperformancecanbecomputed.ToconstructtheGBUChallengeProblemwesoughttospecifytargetandquerysetsforeachofthethreepartitionssuchthatrecognitiondif“cultywouldvarymarkedlywhileatthesametimefactorssuchastheindividualpeopleinvolvedornumberofimagesperpersonremainedthesame.Togaugetherelativedif“cultyassociatedwithrecognizingapairofimages,similarityscoreswerecreatedbyfusingscoresfromthreeofthetopperformingalgorithmsintheFRVT2006;thisfusionprocessisdescribedmorefullyinthenextsection.ThefollowingconstraintswereimposedwhenselectingtheGBUpartitions:DistinctImages:Animagecanonlybeinonetargetorqueryset.Balancedsubjectcounts:Thenumberofimagesperpersonarethesameinalltargetandquerysets.Differentdays:Theimagesinallmatchpairsweretakenondifferentdays.Afterapplyingtheseconstraints,andgiventhetotalnumberofimagesavailable,thenumberofimagesperpersoninthetargetandquerysetswasselectedtofallbetween1and4.Thisnumberdependeduponthetotalavailabilityofimagesforeachperson.Theselectioncriteriaforthepartitionresultsinthefollow-ingproperties.Animageisonlyinonepartition.Therearethesamenumberofmatchfacepairsineachpartitionandthesamenumberofnon-matchpairsbetweenanytwosubjects.Thisimpliesthatanydifferenceinperformancebetweenthepartitionsisnotaresultofdifferentpeople.Thedifferenceinperformanceisaresultofthedifferentconditionsunderwhichtheimageswereacquired.Figures1,2,and3,areexamplesofmatchingfacepairsfromeachofthepartitions.TheimagesincludedintheGBUtargetandquerysetsweredecidedindependentlyforeachperson.Foreachsubject,asubject-speci“csimilaritymatrixisextractedfromalargermatrixcontainingsimilarityscoresfromtheFRVT2006fusionalgorithm.Eachsubject-speci“cmatrixcontainsallsimilarityscoresbetweenpairsofimagesofsubject.FortheGoodpartition,agreedyselectionalgorithmiterativelyaddedmatchfacepairsforsubjectthatmaximizedtheaveragesimilarityscoreforsubject;fortheUglypartition,matchfacepairswereselectedtominimizetheaveragesimilarityscoreforsubject;andfortheBadpartition,facepairsforsubjectwereselectedtomaintainanapproximatelyaveragesimilarityscore.Theselectionprocessforeachsubjectwasrepeateduntilthedesirednumberofimageswereselectedforthatsubject.Sincetheimagesforeachsubjectareselectedindependently,thesimilarityscoreassociatedwithagoodfacepaircanvaryfromsubjecttosubject(similarlyfortheBadandUglypartitions).EachoftheGBUtargetandquerysetscontains1,085imagesfor437distinctpeople.Thedistributionofimagecountsperpersoninthetargetandquerysetsare117subjectswith1image;122subjectswith2images;68subjectswith3images;and130subjectswith4images.Ineachpartitionthereis3,297matchfacepairsand1,173,928non-matchfacepairs.IntheGBUimageset58%ofthesubjectsweremaleand42%female;and69%ofthesubjectswereCaucasian,22%eastAsian,4%Hispanic,andtheremaining5%othergroups;and94%ofthesubjectswerebetween18and30yearsoldwiththeremaining6%over30yearsold.FortheimagesintheGBU,theaveragedistancebetweenthecentersoftheeyesis175pixelswithastandarddeviationof36pixels.III.TFRVT2006FPerformanceresultsfortheGBUChallengeProblemarereportedfortheGBUFRVT2006fusionalgorithm,whichisafusionofthreeofthetopperformersintheFRVT2006.Thealgorithmswerefusedinatwo-stepprocess.Inthe“rststep,foreachalgorithm,themedianandthemedianabsolutedevi-ation(MAD)wereestimatedfromevery1in1023similarityscores(andMADarethemedianandMADfor).ThemedianandMADwereestimatedfrom1in1023similarityscorestoavoidovertuningtheestimatesto 0.00.20.40.60.81.0 False accept rateVerification rate 0.0010.010.11.0 Bad 0.05 0.16 0.47 0.00.20.40.60.81.0 False accept rateVerification rate 0.0010.010.11.0 0.05 0.17 (a)(b)Fig.9.ROCfortheLRPCA-ocularbaselinealgorithmontheGood,theBad,&theUglypartitions.In(a)performanceisfortheleftocularregionthatconsistsofthelefteyeandtwolefteye-browregions;performancein(b)isforcorrespondingrightocularregions.Theveri“cationrateforeachparataFARof0.001ishighlightedbytheverticallinesatFAR=0.001. Fig.6.This“gureshowsacroppedfaceandthethirteenlocalregions.Thecropfacehasbeengeometricallynormalizedandtheselfquotientprocedure Fig.7.This“gureillustratesthecomputationofaself-quotientfaceimage.Thefaceimagetotheleftisacroppedandgeometricallynormalizedimage.TheimageinthemiddleisthegeometricallynormalizedimageblurredbyaGaussiankernel.Theimageontheleftisaself-quotientimage.Thisimageisobtainedbypixel-wisedivisionofthenormalizedimagebytheblurredthecoordinatesofthecentersoftheeyesareprovided;inthefullyautomaticmode,thecentersoftheeyesarelocatedbythebaselinealgorithm.IntheLRPCAalgorithm,thePCArepresentationisbasedonthirteenlocalregionsandthecompletefacechip.Thethirteenlocalregionsarecroppedoutofanormalizedfaceimage.Someofthelocalregionsoverlap,seeFigure6.Thelocalregionsarecenteredrelativetotheaveragelocationoftheeyes,eyebrows,noseandmouth.Thenextstepnormalizesthe14faceregionstoattenuatevariationinillumination.First,selfquotientnormalizationisindependentlyappliedtoeachofthe14regions[9].Theselfquotientnormalizationprocedure“rstsmootheseachregionbyconvolvingitwithatwo-dimensionalGaussiankernelandthendividestheoriginalregionbythesmoothedregion,seeFigure7.Inthe“nalnormalizationstep,thepixelvaluesineachregionarefurtheradjustedtohaveasamplemeanofzeroandasamplestandarddeviationofone.Duringtraining,14distinctPCAsubspacesarecon-structed,oneforeachofthefaceregions.FromeachPCAde-composition,the3through252eigenvectorsareretainedtorepresenttheface.Thedecisiontousetheseeigenvectorswasbaseduponexperimentsonimagessimilartotheimages 0.00.20.40.60.81.0 False accept rateVerification rate 0.0010.010.11.0 0.07 0.24 Fig.8.ROCfortheLRPCAbaselinealgorithmontheGBUpartitions.Theveri“cationrateforeachpartitionataFARof0.001ishighlightedbytheverticallineatFAR=0.001.intheGBUChallengeProblem.Aregioninafaceisencodedbythe250coef“cientscomputedbyprojectingtheregionontotheregions250eigenvectors.Afaceisencodedbyconcatenatingthethe250coef“cientsforeachofthe14regionsintoanewvectoroflength3500.EachdimensioninthePCAsubspaceisfurtherscaled.First,therepresentationiswhitenedbyscalingeachdimen-siontohaveasamplestandarddeviationofoneonthetrainingset.Next,theweightoneachdimensionisfurtheradjustedbasedonFisherscriterion.Thisweightiscomputedbasedontheimagesinthetrainingset.TheFisherscriterionweightemphasizesthedimensionsalongwhichimagesofdifferentpeoplearespreadapart.Theweightattenuatesthedimensionsalongwhichtheaveragedistancebetweenimagesofthesamepersonandimagesofdifferentpeopleareroughlythesame.Whenusedforrecognition,i.e.duringtesting,imagesare“rstprocessedasdescribedaboveandthenprojectedintothe14distinctPCAsubspacesassociatedwitheachofthe14regions.Thecoordinatesofimagesprojectedintothesespaces,250foreachofthe14regions,arethenconcatenatedintoasinglefeaturevectorrepresentingtheappearanceofthatface.Thisproducesonevectorperfaceimage;eachvectorcontains3,500values.ThebaselinealgorithmmeasuressimilaritybetweenpairsoffacesbycomputingthePearsonscorrelationcoef“cientbetweenpairsofthesevectors.TheperformanceofthebaselinealgorithmontheGBUChallengeProblemissummarizedinFigure8.AcomparisonofperformanceoftheFusionandtheLRPCA-baselinealgorithmisgiveninTableI.Arecentareaofinterestinfacerecognitionandbio- 0.10.00.10.20.3 MoreLessMatchNonmatchSimilarity 0.10.00.10.20.3 MatchNonmatch 0.10.00.10.20.3 MatchNonmatch Ugly Fig.5.Histogramofthematchandnon-matchdistributionsfortheGood,theBad,&theUglypartitions.Thegreenbarsrepresentthematchdistributioandtheyellowbarsrepresentthenon-matchdistribution.Thehorizontalaxesindicaterelativefrequencyofsimilarityscores. 0.00.20.40.60.81.0 False accept rateVerification rate 0.0010.010.11.0 Bad 0.15 0.80 Fig.4.ROCfortheFusionalgorithmontheGood,theBad,&theUglypartitions.Theveri“cationrateforeachpartitionataFARof0.001ishighlightedbytheverticallineatFAR=0.001.imagesofothersubjectsintheGBUChallengeProblem.Anexampleisanalgorithmbasedonperson-speci“cPCArepresentations.Inthisexample,duringthegeometricnor-malizationprocess,20slightlydifferentnormalizedversionsoftheoriginalfacewouldbecreated.Aperson-speci“cPCArepresentationisgeneratedfromthesetof20normalizedfaceimages.ThismethodconformswiththeGBUtrainingprotocolbecausethe20faceimagesandthepersonspeci“cPCArepresentationarefunctionsoftheoriginalsinglefaceimage.Whentherearemultipleimagesofapersoninatargetorqueryset,thisapproachwillgeneratemultipleimage-speci“crepresentations.ThistrainingproceduredoesnotintroduceanydependenceuponotherimagesinthetargetsetandconsequentlyispermittedbytheGBUprotocol.V.BTheGBUChallengeProblemincludesabaselinefacerecognitionalgorithmasanentrypointforresearchers.Thebaselineservestwopurposes.First,itprovidesaworkingexampleofhowtocarryouttheGBUexperimentsfollowingtheprotocol.Thisincludestraining,testingandevaluationusingROCanalysis.Second,itprovidesaperformancestan-dardforalgorithmsappliedtotheGBUChallengeProblem.Thearchitectureofthebaselinealgorithmisare“nedim-plementationofthestandardPCA-basedfacerecognitional-gorithm,alsoknownasEigenfaces[7][8].Thesere“nementsconsiderablyimproveperformanceoverastandardPCA-basedimplementation.There“nementsincluderepresentingafacebylocalregions,aselfquotientnormalizationstep,andweightingeigenfeaturesbasedonFischerscriterion.WerefertotheGBUbaselinealgorithmaslocalregionPCAItmaycomeasasurprisetomanyinthefacerecognitioncommunitythataPCA-basedalgorithmwasselectedfortheGBUbenchmarkalgorithm.However,whendevelopingtheLRPCAbaselinealgorithm,weexplorednumerousstandardalternatives,includingLDA-basedalgorithmsandalgorithmscombiningGaborbasedfeatureswithkernelmethodsandsupportvectormachines.ForperformanceacrossthefullrangeoftheGBUChallengeProblem,ourexperimentswithalternativearchitecturesneverresultedinoverallper-formancebetterthantheGBUbaselinealgorithm.A.AStep-by-stepAlgorithmDescriptionThealgorithms“rststepistoextractacroppedandgeometrically-normalizedfaceregionfromanoriginalfaceimage.Theoriginalimageisassumedtobeastillimageandtheposeofthefaceisclosetofrontal.Thefaceregionintheoriginalisscaled,rotated,andcroppedtoaspeci“edsizeandthecentersoftheeyesarehorizontallyalignedandplacedonstandardpixellocations.Inthebaselinealgorithm,thefacechipis128by128pixelswiththecentersoftheeyesspaced64pixelsapart.Thebaselinealgorithmrunsintwomodes:partiallyandfullyautomatic.Inthepartiallyautomaticmode metricsisrecognitionfromtheocularregionoftheface.Thereisinterestinrecognitionfrombothnearinfraredandvisibleimagery.Theregion-baseddesignoftheLRPCAalgorithmallowsforbaseliningocularperformanceontheGBUpartitions.Baselineperformancefortheleftoculariscomputedfromthreeofthe14regions.Theregionsarethelefteyeandtwolefteyebrowregions.Fortherightocularregion,performanceiscomputedfromtherighteyeandtworighteyebrowregions.Thelefteye(resp.righteye)arewithrespectivetothesubject;e.g.,theleftocularregioncorrespondstoasubjectlefteye.PerformancefortheLRPCA-ocularbaselinefortheleftandrightocularregionsisgiveninFigure9.AsummaryofperformanceoftheFusion,theLRPCA-facebaselineandtheLRPCA-ocularbaselinealgorithmsaregiveninTableI.TABLEIERFORMANCEOFTHEFACEBASELINEANDTHEOCULARBASELINEALGORITHMSORTHEOCULARBASELINEPERFORMANCEISGIVENFORBOTHTHELEFTANDTHERIGHTOCULARREGIONSHEVERIFICATIONRATEATAFAR=0.001ISGIVEN LRPCA-ocular PartitionFusionLRPCA-faceleftright Good0.980.640.470.46Bad0.800.240.160.17Ugly0.150.070.050.05 VI.DISCUSSIONANDThispaperintroducestheGood,theBad,&theUglyChallengeProblem.Themaingoalofthechallengeistoencouragethedevelopmentofalgorithmsthatarerobusttorecognizingfrontalfacestakenoutsideofstudiostyleimagecollections.ThethreepartitionsintheGBUChallengeProblememphasizetherangeofperformancethatispossiblewhencomparingfacesphotographedundertheseconditions.ThisstructureallowsforresearcherstoconcentrateonthehardŽaspectsoftheproblemwhilenotcompromisingperformanceontheeasierŽaspects.Partitioningthechallengebylevelsofdif“cultyisthemostprominentfeatureoftheGBUChallengeProblemdesign.AnotheriscontrollingfortherecognizabilityŽofpeoplebyselectingimagesofthesamepeopleforinclusionineachoftheGBUpartitions.Thedatainthethreepartitionsisfurtherbalancedsoastoensurethatforeachpersonthenumberoftargetandqueryimagesineachpartitionisthesame.ThedesignoftheGBUChallengeProblemmeansthatanydifferenceinperformanceobservedbetweenpartitionscannotbeattributedtodifferencesbetweenpeopleornumbersofimagesforindividualpeople.TheuniquedesignoftheGBUChallengeProblemal-lowsresearcherstoinvestigatefactorsthatin”uencetheperformanceofalgorithms.OTooleetal.[10]looksatthedemographiceffectsonthenonmatchdistribution.Beveridgeetal.[11]showsthatthequalityoffaceimagescomesinpairs.Additionalpossiblelinesofinvestigationincludeunderstandingthefactorsthatcharacterizethedifferenceinmatchfacepairsacrossthepartitions.Asecondlineofre-searchischaracterizingtherecognizabilityofaface;e.g.,thebiometriczoo.Athirdlineofresearchisdevelopingmethodsforpredictingperformanceoffacerecognitionalgorithms.ThedesignoftheGBUChallengeProblemencouragesboththedevelopmentofalgorithms,andtheinvestigationofmethodsforunderstandingalgorithmperformance.CKNOWLEDGMENTSTheauthorswishtothanktheFederalBureauofInves-tigation(FBI)andtheTechnicalSupportWorkingGroup(TSWG)fortheirsupportofthiswork.Theidenti“cationofanycommercialproductortradenamedoesnotimplyendorsementorrecommendationbytheColoradoStateUni-versity,theNationalInstituteofStandardsandTechnologyortheUniversityofTexasatDallas.TheauthorsthankJayScallanforhisassistanceinpreparingtheGBUchallenge[1]P.J.Phillips,H.Wechsler,J.Huang,andP.Rauss,TheFERETdatabaseandevaluationprocedureforface-recognitionalgorithms,ŽImageandVisionComputingJournal,vol.16,no.5,pp.295…306,[2]P.J.Phillips,H.Moon,S.Rizvi,andP.Rauss,TheFERETevaluationmethodologyforface-recognitionalgorithms,ŽIEEETrans.PAMIvol.22,pp.1090…1104,October2000.[3]P.J.Phillips,P.J.Flynn,T.Scruggs,K.W.Bowyer,J.Chang,K.Hoffman,J.Marques,J.Min,andW.Worek,Overviewofthefacerecognitiongrandchallenge,ŽinIEEEComputerSocietyConferenceonComputerVisionandPatternRecognition,2005,pp.947…954.[4]P.J.Phillips,W.T.Scruggs,A.J.OToole,P.J.Flynn,K.W.Bowyer,C.L.Schott,andM.Sharpe,FRVT2006andICE2006large-scaleresults,ŽIEEETrans.PAMI,vol.32,no.5,pp.831…846,2010.[5]P.J.Phillips,P.J.Flynn,J.R.Beveridge,W.T.Scruggs,A.J.OToole,D.Bolme,K.W.Bowyer,B.A.Draper,G.H.Givens,Y.M.Lui,H.Sahibzada,J.A.ScallanIII,andS.Weimer,OverviewoftheMultipleBiometricsGrandChallenge,ŽinProceedingsThirdIAPRInternationalConferenceonBiometrics,2009.[6]P.J.Grother,G.W.Quinn,andP.J.Phillips,MBE2010:Reportontheevaluationof2Dstill-imagefacerecognitionalgorithms,ŽNationalInstituteofStandardsandTechnology,NISTIR7709,2010.[7]M.TurkandA.Pentland,Eigenfacesforrecognition,ŽJ.CognitiveNeuroscience,vol.3,no.1,pp.71…86,1991.[8]M.KirbyandL.Sirovich,Applicationofthekarhunen-loevepro-cedureforthecharacterizationofhumanfaces,ŽIEEETrans.PAMIvol.12,no.1,pp.103…108,1990.[9]H.Wang,S.Li,Y.Wang,andJ.Zhang,Selfquotientimageforfacerecognition,ŽinProceedings,InternationalConferenceonImageProcessing,vol.2,2004,pp.1397…1400.[10]A.J.OToole,P.J.Phillips,X.An,andJ.Dunlop,Demographiceffectsonestimatesofautomaticfacerecognitionperformance,ŽinProceedings,NinthInternationalConferenceonAutomaticFaceandGestureRecognition,2011.[11]J.R.Beveridge,P.J.Phillips,G.H.Givens,B.A.Draper,M.N.Teli,andD.S.Bolme,Whenhigh-qualityfaceimagesmatchpoorly,ŽinProceedings,NinthInternationalConferenceonAutomaticFaceandGestureRecognition,2011. AnIntroductiontotheGood,theBad,&theUglyFaceRecognitionChallengeProblemP.JonathonPhillips,J.RossBeveridge,BruceA.Draper,GeofGivens,AliceJ.OToole,DavidS.Bolme,JosephDunlop,YuiManLui,HassanSahibzada,andSamuelWeimer„TheGood,theBad,&theUglyFaceChallengeProblemwascreatedtoencouragethedevelopmentofalgo-rithmsthatarerobusttorecognitionacrosschangesthatoccurinstillfrontalfaces.TheGood,theBad,&theUglyconsistsofthreepartitions.TheGoodpartitioncontainspairsofimagesthatareconsideredeasytorecognize.OntheGoodpartition,thebaseveri“cationrate(VR)is0.98atafalseacceptrate(FAR)of0.001.TheBadpartitioncontainspairsofimagesofaveragedif“cultytorecognize.FortheBadpartition,theVRis0.80ataFARof0.001.TheUglypartitioncontainspairsofimagesconsidereddif“culttorecognize,withaVRof0.15ataFARof0.001.ThebaseperformanceisfromfusingtheoutputofthreeofthetopperformersintheFRVT2006.ThedesignoftheGood,theBad,&theUglycontrolsforposevariation,subjectaging,andsubjectrecognizability.ŽSubjectrecognizabilityiscontrolledbyhavingthesamenumberofimagesofeachsubjectineverypartition.Thisimpliesthatthedifferencesinperformanceamongthepartitionsareresultofhowafaceispresentedineachimage.I.INTRODUCTIONFacerecognitionfromstillfrontalimageshasmadegreatstridesoverthelasttwentyyears.Overthisperiod,errorrateshavedecreasedbythreeordersofmagnitudewhenrecognizingfrontalfacesinstillimagestakenwithconsistentcontrolledilluminationinanenvironmentsimilartoastu-dio[1],[2],[3],[4],[5],[6].Undertheseconditions,errorratesbelow1%atafalseacceptrateof1in1000werereportedintheFaceRecognitionVendorTest(FRVT)2006andtheMultipleBiometricEvaluation(MBE)2010[4],[6].Withthissuccess,thefocusofresearchisshiftingtorecognizingfacestakenunderlessconstrainedconditions.Lessconstrainedconditionsincludeallowinggreatervari-abilityinpose,ambientlighting,expression,sizeoftheface,anddistancefromthecamera.Thetrickindesigningafacerecognitionchallengeproblemisselectingthedegreetowhichtheconstraintsarerelaxedsothattheresultingproblemhastheappropriatedif“culty.Thecomplexityofthistaskiscompoundedbythefactthatitisnotwellunderstoodhowtheabovefactorseffectperformance.TheproblemcannotbetooeasythatitisanexerciseintuningP.J.PhillipsandH.SahibzadaarewiththeNationalInstituteofStandardsandTechnology,100BureauDr.,MS8940GaithersburgMD20899,USA(e-mail:jonathon@nist.gov).PleasedirectcorrespondencetoP.J.Phillips.J.R.Beveridge,B.A.Draper,D.S.Bolme,andY-MLuiarewiththeDepartmentofComputerScience,ColoradoStateU.,FortCollins,CO46556,USA.G.GivensiswiththeDepartmentofStatistics,ColoradoStateU.,FortCollins,CO46556,USA.A.J.OToole,J.Dunlop,andS.WeimerarewiththeSchoolofBehavioralandBrainSciences,GR4.1TheUniversityofTexasatDallasRichardson,TX75083-0688,USAexistingalgorithms,norsohardthatprogresscannotbemade„thethreebearsproblems[2].Traditionally,achallengeproblemisspeci“edbythetwosetsofimagesthataretobecompared.Thedif“cultyoftheproblemisthencharacterizedbytheperformanceofasetofalgorithmstaskedwithmatchingthetwosetsoffaceimages.Tocreateaproblemofadesiredlevelofdif“culty,asetofalgorithmscouldbeonecomponentintheimageselectionprocess.Othersfactorsintheselectionprocessincludelimitingthenumberofimagesperpersonandrequiringthatpairsofimagesofapersonarecollectedondifferentdays.TheGood,theBad,andtheUgly(GBU)challengeprob-lemconsistsofthreepartitionswhicharecalledtheGood,theBad,andtheUgly.Thedif“cultyofeachpartitionisbasedontheperformanceofthreetopperformersintheFRVT2006.TheGoodpartitionconsistsofpairsoffaceimagesofthesamepersonthatareeasytomatch;theBadpartitioncontainspairsoffaceimagesofapersonthathaveaveragematchingdif“culty;andtheUglypartitionconcentratesondif“culttomatchfacepairs.FortheGoodpartition,thenom-inalperformancebasedontheFRVT2006isaveri“cationrate(VR)of0.98atafalseacceptrate(FAR)of0.001.FortheBadandUglypartitions,thecorrespondingVRataFARof0.001are0.80and0.15.Theperformancerangeoverthethreepartitionsisroughlyanorderofmagnitude.ThethreepartitionscapturetherangeofperformanceinherentinlessconstrainedimagesTherearenumeroussourcesofvariation,knownandunknown,infaceimagesthatcaneffectperformance.FourofthesefactorsareexplicitlycontrolledinthedesignoftheGBUchallengeproblem:subjectaging,pose,changeincamera,andvariationsamongfaces.Thedatacollectionprotocoleliminatedorsigni“cantlyreducedtheimpactofthreeofthefactors.Changesintheappearanceofafaceduetoagingisnotafactorbecauseallimageswerecollectedinthesameacademicyear.However,thedatasetcontainsthenaturalvariationsinapersonsappearancethatwouldoccuroveranacademicyear.Becausealltheimageswerecollectedbythesamemodelofcamera,differenceinperformancecannotbeattributabletochangesinthecamera.Changesinposearenotafactorbecausethedatasetconsistsoffrontalfaceimages.OnepotentialsourceofvariabilityinperformanceisthatInstructionsforobtainingthecompleteGBUdistributioncanbefoundathttp://face.nist.gov.InstructionsforobtainingtheLRPCAalgorithmcanbefoundathttp://www.cs.colostate.edu/facerec. (a)(b)(c)Fig.3.Examplesoffacepairsofthesamepersonfromeachofthepartitions:(a)good,(b)challenging,and(c)verychallenging.thedata.Thesimilarityscoreswereselectedtoevenlysampletheimagesintheexperiment.ThefusedsimilarityscoresarethesumoftheindividualalgorithmsimilarityscoresafterthemedianhasbeensubtractedandthendividedbytheMAD.isasimilarityscoreforalgorithmisafusionsimilarityscore,thenFigure4reportsperformanceofthefusionalgorithmoneachofthepartitions.Figure5showsthedistributionofthematchandnon-matchesforthefusionalgorithmonallthreepartitions.Thenon-matchdistributionisstableacrossallthreepartitions.Thematchdistributionshiftsforeachpartition.TheGoodpartitionshowsthegreatestdifferencebetweenthemedianofthematchandnon-matchdistributionsandtheleastdifferencefortheUglypartition.IV.PROTOCOLTheprotocolfortheGBUChallengeProblemisone-to-onematchingwithtraining,modelselection,andtuningcompletedpriortocomputingperformanceonthepartitions.Consequently,underthisprotocol,thesimilarityscoret,qbetweenatargetimageandaqueryimagedoesnotinanywaydependontheotherimagesinthetargetandquerysets.Avoidinghiddeninteractionsbetweenimages,otherthanthetwobeingcomparedatthemoment,providestheclearestpictureofhowalgorithmsperform.Moreformally,anyapproachthatrede“nessimilarityt,qsuchthatitdependsuponthetarget(orquery)imagesetisNOTallowedintheGBUChallengeProblem.Tomaintainseparationoftrainingandtestsets,analgo-rithmcannotbetrainedonimagesofanyofthesubjectsintheGBUChallengeProblem.ItisimportanttonotethatthereareimagesofthesubjectsintheGBUproblemthatareintheFRGCandtheMBGCdatasets.Theseimagesmustbeexcludedfrommodelselection,training,ortuningofanWeillustrateacceptableandunacceptabletrainingproto-colswiththreeexamples.The“rstexampleistrainingofaprincipalcomponentsanalysis(PCA)basedface-recognitionalgorithm.InaPCA-basedalgorithm,PCAisperformedonatrainingsettoproduceasetofEigenfaces.AfaceisrepresentedbyprojectingafaceimageonthesetofEigen-faces.Tomeetthetrainingrequirementsoftheprotocol,imagesofsubjectsintheGBUmustbeexcludedfromthePCAdecompositionthatproducesasetofEigenfaces.ThebenchmarkalgorithminSectionVincludesatrainingsetthatsatis“esthetrainingprotocol.Asecondexampleistakenfromacommontrainingprocedureforlineardiscriminantanalysis(LDA)inwhichthealgorithmistrainedontheimagesinatargetset.TraininganalgorithmonaGBUtargetsettheGBUprotocol.Gen-erally,itiswellknownthattheperformanceofalgorithmscanimprovewithsuchtraining,buttheresultinglevelsofperformancetypicallydonotgeneralize.Forexample,weveconductedexperimentswithanLDAalgorithmtrainedontheGBUtargetimagesandperformanceimprovedoverthebaselinealgorithmpresented,seeSectionV.However,whenwetrainedourLDAalgorithmfollowingtheGBUprotocol,performancedidnotmatchtheLDAalgorithmtrainedonaGBUtargetset.TheGBUprotocoldoespermitimagespeci“crepresenta-tionsaslongastherepresentationdoesnotdependonother (a)(b)(c)Fig.1.Examplesoffacepairsofthesamepersonfromeachofthepartitions:(a)good,(b)challenging,and(c)verychallenging. (a)(b)(c)Fig.2.Examplesoffacepairsofthesamepersonfromeachofthepartitions:(a)good,(b)challenging,and(c)verychallenging.

Related Contents


Next Show more