Face Description with Local Binary Patterns Application to Face Recognition Timo Ahonen - PDF document

1FaceDescriptionwithLocalBinaryPatterns:ApplicationtoFaceRecognitionTimoAhonen,StudentMember,IEEE,AbdenourHadid,andMattiPietik¨ainen,SeniorMember,IEEEAbstractThispaperpresentsanovelandefcientfacialimagerepresentationbasedonlocalbinarypattern(LBP)texturefeatures.ThefaceimageisdividedintoseveralregionsfromwhichtheLBPfeaturedistributionsareextractedandconcatenatedintoanenhancedfeaturevectortobeusedasafacedescriptor.Theperformanceoftheproposedmethodisassessedinthefacerecognitionproblemunderdifferentchallenges.Otherapplicationsandseveralextensionsarealsodiscussed.IndexTermsFacialimagerepresentation,localbinarypattern,component-basedfacerecogni-tion,texturefeatures,facemisalignmentI.INTRODUCTIONAutomaticfaceanalysiswhichincludes,e.g.,facedetection,facerecognitionandfacialexpressionrecognitionhasbecomeaveryactivetopicincomputervisionresearch[1].Akeyissueinfaceanalysisisndingefcientdescriptorsforfaceappearance.DifferentholisticmethodssuchasPrincipalComponentAnalysis(PCA)[2],LinearDiscriminantAnalysis(LDA)[3]andthemorerecent2-DPCA[4]havebeenstudiedwidelybutlatelyalsolocaldescriptorsT.Ahonen,A.Hadid,andM.Pietik¨ainenarewiththeMachineVisionGroup,InfotechOulu,DepartmentofElectricalandInformationEngineering,UniversityofOulu,POBox4500,FIN-90014,Finland.E-mail:tahonen,hadid,mkp@ee.oulu..5thJune2006DRAFT 2havegainedattentionduetotheirrobustnesstochallengessuchasposeandilluminationchanges.Thispaperpresentsanoveldescriptorbasedonlocalbinarypatterntexturefeaturesextractedfromlocalfacialregions.OneoftherstfacedescriptorsbasedoninformationextractedfromlocalregionsistheeigenfeaturesmethodproposedbyPentlandetal.[5].ThisisahybridapproachinwhichthefeaturesareobtainedbyperformingPCAtolocalfaceregionsindependently.InLocalFeatureAnalysis[6],kernelsoflocalspatialsupportareusedtoextractinformationaboutlocalfacialcomponents.ElasticBunchGraphMatching(EBGM)[7]describesfacesusingGaborlterresponsesincertainfaciallandmarksandagraphdescribingthespatialrelationsoftheselandmarks.ThevalidityofthecomponentbasedapproachisalsoattestedbythestudyconductedbyHeiseleetal.inwhichacomponent-basedfacerecognitionsystemclearlyoutperformedglobalapproachesonatestdatabasecontainingfacesrotatedindepth[8].Usinglocalphotometricfeatures[9]forobjectrecognitioninthemoregeneralcontexthasbecomeawidelyacceptedapproach.Inthatsettingthetypicalapproachistodetectinterestpointsorinterestregionsinimages,performnormalizationwithrespecttoafnetransformationsanddescribethenormalizedinterestregionsusinglocaldescriptors.Thisbag-of-keypointsapproachisnotsuitedforfacedescriptionassuchsinceitdoesnotretaininformationonthespatialsettingofthedetectedlocalregionsbutitdoesbearcertainsimilaritiestolocalfeaturebasedfacedescription.Findinggooddescriptorsfortheappearanceoflocalfacialregionsisanopenissue.Ideally,thesedescriptorsshouldbeeasytocomputeandhavehighextra-classvariance(i.e.,betweendifferentpersonsinthecaseoffacerecognition)andlowintra-classvariance,whichmeansthatthedescriptorshouldberobustwithrespecttoagingofthesubjects,alternatingilluminationandotherfactors.Thetextureanalysiscommunityhasdevelopedavarietyofdifferentdescriptorsfortheappearanceofimagepatches.However,facerecognitionproblemhasnotbeenassociatedtothatprogressintextureanalysiseldasithasnotbeeninvestigatedfromsuchpointofview.Recently,weinvestigatedtherepresentationoffaceimagesbymeansoflocalbinarypatternfeatures,yieldinginoutstandingresultsthatwerepublishedintheECCV2004conference[10].Afterthis,severalresearchgroupshaveadoptedourapproach.Inthispaper,weprovideamoredetailedanalysisoftheproposedrepresentation,presentadditionalresultsanddiscussfurther5thJune2006DRAFT 3 110100 8599215454861213 Fig.1.ThebasicLBPoperator.extensions.II.LBPBASEDFACEDESCRIPTIONTheLBPoperator[11]isoneofthebestperformingtexturedescriptorsandithasbeenwidelyusedinvariousapplications.Ithasproventobehighlydiscriminativeanditskeyadvantages,namelyitsinvariancetomonotonicgraylevelchangesandcomputationalefciency,makeitsuitablefordemandingimageanalysistasks.ForabibliographyofLBP-relatedresearch,seehttp://www.ee.oulu./research/imag/texture/.TheideaofusingLBPforfacedescriptionismotivatedbythefactthatfacescanbeseenasacompositionofmicro-patternswhicharewelldescribedbysuchoperator.A.LocalbinarypatternsTheLBPoperatorwasoriginallydesignedfortexturedescription.Theoperatorassignsalabeltoeverypixelofanimagebythresholdingthe3x3-neighborhoodofeachpixelwiththecenterpixelvalueandconsideringtheresultasabinarynumber.Thenthehistogramofthelabelscanbeusedasatexturedescriptor.SeeFigure1foranillustrationofthebasicLBPoperator.Tobeabletodealwithtexturesatdifferentscales,theLBPoperatorwaslaterextendedtouseneighborhoodsofdifferentsizes[12].Deningthelocalneighborhoodasasetofsamplingpointsevenlyspacedonacirclecenteredatthepixeltobelabeledallowsanyradiusandnumberofsamplingpoints.Bilinearinterpolationisusedwhenasamplingpointdoesnotfallinthecenterofapixel.Inthefollowing,thenotationP;willbeusedforpixelneighborhoodswhichmeanssamplingpointsonacircleofradiusof.SeeFigure2foranexampleofcircularneighborhoods.Anotherextensiontotheoriginaloperatoristhedenitionofsocalleduniformpatterns[12].Alocalbinarypatterniscalleduniformifthebinarypatterncontainsatmosttwobitwise5thJune2006DRAFT 4 Fig.2.Thecircular(8,1),(16,2)and(8,2)neighborhoods.Thepixelvaluesarebilinearlyinterpolatedwheneverthesamplingpointisnotinthecenterofapixel.transitionsfrom0to1orviceversawhenthebitpatternisconsideredcircular.Forexample,thepatterns00000000(0transitions),01110000(2transitions)and11001111(2transitions)areuniformwhereasthepatterns11001001(4transitions)and01010011(6transitions)arenot.InthecomputationoftheLBPhistogram,uniformpatternsareusedsothatthehistogramhasaseparatebinforeveryuniformpatternandallnon-uniformpatternsareassignedtoasinglebin.Ojalaetal.noticedthatintheirexperimentswithtextureimages,uniformpatternsaccountforabitlessthan90%ofallpatternswhenusingthe(8,1)neighborhoodandforaround70%inthe(16,2)neighborhood.Wehavefoundthat90.6%ofthepatternsinthe(8,1)neighborhoodand85.2%ofthepatternsinthe(8,2)neighborhoodareuniformincaseofpreprocessedFERETfacialimages.WeusethefollowingnotationfortheLBPoperator:LBPP;R.ThesubscriptrepresentsusingtheoperatorinaP;neighborhood.Superscriptstandsforusingonlyuniformpatterns.B.FacedescriptionwithLBPInthiswork,theLBPmethodpresentedintheprevioussubsectionisusedforfacedescription.Theprocedureconsistsofusingthetexturedescriptortobuildseverallocaldescriptionsofthefaceandcombiningthemintoaglobaldescription.Insteadofstrivingforaholisticdescriptionthisapproachwasmotivatedbytworeasons:thelocalfeaturebasedorhybridapproachestofacerecognitionhavebeengaininginterestlately[6],[8],[13],whichisunderstandablegiventhelimitationsoftheholisticrepresentations.Theselocalfeaturebasedandhybridmethodsseemtobemorerobustagainstvariationsinposeorilluminationthanholisticmethods.Anotherreasonforselectingthelocalfeaturebasedapproachisthattryingtobuildaholisticdescriptionofafaceusingtexturemethodsisnotreasonablesincetexturedescriptorstendtoaverageovertheimagearea.Thisisadesirablepropertyforordinarytextures,becausetexturedescriptionshouldusuallybeinvarianttotranslationorevenrotationofthetextureand,5thJune2006DRAFT 5 Fig.3.Afacialimagedividedinto,andrectangularregions.especiallyforsmallrepetitivetextures,thesmall-scalerelationshipsdeterminetheappearanceofthetextureandthusthelarge-scalerelationsdonotcontainusefulinformation.Forfaceshowever,thesituationisdifferent:retainingtheinformationaboutspatialrelationsisimportant.Thisreasoningleadstothebasicmethodologyofthiswork.Thefacialimageisdividedintolocalregionsandtexturedescriptorsareextractedfromeachregionindependently.Thedescriptorsarethenconcatenatedtoformaglobaldescriptionoftheface.SeeFigure3foranexampleofafacialimagedividedintorectangularregions.Thebasichistogramcanbeextendedintoaspatiallyenhancedhistogramwhichencodesboththeappearanceandthespatialrelationsoffacialregions.Asthefacialregions:Rhavebeendetermined,ahistogramiscomputedindependentlywithineachoftheregions.Theresultinghistogramsarecombinedyieldingthespatiallyenhancedhistogram.ThespatiallyenhancedhistogramhassizewhereisthelengthofasingleLBPhistogram.Inthespatiallyenhancedhistogram,weeffectivelyhaveadescriptionofthefaceonthreedifferentlevelsoflocality:theLBPlabelsforthehistogramcontaininformationaboutthepatternsonapixel-level,thelabelsaresummedoverasmallregiontoproduceinformationonaregionallevelandtheregionalhistogramsareconcatenatedtobuildaglobaldescriptionoftheface.Itshouldbenotedthatwhenusingthehistogram-basedmethods,despitetheexamplesinFigure3,theregions:Rdonotneedtoberectangular.Neitherdotheyneedtobeofthesamesizeorshape,andtheydonotnecessarilyhavetocoverthewholeimage.Forexample,theycouldbecircularregionslocatedattheducialpointslikeintheEBGMmethod.Itisalsopossibletohavepartiallyoverlappingregions.Ifrecognitionoffacesrotatedindepthisconsidered,itmaybeusefultofollowtheprocedureofHeiseleetal.[8]andautomaticallydetecteachregionintheimageinsteadofrstdetectingthefaceandthenusingaxeddivisionintoregions.5thJune2006DRAFT 6Theideaofaspatiallyenhancedhistogramcanbeexploitedfurtherwhendeningthedis-tancemeasure.Anindigenouspropertyoftheproposedfacedescriptionmethodisthateachelementintheenhancedhistogramcorrespondstoacertainsmallareaoftheface.Basedonthepsychophysicalndings,whichindicatethatsomefacialfeatures(suchaseyes)playmoreimportantrolesinhumanfacerecognitionthanotherfeatures[14],itcanbeexpectedthatinthismethodsomeofthefacialregionscontributemorethanothersintermsofextrapersonalvariance.Utilizingthisassumptiontheregionscanbeweightedbasedontheimportanceoftheinformationtheycontain.Forexample,theweightedChisquaredistancecanbedenedas (1)inwhichandarethenormalizedenhancedhistogramstobecompared,indicesandreferto-thbininhistogramcorrespondingtothe-thlocalregionandistheweightforregion.III.EXPERIMENTALANALYSISOurapproachisassessedonthefacerecognitionproblemusingtheColoradoStateUniversityFaceIdenticationEvaluationSystem[15]withimagesfromtheFERET[16]database.PCA[2],BayesianIntra/ExtrapersonalClassier(BIC)[17]andEBGMwereusedascontrolalgorithms.A.ExperimentalsetupToensurethereproducibilityofthetests,thepubliclyavailableCSUfaceidenticationevaluationsystem[15]wasutilizedtotesttheperformanceoftheproposedalgorithm.ThesystemusestheFERETfaceimagesandfollowstheprocedureoftheFERETtestforsemi-automaticfacerecognitionalgorithms[18]withslightmodications.TheFERETdatabaseconsistsofatotalof14051gray-scaleimagesrepresenting1199individ-uals.Theimagescontainvariationsinlighting,facialexpressions,poseangleetc.Inthiswork,onlyfrontalfacesareconsidered.Thesefacialimagescanbedividedintovesetsfollowingly:faset,usedasagalleryset,containsfrontalimagesof1196people.fbset(1195images).Thesubjectswereaskedforanalternativefacialexpressionthaninthefaphotograph.fcset(194images).Thephotosweretakenunderdifferentlightingconditions.dupIset(722images).Thephotosweretakenlaterintime.5thJune2006DRAFT 7dupIIset(234images).ThisisasubsetofthedupIsetcontainingthoseimagesthatweretakenatleastayearafterthecorrespondinggalleryimage.Alongwithrecognitionratesatrank1,twostatisticalmeasuresareusedtocomparetheperformanceofthealgorithms:themeanrecognitionratewitha95%condenceintervalandtheprobabilityofonealgorithmoutperforminganother.Theprobabilityofonealgorithmoutper-forminganotherisdenotedbyP((alg1)(alg2)).Thesestatisticsarecomputedbypermutingthegalleryandprobesets,see[15]fordetails.TheCSUsystemcomeswithimplementationsofthePCA,LDA,BICandEBGMfacerecognitionalgorithms.TheresultsobtainedwithPCA,BICandEBGMareincludedhereforcomparison.B.ParametersoftheLBPmethodTherearesomeparametersthatcanbechosentooptimizetheperformanceoftheLBP-basedalgorithm.TheseincludechoosingthetypeoftheLBPoperator,divisionoftheimagesintoregions:;,selectingthedistancemeasureforthenearestneighborclassierandndingtheweightsfortheweightedstatistic(Equation1).Theextensiveexperimentstondtheparametersfortheproposedmethodaredetailedin[10].WhenlookingfortheoptimalwindowsizeandLBPoperatoritwasnoticedthattheLBPrep-resentationisquiterobustwithrespecttotheselectionofparameters.Changesintheparametersmaycausebigdifferencesinthelengthofthefeaturevector,buttheoverallperformanceisnotnecessarilyaffectedsignicantly[10].Here,theLBPoperatorinpixelwindowswasselectedsinceitisagoodtrade-offbetweenrecognitionperformanceandfeaturevectorlength.Whencomparingdifferentdistancemeasures,themeasurewasfoundtoperformbetterthanhistogramintersectionorlog-likelihooddistance.Thereforethemeasurewaschosentobeused.Tondtheweightsfortheweightedstatistic(Equation1),asimpleprocedurewasadoptedinwhichatrainingsetwasclassiedusingonlyoneofthewindowsatatimeandthewindowswereassignedaweightbasedontherecognitionrate.TheobtainedweightsareillustratedinFigure4(b).Theweightswereselectedwithoututilizinganactualoptimizationprocedureandthustheyareprobablynotoptimal.Despitethat,incomparisonwiththenon-weightedmethod,animprovementbothintheprocessingtimeandrecognitionrate(P((weighted)(non-weighted))=0.976)wasobtained.5thJune2006DRAFT 8 (a)(b)Fig.4.(a)Afacialimagedividedinto7x7windows.(b)Theweightssetfortheweighteddissimilaritymeasure.Blacksquaresindicateweight0.0,darkgray1.0,lightgray2.0andwhite4.0.TABLEITHERECOGNITIONRATESOBTAINEDUSINGDIFFERENTTEXTUREDESCRIPTORSFORLOCALFACIALREGIONS.THEFIRSTFOURCOLUMNSSHOWTHERECOGNITIONRATESFORTHEFERETTESTSETSANDTHELASTTHREECOLUMNSCONTAINTHEMEANRECOGNITIONRATEOFTHEPERMUTATIONTESTWITHA95%CONFIDENCEINTERVAL. MethodfbfcdupIdupIIlowermeanupper Differencehistogram0.870.120.390.250.580.630.68Homogeneoustexture0.860.040.370.210.580.620.68TextonHistogram0.970.280.590.420.710.760.80LBP(nonweighted)0.930.510.610.500.710.760.81 C.ComparinglocalbinarypatternstootherlocaldescriptorsTogainbetterunderstandingonwhethertheobtainedrecognitionresultsareduetogeneralideaofcomputingtexturefeaturesfromlocalfacialregionsorduetothediscriminatorypowerofthelocalbinarypatternoperator,wecomparedLBPtothreeothertexturedescriptors,namelythegray-leveldifferencehistogram,homogeneoustexturedescriptor[19]andanimprovedversionofthetextonhistogram[20].Thedetailsoftheseexperimentscanbefoundin[21].TherecognitionratesobtainedwithdifferentdescriptorsareshowninTableI.Itshouldbenotedthatnoweightingforlocalregionswasusedinthisexperiment.Theresultsshowthatthetestedmethodsworkwellwiththeeasiestfbprobeset,whichmeansthattheyarerobustwithrespecttovariationsoffacialexpressions,whereastheresultswiththefcprobesetshowthat5thJune2006DRAFT 9 0 10 20 30 40 50 0.8 0.85 0.9 0.95 1 RankCumulative score LBP weighted 0 10 20 30 40 50 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 RankCumulative score LBP weighted 0 10 20 30 40 50 0.4 0.5 0.6 0.7 0.8 0.9 1 RankCumulative score LBP weighted 0 10 20 30 40 50 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 RankCumulative score LBP weighted (a)(b)(c)(d) Fig.5.ThecumulativescoresoftheLBPandcontrolalgorithmsonthe(a)fb,(b)fc,(c)dupIand(d)dupIIprobesets.othermethodsthanLBPdonotsurvivechangesinillumination.TheLBPandtextongivethebestresultsinthedupIanddupIItestsets.Webelievethatthemainexplanationforthebetterperformanceofthelocalbinarypatternoperatoroverothertexturedescriptorsisitstolerancetomonotonicgray-scalechanges.Addi-tionaladvantagesarethecomputationalefciencyoftheLBPoperatorandthatnogray-scalenormalizationisneededpriortoapplyingtheLBPoperatortothefaceimage.D.ResultsfortheFERETdatabaseThenalrecognitionresultsfortheproposedmethodareshowninTableIIandtherankcurvesareplottedinFigure5.LBPyieldsclearlyhigherrecognitionratesthanthecontrolalgorithmsinalltheFERETtestsetsandinthestatisticaltest.TheresultsonthefcanddupIIsetsshow5thJune2006DRAFT 10thatespeciallywithweighting,theLBPbaseddescriptionisrobusttochallengescausedbylightingchangesoragingofthesubjectsbutfurtherresearchisstillneededtoachieveevenbetterperformance.ItshouldbenotedthattheCSUimplementationsofthealgorithmswhoseresultsareincludedheredonotachievethesameguresasintheoriginalFERETtestduetosomemodicationsintheexperimentalsetupasmentionedin[15].TheresultsoftheoriginalFERETtestcanbefoundin[18].E.RobustnessofthemethodtofacelocalizationerrorReal-worldfacerecognitionsystemsneedtoperformfacedetectionpriortofacerecognition.Automaticfacelocalizationmaynotbecompletelyaccuratesoitisdesirablethatfacerecognitionworksundersmalllocalizationerrors.Theproposedfacerecognitionmethodcalculateshistogramsoverthelocalregionssoasmallchangeinthepositionofthefacerelativetothegridcauseschangesinthelabelsonlyonthebordersofthelocalregions.Thereforeitcanbeexpectedthattheproposedmethodisnotsensitivetosmallchangesinthefacelocalizationandthatusinglargerlocalregionsincreasestherobustnesstoerrors.TheeffectoflocalizationerrorstorecognitionrateoftheproposedmethodcomparedtoPCAMahCosinewassystematicallytestedasfollows.ThetrainingimagesforPCAandgallery(fa)imageswerenormalizedtosizeusingprovidedeyecoordinates.Thefbsetwasusedasprobes.TheprobeswerealsonormalizedtosizebutarandomvectorX;TABLEIITHERECOGNITIONRATESOFTHELBPANDCOMPARISONALGORITHMS. MethodfbfcdupIdupIIlowermeanupper LBP,weighted0.970.790.660.640.760.810.85LBP,nonweighted0.930.510.610.500.710.760.81PCA,MahCosine0.850.650.440.220.660.720.78Bayesian,MAP0.820.370.520.320.670.720.78EBGM Optimal0.900.420.460.240.610.660.71 5thJune2006DRAFT 11wasaddedtothefacelocation,whereandareuncorrelatedandnormallydistributedwithmean0andstandarddeviation.Tenexperimentswereconductedwitheachprobetotaling11950queriesforeachtestedvalue.TherecognitionratesoftheLBPbasedmethodusingwindowsizesandandPCAMachCosineasafunctionofthestandarddeviationofthesimulatedlocalizationoffsetareplottedinFigure6.Itcanbeseenthatwhennoerrororonlyasmallerrorispresent,LBPwithsmalllocalregionsworkswellbutasthelocalizationerrorincreases,usinglargerlocalregionsproducesbetterrecognitionrate.Mostinterestingly,therecognitionrateofthelocalregionbasedmethodsdropssignicantlyslowerthanthatofPCA.IV.FURTHERWORKUSINGLOCALBINARYPATTERNBASEDFACEDESCRIPTIONSincethepublicationofourpreliminaryresultsontheLBPbasedfacedescription[10],ourmethodologyhasalreadyattainedanestablishedpositioninfaceanalysisresearch.Thisisattestedbytheincreasingnumberofworkswhichadoptedasimilarapproach1[22],[23],[24],[25],[26],[27],[28].Forinstance,localbinarypatternscomputedinlocalregionsforfacedetectionwasusedin[22].Inthatwork,LBPfeaturesfromlocalregionscombinedwithahistogramrepresentingthewholefaceareayieldedanexcellentfacedetectionratewhenusedasfeaturesforasupport1http://www.ee.oulu./research/imag/texture/lbp/bibliography/#faceanalysis 0 2 4 6 8 10 12 0 0.2 0.4 0.6 0.8 1 s of simulated detection offsetRecognition rate LBP 21x21 Fig.6.TherecognitionrateforthefbsetoftwoLBPbasedmethodsandPCAMahCosineasafunctionthestandarddeviationofasimulatedlocalizationerror.5thJune2006DRAFT 12vectormachineclassier.UsingLBPfeaturesforfacialexpressionrecognitionhasbeenstudiedbyFengetal.[23]andShanetal.[24].UsingtheJAFFEandCohn-Kanadefacialexpressionimagedatasets(see[1]),thesepapersshowthatLBPbaseddescriptorscomparefavorablytootherstate-of-the-artmethodsinfacialexpressionrecognition.In[25],Zhangetal.usedAdaBoostlearningalgorithmforselectingasetoflocalregionsandtheirweights.Then,theLBPmethodologywasappliedtotheobtainedregionsyieldinginsmallerfeaturevectorlength.Recently,Lietal.builtahighlyaccurate,illumination-invariantfacerecognitionsystembycombiningnear-infraredimagingwithanLBP-basedfacedescriptionandAdaBoostlearning[26].ComputingLBPfeaturesfromimagesobtainedbylteringafacialimagewith40GaborltersofdifferentscaleandorientationareshowntoyieldexcellentrecognitionrateonalltheFERETsetsin[27].Adownsideofthemethodproposedinthatpaperisthehighdimensionalityofthefeaturevectors.In[28],RodriguezandMarcelproposedanapproachbasedonadapted,client-specicLBPhistogramsforthefacevericationtask.Thereportedexperimentalresultsshowthattheproposedmethodyieldsexcellentperformanceontwofacevericationtestdatabases.V.DISCUSSIONANDCONCLUSIONSInthispaper,anovelandefcientfacialrepresentationisproposed.Itisbasedondividingafacialimageintosmallregionsandcomputingadescriptionofeachregionusinglocalbinarypatterns.Thesedescriptorsarethencombinedintoaspatiallyenhancedhistogramorfeaturevector.Thetexturedescriptionofasingleregiondescribestheappearanceoftheregionandthecombinationofallregiondescriptionsencodestheglobalgeometryoftheface.TheLBPoperatorhasbeenwidelyusedindifferentapplicationssuchastextureclassication,imageretrievaletc.Beforeourstudy,itwasnotobvioustoimaginethatsuchtextureoperatormightbeusefulinrepresentingalsofacialimages.Ourresultsclearlyshowthatfacialimagescanbeseenasacompositionofmicropatternssuchasatareas,spots,linesandedgeswhichcanbewelldescribedbyLBP.Inthisarticle,theproposedmethodologyisassessedwiththefacerecognitiontask.However,asimilarmethodhasyieldedinoutstandingperformanceinfacedetection[22]andfacialexpressionrecognition[23],[24].Wealsobelievethatthedevelopedapproachisnotlimited5thJune2006DRAFT 13tothesefewexamplesasitcanbeeasilygeneralizedtoothertypesofobjectdetectionandrecognitiontasks.Futureworkincludesstudyingmoreadvancedmethodsfordividingthefacialimageintolocalregionsandndingtheweightsfortheseregions.TheAdaBoostmethodpresentedin[25]servesasagoodbasisforthisresearch.Anotherimportanttopicislookingforimagepreprocessingmethodsanddescriptorsthataremorerobustagainstimagetransformationsthatchangetheappearanceofthesurfacetexturesuchasimageblurringcausedbyimagingdevicebeingslightlyout-of-focus.ACKNOWLEDGMENTSThisworkwassupportedbytheAcademyofFinlandandGraduateSchoolinElectronics,TelecommunicationandAutomation.REFERENCES[1]S.Z.LiandA.K.Jain,Eds.,HandbookofFaceRecognition.Springer,2005.[2]M.TurkandA.Pentland,“Eigenfacesforrecognition,”JournalofCognitiveNeuroscience,vol.3,no.1,pp.71–86,1991.[3]K.EtemadandR.Chellappa,“Discriminantanalysisforrecognitionofhumanfaceimages,”JournaloftheOpticalSocietyofAmerica,vol.14,pp.1724–1733,1997.[4]J.Yang,D.Zhang,A.F.Frangi,andJ.Yang,“Two-dimensionalPCA:Anewapproachtoappearance-basedfacerepresentationandrecognition,”IEEETransactionsonPatternAnalysisandMachineIntelligence,vol.26,no.1,pp.131–137,Jan2004.[5]A.Pentland,B.Moghaddam,andT.Starner,“View-basedandmodulareigenspacesforfacerecognition,”inProc.IEEEComputerSocietyConferenceonComputerVisionandPatternRecognition,1994,pp.84–91.[6]P.S.PenevandJ.J.Atick,“Localfeatureanalysis:Ageneralstatisticaltheoryforobjectrepresentation,”Network–ComputationinNeuralSystems,vol.7,no.3,pp.477–500,August1996.[7]L.Wiskott,J.-M.Fellous,N.Kuiger,andC.vonderMalsburg,“Facerecognitionbyelasticbunchgraphmatching,”IEEETransactionsonPatternAnalysisandMachineIntelligence,vol.19,pp.775–779,1997.5thJune2006DRAFT 14[8]B.Heisele,P.Ho,J.Wu,andT.Poggio,“Facerecognition:component-basedversusglobalapproaches,”CompterVisionandImageUnderstanding,vol.91,no.1–2,pp.6–21,2003.[9]K.MikolajczykandC.Schmid,“Aperformanceevaluationoflocaldescriptors,”IEEETransactionsonPatternAnalysisandMachineIntelligence,vol.27,no.10,pp.1615–1630,2005.[10]T.Ahonen,A.Hadid,andM.Pietik¨ainen,“Facerecognitionwithlocalbinarypatterns,”inProc.8thEuropeanConferenceonComputerVision,ser.LectureNotesinComputerScience,vol.3021.Springer,2004,pp.469–481.[11]T.Ojala,M.Pietik¨ainen,andD.Harwood,“Acomparativestudyoftexturemeasureswithclassicationbasedonfeaturedistributions,”PatternRecognition,vol.29,no.1,pp.51–59,1996.[12]T.Ojala,M.Pietik¨ainen,andT.M¨aenp¨a¨a,“Multiresolutiongray-scaleandrotationinvarianttextureclassicationwithlocalbinarypatterns,”IEEETransactionsonPatternAnalysisandMachineIntelligence,vol.24,no.7,pp.971–987,Jul2002.[13]R.GottumukkalandV.K.Asari,“AnimprovedfacerecognitiontechniquebasedonmodularPCAapproach,”PatternRecognitionLetters,vol.25,pp.429–436,March2004.[14]W.Zhao,R.Chellappa,P.J.Phillips,andA.Rosenfeld,“Facerecognition:Aliteraturesurvey,”ACMComputingSurveys,vol.35,no.4,pp.399–458,Dec2003.[15]J.R.Beveridge,D.Bolme,B.A.Draper,andM.Teixeira,“TheCSUfaceidenticationevaluationsystem:Itspurpose,features,andstructure,”MachineVisionandApplications,vol.16,no.2,pp.128–138,Feb2005.[16]P.J.Phillips,H.Wechsler,J.Huang,andP.Rauss,“TheFERETdatabaseandevaluationprocedureforfacerecognitionalgorithms,”ImageandVisionComputing,vol.16,no.10,pp.295–306,Apr1998.[17]B.Moghaddam,C.Nastar,andA.Pentland,“Abayesiansimilaritymeasurefordirectimagematching,”inProc.13thInternationalConferenceonPatternRecognition,1996,pp.II:350–358.[18]P.J.Phillips,H.Moon,S.A.Rizvi,andP.J.Rauss,“TheFERETevaluationmethodologyforfacerecognitionalgorithms,”IEEETransactionsonPatternAnalysisandMachineIntelligence,vol.22,no.10,pp.1090–1104,Oct2000.[19]B.S.Manjunath,J.-R.Ohm,V.V.Vasudevan,andA.Yamada,“Colorandtexture5thJune2006DRAFT 15descriptors,”IEEETransactionsonCircuitsandSystemsforVideoTechnology,vol.11,no.6,pp.703–715,Jun2001.[20]M.VarmaandA.Zisserman,“Textureclassication:Arelterbanksnecessary?”inProc.IEEEComputerSocietyConferenceonComputerVisionandPatternRecognition,2003,pp.691–698.[21]T.Ahonen,M.Pietik¨ainen,A.Hadid,andT.M¨aenp¨a¨a,“Facerecognitionbasedonappearanceoflocalregions,”inProc.17thInternationalConferenceonPatternRecognition,2004,pp.III:153–156.[22]A.Hadid,M.Pietik¨ainen,andT.Ahonen,“Adiscriminativefeaturespacefordetectingandrecognizingfaces,”inProc.IEEEComputerSocietyConferenceonComputerVisionandPatternRecognition,vol.2,2004,pp.797–804.[23]X.Feng,M.Pietik¨ainen,andA.Hadid,“Facialexpressionrecognitionwithlocalbinarypatternsandlinearprogramming,”PatternRecognitionandImageAnalysis,vol.15,no.2,pp.546–548,2005.[24]C.Shan,S.Gong,andP.W.McOwan,“Robustfacialexpressionrecognitionusinglocalbinarypatterns,”inProc.IEEEInternationalConferenceonImageProcessing,2005,pp.II:914–917.[25]G.Zhang,X.Huang,S.Z.Li,Y.Wang,andX.Wu,“Boostinglocalbinarypattern(LBP)-basedfacerecognition,”inProc.AdvancesinBiometricPersonAuthentication,ser.LectureNotesinComputerScience,vol.3338,2004,pp.179–186.[26]S.Z.Li,R.Chu,M.Ao,L.Zhang,andR.He,“Highlyaccurateandfastfacerecognitionusingnearinfraredimages,”inProc.InternationalConferenceonAdvancesinBiometrics,ser.LectureNotesinComputerScience,vol.3832.Springer,2006,pp.151–158.[27]W.Zhang,S.Shan,W.Gao,X.Chen,andH.Zhang,“Localgaborbinarypatternhistogramsequence(LGBPHS):Anovelnon-statisticalmodelforfacerepresentationandrecognition,”inProc.IEEEInternationalConferenceonComputerVision,2005,pp.I:786–791.[28]Y.RodriguezandS.Marcel,“Faceauthenticationusingadaptedlocalbinarypatternhistograms,”inProc.9thEuropeanConferenceonComputerVision,ser.LectureNotesinComputerScience,vol.3954.Springer,2006,pp.IV:321–332.5thJune2006DRAFT