FDDB A Benchmark for Face Detection in Unconstrained Settings Vidit Jain Univers - PDF document

Figure1.ExampleimagesfromBergetal.'sdataset. photographs,however,maynotbedigitallyidenticaltoeachotherbecausetheyareoftenmodied(e.g.,croppedorcontrast-corrected)beforepublication.Thisprocesshasledtothepresenceofmultiplecopiesofnear-duplicateim-agesinBergetal.'sdataset.Notethatthepresenceofsuchnear-duplicateimagesislimitedtoafewdatacollectiondo-mainssuchasnewsphotosandthoseontheinternet,andisnotacharacteristicofmostpracticalfacedetectionap-plicationscenarios.Forexample,itisuncommontondnear-duplicateimagesinapersonalphotocollection.Thus,anevaluationoffacedetectionalgorithmsonadatasetwithmultiplecopiesofnear-duplicateimagesmaynotgeneralizewellacrossdomains.Forthisreason,wedecidedtoidentifyandremoveasmanynearduplicatesfromourcollectionaspossible.Wenowpresentthedetailsoftheduplicatedetec-tion.4.Near-duplicatedetectionWeselectedatotalof3527images(basedonthechrono-logicalordering)fromtheimage-captionpairsofBergetal.[ 2 ].Examiningpairsforpossibleduplicatesinthiscol-lectioninthena¨vefashionwouldrequireapproximately12.5millionannotations.Analternativearrangementwouldbetodisplayasetofimagesandmanuallyidentifygroupsofimagesinthisset,whereimagesinasinglegrouparenear-duplicatesofeachother.Duetothelargenumberofimagesinourcollection,itisunclearhowtodisplayalltheimagessimultaneouslytoenablethismanualidenticationofnear-duplicatesinthisfashion.Identicationofnear-duplicateimageshasbeenstud-iedforwebsearch[ 3 , 4 , 5 ].However,inthewebsearchdomain,scalabilityissuesareoftenmoreimportantthanthedetectionofallnear-duplicateimagesinthecollec-tion.Sinceweareinterestedindiscoveringallofthenear-duplicatesinourdataset,theseapproachesarenotdirectlyapplicabletoourtask.Zhangetal.[ 29 ]presentedamorecomputationallyintensiveapproachbasedonstochasticat-tributerelationalgraph(ARG)matching.Theirapproach Figure3.Near-duplicateimages.(Positive)Thersttwoimagesdifferfromeachotherslightlyintheresolutionandthecolorandintensitydistributions,buttheposeandexpressionofthefacesareidentical,suggestingthattheywerederivedfromasinglephoto-graph.(Negative)Inthelasttwoimages,sincetheposeisdiffer-ent,wedonotconsiderthemasnear-identicalimages. wasshowntoperformwellonarelatedproblemofdetect-ingnear-identicalframesinnewsvideodatabases.TheseARGsrepresentthecompositionalpartsandpart-relationsofimagescenesoverseveralinterestpointsdetectedinanimage.TocomputeamatchingscorebetweentheARGsconstructedfortwodifferentimages,agenerativemodelforthegraphtransformationprocessisemployed.Thisap-proachhasbeenobservedtoachievehighrecallofnear- 3 6.EvaluationToestablishanevaluationcriterionfordetectionalgo-rithms,werstspecifysomeassumptionswemakeabouttheiroutputs.Weassumethat  Adetectioncorrespondstoacontiguousimageregion.  Anypost-processingrequiredtomergeoverlappingorsimilardetectionshasalreadybeendone.  Eachdetectioncorrespondstoexactlyoneentireface,nomore,noless.Inotherwords,adetectioncannotbeconsideredtodetecttwofacesatonce,andtwode-tectionscannotbeusedtogethertodetectasingleface.Wefurtherarguethatifanalgorithmdetectsmultipledisjointpartsofafaceasseparatedetections,onlyoneofthemshouldcontributetowardsapositivedetectionandtheremainingdetectionsshouldbeconsideredasfalsepositives.Torepresentthedegreeofmatchbetweenadetectiondiandanannotatedregionlj,weemploythecommonlyusedratioofintersectedareastojoinedareas:S(di;lj)=area(di)\area(lj) area(di)[area(lj): (2) Tospecifyamoreaccurateannotationfortheimagere-gionscorrespondingtohumanfacesthanisobtainedwiththecommonlyusedrectangularregions,wedeneanellip-ticalregionaroundthepixelscorrespondingtothesefaces.Whilethisrepresentationisnotasaccurateasapixel-levelannotation,itisaclearimprovementovertherectangularannotationsinexistingdatasets.Tofacilitatemanuallabeling,westartwithanautomatedguessaboutfacelocations.Toestimatetheellipticalbound-aryforafaceregion,werstapplyaskinclassierontheimagepixelsthatusestheirhueandsaturationvalues.Next,theholesintheresultingfaceregionarelledusingaood-llimplementationinMATLAB.Finally,amoments-basedtisperformedonthisregiontoobtaintheparametersofthedesiredellipse.Theparametersofalloftheseellipsesaremanuallyveriedandadjustedinthenalstage.6.1.MatchingdetectionsandannotationsAmajorremainingquestionishowtoestablishacor-respondencebetweenasetofdetectionsandasetofan-notations.Whileforverygoodresultsonagivenimage,thisproblemiseasy,itcanbesubtleandtrickyforlargenumbersoffalsepositivesormultipleoverlappingdetec-tions(seeFigure 8 foranexample).Below,weformulatethisproblemofmatchingannotationsanddetectionsasnd-ingamaximumweightedmatchinginabipartitegraph(asshowninFigure 9 ). Figure8.Matchingdetectionsandannotations.Inthisimage,theellipsesspecifythefaceannotationsandtheverectanglesdenoteafacedetector'soutput.Notethatthesecondfacefromlefthastwodetectionsoverlappingwithit.Werequireavalidmatchingtoacceptonlyoneofthesedetectionsasthetruematch,andtoconsidertheotherdetectionasafalsepositive.Also,notethatthethirdfacefromthelefthasnodetectionoverlappingwithit,sonodetectionshouldbematchedwiththisface.Thebluerectanglesdenotethetruepositivesandyellowrectanglesdenotethefalsepositivesinthedesiredmatching. Figure9.Maximumweightmatchinginabipartitegraph.Wemakeaninjective(one-to-one)mappingfromthesetofdetectedimageregionsditothesetofimageregionsliannotatedasfaceregions.Thepropertyoftheresultingmappingisthatitmaximizesthecumulativesimilarityscoreforallthedetectedimageregions. LetLbethesetofannotatedfaceregions(orlabels)andDbethesetofdetections.WeconstructagraphGwiththesetofnodesV=L[D.Eachnodediisconnectedtoeachlabellj2LwithanedgeweightwijasthescorecomputedinEquation 2 .Foreachdetectiondi2D,wefurtherintroduceanodenitocorrespondtothecasewhenthisdetectiondihasnomatchingfaceregioninL.AmatchingofdetectionstofaceregionsinthisgraphcorrespondstotheselectionofasetofedgesME.Inthedesiredmatchingofnodes,wewanteverydetectiontobematchedtoatmostonelabeledfaceregion,andeverylabeledfaceregiontobematchedtoatmostonedetection. 6 [10] W.Kienzle,G.H.Bakr,M.O.Franz,andB.Sch¨olkopf.Facedetection—efcientandrankdecient.InL.K.Saul,Y.Weiss,andL.Bottou,editors,AdvancesinNeuralIn-formationProcessingSystems,pages673–680,Cambridge,MA,2005.MITPress. 7 [11] H.W.Kuhn.TheHungarianmethodfortheassignmentprob-lem.NavalResearchLogisticsQuarterly,2:83–97,1955. 7 [12] S.Z.Li,L.Zhu,Z.Zhang,A.Blake,H.Zhang,andH.Shum.Statisticallearningofmulti-viewfacedetection.InEuropeanConferenceonComputerVision,pages67–81,London,UK,2002.Springer-Verlag. 2 [13] A.Loui,C.Judice,andS.Liu.Animagedatabaseforbench-markingofautomaticfacedetectionandrecognitionalgo-rithms.InIEEEInternationalConferenceonImagePro-cessing,volume1,pages146–150vol.1,Oct1998. 1 [14] K.Mikolajczyk,C.Schmid,andA.Zisserman.Humande-tectionbasedonaprobabilisticassemblyofrobustpartde-tectors.InEuropeanConferenceonComputerVision,pages69–82,2004. 7 [15] A.Y.Ng,M.I.Jordan,andY.Weiss.Onspectralclustering:Analysisandanalgorithm.InAdvancesinNeuralInforma-tionProcessingSystems,pages849–856.MITPress,2001. 4 [16] M.Osadchy,Y.LeCun,andM.L.Miller.Synergisticfacedetectionandposeestimationwithenergy-basedmodels.JournalofMachineLearningResearch,8:1197–1215,2007. 2 [17] J.Rihan,P.Kohli,andP.Torr.OBJCUTforfacedetection.InIndianConferenceonComputerVision,GraphicsandImageProcessing,pages576–584,2006. 2 [18] H.A.Rowley,S.Baluja,andT.Kanade.Neuralnetwork-basedfacedetection.IEEETransactionsonPatternAnalysisandMachineIntelligence,20(1):23–38,January1998. 1 , 2 [19] H.A.Rowley,S.Baluja,andT.Kanade.Rotationinvariantneuralnetwork-basedfacedetection.InIEEEConferenceonComputerVisionandPatternRecognition,page38,Wash-ington,DC,USA,1998.IEEEComputerSociety. 2 [20] H.SchneidermanandT.Kanade.Probabilisticmodelingoflocalappearanceandspatialrelationshipsforobjectrecogni-tion.InIEEEConferenceonComputerVisionandPatternRecognition,page45,Washington,DC,USA,1998.IEEEComputerSociety. 1 [21] H.SchneidermanandT.Kanade.Astatisticalmethodfor3dobjectdetectionappliedtofacesandcars.InIEEEConfer-enceonComputerVisionandPatternRecognition,volume1,pages746–751vol.1,2000. 1 , 2 [22] M.SeshadrinathanandJ.Ben-Arie.Poseinvariantfacede-tection.InVideo/ImageProcessingandMultimediaCommu-nications,2003.4thEURASIPConferencefocusedon,vol-ume1,pages405–410vol.1,July2003. 2 [23] P.SharmaandR.Reilly.Acolourfaceimagedatabaseforbenchmarkingofautomaticfacedetectionalgorithms.InEURASIPConferencefocusedonVideo/ImageProcessingandMultimediaCommunications,volume1,pages423–428vol.1,July2003. 1 [24] K.-K.SungandT.Poggio.Example-basedlearningforview-basedhumanfacedetection.IEEETransactionsonPatternAnalysisandMachineIntelligence,20(1):39–51,1998. 1 , 2 [25] http://mplab.ucsd.edu .TheMPLabGENKIDatabase,GENKI-4KSubset. 1 [26] P.A.ViolaandM.J.Jones.Robustreal-timefacedetec-tion.InternationalJournalofComputerVision,57(2):137–154,May2004. 2 , 7 [27] P.WangandQ.Ji.Multi-viewfaceandeyedetectionusingdiscriminantfeatures.ComputerVisionandImageUnder-standing,105(2):99–111,2007. 2 [28] M.-H.Yang,D.J.Kriegman,andN.Ahuja.Detectingfacesinimages:Asurvey.IEEETransactionsonPatternAnalysisandMachineIntelligence,24(1):34–58,2002. 1 , 2 [29] D.-Q.ZhangandS.-F.Chang.Detectingimagenear-duplicatebystochasticattributedrelationalgraphmatchingwithlearning.InACMInternationalConferenceonMulti-media,pages877–884,2004. 3 A.Guidelinesforannotatingfacesusingel-lipsesToensureconsistencyacrossmultiplehumanannotators,wedevelopedasetofinstructions(showninFigure 11 ).Theseinstructionsspecifyhowtousefaciallandmarkstotanellipsedependingontheposeofthehead.Figure 12 presentsanillustrationoftheresultingellipsesonlinedraw-ingsofahumanhead.Theannotatorswerefutherinstructedtofollowacombinationoftheseguidelinestotellipsestofaceswithcomplexheadposes.TheillustrationsshowninFigure 12 usefaceswithneu-tralexpressions.Apresenceofsomeexpressionssuchaslaughter,oftenchangestheshapeofthefacesignicantly.Moreover,evenbearinganeutralexpression,somefaceshaveshapesmarkedlydifferentfromtheaveragefaceshapeusedintheseillustrations.Suchfaces(e.g.,faceswithsquare-jawordouble-chin)aredifculttoapproximateus-ingellipses.Toannotatefaceswithsuchcomplexities,theannotatorswereinstructedtorefertothefollowingguide-lines:  Facialexpression.Sincethedistancefromtheeyestothechininafacewithfacialexpressionisnotnecessar-ilyequaltothedistancebetweentheeyesandthetopofthehead(anassumptionmadefortheidealhead),theeyesdonotneedtobealignedtotheminoraxisforthisface.  Double-chin.Forfaceswithadoublechin,theaver-ageofthetwochinsisconsideredasthelowestpointoftheface,andismatchedtothebottomextremeofthemajoraxisoftheellipse.  Squarejaw.Forafacewithasquarejaw,theel-lipsetracestheboundarybetweenthefaceandtheears,whilesomepartofthejawsmaybeexcludedfromtheellipse. 9 Figure11.Procedurefordrawingellipsesaroundanaveragefaceregion.Theannotatorswereinstructedtofollowthisowcharttodrawellipsesaroundthefaceregions.Theannotationstepsarealittledifferentfordifferentposes.Here,wepresentthestepsforthreecanonicalposes:frontal,proleandtiltedback/front.Theannotatorswereinstructedtouseacombinationofthesestepsforlabelingfaceswithderived,intermediateheadposes.Forinstance,tolabelaheadfacingslightlytowardsitsrightandtitledback,acombinationofthestepscorrespondingtotheproleandtilted-backposesareused. Figure12.Illustrationsofellipselabelingonlinedrawingsofhumanhead.Theblackcurvesshowtheboundariesofahumanheadinfrontal(left),prole(center),andtilted-back(right)poses.TheredellipsesillustratethedesiredannotationsaspertheprocedureshowninFigure 11 .Notethattheseheadshapesareapproximationstoanaveragehumanhead,andtheshapeofanactualhumanheadmaydeviatefromthismeanshape.Theshapeofahumanheadmayalsobeaffectedbythepresenceoffactorssuchasemotions.TheguidelinesonannotatingfaceregionsinuencedbythesefactorsarespeciedinAppendix A .  Hair.Ignorethehairandttheellipsearoundthehy-potheticalbaldhead.  Occlusion.Hypothesizethefullfacebehindtheoc-cludingobject,andmatchallofthevisiblefeatures. 10