Efficient Shape Matching Using Shape Contexts Greg Mori Member IEEE Serge Belongie Member - PDF document

EfficientShapeMatchingUsingShapeContextsGregMori,MemberIEEESergeBelongie,Member,andJitendraMalik,SeniorMemberIEEEAbstract—Wedemonstratethatshapecontextscanbeusedtoquicklypruneasearchforsimilarshapes.Wepresenttwoalgorithmsforrapidshaperetrieval:representativeshapecontexts,performingcomparisonsbasedonasmallnumberofshapecontexts,andshapemes,usingvectorquantizationinthespaceofshapecontextstoobtainprototypicalshapepieces.IndexTerms—Shape,objectrecognition,opticalcharacterrecognition. NTRODUCTION G.MoriiswiththeSchoolofComputingScience,SimonFraserUniversity,Burnaby,BC,CanadaV5A1S6.E-mail:mori@cs.sfu.ca.S.BelongieiswiththeDepartmentofComputerScienceandEngineering,UniversityofCaliforniaatSanDiego,LaJolla,CA92093.E-mail:sjb@cs.ucsd.edu.J.MalikiswiththeElectricalEngineeringandComputerScienceDivision,UniversityofCaliforniaatBerkeley,Berkeley,CA94720.E-mail:malik@cs.berkeley.edu.Manuscriptreceived27May2004;revised30Mar.2005;accepted31Mar.2005;publishedonline14Sept.2005.RecommendedforacceptancebyR.Basri.Forinformationonobtainingreprintsofthisarticle,pleasesende-mailto:tpami@computer.org,andreferenceIEEECSLogNumberTPAMI-0267-0504.0162-8828/05/$20.002005IEEEPublishedbytheIEEEComputerSociety thisgenre.Severalotherapproachesinthisvein[8],[9]firstattempttofindcorrespondencesbetweenthetwoimagesbeforedoingthecomparison.Thisturnsouttobequiteachallengeasdifferentialopticalflowtechniquesdonotcopewellwiththelargedistortionsthatmustbehandledduetopose/illuminationvariations.Errorsinfindingcorrespondencewillcausedown-streamprocessingerrorsintherecognitionstage.Asanalternative,thereareanumberofmethodsthatbuildclassifierswithoutexplicitlyfindingcorrespondences.Insuchapproaches,onereliesonalearningalgorithmhavingenoughexamplestoacquiretheappropriateinvariances.Theseapproacheshavebeenusedforhandwrittendigitrecognition[10],[11],facerecognition[12],andisolated3Dobjectrecognition[13].Incontrast,techniquesthatperformrecognitionbasedonshapeinformationattempttocaptureaglobalstructureofextractededgeorsilhouettefeatures.Silhouetteshavebeendescribed(andcompared)usingFourierdescriptors[14],skeletonsderivedusingBlum’smedialaxistransform[15],ordirectlymatchedusingdynamicprogramming.Althoughsilhouettesaresimpleandefficienttocompare,theyarelimitedasshapedescriptorsforgeneral3Dobjectsbecausetheyignoreinternalcontoursandaredifficulttoextractfromrealimages.Otherapproaches[16],[17],[18]treattheshapeasasetofpointsinthe2Dimage,extractedusing,say,anedgedetector.Anothersetofmethodscomputecorrespondencesbetweenedgepoints,suchastheworkofCarlsson[19],whichusesorderstructure,andtheworkofJohnsonandHebert[20]andChuiandRangarajan[21].Recentyearshaveseentheemergenceofhybridapproaches[22],[23],[24],[25]thatcaptureappearanceinformationthroughacollectionoflocalimagepatches.Shapeinformationisencodedviaspatialrelationshipsbetweenthelocalpatches.Thelocationsforthelocalpatchesareselectedwithvariousinterestpointoperatorsandarerepresentedeitherasrawpixelvalues[23]orhistogramsofimagegradients[22],[24],termedSIFTdescriptors(ScaleInvariantFeatureTransform).Thislineofworkhasbeendemonstratedtobeeffectiveindetectingrelativelysmallnumbersofcategories.However,theproblemofscalingtolargedatabasesofknownobjectswiththesemethodsremainsopen.Oftheapproachesmentionedabove,theworkbyLowe[22]hasgonethefurthestinaddressingtheissuesoflargedatasets.Theapproachinvolvesefficientlymatchingfeaturesbysearchingk-dtreeswithanalgorithmcalled“Best-Bin-First”[26].Thealgorithmswedevelopinthispaperwillbebasedontheshapecontextpointdescriptor.Inparticular,therepresentativeshape(Section4.1)algorithmisrelatedtotheaboveworkonlocalpatchmodels.Themajordifferencesareinthescopeofthedescriptorandthelocationsatwhichtheyarecomputed.Shapecontextsarearelativelylargescalepointdescriptor.Witharadiusofapproximatelyhalfthediameterofanobject,eachshapecontextcapturesinformationfromalmosttheentireshape.Second,therepresentativeshapecontextsareplacedatrandomlyselectededgepointsspreadovertheentireshape,asopposedtotheinterestingpointsselectedintheotherapproaches.Otherworkonefficientshape-basedretrievalincludesthatbySebastianetal.[27],whoimprovedtheefficiencyofshock-graphshapematchingusingacoarse-levelmatchingphase.Shakhnar-ovichetal.[28]usedavariantofthe“LocalitySensitiveHashing”(LSH)ofIndykandMotwani[29]toquicklyretrievehumanbodyshapes.Fromeetal.[30]alsousedLSHtoperformefficientretrievalofshapes;theirworkinvolved3Dshapeinformationobtainedfromlaserrangescanners.TheworkbyBelongieetal.[3]resultedinextremelygoodperformance,e.g.,99.4percentaccuracyontheMNISThand-writtendigitset,aswellasonavarietyof3Dobjectrecognitionproblems.However,applyingthisdeformablematchingalgorithmtoalargedatabaseofmodelswouldbecomputationallyprohibitive.Todealwiththisproblem,wewilluseatwo-stageapproachtoobjectrecognition:fastpruningfollowedbydetailedInthefollowingsections,wefirstdescribeanewdescriptorthatisanextensionofshapecontextsandthendevelopfastpruningtechniquesbaseduponthisdescriptor.3.1GeneralizedShapeContextsThespatialstructureoftheshapecontexthistogrambins,withcentralbinssmallerthanthoseintheperiphery,resultsinadescriptorthatismorepreciseaboutthelocationofnearbyfeaturesandlesspreciseaboutthosefartheraway.Thissamestructurecanbeappliedtoconstructaricherdescriptorbasedonorientededges.Inthiswork,toeachedgepoint,weattachaunitlengthtangentthatisthedirectionoftheedgeat.Ineachbin,wesumthetangentvectorsforallpointsfallinginthebin.ThedescriptorforapointisthehistogramEachbinnowholdsasinglevectorinthedirectionofthedominantorientationofedgesinthebin.Whencomparingthedescriptorsfortwopoints,weconvertthishistogramtoa,normalizethesevectors,andcomparethemusingthe...GSCÞ¼jjarethecomponentsof,respectively.WecalltheseextendeddescriptorsgeneralizedshapecontextsNotethatgeneralizedshapecontextsreducetotheoriginalshapecontextsifalltangentanglesareclampedtozero.OurexperimentsinSection5willcomparethesenewdescriptorswiththeoriginalshapecontexts. IEEETRANSACTIONSONPATTERNANALYSISANDMACHINEINTELLIGENCE,VOL.27,NO.11,NOVEMBER2005 Fig.1.Shapecontexts.(a)and(b)Samplededgepointsoftwoshapes.(c)Diagramoflog-polarhistogrambinsusedincomputingtheshapecontexts.Weusefivebinsforand12binsfor.(d)-(f)Exampleshapecontextsforreferencesamplesmarkedbyin(a)and(b).Eachshapecontextisalog-polarhistogramofthecoordinatesoftherestofthepointsetmeasuredusingthereferencepointastheorigin.(Dark=largevalue.) Givenalargesetofknownshapes,theproblemistodeterminewhichoftheseshapesismostsimilartoaqueryshape.Fromthissetofshapes,wewishtoquicklyconstructashortlistofcandidateshapeswhichincludesthebestmatchingshape.Aftercompletingthiscoarsecomparisonstep,onecanthenapplyamoretimeconsuming,andmoreaccurate,comparisontechniquetoonlytheshortlist.Weleveragethedescriptivepowerofshapecontextstowardthisgoalofquickpruning.Weproposetwomatchingmethodsthataddresstheseissues.Inthefirstmethod,representativeshapecontexts(RSCs),wecomputeafewshapecontextsforthequeryshapeandattempttomatchusingonlythose.Thesecondmethod,,usesvectorquantizationtoreducethecomplexityoftheshapecontextsfrom60-dimen-sionalhistogramstoquantizedclassesofshapepieces.4.1RepresentativeShapeContextsGiventwoeasilydiscriminableshapes,suchastheoutlinesofafishandabicycle,wedonotneedtocompareeverypairofshapecontextsontheobjectstoknowthattheyaredifferent.Whentryingtomatchthedissimilarfishandbicycle,noneoftheshapecontextsfromthebicyclehavegoodmatchesonthefish—itisimmediatelyobviousthattheyaredifferentshapes.Fig.2demonstratesthisprocess.Thefirstpruningmethod,representativeshapecontexts,usesthisintuition.Inconcreteterms,thematchingprocessproceedsinthefollowingmanner:Foreachoftheknownshapes,weprecomputealargenumber(about100)ofshapecontexts.But,forthequeryshape,weonlycomputeasmallnumberinexperiments)ofshapecontexts.Tocomputetheseshapecontexts,werandomlyselectpointsfromtheshapeviaarejectionsamplingmethodthatspreadsthepointsovertheentireshape.Weuseallthesamplepointsontheshapetofillthehistogrambinsfortheshapecontextscorrespondingtothesepoints.Tocomputethedistancebetweenaqueryshapeandaknownshape,wefindthebestmatchesforeachoftheNotethat,inclutteredimages,manyoftheRSCscontainnoisydataorarenotlocatedontheshape.Hence,foreachoftheknownshapes,wefindthebestRSCs,theoneswiththesmallestdistances.Callthissetofindices.Thedistancebetweenisthen:Q;S kXu2Gi GSC;SCargminGSC;SCisanormalizingfactorthatmeasureshowdiscriminativetherepresentativeshapecontext GSC;SCisthesetofallshapes.Wedeterminetheshortlistbysortingthesedistances.4.2ShapemesThesecondmatchingmethodusesvectorquantizationontheshapecontexts.Withknownshapes,andshapecontextscomputedatsamplepointsontheseshapes,thefullsetofshapecontextsfortheknownshapesconsistsofvectors.Astandardtechniqueincompressionfordealingwithsuchalargeamountofdataisvectorquantization.Vectorquantizationinvolvesclusteringthevectorsandthenrepresentingeachvectorbytheindexoftheclusterthatitbelongsto.Wecalltheseclusters—canonicalshapepieces.Fig.3showstherepresentationofsamplepointsasshapemelabels.Toderivetheseshapemes,alloftheshapecontextsfromtheknownsetareconsideredaspointsinaspace.WeclusteringtoobtainWerepresenteachknownviewasacollectionofshapemes.shapecontextisquantizedtoitsnearestshapemeandreplacedbytheshapemelabel(anintegerin).Aknownviewisthensimplifiedintoahistogramofshapemefrequencies.Nospatialinformationamongtheshapemesisstored.Wehavereducedeachcollectionofshapecontexts(binhistograms)toasinglehistogramwithInordertomatchaqueryshape,wesimplyperformthissamevectorquantizationandhistogramcreationoperationontheshapecontextsfromthequeryshape.Wethenfindnearestneighborsinthespaceofhistogramsofshapemes.WeusetheETH-80ObjectDatabase,theSnodgrassandVanderwartlinedrawings,andtheEZ-GimpyCAPTCHAas1834IEEETRANSACTIONSONPATTERNANALYSISANDMACHINEINTELLIGENCE,VOL.27,NO.11,NOVEMBER2005 Fig.2.Matchingindividualshapecontexts.Threepointsonthequeryshape(left)areconnectedwiththeirbestmatchesontwoknownshapes.distancesaregivenwitheachmatching. Fig.3.(a)Linedrawings.(b)Sampledpointswithshapemelabels.emeswereextractedfromaknownsetof260shapes(26,000generalizedshapecontexts).Notethesimilaritiesinshapemelabels(2,41ontheleftside,24,86,97ontherightside)betweensimilarportionsoftheshapes. ourtestsets.Inthefollowingsections,wepresentgraphsshowingtheperformanceofthetwomethodsonthesetestsets.Thegraphsploterrorrateversuspruningfactor(onaTheerrorratecomputationassumesaperfectdetailedmatchingphase.Thatis,aqueryshapeproducesanerroronlyifthereisnocorrectlymatchingshapeintheshortlistobtainedbythepruningmethod.Theabscissaoneachofthegraphsshowsthepruningfactor,definedtobeShortlist.Forexample,withknownshapes,ifthepruningfactoris26,thentheshortlisthas10shapesinit.Ingeneral,therepresentativeshapecontextsmethodperformsbetteratlargepruningfactors—particularlywhendealingwithocclusion.Missingacoupleofshapecontextswon’tspoilthematching.However,thevectorquantizationusedinshapemesdoesbuyuscomputationalspeedandusingalloftheshapecontextsinthismannerallowslowerrorratestobeobtained.5.1ETH-80ThefirstexperimentinvolvestheETH-80database[4].Thedatabaseconsistsof80uniqueobjects,eachfromoneofeightclasses.Eachobjectisrepresentedby41viewsspacedevenlyovertheupperviewinghemisphere.Wepreparedasetofknownshapesbyselectingoneobjectfromeachoftheeightclassesandusingallofitsviewsinthetrainingset,atotalof328images.Theimagesoftheremainingnineobjectsfromeachclasswereusedasatestset.Thisexperimentwasrepeated10times,eachtimeselectingadifferentsetoftrainingobjects.Weuseanedgedetector[31]toextractlinefeaturesfromtheimages.Theseedgesarethensampledtocreatepointfeaturesforuseinshapecontexts.Weranexperimentsusingthetwopruningmethods.Representativeshapecontextspruningwasdoneusing4,8,12,and16shapecontextsandgeneralizedshapecontexts.Ineachoftheseexperiments,thebest oftheRSCs(i.e.,3,6,9,and12)wereusedtocomputethematchingcost.Shapemepruningwasperformedwithquantizationto25,50,75,100,125,and150shapemes,againusingbothtypesofshapecontexts.ResultsarepresentedinFig.4.Bothofthepruningmethodsaresuccessful:Forexample,apruningfactorofapproximately40(shortlistoflength8)canbeobtainedwithanerrorrateof10percentfortherepresentativeshapecontextsmethod(16RSCsusinggeneralizedshapecontexts)and14percentfortheshapememethod(150shapemesusinggeneralizedshapecontexts).Fig.5showssomeshortlistsontheETH-80datasetusingtherepresentativeshapecontextspruningmethod.Manyoftheerrorsonthisdatasetinvolveobjectsthathavethesamecoarseshape.Forexample,theshapematchingprocessdeemsthetomatoesandapplestobeverysimilar.Relyingsolelyoncoarseshape,withoutcuessuchascolorandtexture,itisdifficulttodifferentiatebetweenthemembersofthesegroupsofobjects.5.2SnodgrassandVanderwartThesecondexperimentusestheSnodgrassandVanderwartlinedrawings[5].Thisdatasetcontainslinedrawingsof260commonlyoccurringobjects.Theyareastandardsetofobjectsthathavebeenfrequentlyusedinthepsychophysicscommunityfortestswithhumansubjects.Sincetheimagesarelinedrawings,nopreproces-singphaseofedgeextractionisneeded.Wesamplepointsfromthelinedrawingsdirectlyanduseelongatedorientedfilterstoestimatelocaltangentdirections.TheSnodgrassandVanderwartdatasethasonlyoneimageperobject.Weusetheseoriginalimagesastheknownsetandcreateasyntheticdistortedsetofimagesforquerying.Thethinplatespline(TPS)model[32]isusedtocreatethesedistortions.Ina2Dviewofaclassof3Dobjects,therearetwosourcesofvariation:posechangeandintraclasschange.WeusethenonlinearTPSmodeltosimulatebothofthesetypesofvariationsimultaneously.WeapplyarandomTPSwarpoffixedbendingenergytoareferencegridandusethiswarptotransformtheedgepointsofalinedrawing.Inadditiontodistortions,wetesttheabilityofourpruningmethodstohandleocclusion.WetakethesetofTPS-distortedobjectsandsubjectthemtorandomocclusions.Theocclusionsaregeneratedusingalinearoccludingcontour.ThequeryobjectsinFig.6showsomedistortedandoccludedSnodgrassandVander-wartimages.Notethattheoccludingcontourisincluded—wewillsamplepointsfromitwhencreatingtheshapecontexts.The260originalSnodgrassandVanderwartimageswereusedasthetrainingset.Wegenerated5,200distortedandoccludedimages(20peroriginalimage)foruseasatestset.Theoccludedimagesweresplitintolevelsofdifficultyaccordingtothepercentageofedgepixelslostunderocclusion.ThesamesetoftestparametersasintheexperimentsontheETH-80datasetwasused.Figs.6and7showtheresultsforourtwopruningmethods.Inthelowocclusionsetting(10percentocclusion),theshapememethodcanachieveapruningfactorof(shortlistoflengththreeoutof260images)withanerrorrateof10percent(150shapemes,originalshapecontexts),whiletherepresentativeshapecontextsmethodhasanerrorrateof4percent(16RSCs,originalshapecontexts). IEEETRANSACTIONSONPATTERNANALYSISANDMACHINEINTELLIGENCE,VOL.27,NO.11,NOVEMBER2005 Fig.4.ErrorrateversuspruningfactoronETH-80dataset,averagedacross10runs.Comparisonofresultsforbestparametersettingsforeachofthefourmethodsisshown.Errorbarsareomittedforclarity,butthestandarddeviationissmall.Themaximumstandarddeviationoverallrunsis0.54percent. Fig.5.ShortlistsfortheETH-80datasetusingtherepresentativeshapecontextsmethod.Thefirstcolumnisthequeryobject.Theremainingfivecolumnsshowclosestmatchestoeachqueryobject. Withextremelydifficultlevelsofocclusion(20percentto30percentand30percentto40percent),RSCscanobtainlargeamountsofpruningwithreasonableerrorrates,whileshapemesareabletooperateatlowerrorrateswithmoderatepruningastheyefficientlyuseallshapecontextsonaqueryshape.Notethat,onthisdataset,thegeneralizedshapecontextsperformslightlyworsethantheoriginalshapecontextdescriptors.ThereasonforthisisthatthesyntheticTPSdistortionsusedtocreatethetestsetcorruptthetangentvectorsusedingeneralizedshapecontexts.TherandomTPSdistortionscontainlocalscalewarpsthatdeformthetangentvectorsgreatly.5.3EZ-GimpyACAPTCHAisaprogram[6]thatcangenerateandgradeteststhatmosthumanscanpass,butcurrentcomputerprogramscan’tpass.CAPTCHAstandsfor“CompletelyAutomatedPublicTuringtesttoTellComputersandHumansApart.”Blum’sgrouphasdesignedanumberofdifferentCAPTCHAs.EZ-Gimpy(Fig.8)isaCAPTCHAbasedonwordrecognitioninthepresenceofclutter.Thetaskistoidentifyasingleword,chosenfromaknowndictionary,thathasbeendistortedandplacedinaclutteredimage.TheCAPTCHAdatasetsprovidemorethanjustacolorfultoyproblemtoworkon.Theypresentchallengingcluttersincetheyareintendedtobedifficultforcomputerprograms.Moreimportantly,thesedatasetsarelarge.Thereare561wordsthatneedtoberecognizedinEZ-Gimpy.Also,sincethesourcecodeforgeneratingtheseCAPTCHAsisavailable(“P”for),wehaveaccesstoapracticallyinfinitesetoftestimages.Thisisincontrasttomanyobjectrecognitiondatasetsinwhichthenumberofobjectsislimitedanditisdifficulttogeneratemanyreasonabletestimages.However,therearedefinitelylimitationstothisdatasetintermsofstudyinggeneralobjectrecognition.Mostnotably,theseare2Dobjectsandthereisnovariationdueto3Dpose.Inaddition,therearenoshadingandlightingeffectsinsyntheticimagesofForourexperiments,atrainingsetofthe561words,eachpresentedundistortedonanunclutteredbackground,wascon-structed.Weappliedtherepresentativeshapecontextspruningmethod,usingthe561wordsasourobjects,followedbydetailedmatching(usingthemethodofBelongieetal.[3])torecognizethewordineachEZ-Gimpyimage.Thisalgorithmisreferredtoas“AlgorithmB”inourpreviousworkonbreakingCAPTCHAs[33].Twodetailsaredifferentfromthoseinthefirsttwoexperiments.First,weconstructedgeneralizedshapecontextsthataretunedtotheshapeofwords:Theyareelliptical,withanouterradiusofaboutfourcharactershorizontallyand ofacharactervertically.Second,thetexturegradientoperator[31]wasusedtoselecttheplacementoftheRSCs,whileCannyedgedetectionisusedtofindedgepixelstofillthebinsoftheshapecontexts.Wegenerated200examplesoftheEZ-GimpyCAPTCHA.Oftheseexamples,ninewereusedfortuningparametersinthetexturegradientmodules.Theremaining191exampleswereusedasatestset.ExamplesoftheEZ-GimpyCAPTCHAimagesusedandthetopmatchingwordsareshowninFig.8,thefullsetoftest1836IEEETRANSACTIONSONPATTERNANALYSISANDMACHINEINTELLIGENCE,VOL.27,NO.11,NOVEMBER2005 Fig.8.ResultsonEZ-Gimpyimages.Thebestmatchingwordsareasfollows:(a)horse,(b)jewel,(c)weight,(d)sound,(e)rice,and(f)space. Fig.7.ErrorrateversuspruningfactorontheSnodgrassandVanderwartdataset.(a)and(b)Variationinperformancewithrespecttotheamountofocclusioninthetestimage.(c)Comparativeresultsforalldifferentmethods.Resultsforthebestparametersettingsfromeachmethodareshown. Fig.6.ShortlistsforthedistortedandoccludedSnodgrassandVanderwartdatasetusingtherepresentativeshapecontextsmethod.Thefirstcolumnisthequeryobject.Theremainingfivecolumnsshowclosestmatchestoeachqueryobject. imagesandresultscanbeviewedathttp://www.cs.sfu.ca/~mori/research/gimpy/ez/.In92percent(176/191)ofthesetestcases,ourmethodidentifiedthecorrectword.ThissuccessratecomparesfavorablywiththatofThayananthanetal.[34]whoperformanexhaustivesearchusingChamfermatchingwithdistortedprototypewords.Ofthe15errorsmade,ninewereerrorsintheRSCpruning.Thepruningphasereducedthe561wordstoashortlistoflength10.Fornineofthetestimages,thecorrectwordwasnotontheshortlist.Intheothersixfailurecases,thedeformablematchingselectedanincorrectwordfromtheshortlist.ThegeneralizedshapecontextsaremuchmoreresilienttotheclutterintheEZ-Gimpyimagesthantheoriginalshapecontexts.Thesamealgorithm,runusingtheoriginalshapecontexts,attainsonlya53percentsuccessrateonthetestset.Previousworkonshapematchingviaadeformabletemplate-basedframeworkhasbeenverysuccessfulforobjectrecognition.However,thesemethodsaretooexpensivecomputationallytobeusedonalargescaleobjectdatabase.Wehaveshownhowashapecontext-basedpruningapproachcanassistbyconstructinganaccurateshortlistinordertoreducethiscomputationalexpense.Weproposedtwomethodsofmatching—oneusingasmallnumberofrepresentativeshapecontextsandtheotherbasedonvectorquantizationofshapecontextsintoshapemes.Wealsopresentedgeneralizedshapecontexts(GSCs),anextensiontoshapecontextswhichmakesuseoflocaltangentinformationatpointlocations.Thesedescriptorscontainmoredetailedinformationabouttheshapeand,whenthelocaltangentcanbereliablyestimated,theyoutperformtheoriginalshapeTheGSCissimilartoLowe’sSIFTdescriptor,whichalsoaggregatesedgeorientationsintoahistogram.However,thespatialstructureofthehistogrambinsofGSCsisverydifferentfromthatofSIFTfeatures.GSCsarelargeinscale,whileSIFTfeaturesarelocaldescriptors.SIFTfeaturesusearegulargridforhistogrambinsanddisregardinformationfarawayfromthecenterofthedescriptor,usingaGaussianweightingtodiscountsamplepoints.Incontrast,theoutermostbinsofGSCsarelargestinsize,reflectingpositionaluncertaintyofusefulcoarseshapecues.Wedemonstratedtheeffectivenessofrepresentativeshapecontextsandshapemes,twoefficientpruningmechanismsbasedonshapecontextsandGSCs,inexperimentsontheETH-80,SnodgrassandVanderwart,andEZ-Gimpydatasets.Portionsofthispaperappearedinpreviouspublications[35],[33].GregMoriwassupportedbygrantsfromNSERC(RGPIN-312230)andtheSFUPresident’sResearchFund.SergeBelongiewassupportedbyUSNationalScienceFoundationCAREER#0448615,TheAlfredP.SloanResearchFellowship,andthroughsubcon-tractsB542001andB547328undertheauspicesoftheUSDepartmentofEnergybytheLawrenceLivermoreNationalLaboratoryundercontractNo.W-7405-ENG-48.Theauthorsthanktheanonymousreviewersfortheirusefulsuggestions.[1]S.Thorpe,D.Fize,andC.Marlot,“SpeedofProcessingintheHumanVisualSystem,”vol.381,pp.520-522,1996.1996.I.Biederman,“Recognition-by-Components:ATheoryofHumanImageUnderstanding,”PsychologicalRev.,vol.94,no.2,pp.115-147,1987.1987.S.Belongie,J.Malik,andJ.Puzicha,“ShapeMatchingandObjectRecognitionUsingShapeContexts,”IEEETrans.PatternAnalysisandMachineIntelligence,vol.24,no.4,pp.509-522,Apr.2002.2002.B.LeibeandB.Schiele,“AnalyzingAppearanceandContourBasedMethodsforObjectCategorization,”Proc.IEEEConf.ComputerVisionandPatternRecognition,vol.2,pp.409-415,2003.2003.J.G.SnodgrassandM.Vanderwart,“AStandardizedSetof260Pictures:NormsforNameAgreement,FamiliarityandVisualComplexity,”J.ExperimentalPsychology:HumanLearningandMemory,vol.6,pp.174-215,1980.1980.L.vonAhn,M.Blum,andJ.Langford,“TellingHumansandComputersApart(Automatically),”CMUTechnicalReportCMU-CS-02-117,Feb.2002.2002.M.TurkandA.Pentland,“EigenfacesforRecognition,”J.CognitiveNeuroscience,vol.3,no.1,pp.71-96,1991.1991.M.Lades,C.Vorbruggen,J.Buhmann,J.Lange,C.vonderMalsburg,R.Wurtz,andW.Konen,“DistortionInvariantObjectRecognitionintheDynamicLinkArchitecture,”IEEETrans.Computers,vol.42,no.3,pp.300-311,Mar.1993.1993.T.Cootes,D.Cooper,C.Taylor,andJ.Graham,“ActiveShapeModels—TheirTrainingandApplication,”ComputerVisionandImageUnderstanding,vol.61,no.1,pp.38-59,Jan.1995.1995.Y.LeCun,L.Bottou,Y.Bengio,andP.Haffner,“Gradient-BasedLearningAppliedtoDocumentRecognition,”Proc.IEEE,vol.86,no.11,pp.2278-2324,Nov.1998.1998.C.BurgesandB.Scholkopf,“ImprovingtheAccuracyandSpeedofSupportVectorMachines,”AdvancesinNeuralInformationProcessingpp.375-381,1997.1997.B.Moghaddam,T.Jebara,andA.Pentland,“BayesianFaceRecognition,”PatternRecognition,vol.33,no.11,pp.1771-1782,Nov.2000.2000.H.MuraseandS.Nayar,“VisualLearningandRecognitionof3-DObjectsfromAppearance,”Int’lJ.ComputerVision,vol.14,no.1,pp.5-24,Jan.1995.995.C.ZahnandR.Roskies,“FourierDescriptorsforPlaneClosedCurves,”IEEETrans.Computers,vol.21,no.3,pp.269-281,Mar.1972.1972.D.Sharvit,J.Chan,H.Tek,andB.Kimia,“Symmetry-BasedIndexingofImageDatabases,”J.VisualComm.andImageRepresentation,June1998.1998.G.Borgefors,“HierarchicalChamferMatching:AParametricEdgeMatchingAlgorithm,”IEEETrans.PatternAnalysisandMachineIntelligence,vol.10,no.6,pp.849-865,1988.1988.D.Huttenlocher,R.Lilien,andC.Olson,“View-BasedRecognitionUsinganEigenspaceApproximationtotheHausdorffMeasure,”Trans.PatternAnalysisandMachineIntelligence,vol.21,no.9,pp.951-955,Sept.1999.999.D.GavrilaandV.Philomin,“Real-TimeObjectDetectionforSmartProc.SeventhInt’lConf.ComputerVision,pp.87-93,1999.1999.S.Carlsson,“OrderStructure,CorrespondenceandShapeBasedCate-ShapeContourandGroupinginComputerVision,pp.58-71,1999.1999.A.E.JohnsonandM.Hebert,“RecognizingObjectsbyMatchingOrientedProc.IEEEConf.ComputerVisionandPatternRecognition,pp.684-689,1997.1997.H.ChuiandA.Rangarajan,“ANewAlgorithmforNon-RigidPointMatching,”Proc.IEEEConf.ComputerVisionandPatternRecognition,vol.2,pp.44-51,June2000.2000.D.G.Lowe,“DistinctiveImageFeaturesfromScale-InvariantKeypoints,”Int’lJ.ComputerVision,vol.60,no.2,pp.91-110,2004.2004.R.Fergus,P.Perona,andA.Zisserman,“ObjectClassRecognitionbyUnsupervisedScale-InvariantLearning,”Proc.IEEEConf.ComputerVisionandPatternRecognition,vol.2,pp.264-271,2003.2003.G.DorkoandC.Schmid,“SelectionofScaleInvariantNeighborhoodsforObjectClassRecognition,”Proc.NinthInt’lConf.ComputerVision,pp.634-640,2003.2003.Y.Amit,D.Geman,andK.Wilder,“JointInductionofShapeFeaturesandTreeClassifiers,”IEEETrans.PatternAnalysisandMachineIntelligence,vol.19,no.11,pp.1300-1305,Nov.1997.1997.J.BeisandD.Lowe,“ShapeIndexingUsingApproximateNearest-NeighbourSearchinHighdimensionalSpaces,”Proc.IEEEConf.ComputerVisionandPatternRecognition,pp.1000-1006,1997.997.T.Sebastian,P.N.Klein,andB.B.Kimia,“Shock-BasedIndexingintoLargeShapeDatabases,”Proc.EuropeanConf.ComputerVision,vol.3,pp.731-746,731-746,G.Shakhnarovich,P.Viola,andT.Darrell,“FastPoseEstimationwithParameterSensitiveHashing,”Proc.NinthInt’lConf.ComputerVision,vol.2,pp.750-757,2003.2003.P.IndykandR.Motwani,“ApproximateNearestNeighbors:TowardsRemovingtheCurseofDimensionality,”Proc.ACMSymp.Theoryofpp.604-613,1998.1998.A.Frome,D.Huber,R.Kolluri,T.Bulow,andJ.Malik,“RecognizingObjectsinRangeDataUsingRegionalPointDescriptors,”Proc.EighthEuropeanConf.ComputerVision,vol.3,pp.224-237,2004.2004.D.Martin,C.Fowlkes,andJ.Malik,“LearningtoFindBrightnessandTextureBoundariesinNaturalImages,”AdvancesinNeuralInformationProcessingSystems,ystems,F.L.Bookstein,“PrincipalWarps:Thin-PlateSplinesandDecompositionofDeformations,”IEEETrans.PatternAnalysisandMachineIntelligence,vol.11,no.6,pp.567-585,June1989.1989.G.MoriandJ.Malik,“RecognizingObjectsinAdversarialClutter:BreakingaVisualCAPTCHA,”Proc.IEEEConf.ComputerVisionandPatternvol.1,pp.134-141,2003.2003.A.Thayananthan,B.Stenger,P.H.S.Torr,andR.Cipolla,“ShapeContextandChamferMatchinginClutteredScenes,”Proc.IEEEConf.ComputerVisionandPatternRecognition,vol.1,pp.127-133,June2003.2003.G.Mori,S.Belongie,andJ.Malik,“ShapeContextsEnableEfficientRetrievalofSimilarShapes,”Proc.IEEEConf.ComputerVisionandPatternvol.1,pp.723-730,2001. IEEETRANSACTIONSONPATTERNANALYSISANDMACHINEINTELLIGENCE,VOL.27,NO.11,NOVEMBER2005