V Jawahar Department of Engineering Science University of Oxford United Kingdom omkarvedaldiaz robotsoxacuk Center for Visual Information Technology International Institute of Information Technology Hyderabad India jawahariiitacin Abstract We invest ID: 20644
Download Pdf The PPT/PDF document "Cats and Dogs Omkar M Parkhi Andrea Veda..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Figure1.AnnotationsintheOxford-IIITPetdata.Fromlefttoright:petimage,headboundingbox,andtrimapsegmentation(blue:backgroundregion;red:ambiguousregion;yellow:fore-groundregion).Thesecondcontributionofthepaperisamodelforpetbreeddiscrimination(Sect.3).Themodelcapturesbothshape(byadeformablepartmodel[23,42]ofthepetface)andtexture(byabag-of-visual-wordsmodel[16,30,38,44]ofthepetfur).Unfortunately,currentdeformablepartmod-elsarenotsufcientlyadvancedtorepresentsatisfactorilythehighlydeformablebodiesofcatsanddogs;nevertheless,theycanbeusedtoreliablyextractstableanddistinctivecomponentsofthebody,suchasthepetface.Themethodusedin[34]followedfromthisobservation:acat'sfacewasdetectedastherststageindetectingtheentireanimal.Herewegofurtherinusingthedetectedheadshapeasapartofthefeaturedescriptor.Twonaturalwaysofcombiningtheshapeandappearancefeaturesarethenconsideredandcompared:aatapproach,inwhichbothfeaturesareusedtoregressthepet'sfamilyandthebreedsimultaneously,andahierarchicalone,inwhichthefamilyisdeterminedrstbasedontheshapefeaturesalone,andthenappearanceisusedtopredictthebreedconditionedonthefamily.Infer-ringthemodelinanimageinvolvessegmentingtheanimalfromthebackground.Tothisend,weimprovedonourpre-viousmethodonofsegmentationin[34]basingitontheextractionofsuperpixels.Themodelisvalidatedexperimentallyonthetaskofdis-criminatingthe37petbreeds(Sect.4),obtainingveryen-couragingresults,especiallyconsideringthetoughnessoftheproblem.Furthermore,wealsousethemodeltobreaktheASIRRAtestthatusestheabilityofdiscriminatingbe-tweencatsanddogstotellhumansfrommachines.2.Datasetsandevaluationmeasures2.1.TheOxfordIIITPetdatasetTheOxford-IIITPetdatasetisacollectionof7;349im-agesofcatsanddogsof37differentbreeds,ofwhich25aredogsand12arecats.Imagesaredividedintotraining,validation,andtestsets,inasimilarmannertothePASCALVOCdata.Thedatasetcontainsabout200imagesforeachbreed(whichhavebeensplitrandomlyinto50fortraining,50forvalidation,and100fortesting).AdetailedlistofbreedsisgiveninTab.1,andexampleimagesaregiveninFig.2.Thedatasetisavailableat[35].Datasetcollection.ThepetimagesweredownloadedfromCatster[4]andDogster[5],twosocialwebsitesded-icatedtothecollectionanddiscussionofimagesofpets,fromFlickr[6]groups,andfromGoogleimages[7].Peo-pleuploadingimagestoCatsterandDogsterprovidethebreedinformationaswell,andtheFlickrgroupsarespe-cictoeachbreed,whichsimpliestagging.Foreachofthe37breeds,about2;000–2;500imagesweredown-loadedfromthesedatasourcestoformapoolofcandidatesforinclusioninthedataset.Fromthiscandidatelist,im-agesweredroppedifanyofthefollowingconditionsap-plied,asjudgedbytheannotators:(i)theimagewasgrayscale,(ii)anotherimageportrayingthesameanimalexisted(whichhappensfrequentlyinFlickr),(iii)theilluminationwaspoor,(iv)thepetwasnotcenteredintheimage,or(v)thepetwaswearingclothes.Themostcommonprobleminallthedatasources,however,wasfoundtobeerrorsinthebreedlabels.Thuslabelswerereviewedbythehumanannotatorsandxedwheneverpossible.Whenxingwasnotpossible,forinstancebecausethepetwasacrossbreed,theimagewasdropped.Overall,upto200imagesforeachofthe37breedswereobtained.Annotations.Eachimageisannotatedwithabreedlabel,apixellevelsegmentationmarkingthebody,andatightboundingboxaboutthehead.Thesegmentationisatrimapwithregionscorrespondingto:foreground(thepetbody),background,andambiguous(thepetbodyboundaryandanyaccessorysuchascollars).Fig.1showsexamplesoftheseannotations.Evaluationprotocol.Threetasksaredened:petfamilyclassication(CatvsDog,atwoclassproblem),breedclas-sicationgiventhefamily(a12classproblemforcatsanda25classproblemfordogs),andbreedandfamilyclassi-cation(a37classproblem).Inallcases,theperformanceismeasuredastheaverageper-classclassicationaccuracy.Thisistheproportionofcorrectlyclassiedimagesforeachoftheclassesandcanbecomputedastheaverageofthediagonalofthe(rownormalized)confusionmatrix.Thismeansthat,forexample,arandomclassierhasaverageac-curacyof1=2=50%forthefamilyclassicationtask,andof1=373%forthebreedandfamilyclassicationtask.Algorithmsaretrainedonthetrainingandvalidationsub-setsandtestedonthetestsubset.Thesplitbetweentrainingandvalidationisprovidedonlyforconvenience,butcanbedisregarded. Figure5.ExamplesegmentationresultsonOxford-IIITPetdataset.ThesegmentationofthepetfromthebackgroundwasobtainedautomaticallyasdescribedinSect.3.3.cation,inwhicha37-classSVMislearneddirectly,usingthesamemethoddiscussedinSect.4.2.Therelativeper-formanceofthedifferentmodelsissimilartothatobservedinSect.4.1and4.2.Flatclassicationisbetterthanhier-archical,butthelatterrequireslessworkattesttime,duetothefactthatfewerSVMclassiersneedtobeevaluated.Forexample,usingtheappearancemodelwiththeimage,head,image-headlayoutsfor37classclassicationyieldsanaccuracyof51.23%,addingtheshapeinformationhi-erarchicallyimprovesthisaccuracyto52.78%,andusingshapeandappearancetogetherinaatclassicationap-proachachievesanaccuracy54.03%.Theconfusionmatrixforthebestresultforbreedclassication,correspondingtothelastentryoftheeightrowofTable4isshowninFig.4. Figure6.Confusionmatrixforbreeddiscrimination.Thever-ticalaxisreportsthegroundtruthlabels,andthehorizontalaxistothepredictedones(theupper-leftblockarethecats).Thematrixisnormalizedbyrowandthevaluesalongthediagonalarereportedontheright.Thematrixcorrespondstothebreedclassierusingshapefeatures,appearancefeatureswiththeimage,head,body,body-headlayoutswithautomaticsegmentations,anda37-classSVM.Thisisthebestresultforbreedclassication,andcorre-spondstothelastentryofrownumber8inTab.4. a b c d e f g hFigure7.Failurecasesforthemodelusingappearanceonly(im-agelayout)inSect.4.2.Firstrow:Catimagesthatwereincor-rectlyclassiedasdogsandviceversa.Secondrow:Bengalcats(b–d)classiedasEgyptianMau(a).Thirdrow:EnglishSetter(f–h)classiedasEnglishCockerSpaniel(e).5.SummaryThispaperhasintroducedtheOxford-IIITPetdatasetforthene-grainedcategorisationproblemofidentifyingthefamilyandbreedofpets(catsanddogs).Threediffer-enttasksandcorrespondingbaselinealgorithmshavebeenproposedandinvestigatedobtainingveryencouragingclas-sicationresultsonthedataset.Furthermore,thebaselinemodelswereshowntoachievestate-of-the-artperformanceontheASIRRAchallengedata,breakingthetestwith42%