239K - views

Layered Shape Synthesis Automatic Generation of Control Maps for NonStationary Textures Amir Rosenberger Tel Aviv University Daniel CohenOr Tel Aviv University Dani Lischinski The Hebrew University F

From left to right input texture exemplar control map extracted from the exemplar a larger control map synthesized by our approach and the resulting new texture Abstract Many inhomogeneous realworld textures are nonstationary and exhibit various lar

Tags : From left right
Embed :
Pdf Download Link

Download Pdf - The PPT/PDF document "Layered Shape Synthesis Automatic Genera..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Layered Shape Synthesis Automatic Generation of Control Maps for NonStationary Textures Amir Rosenberger Tel Aviv University Daniel CohenOr Tel Aviv University Dani Lischinski The Hebrew University F






Presentation on theme: "Layered Shape Synthesis Automatic Generation of Control Maps for NonStationary Textures Amir Rosenberger Tel Aviv University Daniel CohenOr Tel Aviv University Dani Lischinski The Hebrew University F"— Presentation transcript:

ACM Reference Format Rosenberger, A., Cohen-Or, D., Lischinski, D. 2009. Layered Shape Synthesis: Automatic Generation of Control Maps for Non-Stationary Textures. ACM Trans. Graph. 28 , 5, Article 107 (December 2009), 9 pages. DOI = 10.1145/1618452.1618453 http://doi.acm.org/10.1145/1618452.1618453. Copyright Notice Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for pro t or direct commercial advantage and that copies show this notice on the rst page or initial screen of a display along with the full citation. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any component of this work in other works requires prior speci c permission and/or a fee. Permissions may be requested from Publications Dept., ACM, Inc., 2 Penn Plaza, Suite 701, New York, NY 10121-0701, fax +1 (212) 869-0481, or permissions@acm.org. 2009 ACM 0730-0301/2009/05-ART107 $10.00 DOI 10.1145/1618452.1618453 http://doi.acm.org/10.1145/1618452.1618453 LayeredShapeSynthesis:AutomaticGenerationofControlMapsforNon-StationaryTextures AmirRosenberger TelAvivUniversity DanielCohen-Or TelAvivUniversity DaniLischinski TheHebrewUniversity Figure1: Aninhomogeneoustexture,exhibitinganon-uniformmixtureofpeelingpaint,baremetal,andrust.Fromlefttoright:inputtexture exemplar,controlmapextractedfromtheexemplar,alargercontrolmapsynthesizedbyourapproach,andtheresultingnewtexture. Abstract Manyinhomogeneousreal-worldtexturesarenon-stationaryand exhibitvariouslargescalepatternsthatareeasilyperceivedbya humanobserver.Suchtexturesviolatetheassumptionsunderly- ingmoststate-of-the-artexample-basedsynthesismethods.Con- sequently,theycannotbeproperlyreproducedbythesemethods, unlessasuitablecontrolmapisprovidedtoguidethesynthesis process.Suchcontrolmapsaretypicallyeitheruserspeciedor generatedbyasimulation.Inthispaper,wepresentanalternative: amethodforautomaticexample-basedgenerationofcontrolmaps, gearedatsynthesisofnatural,highlyinhomogeneoustextures,such asthoseresultingfromnaturalagingorweatheringprocesses.Our methodisbasedontheobservationthatanappropriatecontrolmap formanyofthesetexturesmaybemodeledasasuperpositionof severallayers,wherethevisiblepartsofeachlayerareoccupied byamorehomogeneoustexture.Thus,givenadecompositionof atextureexemplarintoasmallnumberofsuchlayers,weemploy anovelexample-basedshapesynthesisalgorithmtoautomatically generateanewsetoflayers.Ourshapesynthesisalgorithmisde- signedtopreservebothlocalandglobalcharacteristicsoftheexem- plar'slayermap.Thisprocessresultsinanewcontrolmap,which thenmaybeusedtoguidethesubsequenttexturesynthesisprocess. Keywords: controlmaps,example-basedtexturesynthesis,non- stationarytextures,shapesynthesis 1Introduction Computergeneratedimageryreliesheavilyontexturestoachieve realism.Oneeasywaytoacquirerealistictexturesisbyscanningor takingphotographsofsurfacesandmaterialsthatsurroundusinthe realworld.Therefore,alargenumberofmethodshavebeenpro- posedforsynthesizingtexturesfromexamples,inthelastdecade [Weietal.2009].Manyofthesemethodsareabletoproduceim- pressiveresultswhenappliedtohomogeneoustexturesthatmaybe describedbystationaryMarkovrandomeld(MRF)models.Yet manyrealworldtexturesarehighlyinhomogeneous,andarenot modeledwellbyastationarystochasticprocess. Consider,forexample,therustymetalsurfaceshownontheleftin Figure1.Thetextureonthissurfaceisclearlynon-stationary,andit maybeseenasahighlynon-uniformmixture,orsuperposition,of severaldifferenttextures:peelingpaint,baremetal,andrust.While eachofthesethreetexturesisroughlyhomogeneous,thetexture asawholeisnot.Thisisatypicalsituationformanyrealworld surfaces,whosetextureoftenresultsfromnaturalprocesses,suchas weathering,corrosion,colorcrackingandpeeling,growthofmoss, etc.[DorseyandHanrahan1996;Dorseyetal.1999;Boschetal. 2004;Desbenoitetal.2004;Dorseyetal.2008]. Acommonremedytocopewithsuchtexturesistoguidethesyn- thesisprocessbyacontrolmapthatencodesthelargescalevaria- tionsandthenon-localfeaturesofthedesiredoutputtexture(e.g., [Ashikhmin2001;Hertzmannetal.2001;Zhangetal.2003;Wang etal.2006;Weietal.2008]).However,suchcontrolmapsaretyp- icallyeitheruser-speciedorproducedbyacustomtailoredsimu- lation(e.g.,biologicalorphysically-based). Inthisworkweproposeanewmethodforautomaticallygenerat- ingcontrolmapsfromexamples,gearedatnaturaltexturessuchas theoneinFigure1.Asobservedabove,suchtexturesoftenlook likeasuperpositionofseverallayers,whereeachvisibleregionsof eachlayerareoccupiedbyamorehomogeneoustexture.Theshape ofthetexture-occupiedregionsineachlayerisfarfromarbitrary. Rather,itistheconsequenceofthespecicnaturalprocessthatpro- ducedthistexture,aswellastheshapeofthelayerunderneath.Nei- therglobalstatistics,norsmallneighborhoodsarecapableoffaith- fullycapturingsuchhigherlevelstructures.Suchappearancesmay begeneratedbyspecializedshadersorbyphysically-basedsimu- lations.However,wearenotawareofanygeneralfullyautomatic wayforgeneratingsuchashaderfromaspecicexample. Ourapproachbeginsbydecomposingtheinputexemplarintoa numberoflayers,whichweorderbottomtotop.Anovelexample- based shapesynthesis algorithmisthenusedtogenerateanewset oflayers,whoselocalandglobalcharacteristicsvisuallyresemblethoseoftheexemplar'slayers.Thisalgorithmmakesuseofabidi-rectionalmeasureofsimilaritybetweentheshapesofthelayers,whichisbasedontheshapes'boundaries.Startingfromsomeini-tialoutputshape,weiterativelyoptimizetheshapewithrespecttothissimilaritymeasure.Oncethenewlayersareavailable,atexturetransferprocessbasedon“texture-by-numbers”[Hertzmannetal.2001]isinvoked,resultinginthenaloutputtexture,suchastheresultshowninFigure1.Insummary,themainnoveltyinourapproachliesinexample-basedsynthesisofasuitablecontrolmap,ratherthanworkingdirectlyonthetexture,oronsomeassociatedappearancespace[LefebvreandHoppe2006].Toourknowledge,suchanapproachhasnotbeenexploredbefore.2RelatedWorkExample-basedtexturesynthesishasenjoyedconsiderableresearchattentioninrecentyears.Mostoftherelevantpreviousmeth-odsmayberoughlyclassiedtoparametricmethods[HeegerandBergen1995],andnon-parametricmethods,whichincludepixel-basedmethods[EfrosandLeung1999;WeiandLevoy2000],patch-basedmethods[EfrosandFreeman2001;Kwatraetal.2003],optimization-basedmethods[Kwatraetal.2005],andappearance-spacetexturesynthesis[LefebvreandHoppe2006].Parametricmethodsattempttoconstructaparametricmodelofthetexturebasedontheinputsample,whichhasproventobeachal-lengingtask,andaremostlysuccessfulwithstructurelessstation-arytextures.Non-parametricmethodshavedemonstratedtheabil-itytohandleawidervarietyoftextures,bygrowingthetextureonepixel/patchatatime.Optimization-basedmethodsevolvethetextureasawhole,furtherimprovingthequalityoftheresultsandmakingthesynthesismorecontrollable.Wereferthereaderto[Weietal.2009]foramorecomprehensiveoverviewofexample-basedtexturesynthesis.Whilenon-parametricmethodsaretypicallyabletoreproducesmallscalestructure,theyhaveadifcultycopingwithhighlyin-homogeneoustextures,sincesuchtexturescannotbemodeledbyastationaryMarkovRandomField(MRF)model,whichprovidesthetheoreticalbasisformostofthesemethods.Inordertohandlesuchtexturesandcontrollargescalestructure,Ashikhmin[2001]proposedtoguidethesynthesisprocessbyauser-providedtargetimage,whichspeciesthelocalaveragecolorsacrossthetargettexture.Texture-by-Numbers[Hertzmannetal.2001]extendsthisideafurtherbyaugmentingtheinputexemplarwithalabelmap,whereregionswithdistincttexturearedistinguishedbydifferentlabels.Asuitablelabelmapmaybepaintedmanuallybytheuser,orcreatedautomaticallyusingunsupervisedimagesegmentation.Tosynthesizeanewimage,atargetlabelmapisprovided,whichindicateshowthedifferenttexturesshouldbearrangedinthere-sultingimage.However,thatworkaddressedneithertheissueofautomaticallygeneratingalabelmapfornaturalinhomogeneoustextures,northeautomaticsynthesisofthetargetlabelmap,aswedoinourwork.Manyotherworkssincemadeuseofcontrolmapswhensynthesiz-ingnon-stationarytextures,forexample[Zhangetal.2003;Wangetal.2006;Guetal.2006;Luetal.2007;Weietal.2008].How-ever,inalloftheseworksthecontrolmapforthetargettextureiseitherprovidedbytheuser,orderivedfromaspecicmodeloftex-tureformationacrossa3Dsurface(e.g.,[Luetal.2007]),andwearenotawareofanypreviousattemptsofexample-basedcontrolmapgeneration.Ourshapesynthesisapproachisrelatedtotextureoptimizationtechniques[Wexleretal.2004;Kwatraetal.2005],whichsynthe-sizetexturesbyminimizingatextureenergyfunction.Thisfunc-tionconsistsofasumoflocaltermsmeasuringhowcloseeachsynthesizedtexturepatchistoanexemplarpatch.However,thisformulationdoesnotaccountforthepossibilitythattheremaybemanyotherpatchesintheexemplarthatarenotrepresentedatallinthesynthesizedresult.Whilethismaybeadequateforhomo-geneoustextures,wheremostpatchesaresimilartoeachother,thequalityoftheresultsforinhomogeneoustexturesisoftencompro-mised.Whileitispossibletoinjectsomeglobalstatisticsintotheoptimization[Kopfetal.2007],theresultingprocessstillfailstocapturethelargescaleappearanceofhighlyinhomogeneousnatu-raltexturesthatarethetargetofthiswork.Incontrast,weperformshapesynthesiswithabidirectionalsimilaritymeasure(inspiredbySimakovetal.[2008]andWeietal.[Weietal.2008]),anddemon-stratemorefaithfulreproductionofappearanceinthecomparisonswepresentinSection4.Appearance-spacetexturesynthesis[LefebvreandHoppe2006]isanotheroptimizationmethodthatoperatesinafeaturespace,ratherthanusingthevaluesofpixelsorsmallpatchesdirectly.Apointcorrespondingtoapixelinamoregeneralfeaturespacemayen-codemoreinformation,allowingstructuretobereproducedbetter.Thelayermapthatweassociatewiththeinputexemplarinourapproachcouldbeviewedasafeaturespacecustom-tailoredforsynthesisoflayeredinhomogeneoustextures.Avarietyofmethodsgeneratetexturesofweatheredsurfacesbyassumingandsimulatingaphysicalmodel[DorseyandHanrahan1996;Dorseyetal.1999;Merillouetal.2001;Boschetal.2004;Desbenoitetal.2004;Dorseyetal.2008].Whilesuchmethodshaveproducedsomehighlyrealisticresults,theyarenotgearedto-wardsmatchingaparticularappearancegivenbyanexample.Also,controllingtheresultsofthesynthesistypicallyinvolvesspecify-ingalargenumberofparameters,whicharenotalwaysintuitive.Incontrast,ourapproachisexample-based,ratherthanphysically-Ourapproachsynthesizestheboundariesofthelayershapesbyex-ample.Thus,itisrelatedtotheCurveAnalogiesworkofHertz-mannetal.[2002],whereasimilarframeworkwasappliedtoreproducethestyleofcurvedshapes.However,ourworkusesadifferentsimilaritymeasureandoperatesonadiscretepatch-basedrepresentationofashape'sboundary,ratherthanavector-basedrep-resentation.AlsorelatedistheworkofBahtetal.[2004],whichusesbinaryvoxelgridsinordertosynthesizegeometricdetailsonvolumesurfaces.Thesevoxelgridsaresimilartothebinaryneigh-borhoodsthatweusetooptimizetheshapeboundaries.However,theirgoalistoaddsmaller-scaledetailtoanexistingglobalshape,whilewefocusonsynthesizingtheentireshapefromscratch.3LayeredShapeSynthesisThisworkdealswithexample-basedgenerationofcontrolmapsrepresentedaslayermaps.Alayermapisanimagewherediffer-entpixelvaluesindicatetodifferentlayers.Let:::bethevaluesoflayermappixels,sortedinascendingorder.Then,alayerisdenedasthesetofallpixelswhosevalueisgreaterthanorequalto.Notethatapixelwithvalueactuallybelongstoalllayers;:::;.Onecanthinkofthelayersasstackedontopofeachother,withlayershigherinthestackpartially“conceal-ing”lowerlayers.Eachlayerhasanassociatedforegroundshape,whichweencodeasabinaryimageofthesamedimensionsasthelayermap.Notethattheshapeisalwayscontainedin.AsmaybeseeninFigures1and6,theboundariesofthesenestedshapesarehighlycorrelated,butnotaligned.Intheguresinthispaper,wedisplayvaluescorrespondingtodifferentlayersusinguniquecolors. abcde abcde a 0.01.33.05.36.0 0.02.75.05.4 0.05.05.3 0.04.6 Figure2:Similaritymeasurebetweenpairsofvedifferentshapes.Givenasetofsuchshapes,ourgoalistosynthesizeanewsetofshapes,whilemaintainingbothglobalandlocalsimilaritytotheoriginalones.Forthispurpose,itisimportantthateachshapeoc-cupiesthesamerelativeamountofpixelsinthesynthesizedmapasitdidintheexemplar,andthattheboundariesofthesynthe-sizedshapeslocallyresemblethoseintheexemplar.Wefoundthatrepresentingtheshapebymeansofitsboundarycurvefailstocap-turealloftherelevantinformation.Sucharepresentationcannotpredictthespatialrelationshipbetweendisconnectedcomponentsoftheshape,anddoesnotpreventself-intersections.Instead,werepresentabinaryshapebyacollectionofpatchescenteredontheshape'sboundarypixels(atmultipleresolutions)inordertocap-turesthenecessaryshapeproperties.Ourshapesynthesisapproachemploysoptimizationsimilarlyto[Wexleretal.2004;Kwatraetal.2005],wherethesynthesizedre-sultisiterativelyoptimizedwithrespecttosomemeasureofitssim-ilaritywiththeexemplar.Webeginbyderivingasuitablebidirec-tionalshapesimilaritymeasure,similarlytoSimakovetal.[2008]andWeietal.[2008].Next,wedescribeanovelgreedyoptimiza-tionschemethatiterativelymodiesaninitialshape,soastoin-creaseitssimilaritytoagivenexemplarshape.Finally,wedis-cusshowthismechanismisusedtocreateanentirenewlayermap,whichisasequenceofnestedshapes,fromthelayermapproducedinthelayerdecompositionphasedescribedintheprevioussection.3.1Shapesimilaritymeasurebethesetsofboundarypixelsofshapes,respectfully.Aboundarypixelisapixelinsideashapewithatleastoneofits4-neighborsoutsidetheshape.Letbetwoboundarypixels,andletbetheneighborhoodscenteredaroundthemandrotatedby.Werefertosuchneighborhoodsaspatches.Wedenethesimilarity,,betweentwoboundarypixelsasthedistancebetweentheirneighborhoods(rotatedsuchthatthedistanceisminimized).Formally,)=Sincewedealwithbinaryimages,thenormaboveissimplythenumberofdifferentpixelsbetweentwopatches.Next,wede-nethelocalsimilaritybetweenaboundarypixelandtheboundaryof(another)shapeasthesimilaritybetweenandthepixelmostsimilartoitontheboundary)=Notethatthissimilaritymeasureisnotsymmetric.Whileitensuresthateveryboundarypatchofissimilartoaboundarypatchin Figure3:Iterativeassignmentofboundarypatches.Theexemplarboundarypatches(left)areassignedtothesynthesizedboundarypatches(right).Incaseswheretwopatchesareassignedtothesameone,theassignmentwiththelargerLdifference(redarrow)isdiscardedandwillbeassignedtoanotherpatchinafutureiter-ation(yellowarrow).theremaybeboundarypatchesinthatarenotwellrepresentedin.Forexample,asimpleshapemaybedeemedsimilartoamorecomplexonethatalsohappenstocontainsomesimplefeatures.Thus,werequireabidirectionalsimilaritymeasure,denedas)=)+ B2j;(3)whichistheaveragenumberofdifferentpixelsbetweenabound-arypatchofoneshapetoitsnearestneighborontheother.Figure2showsseveraldifferentshapesandreportstheirpairwisebidirec-tionalsimilarities.3.2ShapeoptimizationArmedwiththesimilaritymeasureabove,weuseanoptimizationprocedurethatiterativelymodiestheboundaryofasynthesizedtomakeitmoresimilartothatoftheexemplarshapeTheoptimizationproceedsfromcoarsetoneresolution.Ateachresolutionwealternatebetweentwomainsteps:(i)matchingeachboundarypatchoftoaboundarypatchof,and(ii)modifyingbyaddingorremovingpixelsbasedontheresultsofthematchingachievedinthepreviousstep.Thisiterativeoptimizationproce-dureresemblesthatofKwatraetal.[2005],buteachofthetwomainstepsdifferssignicantlyfromitscounterpart,becauseweminimizeadifferent(bidirectional)energyfunction,andworkwithbinaryimages,ratherthantextures.Thesetwostepsarediscussedinmoredetailbelow.Boundarypatchmatching.Aspointedoutearlier,wewouldlikeeveryboundarypatchoftoresembleoneof,butwewouldalsolikeeveryboundarypatchoftoberepresentedin.Thus,assumingwehaveanequalnumberofboundarypatchesin,weseekaminimumcostassignment,afundamentalcombinato-rialoptimizationproblem[Schrijver2003].Solvingthisproblemexactlyistooexpensiveforourpurposes(,whereisthenumberofpatches),soweresorttoanapproximatesolutionusingtheiterativegreedyapproachdescribedbelow.denotethesetsofboundarypatchesofrespectively,andassumefornowthatthetwosetshavethesamesize.Eachpatchinisinitiallyassignedtoitsnearestneighbor.Asaresult,somepatchesinmayhavemorethanoneexemplarpatchassignedtothem,whileothersmayhavenone(seeFigure3).Intheformercase,wekeeponlytheassignmentwiththesmallestdifference,anddiscardtherest.Allofthepairsof Figure5:Reningshapeboundarieswithourmulti-resolutionoptimization.Twoinitialshapes(left)areevolvedusingtwodifferentexem-plars(right). Figure4:Left:Shapeadjustment.Boundaryexemplarpatchesaresuperimposedovertheirassignedpositions.Pixelsinregionsofoverlapbetweenthesesuperimposedpatchesmaybeaddedtotheshape(greendot),orremovedfromit(reddot),makingthenewboundarymoresimilartothatoftheexemplar.Right:Matchingpatchesfromthepreviousshape(gold)aresuperimposedagaintoseedthenewshape(blue).patcheswhichhavebeenassignedarethenremovedfromfurtherconsideration,andtheprocessisrepeateduntileverypatchinhasbeenassigned.Ingeneral,differinsize.Typically,thesynthesizedshapeislargerthantheexemplar.Thus,assumingthat,weconstructasetofexemplarpatchesofsizebyincludingeachexemplarpatchtimes,andrandomlyselect-additionalpatchesfrom.Inthiswayweensurethatalltheboundaryfeaturesintheexemplarshapegetanequalchancetoberepresentedinthesynthesizedshape.Shapeadjustment.Afterndingtheassignmentasdescribedabove,ourgoalistomodifytheboundaryofsoastoincreasethesimilarityto(byreducing).Toachievethis,wesu-perimposeeachexemplarpatchoveritscounterpartin.Considerapixel,whichiscoveredbyseveraloverlappingsuper-imposedpatchesfrom.Informally,ifthesepatchesagreethatshouldbepartoftheshape,itisaddedto.Similarly,apixelinsidemightberemovediftheoverlappingpatchesagreethatitshouldnotbelongtotheshape.ThisisillustratedinFigure4(left).Morespecically,considerapixelinthevicinityofthebound-aryof.Itiscoveredbytwogroupsofoverlappingsuperimposedexemplarpatches:onegrouppredictsthatbelongstotheshape,whiletheotheronepredictsthatisoutsidetheshape.Foreachofthesetwopredictionswecomputeascorebysumminguptheweightsofthecorrespondinggroup'spatchesat.Letbeaboundarypixeloftheexemplarpixelassignedtoit.Thentheentireexemplarpatchisassignedthefollowingweight whichisfurthermultipliedbyaGaussianfallofffunction(thustheweightdecreasesawayfromthecenterofthepatch).Thesigmavalueforthisfunctionwaschosentobehalfofthepatchessize.Thegroupwiththehighestscoreatdetermineswhetherbeincludedorexcludedfromtheshape.Whenthesumofweightsaccumulatedateachpixelisbelowathreshold,itsvalueremainsunchanged.Thisisbecausepatchweightsreectadegreeofcertainty,soareasoflowweightaremoresensitivetorandomnessgeneratedbyourapproximatednear-estneighborsearchandthegreedyassignment,suchthatusingthenewvaluesmayproducenoise.Thisthresholdalsodeterminesthenalamountofpixelsintheshapeaftertheiterationisdone.There-fore,itissetdynamicallysothatthe(relative)amountofthepixelsinsidetheshapeisthesameasintheexemplar.Candidatesfromtheintervalal10�2;10�7]aretestedandtheonewhichresultsinthenearestamountischosen.Aftertheupdateiscomplete,theopti-mizationprocedureisrepeateduntilconvergence.Convergenceisreachedwhenthenumberofchangedpixelsfallsbelowathreshold.Asmentionedearlier,theoptimizationproceedsfromcoarsetoneresolution.Theresultcomputedateachresolutionlevelisupsam-pledtoserveasastartingpointforthenext(ner)level.Atcoarserresolutionstheglobalstructuresareformed,whileneresolutionsllinthenedetailsalongtheshape'sboundary.Inourexam-ples,weuse5to6resolutionlevels.Figure5showshowdifferentinitializationsleadtodifferentglobalshapes.However,inalloftheexamplesthesynthesizedshapecontainsboundaryfeaturesthat Figure6:Ourinhomogeneoustexturesynthesisapproach.areverysimilartothosepresentintheexemplar,resultingincloseoverallresemblance.3.3LayermapsynthesisTheshapeoptimizationprocedurepresentedintheprevioussec-tionmaybeuseddirectlytosynthesizetherst(bottom)layer.Arandomlygeneratedshape,withthenumberofforegroundpixelsmatchingthatofthecorrespondingexemplarlayermaybeusedforinitialization.Inordertogeneratethefollowinglayers,however,wemustintroduceanumberofmodications.First,theshapeofeachlayerisnestedinsidetheshapeofthelayerbeneathit.Sec-ond,theboundariesoftwosuccessivelayersaretypicallyhighlycorrelated.Preservingthiscorrelationisimportant,asitisinstru-mentalforfaithfullyreproducingtheappearanceoftheexemplarinthesynthesizedresult.Itisnotobvioushowtoinitializetheshapeofthenextlayersundertheseconditions.Toaddresstheserequirementswebeginthesynthesisofeachlayerbycreatingamaskthatdenesthearea(containedinsidetheshapeofthepreviouslayer),wherethecurrentshapeisallowedtoevolve.Initially,thismaskissettotheentireshape,butweusethemostrecentboundarypatchassignments(fromthelastshapeoptimizationiteration)toshrinkthismaskdowntoabetterinitialguessfortheregioncontaining.Morespecically,weagainsuperimposeboundarypatchesfromtheexemplarovertheirassignedlocationsontheboundaryof,butthistimewetrytopre-dictwhichofthepixelsinsideshouldbelongtothemaskofasillustratedinFigure4(right).Thus,theinteriorofisseededwithpixelswhicharepredictedtobelongtowithasufcientlylargeweight.Themaskisthenshrunktoincludeonlytheseseededpixels.Seededpixelswithhighweightsformtheinitialguessfor,whilethosewithsomewhatsmallerweightsdenetheremain-ingregionofthemask,withinwhichtheshapeisallowedtoevolveinthecourseoftheoptimization.Theinitializationofeachnewlayerisdoneviathisseedingmech-anisminthecoarsestresolution,whereboundarypatchesarelargeenoughtofullycovertheinteriorofthepreviousshape.Asimilarstepisrepeatedatthebeginningofeachresolutionleveltorecreateanaccuratemaskforthecurrentlevelatthenewresolution,andtorenetheshapeboundary.Afterthisstep,shapeoptimizationproceedsasdescribedbefore.Continuouscontrolmaps.Inourexperimentswefoundthatthesubsequenttexturesynthesisprocesscansometimesbeimprovedbyswitchingfromadiscretelayermaptoacontinuouscontrolmap.Specically,foreachpixelinsidetheshapeitscontinuousmapvalueissetto )+arethedistancesfromrespectively(seeFigure7).Thedistancesareobtainedbyperform-ingdistancetransformsover.DistancetransformswerealsousedtocreatecontrolmapsbyLefebvreandHoppe[2006]. Figure7:Thedistancesfromapointinsideashapetotheneigh-boringshapesareusedtoconvertadiscretelabelmap(left)toacontinuousone(right).4ApplicationsandResultsWefoundthelayeredshapesynthesisapproachdescribedintheprevioustexturetobeeffectiveforsynthesisofinhomogeneoustex-tures,suchasthoseresultingfromnaturalagingorweatheringpro-cesses.ThesynthesisprocessforsuchtexturesconsistsofthreesuccessivephasesdepictedinFigure6:layerdecomposition,shapesynthesis,andtexturesynthesis.Thelayerdecompositionphasetakesaninhomogeneoustexturesampleasinput,andgeneratesalayermapwhichencodesthedis-tincthomogeneoustextureregions(layers)presentintheinput,byassigningauniquelabeltoallofthepixelsbelongingtothesamelayer.FollowingthetextureclassicationapproachadvocatedbyVarmaandZisserman[2003],werstsegmenttheexemplar'spix-elsbyperformingK-Meansclusteringonthedimensionalfea-turevectorsformedbyconcatenatingthevaluesofeachpixel'sneighborhood.Wecurrentlyrelyontheusertospecifyasthenumberofdistincttexturesvisibleintheexemplar,typicallybetween3and5.issetto15inallofourexamples.There-sultingclustersshouldroughlycorrespondtothedistincttexturespresentintheexemplar.Pointsclosertotheclustercentersareduetopixelsthatarethemoretypicalrepresentativesofthecorrespond-ingtextures,whilepointsfarawayfromthecentercomefromareasoftransitionbetweentextures.Let;:::;betheresultingclusters,orderedbytheusersuchthatisthebottomlayer(orig-inal“clean”surface),andisthetoplayer(most“weathered”surface).Forthelayermap,wesettheforegroundpixelsofofCi+1[:::.Iftheclustershavebeenorderedproperly,the;:::;expressesapossiblenaturalevolutionandspreadovertimeoftheweatheringphenomenoncapturedbytheexemplar.Inthelastphaseweusethenewlayermapobtainedintheshapesynthesisphase(Section3)tosynthesizeanewinhomogeneoustexture.Thisisdonebyapplyingthe“texture-by-numbers”frame-work[Hertzmannetal.2001].Whileapplyingtexture-by-numbersdirectlyonthelayermapoftenproducessatisfactoryresults,theirvisualqualitymaybefurtherimprovedbyswitchingfromadiscretelayermaptoacontinuousone,asdescribedintheprevioussection.Thecontinuousmapmaybeseenasaheuristicanalogueforthe Figure8:Avarietyofresultsproducedbyourmethod.Left:inputexemplarsandtheirdecompositionstolayers;Middle:synthesizedlayermap;Right:nalsynthesizedresult. Figure9:Terraingenerationbyheightmapsynthesisusingourmethod.Left:inputheightmapanditsdecompositiontolayers;Middle:thesynthesizedlayermap;Right:nalsynthesizedresult. inputourresult[Kopfetal.2007][Kwatraetal.2005]Figure10:Acomparisonofourapproachtotextureoptimizationwithandwithouthistogrammatching.Inthetoprowbothmethodsperformsynthesisdirectlyfromtheexemplar.Inthebottomtworows,weattempttousetextureoptimizationtosynthesizeanewlayermap.weatheringdegreemapof[Wangetal.2006].Weexperimentedwithourmethodonavarietyofnaturalinhomo-geneoustextures.SomeresultsareshowninFigures1,6,and8.Theexamplesarequitevaried,showcasingphenomenasuchascor-rosion,rust,lichen,andpeelingpaint.Theydiffersignicantlynotonlyintheirappearance,butalsointheirunderlyinglayerstructure,asmaybeseenfromthelayermapsextractedbyourmethod.Ourmethodsuccessfullyreproducesthegloballayerstructures,thelo-calnedetailsoftheshapeboundaries,andthenalappearanceofthesetextures.Ourmethodisnotlimitedtosuchtextures,however.Otherinhomogeneoustexturesthatexhibitsalayeredstructurewithnestedshapesmaybesynthesizedaswell.Forexample,wehavesynthesizedaplausiblectionalsatelliteimagefromoneofEarth(bottomrowinFigure8).Sinceweusepatch-basedshapesynthe-sis,somerepetitionsdooccur,buttheyaremostlydifculttospot,astheyareexplicitlylimitedinourapproachbyourfairboundarysamplingandassignmentmechanisms.Anotherapplicationofourapproachisexample-basedterrainsyn-thesis,asdemonstratedinFigure9.Heightmapsusedtorepre-sentterrainsmayalsobeconsideredasnon-stationarytexturesfor Figure11:Inpainting.Left:original,right:ourresult. Figure12:Exampleofusercontrolviaapaintinginterface.whichourlayeredshapesynthesisapproachtsperfectly.Forthegenerationofthelayermap,asimplequantizationoftheheightmapisused.Theboundariesoflayersresemblecontourlinesinatopographicmap.Inthisapplication,whichissimilartotexturesynthesisdescribedbefore,theshapesynthesisphasegeneratesanewtopographiclayoutforthesynthesizedterrainandthetexturesynthesisphaseaddsthenedetails.Thecomputationtimeofourmethodisdominatedbythe“texturebynumbers”phase,whichtakesuptovehoursfora800imageusing55neighborhoods.Thetimeittakestosynthesizeanewlayermapdependsonthetotallengthoftheshapeboundaries.Thecomplexityoftheoptimizationstepislinearinthenumberofnearestneighborcallsforeachboundarypatch.Weuseapproxi-matenearestneighborsearchvialocallysensitivehashing[DatarandIndyk2004].Ittakesourunoptimizedcode20–30secondsonaveragetocompleteoneoptimizationiterationforan800600im-age.Typically,5–10iterationsareusedateachresolutionlevel,sotheexecutiontimeperlayerisupto30minutes.Wecompareourmethodtotwopreviousexample-basedtexturesynthesismethods:textureoptimization[Kwatraetal.2005]andtextureoptimizationwithcolorhistogrammatching[Kopfetal.2007].Figure10showstheresultsofthethreemethodsside-by-side.Thetoprowshowsthenalsynthesisresultonatextureofarustingsurface.Kwatra'smethodissuitedforstationarytextures,andexhibitsmultipleobviousrepetitionsmakingtheresultquitedissimilarfromtheexemplar.Kopf'sresultmatchestheglobalcolorstatisticsoftheexemplar,andproducesabetterresult,butsomerepetitionsarestillapparent,andsomeregionsofthesynthe-sizedtexturedonothaveasimilarcounterpartintheexemplar(suchasthelargeregionoflighterrustnearthecenter).Itisalsointer-estingtoexaminewhetherthesepreviousmethodsareabletosyn-thesizethelayermap,ratherthansynthesizingthetexturedirectly,andthisisdoneontwoexamplesshowninthetwobottomrowsofFigure10.Themiddlerowisalayermapextractedfromacloudyskytexture,whileinthebottomrowthelayermapisfromtheter-raininFigure9.Inbothoftheseexamples,thepreviousmethodsgeneratemorerepetitionsofentireregionsofthelayermap,andinseveralplacestherearedirecttransitionsbetweennon-adjacentlayers,whicharenotpresentintheinputmap.Figure11showsaninpaintingexample,whereaholeislledinsideaninhomogeneoustexture.Whiletheresultisobviouslynotiden-ticaltotheoriginalimage,itisquiteplausible,andthelledregionblendswellwiththeoriginalparts.Thelayermapinsidetheholeisinitializedrandomly.Sinceourmethodmodiestheentirelayermap,aftereachoptimizationstepweresetthelayermapoutsidethe Figure13:Examplesoffailurecases.Top:violationofthelayermodel.Bottom:failuretoreproducespecicshapes.holebacktotheoriginalone.Figure12demonstratesthefeasibilityofcontrollingtheresultofthesynthesisviaapaintinginterface.TheexemplaranditslayermapareshowninthethirdrowofFigure8.Theusersketchesinyellowtheapproximatepositionwhererustshouldbe,andtheresultingsketchservesastheinitializationfortheshapesynthesisphase.Thinstripsofblueandgreenpixelsareautomaticallyaddedbythesystem,sincetheyellowlayershapeissupposedtobenestedinsidetheblueandthegreenlayershapes.Inordertoavoidchang-ingtheusersketchedshapetoomuch,fewerresolutionlevelsareusedbytheshapesynthesismethod,resultinginthemiddleimageinFigure12,whiletherightmostimageisthesynthesizedtexture.Ourapproachmakestwobasicassumptions:(1)thecontrolmapconsistsofanorderedsetlayers,nestedwithineachother;(2)theproposedshapesimilaritymeasurecapturesalltheshapecharacteristicsthatoneaimstoreproduce.Violationofeitheroftheseassumptionsmayleadtoafailure,asdiscussedbelow.(1)Thersttypeoffailureisdemonstratedbythesyntheticexam-pleinthetoprowofFigure13.Herewegreenandblueregionsthatareindependentofeachotherintheinput(e.g.,anaturaltexturewhoseappearanceresultsfromtwoindependentprocesses).Ourapproachassumesalayeredmodelandgeneratesthegreenlayerrst,followedbythebluelayer.Asaresult,therelationsbetweenthegreenandblueregionsarenotpreserved,andseveralbluere-gionsaresynthesizedinsidegreenones.(2)Theproposedsimilaritymeasurecharacterizesshapesbythelo-calappearanceoftheirboundaries,withoutconsideringtheshapeasawhole.Thus,itisbettersuitedforne-scaleunstructuredshapesandfractal-likeboundaries.Morestructuredelementsmightbebetterhandledbyothermodels.Forexample,thelocationsofthedesertsinthebottomrowofFigure8mighthavebeenrepro-ducedbetterusingthecontext-awaretexturesframework[Luetal.2007].ThebottomrowofFigure13showsanothersyntheticexam-plewheretheeasilyrecognizabledistinctshapesintheinputmapappearmixedinthemapgeneratedbyourmethod.5ConclusionWehavepresentedanovelexample-basedmethodforsynthesisofcontrolmapssuitablefornonstationarytextures,suchasthosere-sultingfromweathering.Tothatend,wehavedevelopedanew powerfulexample-basedshapesynthesisalgorithmthatrepresentsshapesasacollectionofboundarypatchesatmultipleresolution,andsynthesizesanewshapefromanexamplebyoptimizingabidi-rectionalsimilarityfunction.Applicationsofourmethodincludesynthesisofnaturaltexturesandterraingeneration.Infutureworkwehopetoextendthemethodofshapesynthesistoalargersetoftextures,forexample,texturesthatdonotexhibitaclearhierarchyoflayers,andtextureswithlargerstructures.Ourcurrentmeasureemphasizesboundarysimilarityoverotherprop-erties,suchasareatoboundarylengthratio,whichismaintainedonlyimplicitly.Wewouldliketogainabetterunderstandingoftherelationsbetweensuchproperties,andexperimentwithvariousextensionsofoursimilaritymeasure.Wewouldalsoliketodiscoveradditionalapplicationsofourshapesynthesisapproach.Inparticular,itwouldbeinterestingtoexploretheapplicabilityofsuchanapproachtothesynthesisof3Dshapes.Acknowledgments:ThisworkwassupportedinpartbygrantsfromtheIsraelMinistryofScience,andfromtheIsraelScienceFoundationfoundedbytheIsraelAcademyofSciencesandHu-manities.Theauthorswouldalsoliketothanktheanonymousre-viewerswhosesuggestionsweregreatlyhelpful.ReferencesSHIKHMIN,M.2001.Synthesizingnaturaltextures.InProc.Symp.Interactive3DGraphics,217–226.HAT,P.,INGRAM,S.,ANDURK,G.2004.Geometrictexturesynthesisbyexample.InSGP'04:Proceedingsofthe2004Eu-rographics/ACMSIGGRAPHsymposiumonGeometryprocess-,ACM,NewYork,NY,USA,41–44.OSCH,C.,PUEYO,X.,MERILLOU,S.,ANDHAZANFARPOUR,D.2004.Aphysically-basedmodelforrenderingrealis-ticscratches.ComputerGraphicsForum23,3(Sept.),361–370.ATAR,M.,ANDNDYK,P.2004.Locality-sensitivehashingschemebasedonp-stabledistributions.InProc.SCG'04,ACMPress,253–262.ESBENOIT,B.,GALIN,E.,ANDKKOUCHE,S.2004.Simu-latingandmodelinglichengrowth.ComputerGraphicsForum,3(Sept.),341–350.ORSEY,J.,ANDANRAHAN,P.1996.Modelingandrenderingofmetallicpatinas.InProc.SIGGRAPH'96,AddisonWesley,ORSEY,J.,EDELMAN,A.,JENSEN,H.W.,LEGAKIS,J.,ANDEDERSEN,H.K.1999.Modelingandrenderingofweatheredstone.InProc.SIGGRAPH'99,ACMPress,225–234.ORSEY,J.,RUSHMEIER,H.,ANDILLION,F.2008.ModelingofMaterialAppearance.ComputerGraphics.MorganKaufmann/Elsevier,Dec..336pages.FROS,A.A.,ANDREEMAN,W.T.2001.Imagequiltingfortexturesynthesisandtransfer.Proc.SIGGRAPH2001,341–346.FROS,A.A.,ANDEUNG,T.K.1999.Texturesynthesisbynon-parametricsampling.Proc.ICCV'992,1033–1038.,J.,T,C.-I.,RAMAMOORTHI,R.,BELHUMEUR,P.,MTUSIK,W.,ANDAYAR,S.2006.Time-varyingsurfaceap-pearance:acquisition,modelingandrendering.ACMTransac-tionsonGraphics25,3(Proc.SIGGRAPH2006),762–771.EEGER,D.J.,ANDERGEN,J.R.1995.Pyramid-basedtextureProc.SIGGRAPH'95,229–238.ERTZMANN,A.,JACOBS,C.E.,OLIVER,N.,CURLESS,B.,ANDALESIN,D.H.2001.Imageanalogies.Proc.SIGGRAPH,327–340.ERTZMANN,A.,OLIVER,N.,CURLESS,B.,ANDEITZ,S.M.2002.Curveanalogies.InProc.13thEurographicsWorkshoponRendering,EurographicsAssociation,233–246.OPF,J.,F,C.-W.,COHEN-O,D.,DEUSSEN,O.,LISCHINSKI,D.,ANDONG,T.-T.2007.Solidtexturesynthesisfrom2dexemplars.ACMTransactionsonGraphics26,3(Proc.SIG-GRAPH2007),2.WATRA,V.,SCHODL,A.,ESSA,I.,TURK,G.,ANDOBICKA.2003.Graphcuttextures:imageandvideosynthesisusinggraphcuts.ACMTransactionsonGraphics22,3(Proc.SIG-GRAPH2003),277–286.WATRA,V.,ESSA,I.,BOBICK,A.,ANDWATRA,N.2005.Textureoptimizationforexample-basedsynthesis.ACMTrans-actionsonGraphics24,3(Proc.SIGGRAPH2005),795–802.EFEBVRE,S.,ANDOPPE,H.2006.Appearance-spacetex-turesynthesis.ACMTransactionsonGraphics25,3(Proc.SIG-GRAPH2006),541–548.,J.,GEORGHIADES,A.S.,GLASER,A.,W,H.,WEIL.-Y.,GUO,B.,DORSEY,J.,ANDUSHMEIER,H.2007.Context-awaretextures.ACMTrans.Graph.26,1,3.ERILLOU,S.,DISCHLER,J.-M.,ANDHAZANFARPOUR,D.2001.Corrosion:simulatingandrendering.InProc.GraphicsInterface2001,CanadianInformationProcessingSociety,167–CHRIJVER,A.2003.CombinatorialOptimization:PolyhedraandEfciency,vol.A.Springer-Verlag,BerlinHeidelberg.IMAKOV,D.,CASPI,Y.,SHECHTMAN,E.,ANDRANI,M.2008.Summarizingvisualdatausingbidirectionalsimilarity.Proc.CVPR2008,IEEEComputerSociety.ARMA,M.,ANDISSERMAN,A.2003.Textureclassication:Arelterbanksnecessary.InProc.CVPR2003,IEEE,691–698.ANG,J.,TONG,X.,LIN,S.,PAN,M.,WANG,C.,BAO,H.,UO,B.,ANDHUM,H.-Y.2006.Appearancemanifoldsformodelingtime-variantappearanceofmaterials.ACMTransac-tionsonGraphics25,3(Proc.SIGGRAPH2006),754–761.EI,L.-Y.,ANDEVOY,M.2000.Fasttexturesynthesisus-ingtree-structuredvectorquantization.Proc.SIGGRAPH2000EI,L.-Y.,HAN,J.,ZHOU,K.,BAO,H.,GUO,B.,ANDHUMH.-Y.2008.Inversetexturesynthesis.ACMTrans.Graph.273,1–9.EI,L.-Y.,LEFEBVRE,S.,KWATRA,V.,ANDURK,G.,2009.Stateoftheartinexample-basedtexturesynthesis.Eurographics2009StateofTheArtReport,April.EXLER,Y.,SHECHTMAN,E.,ANDRANI,M.2004.Space-timevideocompletion.InProc.CVPR2004,vol.1,120–127.HANG,J.,ZHOU,K.,VELHO,L.,GUO,B.,ANDHUM,H.-Y.2003.Synthesisofprogressively-varianttexturesonarbi-trarysurfaces.ACMTransactionsonGraphics22,3(Proc.SIG-GRAPH2003),295–302.