Who Allocated My Memory Detecting Custom Memory Alloca - PDF document

WhoAllocatedMyMemory?DetectingCustomMemoryAllocatorsinCBinariesXiChenAsiaSlowinskaHerbertBosVrijeUniversiteitAmsterdam,TheNetherlandsx.chen@vu.nl,fasia,herbertbg@few.vu.nlAbstract—Manyreversingtechniquesfordatastructuresrelyontheknowledgeofmemoryallocationroutines.Typically,theyinterposeonthesystem'smallocandfreefunctions,andtrackeachchunkofmemorythusallocatedasadatastructure.How-ever,manyperformance-criticalapplicationsimplementtheirowncustommemoryallocators.Examplesincludewebservers,databasemanagementsystems,andcompilerslikegccandclang.Asaresult,currentbinaryanalysistechniquesfortrackingdatastructuresfailonsuchbinaries.WepresentMemBrush,anewtooltodetectmemoryallocationanddeallocationfunctionsinstrippedbinarieswithhighaccu-racy.Weevaluatedthetechniqueonalargenumberofrealworldapplicationsthatusecustommemoryallocators.Asweshow,wecanfurnishexistingreversingtoolswithdetailedinformationaboutthememorymanagementAPI,andasaresultperformananalysisoftheactualapplicationspecicdatastructuresdesignedbytheprogrammer.Oursystemusesdynamicanalysisanddetectsmemoryallocationanddeallocationroutinesbysearchingforfunctionsthatcomplywithasetofgenericcharacteristicsofallocatorsanddeallocators.I.INTRODUCTIONManyreversingtechniquesfordatastructuresdependontheanalysisofmemoryallocatedontheheap[1]–[5].Typ-ically,theyinterposeonthesystem'smallocandfreefunctions,andtrackeachchunkofmemorythusallocatedasdatastructure.Doingsoiswellandgoodforapplicationsthatusethestandardmemoryallocationandde-allocationfunctions,butunfortunatelymanylargerandperformance-criticalprogramsdonot.Instead,theyimplementtheirowncustommemorymanagers,typicallydesignedforefciency.Well-knownexamplesofsuchapplicationsincludetheApachewebserver,thePostgreSQLdatabasemanagementsystem,thegcccompiler,andDropbox.Asreverseengineersdonothaveaccesstosource,theprecisememoryallocationanddealloca-tionfunctionsarenotknown.Asaresult,alltechniquesthatbuildontheinterpositionofsuchfunctionsfail.Theproblemisthattheyonlyseetheallocationsbythesystem'sgeneralpurposeallocators,butnotthesubdivisionoftheseallocationsintosmallerfragmentsbytheapplication'scustommemoryallocator(CMA).Unfortunately,thelargerchunksthatarevisibletothereverseengineerservemerelyasapoolforthemorerelevantallocationsoftheactualdatastructures.Phraseddifferently,thelargechunksthemselvesaremostlymeaningless,whilethesmallerfragmentsarereusedbyvariousfunctionsandsystemcalls.Missingthemmakesitexceedinglydifculttoobserveanymeaningfulaccesspatternsanddetecttheobjectsdesignedbytheprogrammer.Inthispaper,wedescribeasetoftechniquestodetectmem-oryallocationanddeallocationfunctionsinstrippedC/C++binarieswithhighaccuracy.WeimplementedthetechniquesinatoolcalledMemBrushandevaluateditonalargenumberofcustommemoryallocators.ThemaingoalofMemBrushistofurnishexistingreversingtools,disassemblersanddebuggerswithdetailedinformationaboutthememorymanagementAPIimplementedbyaCMA.KnowingtheCMA'sallocation,deallocation,andreallocationroutines,allowsustointerposeonthemandreusethememoryanalysistechniquesforgeneral-purposeallocatorsinapplica-tionsthat`rolltheirown'.Todemonstrateit,weuseMem-BrushtosupportanexistingreversetoolcalledHoward[2].Howardisatooltoextractlow-leveldatastructuresfromastrippedbinary.ThankstoMemBrush,Howardwasabletoextractheapstructuresthatitwouldotherwisenotevensee.Inaddition,researchershaveshownthatknowledgeofmemoryallocationanddeallocationroutinesisusefulforretrottingsecurityinexistingbinaries—forinstancetoprotectagainstmemorycorruption[6]–[11].Currently,thesesecuritymeasuresarepowerlessiftheapplicationusesCMAs.Again,withMemBrushtheseexistingtechniquesshouldsimplywork,regardlessofthememoryallocator.High-leveloverview.ThekeyobservationbehindMemBrushisthatmemoryallocationfunctionshavecharacteristicsthatsetthemapartfromotherroutines.Forinstance,amalloc-likeroutinewillreturnaheapaddressandmalloc'sclientswillusepointersderivedfromthataddresstoaccessmemory,andsoon.MemBrushchecksthesecharacteristicsatruntimetakingcaretolteroutroutinesthatexhibitsimilarbehavior(likewrappers,iterators,etc.)asmuchaspossible.Likealldynamicanalysis,MemBrush'sresultsdependonthecodethatiscoveredatruntime.Specically,itwillnotndCMAroutinesincodethatneverexecutes.Thispaperisnotaboutcodecoveragetechniques.Rather,weusetestsuitestocoverasmuchoftheapplicationaspossible.Fortunately,applicationsthatemployCMAs,typicallyusetheallocationroutinesfrequently—afterall,thatiswhytheyhavethemintherstplace.Thus,ndinginputsthatexercisetheCMAcodeisnotverydifcult,andMemBrushidentiedalmost90%ofalltheCMAroutinesinalltheapplicationswetested.Insummary,MemBrushisabletounearthmostCMAroutinesinarbitrary(gcc-generated)binarieswithahighdegreeofprecision.Whileitistooearlytoclaimthatthe problemofCMAidenticationissolved,MemBrushadvancesthestateoftheartsignicantly.Forinstance,wemanagedtoaccuratelyanalyzethecomplexCMAsystemsusedbytheNginxwebserver,ortheProFTPdleserver.WeimplementedalldynamicanalysistechniquesusingIntel'sPindynamicbinaryinstrumentationframework[12].Ourcurrentimplementationworkswithx86C/C++binariesonLinuxgeneratedbythegccoptimisingcompiler,buttheapproachisnotspecictoanyparticularOSorcompiler.II.BACKGROUNDANDOBSERVATIONSProgrammersincorporatecustommemoryallocatorsintotheirapplicationstoimproveperformance,andinthecaseofregion-basedallocators–toreducetheprogrammingburdenandeliminateasourceofmemoryleaks.Underthehood,CMAsusegeneral-purposememoryallo-cationroutines,suchasmallocandmmap,toallocatelargebuffers,andthendenetheirowncustomfunctionstoallocatethesebuffersintosmallerones.Applicationsusetheresultingblockstostorestructureddataitemssuchasarrays,structs,orC++objects.Whenanapplicationreleasesablock,aCMAdoesnotimmediatelyreturnthememorytothegeneral-purposeallocator.Instead,itmayserveitonafuturerequestbytheapplicationanddefertherealdeallocation(forinstance,untilthetimethatnomorerequestsaretobeexpectedfromtheapplication).Ratherthanaimingforthisorthatcustommemoryal-locator,theobjectiveofMemBrushistodetectanyCMA.InSectionII-A,wethereforeintroducepopulartypesofcustommemoryallocators.Then,inSectionII-B,welisttheessentialcharacteristicsofCMAsthatlaythefoundationforourdetectionalgorithmdescribedinSectionsIII-VI.A.ATaxonomyofCMAsSincecomprehensiveoverviewsofCMAscanbefoundinsurveysbyWilsonetal.[13]andBergeretal.[14],welimitourselvestoasummaryoftheapproachesinthissection.LikeBergeretal.[14],wedistinguishthefollowingvecategories:Per-classallocators(alsoknownasslaballocators).Aper-classallocatorretainsmemorytocontaindataobjectsofthesametype(orsize).ItimplementsthesameAPIasageneral-purposememoryallocator(malloc/free),i.e.,itsupportsal-locationanddeletionofindividualobjects.SlaballocatorsarewidelyusedbymanyUnixandUnix-likeoperatingsystemsincludingFreeBSD[15](“zones”)andLinux[16].Regions(alsoknownasarenas,groups,andzones[17],[18]).Eachobjectallocatedbyanapplicationisassignedtoaregion,i.e.,alargechunkofmemory.Programmerscanonlydeallocateallobjectsfromaregionatonce–individualdeallocationsarenotpossible.Thislimitationfacilitatesallo-cationanddeallocationofmemorywithalowperformanceoverhead,atthecostofanincreasedmemoryusage.ExampleapplicationsusingregionsincludeApache[19](whichreferstothemas“pools”),PostgreSQL[20](whichreferstothemas“memorycontexts”),andNginx[21].Obstacks.Anobstack[22]isamoregenericversionofaregion.Itcontainsastackofobjects,withinwhichanindividualobjectisfreedalongwitheverythingallocatedinthisobstacksincethecreationoftheobject.Anexampleapplicationusingobstacksisthegcccompiler.Custompatterns.ThiscategoryincludesallallocatorsthatimplementthesameAPIasageneral-purposememoryallocator(malloc/free),butaretailoredtotheneedsofaparticularapplication.Forexample,oneoftheallocatorsusedbyNginxfallsintothiscategory.Hybridapproaches.Theresearchcommunityhasproposedvariousapproachestoprovidee.g.,high-speedallocationandcache-levellocality.Forinstance,reaps[14]areacombinationofregionsandgeneral-purposeallocatorsthatextendregionsemanticswithindividualobjectdeletion.B.EssentialCharacteristicsofCMAsHavinglookedatthedifferentcategoriesofCMA,wenowsummarizetheircommonfeatures.ItisimportanttoemphasizethatthesefeaturesaimtocapturethefundamentalbehaviorofCMAsandnotsomeimplementationartifactofspecicvariants.Forinstance,alloftheeightCMAimplementationsthatweanalyzeinSectionVIIIexhibitthesecharacteristics.AswewillseeinSectionsIII-VI,thesecharacteristicsformthebasisforourdetectionalgorithm.Wewilldiscussallocation,deallocation,andreallocationroutinesinturn.Inagenericsense,wewillrefertothesecustomfunctionsasc malloc,c free,andc realloc,respectively.Allocationroutines.c mallocfunctionssubdividelargememorychunksobtainedfromageneral-purposeallocatorintosmallones,andservethesmallonesupontheapplication'srequests.Wemakethefollowingbasicobservationsaboutacustomallocator'sbehavior:(A1)Normally,ac mallocfunctionreturnsapointerpthatreferencesaheapmemoryregion.AswediscussinSectionIII,insomecasesthisruleshouldberelaxed.E.g.,ac mallocdoesnotneedtoliterallyreturnp,butitmightpassitthroughanoutgoingargument.(A2)Applicationsuseporapointerderivedfromp,e.g.,(p+offset),towritetomemory.Herealso,weexpectsomedeviationsfromsuchbehavior.Forinstance,itispossiblethattheoccasionalapplicationallocatesamemoryblockthatitdoesnotuse.However,thisshouldbetheexception,ratherthantherule.Iftheapplication(almost)neverwritestomemoryreferencedbyp,thenthefunctionthatreturnsitdoesnotserveasanallocator.(A3)Unlessthec mallocfunctioninitializesmemorychunkspriortoreturningthem,theapplicationshouldwritetothesechunksbeforereadingthem.(A4)Ac mallocshouldnotreturnthesameobjecttwiceuntilthatchunkisreleasedrstwithacalltoac freefunction.(A5)Sinceweaimtoexcludewrapperfunctions,werequirethatac mallocnotonlychecksandpassesapointerob-tainedfromanotherinternalfunction,butalsoperforms somecomputationstoderivetheaddressofanewlyallocatedobject.Deallocationroutines.Whenanapplicationfreesachunkofmemoryobtainedfromac mallocroutine,c freereclaimsthechunk,sothatitcanbeservedagainonfuturerequests.ThealgorithmsinSectionVarebasedonthefollowingcharacteristicsofdeallocators:(D1)CMAskeeptrackofwhichpartsofmemoryareinuse,andwhichpartsarefree.Theyrecordthelocationsandsizesoffreeblocksinsomekindofmetadata,whichmaybealist,atree,abitmaporanotherdatastructure.Thus,ac freefunctionaccessesthemetadatathatisalsomaintainedbyac mallocfunction.(D2)Whenac freereleasesamemoryregion,theapplicationshouldnotaccessitanymoreunlessthereisabug(andweassumebugsarerare).(D3)Whenac freereleasesamemoryobject,ac mallocmayreturnitonfutureapplication'srequests.(D4)Sinceweaimtoexcludewrapperandinternalhelperfunctions,weselecttheoutermostfunctionthatsharesthemetadatawithac malloc.Theintuitionisthatifafunctiondoesusethemetadata,itshouldbeconsideredapartoftheCMA.Reallocationroutines.Finally,c reallocfunctionsallowapplicationstomodifythesizeofapreviouslyallocatedmemoryblock.Toguaranteethatthenewblockiscontiguousinmemory,c reallocmayhavetorelocateitelsewhere.Weconsiderthefollowingfeaturesofc reallocroutines:(R1)Likec mallocinA1,c reallocfunctionsreturnapointerptoaheapmemoryregion.(R2)Likedeallocationfunctions(D1),c reallocfunctionsalsoaccessthemetadatausedbyac malloc.(R3)Asin(A2)and(A3),applicationsuseporapointerderivedfromptowritetomemory,andwritetotheallocatedmemorybeforereadingit.(R4)Onceac reallocmodiesthesizeofabuffer,futurerepetitionsofthesamerequestdonotrequireanyaction,soalsodonotrelocateit(idempotence).(R5)Ac reallocpreservesthecontentsofamemoryblockuptothelesserofthenewandoldsizes.Thus,iftheblockisrelocated,ac realloccopiestheoldcontentstothenewlocation.WhenR5ndsthatac reallocfunctionrelocatesabuffer,weadditionallyverifyR6–R7below:(R6)Asac realloccombinesac mallocandac free,italsoreleasesamemoryobject,andtheapplicationshouldnotaccessitanymore(asinD2).(R7)Likec freeinD3,ifac reallocreleasesamemoryobject,ac mallocmightreturnitonfutureapplication'srequests.EventhoughtheabovefeaturesreecttheexpectedbehaviorofCMAs,weemphasizethatMemBrushallowsforoccasionaldeviations.Forexample,itispossiblethatanapplicationhasause-after-freebug,andusesachunkofmemoryeventhoughithasbeendeallocatedalready,violatingD2.Also,eventhoughanapplicationshouldnotreaduninitializedmemory(abreach Fig.1.MemBrush:high-leveloverview.ofA3),wemightoccasionallyobservesuchbehavior.Aswewillseelater,wepermitsuchexceptionsaslongastheyarerare.However,inpractice,wedidnotcomeacrossthem.III.ABIRD'SEYEVIEWOFMEMBRUSHWenowdiscusstheCMAdetectionprocedure.MemBrushconsistsofinstrumentationmodulesanddetectionmodules(seeFigure1).Theinstrumentationmodules,3-6,providesupport(suchasdynamicinformationowtracking)forthedetectionmodules,whilethedetectionmodules,7-10,searchfortheCMAroutines.Inthissection,webrieyintroducethevariouscomponents,andinthenextfoursections,weexplainthedetectionmodulesindetail.Inthispaper,wesearchforCMAroutinesthatoperateontopofthemmap/brksystemcallsorthelibclibrary(i.e.,thatinternallycallmalloc/free)toallocatelargechunksofmemory.However,wecancongureMemBrushtodetecttheDougLeaallocator[23]usedbytheGNUClibraryaswell.Todoso,wewouldsimplychoosenottosearchforallocatorsbasedonmalloc,butsolelyonmmap/brk.WeimplementedMemBrushusingIntel'sPindynamicbinaryinstrumentationframework[12].PinprovidesarichAPItomonitorcontextinformation,e.g.,registerormemorycontents,onprograminstructions,function-andsystemcalls.ThemaincomponentsofFigure1arethefollowing:Inputs:12ThemaininputtoMemBrushisa(possibly)strippedx86binary1anditsinputs2.Forthispaper,weusedexistingtestsuitestocoverasmuchoftheapplicationaspossible.Ifneeded,wecanalsoemployacodecoveragetoolforbinarieslikeS2E[24].Callstacktracking:3Toanalyzeifafunction'sbehaviorischaracteristicforaCMAroutine,MemBrushmonitorsthefunctionanditscallees.Forthat,itkeepstrackofthecontextinthefunctioncallstack.OurimplementationfollowsSlowinskaetal.[2].Partialreconstructionofphysicalstackframe:4ToanalyzeCMAroutines,MemBrushneedstoidentifystack-basedprocedurearguments.Like[2],ourimplementationisbasedondynamicanalysis.Inanutshell,wemonitorhowafunctioncalculatespointerstoaccessstackvariablespushedbyitscaller.Ifnecessary,wecanextenditwithastaticanalysispresentedbyElWazeeretal.[25]. Fig.2.Detectionofc mallocfunctions.Additionally,todeterminearstsetofcandidatesforc mallocandc reallocroutines,MemBrushmonitorsthereturnvalueofeachexecutedfunction,andchecksifitisapointerdereferencingaheapmemoryregion.Sinceingccgeneratedbinaries,32-bitreturnvaluesarenormallypassedusingtheEAXregister,MemBrushimplementsthispolicyaswell.Dynamicinformationowtracking(DIFT):5Asweshallexplainlater,thedetectionmodulesrelyondynamicinformationowtracking(fordataowanalysis).Ourtrackerisanextendedversionoflibdft[26].LikemostotherDIFTengines[27],wepropagateinformationondirectowsonly:wecopytagsondatamoveoperations,orthemonALUoperations,andsoon.Wedonotpropagateanyinformationonindirectdataows,suchasconditionalstatements.Pointertracking:6MemBrushmonitorshowtheappli-cationusespointersreturnedbythec mallocandc realloccandidates.Tothisend,thepointertrackingmoduletrackshowpointerstoheapmemoryderivefromotherpointers,andwheretheyarestored.OurimplementationisbasedonSlowinskaetal.[2]whichextendsthegenericDIFTmodule5withpointerpropagationrules.Detectionmodules:78910ThedetectionmodulesidentifytheactualCMAAPI:c malloc,c free,andc realloc.MemBrush'salgorithmscheckforthecharacter-isticfeaturesdiscussedinSectionII-B,andsearchfortheroutinesinturn.Intherststep7,MemBrushdeterminesc mallocroutines.Then8,ittriestondc freefunctionsthatcanbecoupledwiththealreadydetectedallocationfunctions.Inthelaststep9,itidentiesc reallocroutines.Finally10,weperformanadditionalanalysisofthedetectedCMAroutines.IV.CUSTOMALLOCATORDETECTIONTodetectc mallocroutines,MemBrushsearchesforfunc-tionsthatmatchA1-A5fromSectionII-B.Figure2representstheprocedureasalinearpipeline,inwhicheachstagepro-gressivelyltersoutfunctionsthatdonotcomplywiththecorrespondingfeatures.MemBrushstartsbyidentifyingacrudesetofc malloccandidates,i.e.,functionsthatreturnpointersreferencingheapmemoryregions(A1).Whiletheapplicationexecutes,Mem-Brushusesthepointertrackingmodule6totrackallpointersderivedfromtheaddressesreturnedbythegeneral-purposememoryallocators.Thisway,italsofollowsacustomalloca-torcalculatingthelocationsofallocatedobjects.MemBrushmonitorsthereturnvaluesofallfunctionsinvokedatruntime,andselectstheonesthatreturneitheratrackedpointerorasingleconstantthatmightindicateanerror,e.g.,NULL.ToverifyA2,MemBrushtracksallpointersderivedfromthereturnvalueofeachc malloccandidate,andmonitorsiftheyareusedtowritetomemory.ToassessA3,MemBrushadditionallyexaminesiftheapplicationusesthesepointerstowritetoamemorylocationbeforereadingit.Unlesstheallocatorinitializesthememoryitself,thepresenceofsuchread-before-writessuggestseitherthatthecandidateisnoc mallocfunction,or(iftheoccurrenceisrare)thattheapplicationisbuggy.Todealwithallocatorsthatinitializetheirownmemory,MemBrushtagsallmemorylocationswrittenbythecandidatefunction(oritscallees)withauniqueidentier,sothatisabletospottheuninitializedreadslater.Next,weretainfromtheremainingc malloccandidatesonlythosefunctionsthatneverreturnthesamememoryregionagainuntilitisdeallocatedbyac free(A4).Ourapproachdrawsonloadtesting.Thebasicideaisthatweinserta“callloop”thatrepeatsspecicinvocationsofthecandidatefunctionsmanytimes.Aslongasweensurethattheapplicationdoesnotreleasetheallocatedregionwithacalltoac freeroutine,wewouldexpectaproperc malloctoreturnastreamofdistinctaddressesinaccordancewith(A1).Thecandidateprogressestothenextstageifeither(1)it(oroneofitscallees)invokesthegeneral-purposeallocatortoallocateanewmemoryregionandreturnsapointerreferencingit,or(2)itbeginstoreturnanon-pointervalueconsistently,possiblyindicatingthattheapplicationhasrunoutofmemoryandcannotallocateanyextra.Incontrast,wedropthec malloccandidateif(1)theapplicationcrashes,(2)thereturnvalueisapointeralreadyseenduringtheloadtest,or(3)thereturnvalueisneitherapointernoraninvariableerrormessage.Theimplementationreliesonapartialreconstructionofthephysicalstackframeofthec malloccandidate4.First,wepausetheexecutionatacallinstructionthattransfersthecontrolowtothecandidatefunction,andwestoretheCPUcontextofthecallsite.Specically,werecordthevaluesoftheregistersandthestack-basedarguments.Inordertoreplaytheinvocation,MemBrushrepeatedlyresetstheCPUcontexttotherecordedone,restartstheexecutionatthecallinstruction,pausesitagainwhenthefunctionreturns,andexaminesthereturnvalue.Sincethereplayloopmightcorruptthestateoftheapplicationorcauseamemoryleak,werestarttheapplicationafterthisstep.Whileensuringtodothereplayforeverycandidatefunction,MemBrushreplaysanumberofrandomlychoseninvocationsofthecandidate.Finally,welteroutallocatorwrappers(A5).MemBrushclassiesac malloccandidateasawrapperif(1)it(oroneofitscallees)invokesafunctionactuallycategorizedasanallocator,and(2)wheneveritreturnsapointer,itpassesavaluereceivedfromacalleewithoutmodifyingit.Theimplementationbuildsonthecallstack3andpointertrackingmodules6. Fig.3.Detectionofc freefunctions.V.CUSTOMDEALLOCATORDETECTIONTodetectc freeroutines,MemBrushsearchesforfunctionsthatitcancouplewiththealreadyidentiedc mallocroutines.Ac freefunctionmatchesac mallocroutineiftheysharetheirmetadata,andallocate/releasethesamememoryregions.Theprocedureissimilartothatforc mallocfunctionsinthatMemBrushlterscandidatefunctionsinalinearpipelineofstageswhereeachstageveriesoneoftheconditionsD1-D4ofSectionII-B.Figure3illustratesahigh-levelpicture.TherststageisbasedontheobservationthatCMArou-tinessharesomekindofmetadatathatrecordsthepositionsoffreeblocks.Hence,ac freeroutineaccessesdatainmemorywhichc mallocalsousestoderivethereturnvalues(D1).MemBrushrstpinpointsthemetadata,andthenmonitorstheapplicationtoidentifythefunctionsthatreadormodifyit,whichbecomec freecandidates.MemBrushdeterminesthemetadatawhilec mallocfunc-tionsexecute.First,whenac mallocaccessesaheaporstaticmemorylocationforthersttime,MemBrushtagsitwithauniqueidentier.Then,itemploystheDIFTmodule5tomaintainadataowgraphwhichrecordshowthesevaluespropagateandhowtheyarecombined.Whenthec mallocroutinereturns,MemBrushpinpointsthemetadata:itconsultsthegraph,andlistsallmemorylocationsthatcontributedtothecalculationofthereturnvalue.Observethatthemetadatamightrepresenteitherpointersorindices/offsetswhichaCMAusestocomputetheaddressesofallocatedregions.AsMemBrushemploysagenericDIFTapproach,itisimpervioustosuchimplementationdetails.Thenexttwostagesbuildontheobservationthatc mallocandc freeroutineshandlethesamememoryregions.First,MemBrushveriesthatonceac freecandidatereleasesabuffer,theapplicationdoesnotaccessitanymore(D2).Then,ittriestomaketheCMAserveagainamemorychunkthathasjustbeenreclaimedbyac freecandidate(D3).Bothstepsrequirethat,foreachc freeinvocation,MemBrushpinpointsatleastonematchingc mallocinvocation,i.e.,ac mallocwhichallocatedabufferreclaimedbyacalltothec freecandidate.Inanutshell,MemBrushhastwowaystocouplec mallocandc freeinvocations.Therstonereliesonanaccuratepa-rametermatchbetweenthetwofunctions.MemBrushrequiresthatalltheargumentsofthec freecandidateareeithertheargumentsorthereturnvalueofapastc mallocinvocation.Inthesecond(moregeneric)method,ac mallocandac freeinvocationmatchiftheyusethesamemetadata.Observethatthemappingneednotbeone-to-one.Forinstance,forregionbasedallocators,weexpectmultiplec mallocinvocationstomatchasinglec freecandidate.FollowingD2,MemBrushrequiresthatonceac freecan-didatereleasesabuffer,theapplicationdoesnotaccessitanymore.Unlessthereisause-after-freebugintheapplication,thepresenceofsuchaccessessuggeststhatthecandidateisnotac freefunction.Inpractice,wetoleratesomeuse-after-freeaccessestoallowforbugsinthecode,butthenumberofsuchaccessesshouldbelessthan.Inourexperiments,weused=1%.Toanalyzeaninvocationofac freecandidate,MemBrushidentiesamatchingc mallocinvocation,andmonitorsallaccessestotheassociatedheapbuffer.Iftheapplicationstillusesthisbufferafterthec freecandidatereturns,itmeansthatthecandidatefunctiondidnotactuallyreleasethememory,soitdoesnotprogresstothenextstep.D3statesthatwhenc freereclaimsachunkofmemory,theCMAmayserveitagainonfuturerequests.Toverifyac freecandidate,wetrickc mallocintoreallocatingthereclaimedmemory.Whenthecandidatedeallocatorreturns,wesearchthecurrentexecutiontraceforac mallocinvocationthatallocatedabufferinthememorythatwasapparentlyjustfreed,andwereplayitmanytimesinacallloop,asexplainedinSectionIV.Weretainthec freecandidateiftheallocatorreturnsthesamepointerastheinvocationbeingreplayed.Incontrast,wedropthecandidateifthec mallocfunctionfailstoreallocatethatmemoryregion—becauseitcrashes,returnsanerrormessage,orrequestsmorememoryfromthegeneral-purposeallocator.AsinSectionIV,werestarttheapplicationafterthisstep.Finally,wedecidewhichfunctionsformtheCMAinter-face(D4).Ifmultiplefunctionsinthesamecallstackreachedthisstep,wepicktheoutermostone.TheintuitionisthatfunctionsabovetheCMAinterfaceneverdirectlyaccessthemetadata.Thus,ifafunctionusesit,itmustbeCMA-related.VI.CUSTOMREALLOCATORDETECTIONTodetectc reallocroutines,weagaingenerateasetofcandidatescandidates,andthenverifythemagainstR1-R7ofSectionII-Binpipeline-fashion.Figure4presentsanoverviewofthealgorithm.Wewillseethatdetectionofreallocationroutinesreusesmanystepsoftheprevioussections.Thismakessense,becauseareallocationcombinespropertiesofdeallocationandallocation.First,weidentifyc realloccandidatesasthosefunctionsthatreturnpointerstoheapobjects,andthatsharethemetadatawithc mallocroutines(R1andR2).TheimplementationofthisstagesdrawsheavilyonthechecksforA1andD1.Next,toverifyiftheapplicationusesapointerreturnedbyac realloccandidatetowritetothereallocatedheapbufferinawrite-before-readfashion(R3),wereusethevericationofA2andA3.R4requiresthatifac realloccandidaterepeatedlyservesaspecicrequest,onlytherstinvocationshouldtriggeranactionandmayrelocatethebuffer.Again,weconrmthisbehaviorbyreplayingtheinvocations.Specically,whenthe Fig.4.Detectionofc reallocfunctions.candidatereturns,MemBrushreplaysthisinvocationmanytimesinacallloop,andretainsthecandidateonlyifthereturnedvalueremainsconstant.Next,weanalyzeifaninvocationofac realloccandidaterelocatesamemoryblocktomodifyitssize(R5).Asimpletestcouldcheckifapointerreturnedbythecandidateindicatesanobjectallocatedbyac mallocfunctionthatisnotyetfreed.Observe,however,thatthisrequiresanabilitytoaccuratelypinpointallobjectsreleasedbyc freeroutines.AsweexplaininSectionIX,thereexistCMAimplementationswhichmakeitverychallenging.MemBrush,ontheotherhand,leveragesthefactthatc reallocpreservesthecontentsofreallocatedmemoryblocks.Thus,whenac reallocfunctionrelocatesanobject,italsocopiestheoldcontents.Todetectthecopyoperation,Mem-BrushusestheDIFTmodule5.Itmonitorsifthec realloccandidate(oranyofitscallees)copiesdatafromabufferalreadyallocatedbyac malloc.Incaseofarelocation,MemBrushexpectsacopyofacontiguousblockfromanaddressreturnedbyac malloctothereturnvalueofthecandidate.Thesourceofthisoperationisthereallocatedbuffer.Whenthepreviousstageconcludesthataninvocationofac realloccandidaterelocatesabuffer,wealsoconrmthattheapplicationdoesnotaccessthereallocatedbufferanymore(R6),andthatthememoryblockisinfactfreed(R7).ThischeckisidenticaltothevericationofD2andD3—again,wemonitorthereleasedmemory,andwetrickc mallocroutinesintoreallocatingit.Thereallocatedbufferdeterminesthec mallocinvocationweneedtoreplay.VII.ADDITIONALANALYSISOFTHECMAROUTINESWenowunearthadditionalcharacteristicsofCMAs.First,wedescribeMemBrush'sheuristictoestimatethesizeofbuffersrequestedthroughc malloc/c reallocfunctions,andthenwediscusshowwedistinguishbetweenthedifferenttypesofallocatorsfromSectionII-A.A.BufferSizeEstimationBeforewedescribeMemBrush'sproceduretoestimatehowmuchmemorytheapplicationrequestsfromacustomallocatorroutine,observethatitisnotatrivialtask.Afterall,sincetheapplicationmaywellallocatemorememorythanitwillneedduringourtests,wecannotjustmonitorhowmuchofthebufferisactuallyused.MemBrush,instead,rstcollectsanumberofsamplec malloc1invocationsalongwithanupperboundaryonthesizeoftheallocatedbuffers.Then,ittriestodeviseaformulacapturingtherelationbetweenanargumentofthec mallocfunctionandtheassociatedsize.Thecollectionofsamplesisagainbasedonthereplaymech-anism.MemBrushreplaysanumberofac mallocfunctioninvocationsmanytimes,andforeachofthem,itmonitorsthestreamofreturnedvalues.Whentheallocatorservesrequestsfromthesameregionobtainedfromthegeneralpurposeallocator,MemBrushmeasuresthedistancesbetweenthem.Theyrepresenttheupperboundonthesizeoftheallocatedbuffers.Additionally,ifMemBrushndsthattheCMAstoresthemetadatabetweenthechunksreturnedtotheapplication,itexcludesthesebytesfromthedistancemeasurement.Observethat,weshouldonlyincludethedistancesbetweenmemorychunksadjacenttoeachother,lestwesignicantlyoverestimatetheupperboundontheirsize.Tothisend,MemBrushwaitsforthec mallocfunctiontoinvokethegeneral-purposeallocatortoallocateanewmemoryregion,andservetherequestsfromit(refertothevericationofA4inSectionIV).Thisway,wearecertainthatwekeeptrackofallthebuffersallocatedinthatregion,soourestimationoftheirsizeisasaccurateaspossible.Inthesecondstep,foreachc mallocroutine,MemBrushtriestoderiveaformuladescribingthesizeofanallocatedbufferasafunctionofanargumentofthec malloc.Specif-ically,whenwedenotethesizeoftheallocationrequestandthevalueofoneoftheargumentsofthec mallocfunctionbysizeandarg,respectively,weassumethattheCMAusesoneofthefollowingformulas:size=a1arg+b1orsize=a22arg+b2:Next,foreachargumentvariableoftheallocator,arg,weconsiderallthecollectedpairsofthemaximumestimatedsizeandarg,(max size;arg),andwesearchforvaluesofa1,b1,a2,andb2suchthatmax sizea1arg+b1andmax sizea22arg+b2:Finally,weselect(a1andb1)or(a2andb2)thattthesamplesbest,i.e.,minimizethecumulativedistancebetweenthevaluesoftheformulaandtheboundarysizes.AsweshowinSectionVIII,MemBrush'smechanismyieldsgoodresultsinpractice.Itdoesnotworkonlyiftheobjectsizeisdeterminedwhentheapplicationinitializesaninstanceofanallocator,andnotwhenitallocatesabuffer.Then,differentinvocationsoftheallocatorfunctionresultindifferentallocationsizes,yetwecannotndarelationbetweenthemandthefunction'sarguments.1Wefollowexactlythesameprocedureforc reallocroutines. AllocatorEqual-sizedIndividualMultiplechunksobjectdeallocation Per-classXXRegionsaXObstacksaXCustompatternsXHybridapproachesXX aWeuseadditionalcriteriatodistinguishregionsfromobstacks.TABLEIMEMBRUSH'SCRITERIATOCLASSIFYCMAS.B.ClassicationofCMAsToclassifyCMAs,weexaminetwocharacteristics:thesizesofallocatedbuffers,andtherelationbetweentheallocationanddeallocationroutines.Additionally,weneedameanstodistinguishgenericregionsfromobstacks.First,wecheckifaCMAsplitsaregionobtainedfromageneral-purposeallocatorintoequal-sizedchunks.Tothisend,wemonitorobjectswhoseaddressesarederivedfromthebaseofaparticularmalloc/mmapbuffer,andwecomparetheirsizes.Next,weassessifadeallocatorreleasesindividualormultipleobjectsatonce.Tonditout,wecheckhowmanyc mallocinvocationsmatchasingleinvocationofac free(refertoStep1inSectionV).TableIsummarizesthedecisionprocedure.Asthebasiccriteriaarestringentenoughtodistinguishallallocatortypesexceptfromobstacks,weadoptjustoneextraone.Observethat,sinceobstacksallowforthefreeingofobjectsallocatedsincethecreationofanyobjectintheregion,allocationsfollowingacalltoac freefunctiondonotnecessarilystartatthebottomoftheregion,butatanylocationinsideit.Thus,wemonitorstreamsofaddressesofobjectswithinindividualregions,andwecheckiftheirincreasingsubsequencesstartatthesamelocation.Eventhoughitwasnotnecessaryinourexperiments,wecouldadditionallyvalidatetheper-classallocators.Insteadofcomparingonlythesizesofallocatedobjects,wecanalsoexaminetheirlow-leveldatastructures.WedemonstratethisprocedureinSectionVIII-C.VIII.EVALUATIONInthissection,weevaluateMemBrush.Wediscussitsaccuracy(SectionVIII-A),presentsomestatisticsillustratingthedetectionprocedure(SectionVIII-B),andnallywedemonstratethepracticalbenetsofapplyingMemBrushtoanexistingbinaryanalysistechniqueforreversingdatastructures(SectionVIII-C).A.AccuracyofMemBrush'sDetectionAlgorithmInthissection,weevaluatetheaccuracyofMemBrush.Westartwithanoverviewoftheapplicationswetested,andwereporthowwellMemBrushmanagedtopinpointtheCMAroutines.Then,wecontinuewithaclassicationofCMAs.Finally,wediscusstheaccuracyofMemBrush'sheuristic ApplicationAllocatorsDeallocatorsReallocatorsTPsFPsTPsFNsTPsFNs apache3/5-4/6-0/1-nginx7/7-2/2-0/0-smbget(samba)1/1-1/1-1/1-wget1/1-1/1-1/1-proftpd6/6-5/5-0/0- 400.perlbench14/16-5/5-0/0-401.bzip20/0-0/0-0/0-403.gcc14/1745/5-0/0-429.mcf0/0-0/0-0/0-446.gobmk0/0-0/0-0/0-456.hmmer0/0-0/0-0/0-458.sjeng0/0-0/0-0/0-462.libquantum0/0-0/0-0/0-464.h26ref0/0-0/0-0/0-471.omnetpp0/0-0/0-0/0-473.astar0/0-0/0-0/0-483.xalancbmk6/6-6/6-0/0- Total:52/59429/31-2/3- TABLEIITHEACCURACYOFMEMBRUSH'SALGORITHM.THETOPPARTOFTHETABLEREPORTSTHERESULTSFOR5REAL-WORLDAPPLICATIONS,ANDTHEBOTTOMONE—FORTHESPECINT2006BENCHMARKINGSUITE.toestimatethesizeofbuffersrequestedthroughc mallocfunctions.TheaccuracyoftheCMAroutinesdetection.TableIIpresentsanoverviewoftheapplicationsweanalyzedwithMemBrush.Thelistcontainsvereal-worldprograms,includ-ingtheApacheandNginxwebservers,smbgetfromtheSambanetworkingtool,theProFTPDleserver,andwget(conguredtousethelocklessallocator[28]).Additionally,weappliedMemBrushtotheSpecINT2006benchmarkingsuite.ToverifyMemBrush'saccuracy,wecomparetheresultstotheactualCMAroutinesintheprograms.Thus,alltheresultspresentedinthissectionwereobtainedforbinariesforwhichwecouldalsoconsultthesourcecodeandgetthegroundtruth.Foreachapplication,wereportthenumberofdetectedCMAroutinescomparedtothenumberoftheCMAroutinesintheapplication(TPs),andthenumberoffalsepositives(FPs).Overall,MemBrushdetectedcorrectly52outof59c mallocfunctions(88%),29outof31c freeroutines(94%),and2outof3c reallocfunctions(67%).Aswediscussbelow,manyfalsenegativesstemfromcompileroptimizations,andwecouldpreventlotsofthem.Asfarasthefalsepositivesareconcerned,therewerefour.Eventhoughstrictlyspeaking,thesefunctionsarefalsepositives,inpracticetheywerewrap-persofaninlinedallocator.Thus,byjustlookingatthebinary,MemBrushhasnomeanstoprovidemoreaccurateresults[29],andtheidentiedfunctionsdoprovidetheapplicationwithmemorychunksactingasproperallocators.Forthefalsenegatives,weoftenmissedacustomal-locatorbecausewedidnotevenclassifyitasac malloccandidateintherststep.Weidentiedtworeasonsforthis:(1)theallocatorpassesapointerinanoutgoingar-gument,andnotinthereturnvalue,or(2)insteadofa 2 4 6 8 Per-class Regions Custom Hybrid Obstacks 2 4 6 8 10 12 14 apache nginx samba wget proftpd perlbench gcc xalancbmk Fig.5.TheaccuracyofMemBrush'sproceduretoclassifyCMArou-tines.Thebottompartofthegraphpresentstheallocatorsthatwereclassiedcorrectly,andthetoponesummarizesmisclassications.pointertoaheapobject,theallocatorreturnsanoffset,whichtheapplicationaddstothebaseofabuffer(oftenusingamacro)beforeaccessingthememory.E.g.,inApache,theapr_rmm_malloc,apr_pool_create_excustomallo-cators,andalsotheapr_rmm_reallocreallocator,showthisbehavior.Thesameholdsforthetwomissingallocatorsin400.perlbench,andoneofthemissesin403.gcc.Inordertoreducetherstsourceoffalsenegatives,wecouldextendMemBrushtoconsiderresultsreturnedinparametersalso,usingthetechniquesdescribedbyElWazeeretal.[25].Tohandletheallocatorsreturninganoffsetinsteadofapointer,wecouldusedynamicinformationowtrackingtotellifthevaluereturnedbyafunctionislaterusedtoderiveapointerdereferencingheapmemory.Weleaveitasafuturework.Theremainingtwofalsenegativesin403.gccstemfromcompileroptimizations.Intherstcase,theapplicationalwaysjumpsto,andnevercalls,oneofthecustomallocators.Inthesecondcase,thealloc_pageroutineisinlined.MemBrushdetectedfourfunctions,whichare,strictlyspeaking,wrappersofalloc_page,butinpracticebehaveasallocators.Weformallyclassiedthemasfalsepositives,eventhoughtheywouldbeusefulresultsinpractice.ThetwomissesinthecustomdeallocatordetectioninApachearecausedsolelybythefalsenegativesintheallocatordetection.apr_rmm_mallocandapr_pool_create_exaretheonlyallocatorsthatcanreallocatethememoryreleasedbyapr_rmm_freeandapr_pool_destroy,respectively.Sincewedidnotdetecttheallocators,wedidnotmanagetotrickthemintoreallocatingthejustreclaimedmemoryeither.AsaresultthetwodeallocatorcandidatesdidnotpasstheD3lter.Insummary,weseethatMemBrush'salgorithmproveseffectivewithveryfewfalsepositives.Thereasonforalltheimportantfalsenegativesisthatwedonotidentifythevaluesreturnedbyafunctionaccuratelyenough.However,wecanemployexistingtechniquestofurtherimprovetheprocedure.TheaccuracyoftheCMAclassication.Figure5presentsthetypesofcustommemoryallocatorsclassiedbyMem-Brush.Thebottompartofthegraphcontainscorrectlyclas-siedfunctions,andthetopone–misclassications.Inthe403.gccbenchmark,MemBrusherroneouslymistookobstacksforregionbasedallocators.Eventhoughtheseallocatorsareconceptuallyobstack-based,eachobstackisimplementedasalistofchunks,andnotasaregionsplitintoindividualbuffers. 0 500 1000 1500 2000 2500 3000 3500 in A1 A2 A3 A4 A5 out apache nginx samba wget proftpd 400.perlbench 401.bzip2 403.gcc 429.mcf 446.gobmk 456.hmmer 458.sjeng 462.libquantum 464.h264ref 471.omnetpp 473.astar 479.xalancbmk Fig.6.ThenumberofallocatorcandidatesanalyzedbyMemBrushwhenverifyingcharacteristicsA1-A5.InApache,thereare35func-tionsaftertheA5step,andastheybelongtodifferentsharedlibraries,theymapto3functionsinthelibaprlibapr-utillibraries.TheCMAinsertsnewnodesinthelistwheneveranallocationoccurs,anddeletesanumberofthemostrecentlyaddedonesupondeallocation.Thus,theaddressesofallocatedchunks,i.e.,listelements,donotformincreasingsubsequencesasweexpected(refertoSectionVII-B).However,asobstacksareamoregenericversionofregions,wearenottooconcernedwiththismisclassication.Theaccuracyofthebuffersizeestimation.Ingeneral,MemBrusheitheraccuratelyestimatedhowmuchmemorytheapplicationrequestsfromacustomallocatorroutine,ordidnotprovideanyresults.Itmeans,thatMemBrush'sanalysisisaccurate,andtheresultsarenotmisleading.MemBrushdidnotmanagetodealwith7outof59allocators.Aswementionedalready,inallthesecases,theapplicationdeterminesthesizeofthebufferswhencreatinganallocator,andnotwhenallocatinganobject.Examplesincludethengx_array_pushfunctioninnginx,andtheapr_array_pushfunctioninApache.Foralltheremainingallocators,wefoundthatthesizeoftheallocationiseitheroftheform(arg+b)oritisaconstant.B.EffectivenessandNecessityofFilteringStagesWenowpresentsomestatisticsillustratingtheanalysisprocedure.Duetospaceconstraints,welimitthediscussiontothedetectionoftheallocationroutines.Figure6showshowmanyallocatorcandidatesMemBrushanalyzedineachstepofitsdetectionprocedure.Foralltheapplications,theA1lteridentiesupto430c malloccandidates(withamedianof78),andtheirnumbergraduallydropsasMemBrushproceeds.Eachtime,itndsatleast1wrapperfunction(193for483.xalancbmk,withamedianof14),ofteninvokingthegeneral-purposeallocator.C.PracticalBenets-aShowCaseInthissection,wedemonstratethebenetsofapplyingMemBrushtoabinaryanalysis.WeshowthatbyfurnishinganexistingreverseengineeringtoolwithinformationabouttheinterfaceimplementedbyaCMA,wesignicantlyincreasetheaccuracyoftheanalysis. Howard[2]isatooltoreversedatastructuresinstrippedbinaries.Toanalyzethememoryallocatedontheheap,itinterposesonthesystem'smallocandfreefunctions,andtrackseachchunkofmemorythusallocatedasadatastructure.Thus,whenthebinaryusesaCMA,Howarddoesnotanalyzethedatastructuresatthegranularityusedbytheapplication,anditsaccuracyislow.However,withtheknowledgeacquiredbyMemBrush,HowardcaninterposeontheroutinesusedbytheCMA,andfurtherperformitsanalysis.Asanexample,weanalyzeheapmemoryinthesm-bgetutilityinSamba.Asthecorememoryallocator,itusestalloc[30],ahierarchical,referencecountedmem-orypoolsystem.MemBrushdetectstwoCMAroutines:the__talloc()allocatorandthe__talloc_free()dealloca-tor.TableIIIpresentstheresultsobtainedbyHowardintwocases:(1)whenitanalyzesbuffersallocatedbythegeneralpurposeallocationroutines,and(2)whenitalsointerposesonthe__talloc()and__talloc_free()functionsfoundbyMemBrush.Wesplittheresultsintofourcategories:OK:Howardidentiedtheentiredatastructurecorrectly(i.e.,acorrectlyidentiedstructureeldisnotcountedseparately).Flattened:eldsofanestedstructurearecountedasanormaleldoftheouterstructure.Missed:Howardmisclassiedthedatastructure.Unused:singleelds,variables,orentirestructuresthatwereneveraccessedduringourtests.Asexpected,whenweusethevanillaversionofHoward,allthememorythatbelongstotheheapbuffersthatarelaterusedbytheCMA,iserroneouslyclassiedasarrays.Thus,wegetmeaningfulresultsonlyfortheremaining58:5%ofthearraysand53:2%ofthestructsallocatedontheheap.Incontrast,whenwecombineHowardwithMemBrush,theaccuracyoftheanalysisincreasessignicantly.Now,93:2%ofthearraysand91:3%ofthestructvariablesallocatedontheheapareclassiedcorrectly.Wecounted8:7%attenedstructures.Theyareallcausedbyalargetevent_reqstruc-turecontainingtwonestedsubstructures.Astheaddressesofthesubstructureseldsarealwayscalculatedrelativetothebeginningoftevent_req,Howardhadnomeansofclassifyingtheseregionsasindividualstructures.TheresultsshowthatbyusingMemBrush,Howardisabletoanalyzethedatastructuresactuallyusedbysmbget,insteadofthelargebuffersfurthersplitbytheCMAroutines.IX.LIMITATIONSMemBrushisnotawless.Inthissection,wediscusssomegenericlimitationswehaveidentied.Compileroptimizations.Ingeneral,MemBrushdetectsCMAroutinesatruntime,sotheanalysisresultscorrespondtotheoptimizedcode,whichmaybedifferentfromwhatisspeciedinthesource.ThisisknownasWYSINWYX(WhatYouSeeIsNotWhatYoueXecute)[29],anditmightleadtoinaccuracies.Forinstance,inthe403.gccbenchmark,MemBrushhasnomeanstoidentifyaninlinedallocator, CategoryWithoutMemBrushWithMemBrushArraysStructsArraysStructs Theresultsinthenumberofvariables:OK58.5%53.2%93.2%91.3%Flattened0%0%0%8.7%Missed41.5%46.8%6.8%0%Unused0%0%0%0% Theresultsinthenumberofbytes:OK60.4%51.7%92.4%90.2%Flattened0%0%0%9.8%Missed39.6%48.3%7.6%0%Unused0%0%0%0% TABLEIIITHEACCURACYOFTHEDATASTRUCTUREANALYSISWITHOUTANDWITHMEMBRUSH'SDETECTIONOFCMAFUNCTIONS.leadingtothefourfunctionsformallyclassiedasfalsepositives.Observethatanalyzingthecodethatexecutesisofcoursetherightthingtodo.Otherwise,wewouldnotbeabletoanalyzetherealbehaviorofthebinaryorperformproperforensics.Functionparameteridentication.InordertoidentifytheCMAroutinecandidates,andlateraccuratelymatchc freeandc mallocinvocations,MemBrushmonitorsthereturnvalueandtheargumentsoffunctions.OurcurrentimplementationassumesthatfunctionspassthereturnvalueusingtheEAXregister,andtheparametersusingthestack.AswesawinSectionVIII-A,thisisnotalwaysenough.However,wecouldextendourtechniqueasproposedbyElWazeeretal.[25].Identicationofthebuffersreleasedwithac freeroutine.EventhoughMemBrushcanaccuratelydetectc freeroutines,thereexistCMAimplementationswhichmakeitverychallengingtopinpointallthememorythatisfreed.Forinstance,whenoneofthedeallocatorsintheApachewebserverreleasesapool,italsoreclaimsallitssubpools,whichareseparateregionsobtainedfromthegeneralpurposeallocator.Findingoutinanimplementation-agnosticwayisdifcult.X.RELATEDWORKCustommemoryallocationisamatureeld.ManyrealworldapplicationsuseCMAs,typicallytoimproveruntimeperformance.Well-knownexamplesincludetheApacheandNginxwebservers,thegcccompiler,amongmanyothers.Manyresearchprojects,like[31]–[34],proposenewmemorymanagersdesignedforlowoverhead,andhigh-performancememoryallocation.Otherapproaches,e.g.,DieHard[35],Hound[36]andCling[37],usecustommemorymanagerstailoredtoimprovethememorysafetyofappli-cationsusingthem.Theyhelpmitigateheapcorruptions,danglingpointersorreadsofuninitializeddata.Manyapproachesthatdetectbufferoverows,use-after-freeordouble-freeattacks[6]–[11]relyoninformationabouttheprograms'datastructures—specically,thebuffersthattheyshouldprotect.Thus,inthepresenceofCMAs,theirscopeislimitedtomemorychunksobtainedfromthegeneral-purpose allocators.TheywouldalldirectlybenetfromMemBrush—toofferanergrainedprotection,andtodetectattacksontheactualdatastructuresusedbyapplications.Themostimportantoutcomeofourliteraturestudy,isthatthereis,toourknowledge,noworkondetectionofcustommemoryallocationroutines.XI.CONCLUSIONCustommemoryallocatorsareverycommoninreal-worldapplications,wheretheyareusedinsteadofthestandardallocationfunctionsforperformancereasons.Unfortunately,manyexistingbinaryanalysistechniquesdependontheabilitytointerceptthememoryallocationfunctions.Uptonowthiswasnotpossible.Inthispaper,wepresentedasetoftechniquesforidentifyingcustommemoryallocation,deallocation,andreallocationfunctions.Eachofthesethreecategoriesishandledbyaseparatepipelineofltersthataimtotestfundamentalpropertiesthatmostholdforalmostanyimplementation.WeevaluatedourtechniquesonadiversesetofcustommemoryallocatorimplementationsandverifytheiraccuracyonbothSpecIntandseveralreal-worldapplicationsthatareknowntousecustommemoryallocators.Inpracticallyallcases,weshowedthatwecanndtheallocationroutineswithgreataccuracy.Finally,weshowedthattheoutcomeofourresearchisimmediatelyusefulbyusingtheresultsintheHowarddatastructureextractiontool.ACKNOWLEDGMENTSThisworkissupportedbytheEuropeanResearchCouncilthroughprojectERC-2010-StG259108-ROSETTA,andtheEUFP7SysSecNetworkofExcellence.REFERENCES[1]C.JungandN.Clark,“DDT:designandevaluationofadynamicprogramanalysisforoptimizingdatastructureusage,”inProc.ofthe42ndAnnualIEEE/ACMInternationalSymposiumonMicroarchitecture,ser.MICRO-42,2009.[2]A.Slowinska,T.Stancescu,andH.Bos,“Howard:adynamicexcavatorforreverseengineeringdatastructures,”inProc.ofthe18thAnnualNetwork&DistributedSystemSecuritySymposium,ser.NDSS'11,2011.[3]Z.Lin,X.Zhang,andD.Xu,“Automaticreverseengineeringofdatastructuresfrombinaryexecution,”inProc.ofthe17thAnnualNetworkandDistributedSystemSecuritySymposium,ser.NDSS'10,2010.[4]G.BalakrishnanandT.Reps,“Analyzingmemoryaccessesinx86binaryexecutables,”inProc.Conf.onCompilerConstruction,ser.CC'04,2004.[5]T.RepsandG.Balakrishnan,“Improvedmemory-accessanalysisforx86executables,”inCC'08/ETAPS'08:Proc.oftheJointEuropeanConferencesonTheoryandPracticeofSoftware17thinternationalconferenceonCompilerconstruction,2008.[6]“Valgrind.”[Online].Available:http://valgrind.org[7]B.Perence,“ElectricFence.”[Online].Available:http://perens.com/FreeSoftware/ElectricFence[8]R.HastingsandB.Joyce,“Purify:Fastdetectionofmemoryleaksandaccesserrors,”in1992WinterUSENIXConference,1992.[9]D.DhurjatiandV.Adve,“EfcientlyDetectingAllDanglingPointerUsesinProductionServers,”inProc.oftheInternationalConferenceonDependableSystemsandNetworks,ser.DSN'06,2006.[10]J.Caballero,G.Grieco,M.Marron,andA.Nappa,“Undangle:earlydetectionofdanglingpointersinuse-after-freeanddouble-freevulner-abilities,”inProc.ofthe2012InternationalSymposiumonSoftwareTestingandAnalysis,ser.ISSTA'12,2012.[11]A.Slowinska,T.Stancescu,andH.Bos,“BodyArmorforBinaries:preventingbufferoverowswithoutrecompilation,”inProc.ofUSENIXAnnualTechnicalConference,ser.USENIXATC'12,2012.[12]Intel,“Pin-ADynamicBinaryInstrumentationTool,”http://www.pintool.org/,2011.[13]P.R.Wilson,M.S.Johnstone,M.Neely,andD.Boles,“DynamicStorageAllocation:ASurveyandCriticalReview,”1995.[14]E.D.Berger,B.G.Zorn,andK.S.McKinley,“Reconsideringcustommemoryallocation,”inProc.ofthe17thACMSIGPLANconferenceonObject-orientedprogramming,systems,languages,andapplications,ser.OOPSLA'02,vol.37,no.11,2002.[15]TheFreeBSDProject,“FreeBSDKernelDe-veloper'sManual.ZONE(9).”[Online].Available:http://www.freebsd.org/cgi/man.cgi?query=uma[16]M.T.Jones,“AnatomyoftheLinuxSlabAllocator,”2007.[Online].Available:http://www.ibm.com/developerworks/linux/library/l-linux-slab-allocator/[17]D.T.Ross,“TheAEDfreestoragepackage,”CommunicationsoftheACM,vol.10,no.8,1967.[18]D.R.Hanson,“Fastallocationanddeallocationofmemorybasedonobjectlifetimes,”Software:PracticeandExperience,vol.20,no.1,1990.[19]TheApacheSoftwareFoundation,“DevelopingmodulesfortheApacheHTTPServer2.4.”[Online].Available:http://httpd.apache.org/docs/2.4/developer/modguide.html[20]ThePostgreSQLGlobalDevelopmentGroup,“PostgreSQL9.2.4Documentation.Section43.3.MemoryManagement.”[Online].Available:http://www.postgresql.org/docs/9.2/static/spi-memory.html[21]nginx,“nginxdocumentation.”[Online].Available:http://nginx.org/en/docs/[22]“TheGNUClibrary.Obstacks.”[Online].Available:http://www.gnu.org/software/libc/manual/html node/Obstacks.html[23]DougLea,“AMemoryAllocator.”[Online].Available:http://g.oswego.edu/dl/html/malloc.html[24]V.Chipounov,V.Kuznetsov,andG.Candea,“S2E:Aplatformforinvivomulti-pathanalysisofsoftwaresystems,”inProc.ofthe16thIntl.ConferenceonArchitecturalSupportforProgrammingLanguagesandOperatingSystems,ser.ASPLOS'11,2011.[25]K.ElWazeer,K.Anand,A.Kotha,M.Smithson,andR.Barua,“ScalableVariableandDataTypeDetectioninaBinaryRewriter,”inProc.ofthe34thACMSIGPLANConferenceonProgrammingLanguageDesignandImplementation,ser.PLDI'13,2013.[26]V.P.Kemerlis,G.Portokalidis,K.Jee,andA.D.Keromytis,“libdft:PracticalDynamicDataFlowTrackingforCommoditySystems.”inProc.ofthe8thAnnualInternationalConferenceonVirtualExecutionEnvironments,ser.VEE'12,2012.[27]G.Portokalidis,A.Slowinska,andH.Bos,“Argos:anEmulatorforFingerprintingZero-DayAttacks,”inProc.ofthe1stACMEuropeanConferenceonComputerSystems2006,ser.EuroSys'06,2006.[28]Lockless,“LocklessPerformance.”[Online].Available:http://locklessinc.com[29]G.BalakrishnanandT.Reps,“WYSINWYX:WhatYouSeeIsNotWhatyoueXecute,”ACMTrans.Program.Lang.Syst.,vol.32,pp.23:1–23:84,2010.[30]Samba,“tallocDocumentation.”[Online].Available:http://talloc.samba.org/talloc/doc/html/index.html[31]S.Schneider,C.D.Antonopoulos,andD.S.Nikolopoulos,“Scalablelocality-consciousmultithreadedmemoryallocation,”inProc.ofthe2006IntlSymposiumonMemoryManagement,ser.ISMM'06,2006.[32]A.JulaandL.Rauchwerger,“Twomemoryallocatorsthatusehintstoimprovelocality,”inProc.ofthe2009InternationalSymposiumonMemoryManagement,ser.ISMM'09,2009.[33]R.LiuandH.Chen,“SSMalloc:alow-latency,locality-consciousmemoryallocatorwithstableperformancescalability,”inProc.ofthe3rdACMAsia-PacicWorkshoponSystems,ser.ApSys'12,2012.[34]S.Lyberis,P.Pratikakis,D.S.Nikolopoulos,M.Schulz,T.Gamblin,andB.R.deSupinski,“Themyrmicsmemoryallocator,”inProc.ofthe2012Intl.SymposiumonMemoryManagement,ser.ISMM'12,2012.[35]E.D.BergerandB.G.Zorn,“DieHard:probabilisticmemorysafetyforunsafelanguages,”inProc.ofthe2006ACMConferenceonProgramminglanguagedesignandimplementation,ser.PLDI'06,2006.[36]G.Novark,E.D.Berger,andB.G.Zorn,“Efcientlyandpreciselylocatingmemoryleaksandbloat,”inProc.ofthe2009ConferenceonProgramminglanguagedesignandimplementation,ser.PLDI'09,2009.[37]P.Akritidis,“Cling:Amemoryallocatortomitigatedanglingpointers,”inProc.ofthe19thUSENIXconferenceonSecurity,ser.SSYM'10,2010.