April Mengzhi Wang Main Memory Databases Mengzhi Wang April Advanced Topics in Database - PDF document

15-823April5,2001/MengzhiWang1MainMemoryDatabasesMengzhiWangApril5,200115-823AdvancedTopicsinDatabaseSystemsPerformance 15-823April5,2001/MengzhiWang2WhatisMMDB?•Diskresidentdatabases(DRDB)Memoryresidentdatabases(MRDB)Mainmemorydatabases(MMDB) 15-823April5,2001/MengzhiWang3Memoryvs.DiskAccesstime–MemoryisfasterthandiskAccesspattern 15-823April5,2001/MengzhiWang5OutlineOverviewMMDBtechniquesDataorganizationQueryprocessingRecovery 15-823April5,2001/MengzhiWang6MMDBtechniquesAssumption 15-823April5,2001/MengzhiWang7DataOrganizationTraditionalDRDB–Relations:slottedpagesTwoindirectionsforeasymovingExploitationofthediskaccesspattern–Indices:B-treeShallow:pointerchasingisexpensivefordisks 15-823April5,2001/MengzhiWang8DataOrganization:RelationsRelations–Stillorganizedinpages?–Howtodealwithvariablelengthfields?–Howtodealwithforeignkeys?Wemaystillwantpagesaround–Unitsforrecovery–Slottedpages?Maybenot 15-823April5,2001/MengzhiWang9DataOrganization:RelationsTupleId:addressForeignkey:Pointers EmpNameDept DeptName JohnMichaelGeorgeSalesDevelopment 15-823April5,2001/MengzhiWang10DataOrganization:RelationsFieldvalues–Pointstoadomain–NoneedtoworryaboutvariablelengthfieldsVariablelengthfields–Pointerstoheapspace 15-823April5,2001/MengzhiWang11DataOrganization:IndicesB-tree–Shallow:goodfordisks–Spaceutilization:Always�50%Keys~50%amongthem 15-823April5,2001/MengzhiWang12DataOrganization:IndicesKeyvaluesinindices–Storekeyvaluesorpointerstotuples?Indexstructure–B-treeorsomethingelse? 15-823April5,2001/MengzhiWang13DataOrganization:T-treeProposedbyLehmanandCareyModifiedbinaryAVLtrees–Twopointersandmorethanonekeyvaluesineachnode. k’1k23…kn’’ k1k2k3kn k’ k’’ 15-823April5,2001/MengzhiWang14DataOrganization:T-treeBalancedbyrotatingnodesNumberofrotationreducedbyallowingalittlevariationinnumberofkeysineachnodesAdvantages–Spaceefficient–Logarithmperformance 15-823April5,2001/MengzhiWang15OutlineOverviewMMDBtechniques–DataorganizationQueryprocessing–RecoveryCasestudiesMMDBrevisited 15-823April5,2001/MengzhiWang16QueryProcessingCostmodel–DRDB:I/O–MMDB:Computations 15-823April5,2001/MengzhiWang17QueryProcessingJoins:pointerjoin EmpNameDept DeptName JohnMichaelGeorgeSalesDevelopment 15-823April5,2001/MengzhiWang18QueryProcessingTemporaryresults–Usepointerstotuplesinsteadofcopyingdata EmpNameDeptJohnMichaelGeorge DeptNameSalesDevelopment temp 15-823April5,2001/MengzhiWang19OutlineOverviewMMDBtechniques–Dataorganization–QueryprocessingRecoveryCasestudiesMMDBrevisited 15-823April5,2001/MengzhiWang20RecoveryRecoverability–Frequencyoffailures–DatalossPerformance–Transactionsrunningfaster–Committingstillslow 15-823April5,2001/MengzhiWang21RecoveryDifferentConsiderations–Commitprocess–Checkpointing–Reload 15-823April5,2001/MengzhiWang22CommitProcessTransactions:ACID–Durability:LogforcedtodisksatcommitProblem:LogI/ObecomesbottleneckHowlongdoweneedtokeepthelog?–Untilthenextcheckpoint 15-823April5,2001/MengzhiWang23CommitProcessSolution1:–Usestablememoryforlogtails StableMemory MainMemory Logtails 15-823April5,2001/MengzhiWang24CommitProcessSolution2:Groupcommit–Accumulateloguntilpageisfull–Writealogpageoutonlyonce–AlsousedinDRDB 15-823April5,2001/MengzhiWang25CommitProcessSolution3:Precommit–Releasethelockafterlogsinlogbuffer–Reduceblockingtimeofothertransactions 15-823April5,2001/MengzhiWang26CheckpointingCheckpointsinDRDB–Bringpagesondiskuptodate–ReducetheworkofrestartprocessCheckpointsinMMDB–Makeacopyofthedataondisks–Truncatethelogs 15-823April5,2001/MengzhiWang27CheckpointingGoal–Littleinterferencewithusertransactions.Framework–Twocopiesofdataondisk–Ping-pongalgorithmTechniques–Non-fuzzycheckpointing–Fuzzycheckpointing 15-823April5,2001/MengzhiWang28CheckpointingNon-fuzzycheckpointing–Actionconsistentortransactionconsistent–Locksimposedbycheckpointingprocess–Increasinglockcontentions Actionconsistentpoints BT1ET1 Updatex1Updatex2 Transactionconsistentpoints time 15-823April5,2001/MengzhiWang29CheckpointingFuzzycheckpointing–Flushpagesregardlessoftheirstates–Redotheoperationsduringrecovery BT1ET1 Updatex1Updatex2 Transactionconsistentpoints Actionconsistentpoints time Fuzzycheckpointingflushespagesregardlessoftheirstates 15-823April5,2001/MengzhiWang30ReloadGoal:fastApproach:–Simple:acceptworkafterallthedataisloaded–Fast:workafterasmallamountofdataisreloaded 15-823April5,2001/MengzhiWang31MMDBSummaryDateorganization–UsepointersasmuchaspossibleRecovery–FocusonperformanceAssumption:–Memoryischeapatrandomaccesses 15-823April5,2001/MengzhiWang32OutlineOverviewMMDBtechniques–Dataorganization–Queryprocessing–RecoveryCasestudiesMMDBrevisited 15-823April5,2001/MengzhiWang33CaseStudy:TimestenTimesten(http://www.timesten.com –Mainmemorydatabasesystems–Traditionalsystemarchitecture–T-treeforindices–TentimesfasterthanDRDB 15-823April5,2001/MengzhiWang34Casestudy:Dalihttp://www.bell-labs.com/project/dali/ MainmemorydatastoragesystemGoal:highperformanceSimilartoObjectStore–Datamappedtomemory–AccessthroughC++API–Limitedquerycapabilities 15-823April5,2001/MengzhiWang35Casestudy:DaliStoragemanager–Dataorganizedasdatafiles–Mappedtovirtualmemoryaddressspace–Pointersasoffsetsinfiles–Dividedintosegments–Differentrecoverymechanismfordifferentdata 15-823April5,2001/MengzhiWang36Casestudy:DaliTransactions:Multi-levelrecovery–Physicalundoandredologsinmemoryforongoingactions–Logicalredologsinsystemlogatprecommit–SystemlogflushedtodiskatcommitCheckpointing:–Fuzzy 15-823April5,2001/MengzhiWang37Casestudy:DaliSecurity–ProcessdeathReleaselatchesondatastructuresRollbacktransactions–ApplicationerrorsMemoryprotectionCodewords 15-823April5,2001/MengzhiWang38Casestudy:Monethttp://www.cwi.nl/ MainmemorydatabasesDesigned–Fordatawarehouseapps–Forcacheperformance 15-823April5,2001/MengzhiWang39Casestudy:MonetExampleSelectname,salaryFromemployeeWhereage35Observation:–Smallstridesoffersbetterlocality 15-823April5,2001/MengzhiWang40Casestudy:MonetVerticaldecomposingSpecialhandlingofstrings EmpNameDeptJohnMichaelGeorgeOlddesign Newdesign EmpTID DeptTID NameTIDJohnMichaelGeorge S2S3 15-823April5,2001/MengzhiWang41OutlineOverviewMMDBtechniques–Recovery–Dataorganization–QueryprocessingCasestudiesMMDBrevisited 15-823April5,2001/MengzhiWang42MMDBRevisitedAssumption–MemoryaccessesareuniformlycheapToday’ssituation–Memoryaccessesareexpensive–MemoryaccessesarenotuniformCacheperformanceisimportant 15-823April5,2001/MengzhiWang43DataOrganization:RelationsPointerstodomain?Foreignkeyaspointers?Variablelengthfields? 15-823April5,2001/MengzhiWang44DataOrganization:RelationsRowmajororcolumnmajor?Maybemixed–Groupfrequentlyaccessedfieldstogether 15-823April5,2001/MengzhiWang45DataOrganization:IndicesKeysaspointersinindices?T-treeorB-treeorsomethingelse?T-treesarenotgood–DeeptreesleadtoheavypointerchasingCSB-treePrefetchingB+-trees 15-823April5,2001/MengzhiWang46QueryProcessingCache-consciousprocessing–Similartechniquesformemory-diskhierarchyPartition–Blocking 15-823April5,2001/MengzhiWang47SummaryPreviousMMDBwork–Recovery–Indexstructures–ProtectionToday’sfocus–Cacheawareness 15-823April5,2001/MengzhiWang48ReferencesH.Garcia-Molina,K.Salem.Mainmemorydatabasesystems:anoverview.TKDE4(6),P.Bohannonetal.ThearchitectureoftheDalimain-memorystoragemanager.JournalofMultimediaToolsandApplications,1997.TobinJ.Lehman,MichaelJ.Carey.Astudyofindexstructuresformainmemorydatabasemanagementsystems.VLDB’86.K.Salem,H.Garcia-Molina.Checkpointingmemory-residentdatabases.ICDE89