Catac ysm Handling Extreme Overloads in Internet Ser vices Bhuv an Ur gaonkar and Prashant - PDF document

Cataclysm:HandlingExtremeOverloadsinInternetServicesBhuvanUrgaonkarandPrashantShenoyDepartmentofComputerScience,UniversityofMassachusetts,AmherstMA01003bhuvan,shenoy
@cs.umass.eduAbstractthispaperwepresentCataclysm,acomprehensiveapproachforhandlingextremeoverloadsinhostedInternetapplications.Theprimarycontributionofourworkistodevelopanoverloadcontrolapproachthatbringstogetheradmissioncontrol,dy-namicprovisioningofplatformresources,andadaptivedegra-dationofQoSintooneintegratedsystem.Cataclysmprovidesseveraldesirablefeaturesunderoverloads,suchaspreferentialadmissionofimportantrequests,theabilitytohandlediverseworkloads,andrevenuemaximizationatmultipletime-scalesviadynamicprovisioningandsize-basedadmissioncontrol.Cataclysmcantransparentlytradeofftheaccuracyofitsde-cisionmakingwiththeintensityoftheworkloadallowingittohandleincomingratesofseveraltensofthousandsofre-quests/second.WeimplementaprototypeCataclysmhostingplatformonaLinuxclusteranddemonstratethebenetsofourintegratedapproachusingavarietyofworkloads.1Introduction1.1MotivationDuringthepastdecade,therehasbeenadramaticincreaseinthepopularityofInternetapplicationssuchasonlinenews,on-lineauctionsandelectroniccommerce.ItiswellknownthattheworkloadseenbyInternetapplicationsvariesovermultipletime-scalesandofteninanunpredictablefashion[12,29].Cer-tainworkloadvariationssuchastime-of-dayeffectsareeasytopredictandhandlebyappropriatecapacityprovisioning[15].Othervariationssuchasashcrowdsareoftenunpredictable.OnSeptember11th2001,forinstance,theworkloadonapop-ularnewswebsiteincreasedbyanorderofmagnitudeinthirtyminutes,withtheworkloaddoublingeverysevenminutesinthatperiod[29].Theloadone-commerceretailwebsitescanincreasedramaticallyduringthenaldaysofthepopularhol-idayseason.Similarly,theloadononlinebrokeragewebsitescanbeseveraltimesgreaterthantheaverageloadduringanunexpectedmarketcrash.Inthispaper,wefocusonhandlingextremeoverloadsseenbyInternetapplications.Informally,anextremeoverloadisascenariowheretheworkloadunexpectedlyincreasesbyuptoanorderofmagnitudeinafewtensofminutes.Ourgoalsare(i)todesignasystemthatremainsoperationaleveninthepresenceofanextremeoverloadandevenwhentheincom-ingrequestrateisseveraltimesgreaterthansystemcapacity,and(ii)tomaximizethenumberofrequestsservicedbytheapplicationduringsuchanoverload.WeassumethatInternetapplicationsorservicesrunonahostingplatform—essentiallyaserverclusterthatrentsitsresourcestoapplications.Ap-plicationproviderspayforserverresources,andinturn,areprovidedperformanceguarantees,expressedintheformofaservicelevelagreement(SLA).Ahostingplatformcantakeoneormoreofthreeactionsduringanoverload:(i)addcapacitytotheapplicationbyallocatingidleorunder-usedservers,(ii)turnawayexcessrequestsandpreferentiallyserviceonly“im-portant”requests,and(iii)degradetheperformanceofadmit-tedrequestsinordertoservicealargernumberofaggregaterequests.Thersttwoapproacheshavebeenstudiedintheliterature.Therstapproachinvolvesdynamicprovisioningtomatchap-plicationcapacitytotheworkloaddemand[9,24,27].Thesecondapproachinvolvespolicingintheformofadmissioncontrol,whichlimitsthenumberofadmittedrequestssothatthecontractedperformanceguaranteesaremet[11,14,32,35].Thenotionofprovidingpreferentialtreatmentto“important”requestshasalsobeenstudied,althoughfromtheperspectiveofmaximizingrevenue(e.g.,bygivinghigherprioritytocer-tainrequests,suchasthoseinvolvingnancialtransactions[6]).Inthispaper,wearguethatacomprehensiveapproachtohandlingextremeoverloadsshouldinvolveasynergisticcom-binationofalloftheabovetechniques.Ahostingplatformshould,wheneverpossible,allocateadditionalcapacitytoanapplicationinordertohandleincreaseddemands.Theplat-formshoulddegradeperformanceinordertotemporarilyin-creaseeffectivecapacityduringoverloads.WhennocapacityadditionispossibleorwhentheSLAdoesnotpermitanyfur-therperformancedegradation,theplatformshouldturnawayexcessrequests.Whiledoingso,theplatformshouldpreferen-tiallyadmitimportantrequestsandturnawaylessimportantre-queststomaximizeoverallutility.Forinstance,smallrequestsmaybepreferredoverlargerequests,ornancialtransactionsmaybepreferredovercasualbrowsingrequests.Itisimportanttonotethatsuchacomprehensiveapproachtohandlingsevereoverloadsinvolvesmorethantheimplemen-1 tationofseparatemechanismstoachieveeachoftheabovegoals.Mechanismssuchasdynamicprovisioningandadmis-sioncontrolcanbecoupledinusefulandnon-trivialwaystofurtherimprovethehandlingofextremeoverloads.Forin-stance,theadmissioncontrollercanpro-activelyinvokethedynamicprovisioningmechanismwhentherequestdroprateexceedsacertainthreshold.Thedynamicprovisioningmech-anisminturncanprovideusefulinformationtotheadmissioncontrollerregardingtheprovisionedcapacitysothatthelat-tercansetappropriateperformancethresholdsforadmittedre-quests.Suchanintegrationofmechanismscanenhancetheabilityoftheplatformtohandleoverloads.Anorthogonalgoalforthehostingplatformisrobustnessundersevereoverloads.Robustness—theabilitytoremainop-erationalunderoverloads—-requiresthehostingplatformtobebothextremelyagileandefcient.Agilityrequiresaquickresponseinthefaceofasuddenworkloadspike.Efciencyre-quirestheabove-mentionedmechanisms,andinparticulartheadmissioncontroller,tohaveverylowoverheads.Sinceanex-tremeoverloadmayinvolverequestratesthatareuptoanorderofmagnitudegreaterthanthecurrentlyallocatedcapacity,theadmissioncontrollermustbeabletoquicklyexaminerequestsanddiscardalargefractionoftheserequests,whennecessary,withminimaloverheads.Whereaspriorapproachesforhandlingoverloadshavecon-sideredindividualmechanismssuchasprovisioningandad-missioncontrol,inthispaper,wefocusonanintegratedap-proach,withaparticularemphasisonhandlingextremeover-loads.ResearchContributionsofthisPaperInthispaper,wepresentCataclysm,ahostingplatformthatisdesignedtohandleextremeoverloadsinInternetapplica-tions.Thekeycontributionofourworkistointegratetech-niquessuchasadmissioncontrol,dynamicresourceprovision-ing,andadaptivedegradationofQoSintooneintegratedsys-temforhandlingoverloads.Unlikerecenteffortsonadmis-sioncontrol[19,32],ourtechniquesarespecicallyfocusedonhandlingextremeoverloads.Weproposeverylowover-headadmissioncontrolmechanismsthatcanscaletoveryhighrequestratesunderoverloads.Ourmechanismscanpreferen-tiallyadmitimportantrequestsduringoverloadandtranspar-entlytradeoffaccuracyoftheirdecisionmakingwiththein-tensityoftheworkload,enablingthemtoscaletoincomingratesofuptoafewtensofthousandsofrequests/s(notalloftheserequestsarenecessarilyadmittedandserviced;thead-mittedfractiondependsontheavailablecapacity).MultipleadmissioncontrollerscanbeemployedinlargerInternetappli-cationsforgreaterscalability.OurdynamicprovisioningmechanismemploysaG/G/1-basedqueuingtheoreticmodelofareplicableapplicationinconjunctionwithonlinemeasurementstodynamicallyvarythenumberofserversallocatedtoeachapplication.Theprovi-sioningtechniquecanbereactivelyinvokedbytheadmissioncontrollerwhentherequestdropratesexceedcertainvalues.Anovelfeatureofourplatformisitsabilitytonotonlyvarythenumberofserversallocatedtoanapplicationbutalsoothercomponentssuchastheadmissioncontrollerandtheloadbal-ancingswitches.Further,theadmissioncontrollerandprovi-sioningmechanismcooperatewithoneanother,andtherebyenhancetheabilityoftheplatformtocounteroverloads.WehaveimplementedaprototypeCataclysmhostingplat-formonaclusterofLinuxservers.Wedemonstratetheeffec-tivenessofourintegratedoverloadcontrolapproachviaanex-perimentalevaluation.Ourresultsshowthat(i)preferentiallyadmittingrequestsbasedonimportanceandsizecanincreasetheutilityandeffectivecapacityofanapplication,(ii)reactiveprovisioningisbothagileandeffectiveatdivertingplatformresourcestowheretheyareneededmost,whilemaximizingplatformrevenue.Therestofthispaperisorganizedasfollows.Section2pro-videsanoverviewoftheproposedsystem.Sections3,4and5describethemechanismsthatconstituteouroverloadmanage-mentsolution.Section6describestheimplementationofourprototype.InSection7wepresenttheresultsofourexperi-mentalevaluation.Section8presentsrelatedworkandSection9concludesthispaper.2SystemOverviewInthissection,wepresentthesystemmodelforourCataclysmhostingplatformandthemodelassumedforInternetapplica-tionsrunningontheplatform.2.1CataclysmHostingPlatformTheCataclysmhostingplatformconsistsofaclusterofcom-modityserversinterconnectedbyamodernLANtechnologysuchasgigabitEthernet.OneormorehighbandwidthlinksconnectthisclustertotheInternet.Eachnodeinthehostingplatformcantakeononeofthreeroles:cataclysmserver,cat-aclysmsentry,orcataclysmcontrolplane(seeFigure1).CataclysmServers:CataclysmserversarenodesthatrunInternetapplications.Thehostingplatformmayhostmultipleapplicationsconcurrently.Eachapplicationisassumedtorunonasubsetofthenodes,andanodeisassumedtorunnomorethanoneapplicationatanygiventime(thisisreferredtoasdedicatedhostingintheliterature).Notallcataclysmserversneedtobeassignedtoapplications—asubsetoftheserversmaybeunassignedandformthefreeserverpool.Thenum-berofserversassignedtoanapplicationcanchangeovertimedependingonitsworkload.Anapplication'sserverpoolcanbeincreasedeitherbyallocatingserversfromthefreepoolorbydeallocatingunder-utilizedserversfromanotherapplicationandreassigningthemtotheoverloadedapplication.Eachserveralsorunsthecataclysmnucleus—asoftwarecomponentthatperformsonlinemeasurementsofapplication-specicresourceusages,whicharethenconveyedtotheothertwocomponentsthatwedescribenext.CataclysmSentry:Eachapplicationrunningontheplat-formisassignedoneormoresentries.Asentryguardsthe2 Cataclysm servers for S1Cataclysm servers for S2Cataclysm Control Plane CataclysmNucleusrequests requestsApplication clients across the Internet CataclysmAdmission ControlLoad Balancing ProvisioningOS KernelNucleusFree Server PoolFigure1:TheCataclysmHostingPlatformArchitecture.serversassignedtoanapplicationandisresponsiblefortwotasks.First,thesentrypolicesallrequeststoanapplication'sserverpool—incomingrequestsaresubjectedtoadmissioncontrolatthesentrytoensurethatthecontractedperformanceguaranteesaremet;excessrequestsareturnedawayduringoverloads.Second,eachsentryimplementsalayer-7switchthatperformsloadbalancingacrossserversallocatedtoanap-plication.Therehasbeensubstantialresearchonloadbalanc-ingtechniquesforclusteredInternetapplications[7];anysuchlayer-7loadbalancingtechniquemaybeusedinthesentry.Whereasasinglesentrysufcesforsmallapplications,largeapplicationsrequiremultiplesentries,sinceasinglesen-tryserverwillbecomeabottleneckwhenguardingalargenumberofservers.Justasthenumberofserversallocatedtoanapplicationvarywiththeload,ourhostingplatformcandynamicallyvarythenumberofsentriesdependingonthein-comingrequestrate(andthecorrespondingloadonthesen-tries).Whenasentryisassignedordeallocated,theappli-cation'sserverpoolisrepartitionedandeachremainingsen-tryisassignedresponsibilityforamutuallyexclusivesubsetofnodes.Eachsentrythenindependentlyperformsadmissioncontrolandloadbalancingonarrivingrequests,therebycol-lectivelymaintainingtheSLAfortheapplicationasawhole.Around-robinDNSschemeisusedtopartition(andlooselybalance)theincomingrequestsacrossmultiplesentries.CataclysmControlPlane:Thecontrolplaneisresponsi-bleforprovisioningserversandsentriesforindividualappli-cations.Ittrackstheresourceusagesonnodes,asreportedbyCataclysmnuclei,anddeterminestheresources(intermsofthenumberofserversandsentries)tobeallocatedtoeachappli-cation.Thecontrolplaneandthesentriescooperatewithoneanotherduringanoverload.Asentrycaninvoketheprovision-ingmechanismsinthecontrolplanewhenanoverloadisde-tected,andthecontrolplanecanprovideprovisioning-relatedinformationtosentriesforadmissioncontrol.2.2ModelforInternetApplicationsTheInternetapplicationsconsideredinthisworkareassumedtobeinherentlyreplicable.Thatis,theapplicationisassumedtorunonaclusterofservers,anditisassumedthatrunningtheapplicationonalargernumberofserversresultsinanef-fectiveincreaseincapacity.Many,butbynomeansall,In-ternetapplicationsfallintothiscategory.Vanillaclusteredwebserversareanexampleofareplicableapplication.Multi-tieredInternetapplicationsarepartiallyreplicable.Atypi-calmulti-tieredapplicationhasthreecomponents:afront-endHTTPserver,amiddle-tierapplicationserver,andaback-enddatabaseserver.Thefront-endHTTPserveriseasilyreplica-blebutisnotnecessarilythebottleneck.Themiddle-tier—afrequentbottleneck—canbeimplementedindifferentways.Onepopulartechniqueistouseserver-sidescriptingsuchasApache'sphpfunctionality,ortousecgi-binscriptinglan-guagessuchasperl.1Ifthescriptsarewrittencarefullytohandleconcurrency,itispossibletoreplicatethemiddle-tieraswell.MorecomplexapplicationsuseJavaapplicationserverstoimplementthemiddle-tier.DynamicreplicationofJavaap-plicationserversismorecomplexandtechniquesfordoingsoarebeyondthescopeofthispaper.Dynamicreplicationofback-enddatabasesisanopenresearchproblem.Conse-quently,mostdynamicreplicationtechniquesintheliterature,includingthiswork,assumethatthedatabaseissufcientlywellprovisionedanddoesnotbecomeabottleneckevendur-ingoverloads.GivenareplicableInternetapplication,weassumethattheapplicationspeciesthedesiredperformanceguaranteesintheformofaservicelevelagreement.AnSLAhastwocompo-nents:(1)adescriptionoftheQoSguaranteesthattheplatformwillprovidetotheapplication,and(2)therevenueschemethatwillbeusedbytheplatformtobilltheapplication(whichmayincludepenaltiesforviolationsofthecontractedperformanceguarantees).TheSLAisdenedasfollows:ResptimeR ifarrivalrate\n\rifarrivalrate\nifarrivalrate\n(1)Revenue "!$#&%' andarrivals% "!$#&%'andarrivals%( "!$#&%'andarrivals)*\rotherwise(2)Essentially,theSLAspeciesthedesiredresponsetimesfordifferentarrivalratesandalsotherevenueearnedbytheplat-formformeetingthedesiredresponsetimeatthecorrespond-ingarrivalrate.Figure2illustratesonesuchSLA.Addition-ally,theSLAmayalsospecifyalowerboundonthenumberofserversthatanapplicationshouldbeassigned.1Acommontechniqueforimplementingamulti-tieredapplicationistousetheso-called“LAMP”servers—Linux,Apache,mysqlandphp.3 arrival rate avg. resp. time 1000 - 10000&#x 100;� 100001 secQoS TableRevenue Table3 sec arrival rate1000 - 10000&#x 100;� 10000revenue per0.05 if avg. RT 1 sec+0.05 if avg. RT 2 sec,-Figure2:Asampleservicelevelagreement.3CataclysmSentryDesignInthissection,wedescribethedesignofaCataclysmsentrywhichisresponsiblefortwotasks—admissioncontrolandloadbalancing.Asindicatedearlier,theloadbalancingtechniqueusedinthesentryisnotafocusofthiswork,andweassumethesentryemploysalayer-7loadbalancingalgorithmsuchas[26].Givenourfocusonextremeoverloads,ourdesignoftheadmissioncontrollerfocusesontwokeyissues:(i)toensureverylowoverheadadmissioncontroltestsinordertoscaletoveryhighrequestratesseenduringoverloads,and(ii)tomax-imizetheutilityandthenumberofrequestsservicedduringoverloadsbypreferentiallyadmittingmoreimportantrequests.Intherestofthissection,weelaborateonthesetwoissuesindetail.AdmissionControlBasicsEachInternetapplicationisassumedtoconsistof.requestclasses:/0/(0/21.Thesentrymapseachincomingrequesttoaclass.Theclassofarequestdeterminesitsimportance—requestsmappedtoclass/aretreatedasmostimportantandthosein/21astheleastimportant.Thenum-berofrequestclasses.andthefunctionthatmapsrequeststoclassesisapplication-dependent.Toillustrate,avanillawebservermaydenetwoclassesandmaymapallrequestssmallerthanacertainsize3toclass/andlargerrequeststo/.Incontrast,anonlinebrokeragewebsitemaydenethreeclassesandmaymapnancialtransactionsto/,othertypesofcus-tomerrequestssuchasbalanceinquiriesto/,andcasualbrowsingrequestsfromnon-customersto/24.Eachclasshasaqueueassociatedwithit;incomingrequestsareappendedtothecorrespondingclass-specicqueue(seeFigure3).Theadmissioncontrollerprocessesqueuedrequestsinthedecreasingorderofimportance—requestsin/aresubjectedtotheadmissioncontroltestrst,andthenthosein/andsoon.Doingsoensuresthatrequestsinclass/25aregivenhigherprioritythanthoseinclass/76,89).Theadmissioncontroltest—whichisdescribedindetailinthenextsection—admitsrequestssolongasthesystemhassufcientcapacitytomeetthecontractedSLA.Notethat,ifrequestsinacertainclass/:5failtheadmissioncontroltest,allqueuedrequestsinlessimportantclassescanberejectedwithoutanyfurthertests.RequestswithineachclasscanbeprocessedeitherinFIFOorderorinorderoftheirservicetimes.Intheformercase,allrequestswithinaclassareassumedtobeequallyimportant,whereasinthelattercasesmallerrequestsaregivenpriorityoverlargerrequestswithineachclass.Admittedrequestsarehandedtotheloadbalancer,whichthenforwardsthemtooneoftheCataclysmserversintheapplication'sserverpool.Observethattheaboveadmissioncontrolstrategymeetsoneofourtwogoals—itpreferentiallyadmitsonlyimportantrequestsduringanoverloadandturnsawaylessimportantre-quests.However,thestrategyneedstoinvoketheadmissioncontroltestoneachindividualrequest,resultinginacomplex-ityof&#x?000;;=,where&#x?000;isthenumberofqueueduprequests.Further,whenrequestswithinaclassareexaminedinorderofservicetimesinsteadofFIFO,thecomplexityincreasesto&#x?000;&#x?000;;=duetotheneedtosortrequests.Sincetheincom-ingrequestratecanbeseveraltimeshigherthancapacitydur-inganextremeoverload,runningtheadmissioncontroltestoneveryrequestorsortingrequestspriortoadmissioncontrolmaybesimplyinfeasible.Consequently,inwhatfollows,wepresenttwostrategiesforverylowoverheadadmissioncontrolthatscalewellduringoverloads.3.2EfcientBatchProcessingOnepossibleapproachforreducingtheadmissioncontroloverheadistoprocessrequestsinbatches.Observethatre-questarrivalstendtobeveryburstyduringsevereoverloads,withalargenumberofrequestsarrivinginashortdurationoftime.Theserequestsarequeuedupintheappropriateclass-specicqueuesatthesentry.Ourtechniqueexploitsthisfea-turebyconductingasingleadmissioncontroltestonanentirebatchofrequestswithinaclass,insteadofdoingsoforeachindividualrequest.Suchbatchprocessingcanamortizethead-missioncontroloverheadoveralargernumberofrequests,es-peciallyduringoverloads.Toperformefcientbatch-basedadmissioncontrol,wede-neHbucketswithineachrequestclass.Eachbuckethasarangeofrequestservicetimesassociatedwithit.Thesentryestimatestheservicetimeofarequestandthenhashesitintothebucketcorrespondingtothatservicetime.Toillustrate,arequestwithanestimatedservicetimeintherange\rishashedtobucket1,thatwithservicetimeintherangetobucket2,andsoon.Bydeninganappropriatehashingfunction,hashingarequesttoabucketcanbeimplementedefcientlyasaconstanttimeoperation.Theadmissioncontrolisinvokedperiodicallyoneachre-questclass.Theadmissioncontrolthenconsiderseachnon-emptybucketinthatclassandinvokesasingleadmissioncon-troltestonallrequestsinthatbucket(i.e.,allrequestswithinabucketaretreatedasabatch).Consequently,nomorethanHadmissioncontroltestsareneededwithineachclass,oneforeachbucket.Sincethereare.requestclasses,thisreducestheadmissioncontroloverheadto;=,whichissubstan-tiallysmallerthanthe&#x?000;;=overheadforadmittingindividualrequests.Sincesuccessivebucketscontainrequestswithpro-4 ClassifierClass GoldAdmission ControlOPEQSRUTWVYXSV[ZC\^]_`a?bScedEfhgSf ikjlm?npoqrEsStUuWvhwSv xzy|{~} €Figure3:Class-baseddifferentiationforadmissioncontrol.gressivelylargerservicetimes,thetechniqueimplicitlygivesprioritytosmallerrequests.Moreover,nosortingofrequestsisnecessary—thehashingimplicitly“sorts”requestswhenmap-pingthemintobuckets.Havingprovidedtheintuitionbehindbatch-basedadmis-sioncontrol,wediscussthehashingprocessandtheadmis-sioncontroltestindetail.Inordertohasharequestintoabucket,thesentrymustrstestimatetheinherentservicetimeofthatrequest.Theinherentservicetimeofarequestisthetimeneededtoservicetherequestonalightlyloadedserver(i.e.,whentherequestdoesnotseeanyqueuingdelays).Theinherentservicetimeofarequestisdenedtobe‚5~ƒJ„… †k… ƒz‡ˆ‰EŠ‹ŒŽ@‘‡‘(3)where‰EŠ‹isthetotalCPUtimeneededtoservice, ‘‡‘istheIOtimeoftherequest(whichincludesthetimetofetchdatafromdisk,thetimetherequestisblockedonadatabasequery,thenetworktransfertime,etc.),andŽisanempiri-callydeterminedconstant.Section3.4discussesonlinemea-surementtechniquesforestimatingtheseparameters.Thein-herentservicetimeisthenusedtohashtherequestintoanappropriatebucket—therequestmapstoabucketsuchthat35%‚5pƒL„(…[†’… ƒ“‡%35p”.Thespecicadmissioncontroltestforeachbatchofre-questswithinabucketisasfollows.Let•denotethebatchsize(i.e.,thenumberofrequests)inabucket.Let–denotetheestimatedqueuingdelayseenbyeachrequestinthebatch.ThequeuingdelayisthetimetherequesthastowaitataCat-aclysmserverbeforeitreceivesservice;thequeuingdelayisafunctionofthecurrentloadontheserveranditsestimationisdiscussedinSection3.4.Let—denotetheaveragenumberofrequests(connections)thatarecurrentlybeingservicedbyaserverintheapplication'sserverpool.Thenthe•requestswithinabatchareadmittedifandonlyifthesumofthequeu-ingdelayseenbyarequestanditsactualservicetimedoesnotexceedthecontractedSLA.Thatis,–Œ™˜—Œ›šz•œ:Ÿž@‚%*!$ S¡p‘(4)where‚istheaverageinherentservicetimeoftherequestsinabatch,œisthenumberofserversallocatedtotheapplication,and! S¡e‘isthedesiredresponsetime.ThetermŒ£¢¤ƒ"¥@‚isanestimateoftheactualservicetimeofthelastrequestinthebatch,andisdeterminedbyscalingtheinherentservicetime‚bytheserverload—whichisthenumberoftherequestscurrentlyinservice,i.e.,—,plusthenumberofrequestsfromthebatchthatmightbeassignedtotheserveri.e,¢¤ƒ¥.Ratherthanactuallycomputingthemeaninherentservicetimeoftherequestinabatch,itisapproximatedas‚¦Œ35p”(’§M,whereKistheservicetimerageassociatedwiththebucket.Asindicatedabove,theadmissioncontrolisinvokedforeachclassperiodically—theinvocationismorefrequentforimportantclassesandlessfrequentforlessimportantclasses.Thisensuresthatimportantrequestsarealwaysgivenpriority(bybeingadmittedrst)overlessimportantrequests.Further,itreducesthechancesofadmittinglessimportantrequestsintothesystemandhavethemdenyservicetomoreimportantre-queststhatarriveshortlythereafter.Let¨k¨¨G1denotetheperiodofinvocationforthe.classes,¨©%¨%(¨ª1.Sincearequestmaywaitinabucketforupto¨5timeunitsbe-foreadmissioncontrolisinvokedforitsbatch,theabovetestismodiedas–Œ˜—Œš•œ2Ÿž@‚%«! S¡e‘$¬¨5(5)Intheeventthisconditionissatised,allrequestsinthebatchareadmittedintothesystem.Otherwiserequestsinthebatcharedropped.Techniquesforestimatingparameterssuchasthequeuingdelay,inherentservicetime,andthenumberofexist-ingconnectionsarediscussedinSection3.4.3.3ScalableThreshold­basedAdmissionCon­trolBatchprocessingeliminatestheneedtoinvoketheadmissioncontroltestforindividualrequestsandreducestheoverheadtoonetestperbatch.Whereasthisenhancesthescalabilityofthesentry,itmaystillimposesignicantprocessingdemandsdur-ingextremeoverloads.Inthissection,wepresenttechniquesforfurtherreducingtheadmissioncontroloverhead.Ourtech-niquetradesefciencyoftheadmissioncontrolforaccuracyandreducestheoverheadtoafewarithmeticoperationsperrequest.Thekeyideabehindthistechniqueistoperiodicallypre-computethenumberofrequeststhatshouldbeadmittedineachclassandthensimplyenforcetheselimitswithoutconductinganyadditionalper-requesttests.Thetechniqueestimatesthearrivalrateineachclassandtheaverageservicetimeofare-quest,andusesthesevaluestopre-computeathresholdforthenumberofrequeststhatcanbeadmittedineachclass.Asanexample,assumethatthesystemcapacityis100requests/s,andthatanapplicationhastworequestclasseswitharrivalratesof®­J¯requests/sand°¯J\rrequests/s.Inthiscase,thethresholdissetsothatallrequestsfromclass/andonlyhalftherequestsinclass/areadmitted.Iftheincom-ingrequestrateschangeto›IC¯“\rand±­J¯,thenanew5 thresholdiscomputedthatadmitstwooutofeverythreere-questsin/andturnsawayallrequestsin/.Observethat,suchastrategycanbeimplementedveryefciently.Oncearequestismappedontoaclass,theadmissioncontrolleronlyneedstodetermineiftherequestisabovethethresholdinordertoadmitit(thustheadditionaloverheadbeyondrequestclas-sicationisonlyafewarithmeticoperations).Adrawbackoftheapproachisthattheactualarrivalratesmaydifferfromtheestimates,resultinginerrors—ifthethresholdissettoolow,theSLAmaybeviolatedandifitistoohigh,morerequestsmaybeturnedawaythannecessary.Thethresholdisdenedtobeapair(class,fraction²†k³Š)andhasthefollowingmeaning:allrequestsinclasseslessim-portantthanshouldbedropped,requestsinclassshouldbeadmittedwithprobability²†k³Š,andallrequestsinclassesmoreimportantthanshouldbeadmitted.Wedeterminetheseparametersbasedonobservationsofarrivalratesandservicetimesforrequestsofvariousclassesoverperiodsofmoderatelength(weuseperiodsoflength´L\rsec).DenotingthearrivalratestoclassesIJ(k.by(µ1andtheobservedaver-ageservicetimesby3(03C1,thethreshold-pair(,²†k³Š)isobtainedsuchthat6’¶h5·6’¶ª6N@C36¸¹I(6)and²†’³Š@5@C35Œ6’¶Y5·6’¶6@z36ºI(7)Thus,admissioncontrolnowmerelyinvolvesapplyingthein-expensiveclassicationfunctiononanewrequesttodetermineitsclassandthenusingtheequallylightweightthresholdingfunctiontodecideifitshouldbeadmitted.Thethreshold-basedandthebatch-basedadmissioncontrolstrategiesneednotbemutuallyexclusive.Thesentrycanem-ploythemoreaccuratebatch-basedadmissioncontrolstrategysolongastheincomingrequestratepermitsoneadmissioncontroltestperbatch.Iftheincomingrateincreasessigni-cantly,theprocessingdemandsofthebatch-basedadmissioncontrolmaysaturatethesentry.Insuchanevent,whentheloadatthesentryexceedsathreshold,thesentrycantradeac-curacyforefciencybydynamicallyswitchingtoathreshold-basedadmissioncontrolstrategy.Thisensuresgreaterscal-abilityandrobustnessduringoverloads.Thesentryrevertstothebatch-basedadmissioncontrolwhentheloaddecreasesandstaysbelowthethresholdforasufcientlylongduration.3.4OnlineParameterEstimationThebatch-basedandthreshold-basedadmissioncontrolre-quireestimatesofanumberofsystemparameters.Thesepa-rametersareestimatedusingonlinemeasurements.Thenu-cleirunningonthecataclysmserversandsentriescollectivelygatherandmaintainvariousstatisticsforadmissioncontrol.Thefollowingstatisticsaremaintained:»Arrivalrate5:Thearrivalratesineachrequestclassaremeasuredatthesentry.Sinceeachrequestismappedontoaclassatthesentry,itistrivialtousethisinformationtomeasuretheincomingarrivalratesineachclass.»Queuingdelay–:Thequeuingdelayincurredbyare-questismeasuredatthecataclysmserver.ThequeuingdelayismeasuredasthedifferencebetweenthetimetherequestarrivesattheserverandthetimeitisacceptedbytheHTTPserverforservice(weassumethatthedelayincurredatthesentryisnegligible).Thenucleicanmea-surethesevaluesbyappropriatelyinstrumentingtheop-eratingsystemkernel.Thenucleiperiodicallyreporttheobservedqueuingdelaystothesentry,whichthencom-putesthemeandelaysacrossallserversintheapplica-tion'spool.»Numberofrequestsinservice—:Thisparameterismea-suredattheCataclysmserver.Byappropriatelyinstru-mentingtheOSkernel,thenucleicantrackthenumberofactiveconnectionsservicedbytheapplicationandperiod-icallyreportthemeasuredvaluestothesentry.Thesentrythencomputesthemeanofthereportedvaluesacrossallserversfortheapplication.»Requestservicetime3:Thisparameterisalsomeasuredattheserver.Theactualservicetimeofarequestismea-suredasthedifferencebetweenthearrivaltimeattheserverandthetimeatwhichthelastbyteoftheresponseissent.Themeasurementoftheinherentservicetimeismorecomplex.DoingsorequirescarefulinstrumentationoftheOSkernelandsomeinstrumentationoftheappli-cationitself.Thisinstrumentation—discussedfurtherinSection6—enablesthenucleustocomputetheCPUpro-cessingtimeforarequestaswellasthedurationforwhichtherequestedisblockedonI/O.Together,thesevaluesde-terminetheinherentservicetime(seeEquation3).»ConstantŽ:TheconstantŽinEquation3ismeasuredus-ingofinemeasurementsontheCataclysmservers.WeexecuteseveralrequestswithdifferentCPUdemandsanddifferent-sizedresponsesunderlightloadconditionsandmeasuretheirexecutiontimes.WealsocomputetheCPUdemandsandtheI/Otimesasindicatedabove.Thecon-stantŽisthenempiricallydeterminedasthevaluethatminimizesthedifferencebetweentheactualexecutiontimeandtheinherentservicetimeinEquation3.Thesentryusespaststatisticstoestimatetheinherentser-vicetimeofanincomingrequestinordertomapitontoabucket.Todoso,thesentrymaintainsahashtableformain-tainingtheusagestatisticsfortherequestsithasadmittedsofar.EachentryinthistableconsistsoftherequestedURL(whichisusedtocomputetheindexoftheentryinthetable)andavectoroftheresourceusagevaluesforthisrequestasre-portedbythevariousservers.Requestsforstaticcontentpos-sessthesameURLeverytimeandsoalwaysmaptothesame6 entryinthehashtable.TheURLforrequestsfordynamiccon-tent,ontheotherhand,maychange(e.g.theargumentstoascriptmaybespeciedaspartoftheURL).Forsuchrequests,wegetridoftheargumentsandhashbasedonthenameofthescriptinvoked.Theresourceusagevaluesforrequeststhatinvokethesescriptsmaychangedependingonthearguments.Wemaintainexponentiallydecayedaveragesoftheirusages.4ProvisioningforCataclysmsAlthoughtheadmissioncontrolmechanismsmaintaintheSLAofadmittedrequestsevenduringoverloads,asignicantfrac-tionoftherequestsmaybeturnedawayduringextremeover-loads.Insuchsituations,thenumberofrequeststhatareturnedawaycanbereducedbyincreasingthecapacityoftheapplica-tion.TheCataclysmcontrolplaneimplementsaprovisioningmechanismtodynamicallyvarythenumberofserversallo-catedtotheapplications.Theapplication'sserverpoolisin-creasedduringoverloadsbyallocatingserversfromthefreepoolorbyreassigningunder-usedserversfromotherapplica-tions.Thecontrolplanecanalsodynamicallyprovisionsentryserverswhentheincomingrequestratesimposessignicantprocessingdemandsontheexistingsentries.TherestofthissectiondiscussesourtechniquesfordynamicallyprovisioningCataclysmserversandsentries.4.1Queuing­theoreticModelforReplicableAp­plicationsThedynamicprovisioningproblemcanbeformulatedasacon-strainedoptimizationproblemwhosegoalistodetermineapartitioningoftheserversamongthehostedapplicationthatwillmaximizetheplatform'sexpectedrevenue.Thekeychal-lengeintheformulationofthisoptimizationproblemisthatofdeterminingtheutilityofassigningacertainnumberofserverstoanapplication.Utility-basedprovisioninghasbeenstudiedintheMusesystem[9],whereaneconomicapproachisusedforformulatingtheoptimizationproblem.Chandraetal.[8]modelaserverresourcethatservicesmultipleapplicationsasaGPSsystemanddeviseanoptimizationproblemtoderivere-sourceallocations.Ourapproachisalsoaninstanceofutility-basedprovisioningwhereweuseaqueuing-theoreticmodelforanapplication.Weuseawellknownqueuingtheoryresulttodeviseamodelfordeterminingtheutilityofaserversetofagivensizeforareplicableapplicationunderacertainworkload[20].Ourmodeldoesnotmakeanyassumptionsaboutthenatureoftherequestarrivalprocessesortheservicetimes.OurabstractionofasinglereplicaofaserviceisaG/G/1queuingsystem.ThefollowingupperboundisknownontheaverageresponsetimeforaG/G/1queuingsystem:¼!K%«¼‚KŒ=@¾½‘Œ½¿MÀ@ª¬ÂÁ(8)Here¼!Kistheaverageresponsetime,¼‚Kistheaverageservicetime,istherequestarrivalrate,ÁÃ@¼‚Kistheutilization,and½‘and½¿arethevarianceofinter-arrivaltimeandthevarianceofservicetimerespectively.Rewritingin-equality(8),wehavethefollowing:ĸ°Å¼‚KŒ½‘Œ½¿MÀ@G¼!K¬¼‚KÆ(9)Theaboveinequalitygivesalowerboundontherequestar-rivalrateforwhichoneapplicationreplicawillbeabletopro-videanaverageresponsetimeof¼!K.Wedenotethisby"5~ƒhenceforth.Wedenetheutilityofœserversforanapplica-tionÇ,È=œastheexpectedrevenuethatœserverswouldgeneratefor3.RecallfromtheSLAdenedinSection2thattodeterminethis,weneedtopredicttheoverallarrivalratetotheserviceduringthenextprovisioningcycle(simplycy-clehenceforth).HavingpredictedthisrateŠ†k…,theresponsetimetarget¼!KisreadfromtheentryintheQoStableoftheSLAcorrespondingtoanarrivalrateofŠ†’….WeusetheRev-enueTableoftheSLAtodeterminetherevenueperadmittedrequestÇÉ;forthisarrivalrate.ThuswehaveÈ=œœ@C5pƒ&#x?ǵ;@(À@#Š†’³kÊ(10)#µŠ†’³kÊdenotesthelengthofacycle.Also,becauseofthelowerboundËLÌ:5~ƒonthenumberofserversassignedto3,weonlyneedtocomputeÈ©œforœinthefollowingrangeËJÌ25~ƒ%œ%'Í©Î^ϊÑÐÓÒW²Ô¸Š†’…µ5pƒÖÕ(11)HereÒisthetotalnumberofcataclysmserversinthecluster.Theseutilitycurvesarethenusedtodeterminethepartitioningofthecataclysmserversamongapplicationsthatwouldmaxi-mizetheexpectedrevenueoverthenextcycle.ThisisachievedbysolvingthefollowingconstrainedoptimizationproblemMaximize:בÈ=œ‘Subjectto:בœ‘Ø%ÒÒdenotesthenumberofhost-serversintheplatform.Figure4outlinesthestepsinvolvedindeterminingthisoptimalpar-titioning.Thenalstepinthisprocedureissolvingtheaboveoptimizationproblem.Forthereplicableapplicationsthatweconsiderinourwork(seeSection2.2),theutilitycurvesarelinear.Asimplegreedyheuristiccanbeusedtosolvethisprob-lemoptimally.Thisheuristicbeginsbyassigningtoeachappli-cationanumberofserversequaltothelowerboundguaranteedinitsSLA.Itthenproceedsinsteps,eachresultinginassign-ingoneadditionalservertosomeapplication.Ineachstep,theheuristicdeterminestheapplicationthatwouldexperiencethemostgaininrevenueduetoanadditionalserverandthathasnotyetreacheditsupperboundonnumberofservers.Thisprocesscontinuestill(1)alltheservershavebeenassignedor(2)allapplicationshavereachedtheirupperboundsor(3)noapplicationwouldexperienceapositivegaininrevenueduetoanadditionalserver.7 ...Service 1Service nQueuingÙ?Ú^ÛeÜÝÞWß~àUáâãSäåæJçæ“çEèéJêé“êEëìJíì“íEî(1) Determine using queuingïSðñ(2) Derive utility curves using the Optimizationserver allocationsutility curvesSLASLAòóôõöS÷øùSúûQueuingFigure4:Dynamicprovisioninginthecontrolplane.4.2SentryProvisioningIngeneral,allocationanddeallocationofsentriesissigni-cantlylessfrequentthanthatofCataclysmservers.Further,thenumberofsentriesneededbyanapplicationismuchsmallerthanthenumberofserversrunningit.Consequently,asimpleprovisioningschemesufcesfordynamicallyvaryingthenum-berofsentriesassignedtoanapplication.OurschemeusestheCPUutilizationoftheexistingsentryserversasthebasisforallocatingadditionalsentries(ordeallocatingactivesentries).Iftheutilizationofasentrystaysinexcessofapre-denedthresholdüFýü‰EŠ‹foracertainperiodoftime,itrequeststhecontrolplaneforanadditionalsentryserver.Uponreceivingsuchrequestsfromoneormoresentriesofanapplication,thecontrolplaneassignseachanadditionalsentry.Similarly,iftheutilizationofasentrystaysbelowathresholdBEDCþ‰EŠ‹,itisre-turnedtothefreepoolwhileensuringthattheapplicationhasatleastonesentryremaining.Wheneverthecontrolplaneassigns(orremoves)asentryservertoanapplication,itrepartitionstheapplication'sserverspoolequallyamongthevarioussentries.TheDNSentryfortheapplicationisalsoupdateduponeachallocationordeallocation;around-robinDNSschemeisusedtolooselypartitionincomingrequestsamongsentries.Sinceeachsentrymanagesamutuallyexclusivepoolofservers,itcanindependentlyperformadmissioncontrolandloadbalanc-ingonarrivingrequests;theSLAiscollectivelymaintainedbyvirtueofmaintainingitateachsentry.5IntegratingPolicingandProvisioningAnovelfeatureofourhostingplatformisthecooperationbe-tweentheadmissioncontrollerandtheprovisioningmecha-nisminthecontrolplane.Thetwomechanismsoperateatdif-ferenttimescales—whiletheadmissioncontrolhandlesover-loadsatrequestarrivaltimescales(milliseconds),theprovi-sioningmechanismhandlesoverloadsatthetimescaleofmin-utes.Nevertheless,cooperationbetweenthesemechanismsen-hancesthecapabilityoftheplatforminhandlingoverloads.Thedynamicprovisioningmechanismnormallyoperatesinapredictivefashion.Itisinvokedperiodically(onceevery30minutesinourprototype)andusestheworkloadobservationsintheprevioustimeperiodtoreallocateserverstoapplicationsifnecessary.Sinceoverloadsareoftenunanticipated,asentryofanoverloadedapplicationcandynamicallyinvokethepro-visioningmechanismwhenevertherequestdroprateexceedsacertainpre-denedvalue.Insuchascenario,theprovisioningmechanismoperatesinareactivemodetocountertheover-load.Themechanismusesrecentworkloadmeasurementsfortheoverloadedapplicationtorecomputeitsutilitycurve(un-likethepredictivemodewhereallutilitycurvesarerecom-puted,onlythecurvefortheoverloadedapplicationisrecom-putedinthereactivemode).Theprovisioningmechanismthenallocatesadditionalserverstotheoverloadedapplication.Un-desirableoscillationsinsuchallocationsarepreventedusingtwoconstraints:(i)alimitofÿisimposedonthenumberofserversthatcanbeallocatedtoanapplicationinasinglestepinthereactivemodeand(ii)adelayoftimeunitsisimposedonthedurationbetweentwosuccessiveinvocationsoftheprovi-sioningmechanisminthereactivemode(issetto5minutesinourprototype).Observethat,whileprovisioningofserverstoapplicationscanbebothpredictiveandreactive,provisioningofsentriesisalwayspurelyreactive—allocationofsentriesisalessfrequenteventthantheallocationofserversanditsufcestodosoinapurelyreactivefashion.Reactiveprovisioninginvolvescommunicationfromthesentrytothecontrolplane.Communicationfromthecon-trolplanetothesentryisalsopresentinourhostingplatform.Clearly,aftereachprovisioningstep,thecontrolplanemustprovidethenewallocationstoeachsentry.Inaddition,thepro-visioningmechanismconveystheresponsetimetargetstothesentriesaftereachinvocation.RecallthattheQoStableintheSLApermitsdegradedresponsetimetargetsforhigherarrivalrates.TheprovisioningmechanismmaydegradetheresponsetimetotheextentpermittedbytheSLA,addmorecapacity,orabitofboth.Theoptimizationdrivesthesedecisions,andtheresultingtargetresponsetimesareconveyedtotheadmissioncontrollers.Thus,theseinteractionsenableintegrationofad-missioncontrol,provisioningandadaptiveperformancedegra-dationinthehostingplatform.WeexperimentallydemonstratevariousaspectsofthisintegrationinSection7.6ImplementationConsiderationsWeimplementedaprototypeCataclysmhostingplatformonaclusterof20Pentiummachinesconnectedviaa1Gbpseth-ernetswitchandrunningLinux2.4.20.Eachmachineintheclusterwasusedforrunningoneofthefollowingentities:(1)anapplicationreplica,(2)acataclysmsentry,(3)thecataclysmprovisioning,(4)aworkloadgeneratorforanapplication.Inthissectionwediscussourimplementationofthecataclysm8 sentryandprovisioning.6.1CataclysmSentryWeusedKernelTCPVirtualServer(ktcpvs)version0.0.14[22]toimplementthepolicingmechanismsdescribedinSec-tion3.ktcpvsisanopen-source,Layer-7loadbalancerim-plementedasaLinuxmodule.Itlistensforclientrequestsatawell-knownportanddistributesarrivingrequestsamongbackendservers.ItacceptsTCPconnectionsfromclients,opensseparateconnectionswithservers(oneforeachconnec-tionfromaclient)andtransparentlyrelaysdatabetweenthese.WemodiedktcpvstoimplementallthesentrymechanismsdescribedinSections3-5.ktcpvshasamulti-threadeddesign,withanumberofker-nelthreadswaitingfornewconnectionstoarrive.Uponthear-rivalofaconnection,oneofthethreadsacceptsit,opensasep-arateTCPconnectionwithoneoftheserversandcopiesdatabackandforthbetweentheseconnections.Whenthisconnec-tioncloses,thethreadgoesbacktowaitingfornewarrivals.Wemodifyktcpvssoitconsistsofthreeclassesofkernelthreads—(1)oneaccepterthread,(2)onepolicerthreadperclasssupportedbytheservice,and(3)alargenumberofre-layerthreads.Theaccepterthreadacceptsanewlyarrivedcon-nectionandcreatesaconnectionobjectcontaininginformationthatwouldbeneededfortransferringdataoverthisconnec-tionifitgetsadmitted.Itdeterminestheclassthatthisrequestbelongsto.2Thisobjectisthenmovedtothewaitqueuecorrespondingtotherequest'sclass.AwaitqueueisaFIFOqueueusedasanintermediatestorageforacceptedrequestsofaclassbeforetheadmissioncontrolactsonthem.Thepolicerthreadcorrespondingtoacertainclasswakesuponceeveryprocessingintervalforthatclass(Section3,seeFigure3)andinvokestheadmissioncontroltestonthecurrentbatchofre-questsinthatclass'swaitqueue.Theadmittedconnectionsaremovedtotheadmitqueue,asingleFIFOqueueforadmit-tedconnections.Theconnectionsdroppedbytheadmissioncontrolareclosed.Therelayerthreadswaitforconnectionsarrivingintotheadmitqueue.Afterpickingaconnectionfromtheadmitqueue,arelayerthreadopensanewconnectionwiththeleastloadedserver(chosenasdescribedinSection3)andhandlestherelayingofdatabetweentheseconnections.Itre-turnstowaitingforthearrivalofnewadmittedconnectionsintotheadmitqueueonceitisdoneservicingthisconnection.Inourexperiments,wehaduptothreeclasses,whichmeantuptothreepolicerthreadsandthreewaitqueues.Thenumberofrelayerthreadscouldchangedynamicallydependingontherequestarrivalrate.Weimposedlowerandupperboundsof
J\randMJ¯L\rrespectivelyonthenumberofrelayerthreadsinourexperiments.2Inourcurrentimplementation,theclassofarequestisdecidedbasedontheURLrequested.Thismappingisreadbythesentryfromacongurationleatstartuptime.Ingeneralarequest'sclassmaybedeterminedbasedonothercriterionsuchasclientIPaddress.6.2CataclysmProvisioningCataclysmprovisioningwasimplementedasauser-spacedae-monrunningonadedicatedmachine.Atstartup,itreadsin-formationneededtocommunicatewiththesentriesandinfor-mationabouttheserversintheclusterfromacongurationle.ItcommunicateswiththesentriesoverTCPsockets.Thesentriesgatherandreportvariousstatisticsneededbythepro-visioningalgorithmoverthesesockets.Theprovisioningal-gorithmcanbeinvokedinoneoftwowaysdescribedinSec-tions4and5.Thedefaultktcpvsimplementationprovidesaexibleuser-spaceutilitytcpvsadmusingwhichthesetofbackendserversthatitforwardstheincomingrequeststocanbechangeddynamically.Wemakeuseofthisutilityinourimplementation.Afterdeterminingapartitioningofthecluster'sserversamongthehostedapplications,theprovision-ingdaemonremotelylogsontothenodesrunningthesentriesandusestcpvsadmwiththeappropriateargumentstoaffecttheapplications'serversetsinthispartitioning.Inourexperi-ments,wechosethelengthofthedefaultprovisioningdurationtobe´J\rmin.7EvaluationInthissectionwepresenttheexperimentalsetupfollowedbytheresultsofourexperimentalevaluation.7.1ExperimentalSetupOurprototypewasbuiltonaclusterof20Pentiummachinesconnectedbya1GbpsEthernetswitch.Thecataclysmsen-trieswererunondual-processor1GHzmachineswith1GBRAM.Cataclysmprovisioningwasrunonadual-processor450MHzmachinewith1GBRAM.Themachinesusedascat-aclysmservershad2.8GHzprocessorsand512MBRAM.Fi-nally,theworkloadgeneratorswererunonmachineswithpro-cessorspeedsvaryingfrom450MHzto1GHzandwithRAMsizesintherange128MB-512MB.AllthemachinesranLinux2.4.20.InourexperimentsweconstructedreplicableapplicationsusingtheApache1.3.28webserverwithPHPsupporten-abled.Thelesetservicedbythesewebserverscomprisedlesofsizevaryingfrom1kBto256kBtorepresenttherangefromsmalltextlestolargeimageles.Inaddition,thesewebservershostedPHPscriptsworthdifferentamountsofCPUcomputation.Thedynamiccomponentoftheworkloadoftheseapplicationsconsistedofrequestsforthesescripts.TheMaxClientlimitforApachewasincreasedfromthedefaultI¯“\rto¯“\rJ\r.RecallfromSection3thatknowingarequest'sCPUre-quirementwascrucialforthedeterminationofitsinherentsize.WemakeminormodicationstoApacheandtheLinuxCPUschedulertoenablethemeasurementofper-requestCPUus-age.Inalltheexperiments,theSLApresentedinFigure2wasusedfortheapplications.RecallfromSection3thatknowingarequest'sCPUre-quirementwascrucialforthedeterminationofitsinherent9 0501001502000100200300400500Arrival rate (req/sec)Time (sec)Arrival rateGOLDSILVERBRONZE(a)Arrivalrates0501001502000100200300400500Admission rate (req/sec)Time (sec)Admission rateGOLDSILVERBRONZE(b)Admissionrates00.20.40.60.810100200300400500Fraction admittedTime (sec)Fraction admittedGOLDSILVERBRONZE(c)Fractionadmitted0500100015002000250030003500400045005000010020030040050095th % resp. time (msec)Time (sec)95th percentile response timeGOLDSILVERBRONZE(d)95thresp.timeFigure5:Demonstrationofclass-baseddifferentiationachievedbytheadmissioncontrol.size.Sincethenotionofarequestisspecictotheser-vice(theApachewebserverinourcase)andtheoperat-ingsystemisobliviousofit,wemadeaminormodica-tiontoApachetoenableper-requestCPUusagemeasurement.Thisconsistedofinsertinginvocationsoftwonewsystemcallsbeginrecordcpu()andendrecordcpu()intheApachesourcecodebeforerequestprocessingstartsandafteritendsrespectively.TheLinuxCPUsched-ulercodewasmodiedsothattheCPUtimeusedbyaprocess(andallitsdescendants)isrecordedbetweenwhenitcallsbeginrecordcpu()tillwhenitcallsendrecordcpu().Theworkloadgeneratorsfortheseapplicationswerebasedonhttperf[25],anopen-sourcewebworkloadgenerator.httperfoffersavarietyofworkloadgenerators.Whilerun-ning,itkeepstrackofanumberofperformancemetricsthataresummarizedintheformofstatisticsthatarereportedattheendoftherun.Weenhancehttperftorecordaspeci-edpercentileoftheresponsetimeexperiencedbytherequestsduringarun.7.2Class­basedDifferentiationOurrstexperimentinvestigatestheefcacyofthemecha-nismsemployedbytheCataclysmsentrytoprovideclass-baseddifferentiationtorequests.TheCataclysmprovision-ingwaskeptturnedoffinthisexperiment.WeconstructedareplicatedwebserverconsistingofthreeApacheservers.Thisapplicationsupportedthreeclassesofrequests—Gold,SilverandBronzeindecreasingorderofsignicance.TheclassofarequestcouldbeuniquelydeterminedfromitsURL.Theinvo-cationperiodoftheadmissioncontrolwassetto\r,¯J\randI(\rJ\rmsecforthethreeclassesrespectively.TheworkloadconsistedofrequestsformembersofasetofPHPscripts.ThecapacityofeachApacheserverforthisworkload(i.e.,therequestarrivalrateforwhichtheG¯‡?„per-centileresponsetimeoftherequestswasbelowtheresponsetimetarget)wasdeterminedofineandwasfoundtobenearlyL\rrequests/sec.Figure5(a)showstheworkloadusedinthisexperiment.Requestarrivalratesforthethreeclassesstartedatlowvaluesbutgraduallypickeduptocreateanoverload,rstincreasingataslowandsteadyrateandthensuddenlyattain-ingahighvalue.ThedurationsduringwhichthepeakloadsforthethreeclassespersistedwerechosentohavethenestedstructureasseeninFigure5(a)—arrivalrateofBronzerequestswasthersttopeakandthelasttodropdown,Silverrequestspeakednextetc—toclearlybringouttheclass-baseddifferen-tiationachievedbythesentry.Figures5(b)and5(d)showtheratesatwhichtheserequestswereadmittedandtheG¯‡?„per-centileresponsetimeofadmittedrequestsrespectivelyduringtheexperiment.Nearlyalltherequestsarrivingtillt=I(´L\rsecwereadmittedbythesentry.Betweent=I(´J\rsecandt=IL¯sec,theBronzerequestsweredroppedalmostexclusively.Att=IL¯secthearrivalrateofSilverrequestsshotupandreachednearlyIMJ\rrequests/sec.TheadmissionrateofBronzerequestsdroppedtoalmostzerotoadmitasmanySilverrequestsaspos-sible.Att=MI(\rsec,thearrivalrateofGoldrequestsshotuptoM“\rL\rrequests/sec.ThesentrytotallysuppressedallarrivingBronzeandSilverrequestsnowandletinonlyGoldrequestsaslongastheincreasedarrivalrateofGoldrequestspersisted.Figure5(c)isanalternaterepresentationofthesystembehav-iorinthisexperimentanddepictsthevariationofthefractionofrequestsofthethreeclassesthatwereadmitted.Figure5(d)depictstheperformanceofadmittedrequests.Wendthatthesentrywasverysuccessfulinmaintainingtheresponsetimebe-lowI(\rL\rJ\rmsec.FromtheadmissionratesshowninFigure5(b)wendthattheadmissioncontrolwassomewhatconservativeandadmittedrequestsatslightlylowerratethanthatcapacityrevealedbyofinemeasurementofsystemcapacity.ObservethatthefewinstanceswhentheL¯‡?„percentileresponsetimeforsomeclasswasworsethanI\rJ\rL\rmseccorrespondtowhenveryfewrequestsofthatclasswereadmittedimplyingtherewasnotenoughdatafortheseobservationstobeconsideredstatisticallysignicant.7.3CataclysmProvisioningWeconductedanexperimentwithtwowebapplicationshostedonourCataclysmplatform.Thecontrolplaneranonadedi-catednode.Eachapplicationhaditssentryrunningonasep-aratenode.ThetotalnumberofcataclysmserversavailableinthisexperimentwasIJI.TheSLAsforboththeapplicationswereidenticalandaredescribedinFigure2.Further,theSLAsimposedalowerboundof´onthenumberofserversthateachapplicationcouldbeassigned.Theworkloadgeneratorsfortheapplicationsranonfournodesinthecluster.Thedefaultprovi-10 0 500 1000 1500 2000 0 1000 2000 3000 4000 5000Rate (req/sec)Time (sec)Application 1: Arrival and admission rates[R,N=4][R,N=5][D,N=8][R,N=7][D,N=5]Arrival rateAdmission rate(a)Arrivalandadmissionrates05001000150020002500300001000200030004000500095th % resp. time (msec)Time (sec)Application 1: 95th resp time95th resp. timeresp. time target(b)95thresp.time 0 500 1000 1500 2000 0 1000 2000 3000 4000 5000Rate (req/sec)Time (sec)Application 2: Arrival and admission rates[T,N=4][D,N=6]Arrival rateAdmission rate(c)Arrivalandadmissionrates05001000150020002500300001000200030004000500095th % resp. time (msec)Time (sec)Application 2: 95th resp time95th resp. timeresp. time target(d)95thresp.timeFigure6:Dynamicprovisioningandadmissioncontrol:Per-formanceofApplications1and2.D:Defaultinvocationofprovisioning,T:Provisioningtriggeredbyexcessivedrops,[N=n]:sizeoftheserversetisnnow.Onlyselectedprovi-sioningeventsareshown.sioningdurationusedbythecontrolplanewas´J\rmin.Inthisexperiment,whenmigratingaserverfromoneapplicationtoanother,wedidnothavethecontrolplaneactuallyreboottheserverandinstallthedataforthenewapplication.Weinstalledallthecontentservedbythetwoapplicationsonallthecata-clysmserversinthecluster.Therefore,migrationwasachievedsimplybyhavingthecontrolplanesendmessagestothesen-triesforthedonorandthereceiverapplicationsoftheserverbeingmoved.Asaresult,servermigrationtimewasnegligi-ble.Theworkloadsforthetwoapplicationsconsistedofre-questsforanassortmentofPHPscriptsandlesinthesizerangeIkB-ICM
kB.Requestsweresentatasustainablebaseratetothetwoapplicationsthroughouttheexperiment.Overloadswerecreatedbysendingincreasednumberofrequestsforasmallsubsetofthescriptsandstaticles(tosimulateasubsetofthecontentbecomingpopular).Theexperimentbeganwiththetwoapplicationsrunningon´serverseach,correspondingtothelowerboundintheirSLAs.Thefreepoolcontained¯serversatthistime.Avalueof¯J\r%wasusedforthethresholdz†k³Šbythesentries.Figures6(a)and6(c)depictthearrivalratestothetwoapplications.TherequestarrivalrateforAp-plication2waskeptatasustainableI(\rL\rrequests/sec.Thear-rivalrateforApplication1wasmadetoincreaseinastep-likefashionstartingfromI(\rJ\rrequests/sec,doublingroughlyonceevery¯mintillitreachedapeakvalueofIJ\rL\rrequests/sec.AtthispointApplication1washeavilyoverloadedwiththearrivalrateseveraltimeshigherthansystemcapacity(whichwasroughlyL\rrequest/secperserverassignedtotheserviceasdeterminedbyofinemeasurements).AlsoshowninFig-ure6(a)aretheworkingofthesentryforApplication1andtheprovisioning.Att=ýI\rsecthesentry,havingobservedmorethan¯J\r%oftherequestsbeingdroppedoverthelast¯min,triggeredtheprovisioningalgorithmasdescribedinSection5.TheprovisioningalgorithmrespondedbypullingoneserverfromthefreepoolandaddingittoApplication1.Att=IMI\rsec,thecontinuouslyincreasingrateofarrivalstoApplication1resultedintheadditionofanotherserverfromthefreepool.ObserveinFigure6(a)theincreasesintheadmissionratescor-respondingtotheseadditionalserversbeingmadeavailabletoApplication1.Thenextinterestingeventwasthedefaultinvo-cationofprovisioningatt=I
J\rJ\rsec.Theprovisioningalgo-rithmaddedallthe´serversremaininginthefreepooltotheheavilyoverloadedApplication1.Also,basedonrecentobser-vationofarrivalrates,itpredictedanarrivalrateintherangeI\rJ\rJ\r-I\rJ\rL\rJ\rrequests/secanddegradedtheresponsetimetar-getforApplication1toMJ\rJ\rJ\rmsecbasedonitsQoStable(seeFigure2).Inthesecondhalfoftheexperiment,theoverloadofAp-plication1subsidedandApplication2gotoverloaded.Fig-ure6(c)depictsthearrivalandadmissionratesforApplication2.Tillt=ML­I(\rsec,Application2waswell-provisionedforitsworkloadanditssentryadmittedalmostalltherequests.Afterthis,theservicewasexposedtoincreasinglyhigharrivalratespeakingatapproximatelyIMJ\rJ\rrequests/secthatsustainedfortheremainderoftheexperiment.Thefunctioningoftheprovi-sioningwasqualitativelysimilartowhenService1wasover-loaded.ThesentryofApplication2triggeredtheprovisioningatt=´ÉIJ\rsecresultinginoneserverbeingmovedfromAppli-cation1.Att=´L\rJ\rsec,theseconddefaultinvocationofprovi-sioningoccurred.ThisresultedinMmoreserversbeingmovedfromApplication1toApplication2.Theresponsetimetar-getforApplication1wasreducedbacktoI\rJ\rJ\rmsecbecauseitsarrivalratewaspredictedtobelessthanI\rJ\rL\rrequests/sec,whilethatforApplication2wasdegradedtoMJ\rJ\rL\rmsec.Figures6(b)and6(d)showtheL¯‡?„percentileresponsetimesforthetwoservicesduringtheexperiment.Therearetwokeyobservations.First,thecontrolplanewasabletopre-dictchangestoarrivalratesanddegradetheresponsetimetar-getaccordingtotheSLAresultinginanincreasednumberofrequestsbeingadmitted.Secondly,thesentrieswereabletokeeptheadmissionrateswellbelowsystemcapacitytoachieveresponsetimeswithintheappropriatetargetwithonlysporadicviolations(whichwereonfewerthan%oftheoccasions).7.4ScalingtoHandleExtremeOverloadsWeconductedanexperimenttoinvestigatetheimprovementinthescalabilityofthesentryduetothethreshold-basedad-missioncontrol.WemeasuredtheCPUutilizationatthesen-tryserverfordifferentrequestarrivalratesforboththebatch-basedandthethreshold-basedadmissioncontrol.Figure711 02040608010005000100001500020000CPU usage (%)Arrival rate (req/sec)Scalability of the Admission ControlBatch-basedThreshold-basedFigure7:Scalabilityoftheadmissioncontrol.050100150200050100150200250300350400450500Admission rate (req/sec)Time (sec)Admission rateGOLDSILVERBRONZE05001000150020002500300035004000450005010015020025030035040045050095th % resp. time (msec)Time (sec)95th percentile response timeGOLDSILVERBRONZE(a)Admissionrates(b)95thpercentileresponsetimeFigure8:Performanceofthethreshold-basedadmissioncon-trol.Att=I(´G¯sec,thethresholdwassettorejectallBronzere-quests;att=I
J\rsec,itwasupdatedtorejectallBronzeandSil-verrequests;att=MýI(\rsecitwasupdatedtoalsorejectGoldre-questswithaprobability\r¯;nally,att=´ J\rsec,itwasagainsettorejectonlyBronzerequests.showsourobservationsofCPUutilizationwithG¯%con-denceintervals.Sincewewereinterestedonlyintheoverheadsoftheadmissioncontrolandnotinthedatacopyingoverheadsinherentinthedesignofthektcpvsswitch,weforcedthesentrytodropallrequestsafterconductingtheadmissioncontroltest.WeincreasedtherequestarrivalratestilltheCPUatthesentryserverbecamesaturated(nearlyJ\r%utilization).Weobservemorethanafour-foldimprovementinthesentry'sscalability—whereasthesentryCPUsaturatedatL\rL\rJ\rrequests/secwiththebatch-basedadmissioncontrol,itwasabletohandleal-mostIJ\rL\rJ\rrequests/secwiththethreshold-basedadmissioncontrol.Asecondexperimentwasconductedtoinvestigatethedegradationinthedecisionmakingduetothethreshold-basedadmissioncontroller.WerepeatedtheexperimentreportedinSection7.2(Figure5)butforcedthesentrytoemploythethreshold-basedadmissioncontroller.ThethresholdsusedbytheadmissioncontrolwerecomputedonceeveryIC¯sec.Fig-ure8(a)showschangesintheadmissionratesforrequestsofthethreeclasses.Att=I´L¯sec,afterobservingthehighar- 0 5000 10000 15000 20000 25000 30000 35000 40000 45000 50000 0 100 200 300 400 500 600Arrival rate (req/sec)Time (sec)Arrival rates[S=1][S=2][S=3]Total arrival rateArrival rate to sentry 1 0 20 40 60 80 100 0 100 200 300 400 500 600CPU utilization (%)Time (sec)Sentry 1: CPU utilization[S=1][S=2][S=3][S=1][S=2][S=3]CPU utilization(a)Arrivalrates(b)CPUutilization:Sentry1.Figure9:Dynamicprovisioningofsentries.[S=n]meansthenumberofsentriesisnnow.rivalrateofBronzerequests,thethresholdwassettorejectallBronzerequests.Followingthis,att=I
L\rsec,whenre-queststoallthreeclasseswereobservedtoarriveathighrates,thethresholdwasrecomputedtorejectallBronzeandSilverrequests.Att=MI\rsecitwasmadetighter—allBronzeandSilverrequestsweredroppedwhileGoldrequestswereadmit-tedwithaprobabilityof\r¯.Att=´L\rsec,aftertheoverloadduetoGoldandSilverrequestshadsubsided,thethresholdchangedbacktorejectonlyBronzerequests.Theimpactoftheinaccuraciesinherentinthethreshold-basedadmissioncon-trollerresultedindegradedperformanceduringperiodswhenthethresholdchosenwasincorrect.Weobservetwosuchpe-riods(ICM“\r-I(´G¯secandIJ\r-MýI(\rsec)duringwhichtheG¯‡?„percentileoftheresponsetimedeterioratedcomparedtothetargetofI(\rL\rJ\rmsec.Theresponsetimesduringtherestoftheexperimentwerekeptundercontrolduetothethresholdget-tingupdatedtoastrictenoughvalue.Finally,weconductedanexperimenttodemonstratetheabilityofthesystemtodynamicallyprovisionadditionalsen-triestoaheavilyoverloadedservice.Figure9showstheout-comeofourexperiment.TheworkloadconsistedofrequestsforsmallstaticlessenttothesentrystartingatG\rJ\rL\rre-quests/secandincreasingbyL\rL\rJ\rrequests/seceveryminuteandisshowninFigure9(a).IftheCPUutilizationofthesen-tryserverremainedabove
J\r%formorethan´J\rsec,arequestwasissuedtothecontrolplaneforanadditionalsentry.Fig-ure9(b)showsthevariationoftheCPUutilizationattherstsentry.Att=MI\rsec,asecondsentrywasaddedtotheservice.SubsequentrequestsweredistributedequallybetweenthetwosentriescausingthearrivalrateandtheCPUutilizationattherstsentrytodrop.Athirdsentrywasaddedatt=ªM“\rsec,whenthetotalarrivalratetotheservicehadreached´GM“\rJ\rL\rrequests/secoverwhelmingboththeexistingsentries.12 02040608010012014016018020005001000150020002500Arrival rate (req/sec)Time (sec)Arrival ratetextimagedynamic(a)Arrivalrates02040608010012014016018005001000150020002500Admission rate (req/sec)Time (sec)Admission ratetextimagedynamic(b)Admissionrates01002003004005006007008000500100015002000250095th % resp. time (msec)Time (sec)95th percentile response timetextimagedynamic(c)95th,admissioncontrol010002000300040005000600070008000900010000050010001500200025003000350095th % resp. time (msec)Time (sec)95th percentile response timetextimagedynamic(d)95th,noadmissioncontrolFigure10:Revenuemaximizationviasize-awareadmissioncontrol.RevenueMaximizationviaSize­awareAd­missionControlWeconductedanexperimentwithanapplicationconstructedasintherstexperimenttodemonstratetheimpactofthesize-awareworkingoftheadmissioncontrolonsystemper-formance.Theworkloadconsistedoftextles(IkB-
kB),im-ageles(kB-ICM
kB)anddynamicrequestsforacollectionofPHPscripts.Figures10(a),(b)depicttheworkloadusedintheexperimentandtherateatwhichrequestsofthethreekindswereadmittedbythesentry.Therearethreeregionsofoverloads—rstbetweent=I\rJ\rsecandt=IMJ¯J\rsecduetotheincreasedarrivalratesofbothstaticanddynamic,secondbe-tweent=ICMJ¯J\rsecandt=IJ\rJ\rsecduetohighincidenceofthedynamicrequestsandnallyaftert=IL\rJ\rsecduetoarrivalofexcessivestaticrequests.Becausetheadmissioncontroluti-lizesinformationabouttheresourcerequirementsofthein-comingrequestsgatheredusingtheprolingdescribedinSec-tion3.4,itisabletoselectivelyadmitalmostalltherequestsforthelightweighttextles.Requestsforimagelesthatarethemostresourcedemandinginthisworkloadarepunishedmostseverely.Thisdemonstratestheabilityoftheadmissioncontroltomaximizethenumberofadmittedrequestsinthefaceofdiverseworkloads.AcomparisonofFigures10(c)and(d)illustratestheefcacyoftheadmissioncontrolinmaintain-ingresponsetimetargets.Withouttheadmissioncontrol,theresponsetimesareobservedtodegradeuncontrollablyunderoverloadconditionsasseeninFigure10(d).8RelatedWorkPreviousliteratureonissuesrelatedtooverloadmanagementinplatformshostingInternetservicesspansseveralareas.Inthissectionwedescribetheimportantpiecesofworkonthesetopics.AdmissionControlforInternetServices:Manypapershavedevelopedoverloadmanagementsolutionsbasedondo-ingadmissioncontrol.Severaladmissioncontrollersoperatebycontrollingtherateofadmissionbutwithoutdistinguish-ingrequestsbasedontheirsizes.Voigtetal.[33]presentkernel-basedadmissioncontrolmechanismstoprotectwebserversagainstoverloads—SYNpolicingcontrolstherateandburstatwhichnewconnectionsareaccepted,prioritizedlistenqueuereordersthelistenqueuebasedonpre-denedconnec-tionpriorities,HTTPheader-basedcontrolenablesratepolic-ingbasedonURLnames.WelshandCuller[35]proposeanoverloadmanagementsolutionforInternetservicesbuiltus-ingtheSEDAarchitecture.Asalientfeatureoftheirsolutionisfeedback-basedadmissioncontrollersembeddedintoindi-vidualstagesoftheservice.TheadmissioncontrollersworkbygraduallyincreasingadmissionratewhenperformanceissatisfactoryanddecreasingitmultiplicativelyuponobservingQoSviolations.TheQGuardsystem[17]proposesanadap-tivemechanismthatexploitsratecontrolsforinboundtofendoffoverloadandprovideQoSdifferentiationbetweentrafcclasses.Thedeterminationoftheseratelimits,however,isnotdynamicbutisdelegatedtotheadministrator.Iyeretal.[16]proposeasystembasedontwomechanisms—usingthresholdsontheconnectionqueuelengthtodecidewhentostartdroppingnewconnectionrequestsandsendingfeedbacktotheproxyduringoverloadswhichwouldcauseittore-strictthetrafcbeingforwardedtotheserver.However,theydonotaddresshowthesethresholdsmaybedeterminedon-line.CherkasovaandPhaal[11]proposeanadmissioncon-trolschemethatworksatthegranularityofsessionsratherthanindividualrequestsandevaluateitusingasimplesimu-lationstudy.Thiswasbasedonasimplemodeltocharacterizesessions.Theadmissioncontrollerwasbasedonrejectingallsessionsforasmalldurationiftheserverutilizationexceededapre-speciedthresholdandhassomesimilaritytoourap-proximateadmissioncontrol,exceptweuseinformationaboutthesizesofrequestsinvariousclassestodeterminethedropthreshold.Severaleffortshaveproposedsolutionsbasedonanalyti-calcharacterizationoftheworkloadsofInternetservicesandmodelingoftheservers.In[19],KanodiaandKnightlyuti-lizeamodelingtechniquecalledserviceenvelopstodeviseanadmissioncontrolforwebservicesthatattemptstodifferentresponsetimetargetsformultipleclassesofrequests.LiandJamin[23]presentameasurement-basedadmissioncontroltodistributebandwidthacrossclientsofunequalrequirement.Akeydistinguishingfeatureoftheiralgorithmistheintroduc-tionofcontrolledamountsofdelayintheprocessingofcer-tainrequestsduringoverloadstoensuredifferentclassesofre-questsarereceivingtheappropriateshareofthebandwidth.13 KnightlyandShroff[21]describeandclassifyabroadclassofadmissioncontrolalgorithmsandevaluatetheaccuracyofthesealgorithmsviaexperiments.Theyidentifykeyaspectsofadmissioncontrolthatenableittoachievehighstatisticalmultiplexinggains.Twoadmissioncontrolalgorithmshavebeenproposedre-centlythatutilizemeasurementsofrequestsizestoguidetheirdecisionmaking.VermaandGhosal[32]proposeaservicetimebasedadmissioncontrolthatusespredictionsofarrivalsandservicetimesintheshort-termfuturetoadmitasub-setofrequeststhatwouldmaximizetheprotoftheserviceprovider.In[14],theauthorspresentanadmissioncontrolformulti-tiere-commercesitesthatexternallyobservesexecu-tioncostsofrequests,distinguishingdifferentrequeststypes.Ourmeasurement-basedadmissioncontrolisbasedonsimilarideas,althoughthetechniquesdifferinthedetails.DynamicProvisioningandManagingResourcesinClus-ters:Theworkondynamicprovisioningofaplatform'sre-sourcesmaybeclassiedintotwocategories.Somepapershaveaddressedtheproblemofprovisioningresourcesatthegranularityofindividualserversasinourwork.Ranjanetal.[27]considertheproblemofdynamicallyvaryingthenumberofserversassignedtoasingleservicehostedonadatacenter.Theirobjectiveistominimizethenumberofserversneededtomeettheservice'sQoStargets.Thealgorithmisbasedonasimpleschemetoextrapolatethecurrentsizeoftheserversetbasedonobservationsofutilizationlevelsandworkloadstodeterminetheserversetoftherightsizeandisevaluatedviasimulations.TheOceanoprojectatIBM[3]hasdevel-opedaserverfarminwhichserverscanbemoveddynamicallyacrosshostedapplicationsdependingontheirchangingneeds.Themainfocusofthispaperwasontheimplementationis-suesinvolvedinbuildingsuchaplatformratherthantheexactalgorithmsforprovisioning.Otherpapershaveconsideredtheprovisioningofresourcesatnergranularityofresources.Muse[9]presentsanarchi-tectureforresourcemanagementinahostingcenter.Museemploysaneconomicmodelfordynamicprovisioningofre-sourcestomultipleapplications.Inthemodel,eachapplica-tionhasautilityfunctionwhichisafunctionofitsthroughputandreectstherevenuegeneratedbytheapplication.Thereisalsoapenaltythattheapplicationchargesthesystemwhenitsgoalsarenotmet.Thesystemcomputesresourceallocationsbyattemptingtomaximizetheoverallprot.ClusterReserves[4]hasalsoinvestigatedresourceallocationinserverclusters.Theworkassumesalargeapplicationrunningonacluster,wheretheaimistoprovidedifferentiatedservicetoclientsbasedonsomenotionofserviceclass.ThisisachievedbymakingtheOSschedulersprovidexedresourcesharestoap-plicationsspanningmultiplenodes.TheCluster-OnDemand(COD)[10]workpresentsanautomatedframeworktoman-ageresourcesinasharedhostingplatform.CODintroducesthenotionofavirtualcluster,whichisafunctionallyisolatedgroupofhostswithinasharedhardwarebase.AkeyelementofCODisaprotocoltoresizevirtualclustersdynamicallyincooperationwithpluggablemiddlewarecomponents.Chandraetal.[8]modelaserverresourcethatservicesmultipleappli-cationsasaGPSsystemandpresentsonlineworkloadpredic-tionandoptimization-basedtechniquesfordynamicresourceallocation.In[30]theauthorsaddresstheproblemofprovid-ingresourceguaranteestodistributedapplicationsrunningonasharedhostingplatform.[31]proposesaresourceoverbookingbasedschemeformaximizingrevenueinasharedplatform.Analternateapproachforimprovingperformanceofover-loadedwebserversisbasedonre-designingtheschedulingpolicyemployedbytheservers.SchroederandHarchol-Balter[28]proposetoemploytheSRPTalgorithmbasedonschedulingtheconnectionwiththeshortestremainingtimeanddemonstratethatitleadstoimprovedaverageresponsetime.Whileschedulingcanimproveresponsetimes,underextremeoverloadsadmissioncontrolandtheabilitytoaddextracapac-ityareindispensable.Betterschedulingalgorithmsarecom-plementarytooursolutionsforhandlingoverloads.DesignofEfcientLoadBalancers:Ouradmissioncon-trolschemeisnecessarilybasedontheuseofaLayer-7switchandhencethescalabledesignofsuchswitchesisimportanttoourimplementation.Paietal.[26]designlocality-awarere-questdistribution(LARD),astrategyforcontent-basedrequestdistributionthatcanbeemployedbyfrontserversinnetworkserverstoachievehighlocalityinthebackendserversandgoodloadbalancing.TheyintroduceaTCPhandoffprotocolthatcanhandoffanestablishedTCPconnectioninaclient-transparentmanner.AloadbalancerbasedonTCPhandoffhasbeenshowntobemorescalablethanthektcpvsloadbalancerwehaveused.In[5]ahighlyscalablearchitectureforcontent-awarerequestdistributioninWebserverclusters.ThefrontswitchisaLayer-4switchthatdistributedrequeststoanumberofback-endnodes.Content-baseddistributionisperformedbytheseback-endservers.[7]providesacompre-hensivesurveyofthemainmechanismstosplittrafcamongtheserversinacluster,discussingboththevariousarchitec-turesandtheloadsharingpolicies.OuradmissioncontrolandloadbalancingschemesareindependentoftheactualswitchimplementationsolongasitisLayer-7andhencemaybeim-plementedinanyoftheaforementionedscalableswitches.ModelingofInternetServices:Accuratemodelingofap-plicationsiscrucialfortranslatingQoSneedstoresourcere-quirements.Doyleetal.[13]presentamodel-basedutilityresourcemanagementfocusingoncoordinatedmanagementofmemoryandstorage.Theydevelopananalyticalmodelforawebservicewithstaticcontent.In[18]Kamraetal.useanM/GI/1processorsharingqueueasanabstractionfora3-tiere-commerceapplication.SLAsandAdaptiveQoSDegradation:TheWSLAprojectatIBM[34]addressesservicelevelmanagementissuesandchallengesindesigninganunambiguousandclearspeci-cationofSLAsthatcanbemonitoredbytheserviceprovider,customerandevenbyathird-party.AbdelzaherandBhatti[1]proposetodealwithserveroverloadsbyadaptingdeliveredcontenttoloadconditions.ThisisadifferentkindofQoS14 degradationthanwhatwehaveproposedinourwork,butitcanbeintegratedintoaCataclysmplatformbydeningappro-priateSLAsbasedonit.9ConclusionsandFutureWorkInthispaperwepresentedCataclysm,acomprehensiveap-proachforhandlingextremeoverloadsinahostingplatformrunningmultipleInternetservices.Theprimarycontributionofourworkwastodevelopanoverloadmanagementsolutionthatbroughttogetherthetechniquesofadmissioncontrol,dy-namicprovisioningoftheplatform'sresourcesandadaptivedegradationofQoSintooneintegratedsystem.Cataclysmpro-videsseveraldesirablefeaturesunderoverloads,suchaspref-erentialadmissionofmoreimportantrequests,theabilitytohandlediverseworkloadsandrevenuemaximizationatmul-tipletime-scalesviadynamicprovisioningofresourcesandsize-basedadmissioncontrol.Thecataclysmsentrycantrans-parentlytradeofftheaccuracyofitsdecisionmakingwiththeintensityoftheworkloadallowingittohandleincomingratesofuptoIJ\rL\rJ\rrequests/second.WeimplementedaprototypeCataclysmhostingplatformonaLinuxclusteranddemon-stratedthebenetsofourintegratedapproachusingavarietyofworkloads.Aspartoffuturework,weplantoextendouroverloadman-agementtechniquestocomplex,multi-tieredInternetapplica-tions.erences[1]T.AbdelzaherandN.Bhatti.WebContentAdaptationtoImproveServerOverloadBehavior.InProceedingsoftheWorldWideWebConference(WWW8),Tornoto,1999.[2]M.Andreolini,M.ColajanniandM.Nuccio.Scalabilityofcontent-awareserverswitchesforcluster-basedWebinformationsystems.InProc.of12thInt'lWorldWideWebConf.(WWW2003),Budapest,Hun-gary,May2003.[3]K.Appleby,S.Fakhouri,L.Fong,M.Goldszmidt,S.Krishnakumar,D.Pazel,J.PershingandB.Rochwerger.Oceano-SLA-basedManage-mentofaComputingUtility.InProceedingsoftheIFIP/IEEESympo-siumonIntegratedNetworkManagement,May2001.[4]M.Aron,P.Druschel,andW.Zwaenepoel.ClusterReserves:AMech-anismforResourceManagementinCluster-basedNetworkServers.InProceedingsoftheACMSIGMETRICSConference,SantaClara,CA,June2000.[5]M.Aron,D.Sanders,P.DruschelandW.Zwaenepoel.ScalableContent-AwareRequestDistributioninCluster-basedNetworkServers.InPro-ceedingsoftheUSENIX2000AnnualTechnicalConference,SanDiego,CA,June2000.[6]N.BhattiandR.Friedrich.WebServerSupportforTieredServices.InIEEENetwork,vol.13,no.5,pp.64-71,Sept.1999.[7]V.Cardellini,E.Casalicchio,M.ColajanniandP.Yu.ThestateoftheartinlocallydistributedWeb-serversystems.InACMComputingSurveys(CSUR)archiveVolume34,Issue2,Pages263-311,June2002.[8]A.Chandra,W.GongandP.Shenoy.DynamicResourceAllocationforSharedDataCentersUsingOnlineMeasurementsInProceedingsoftheEleventhInternationalWorkshoponQualityofService(IWQoS2003),Monterey,CA,June2003.[9]J.Chase,D.Anderson,P.Thakar,A.Vahdat,andR.Doyle.ManagingEnergyandServerResourcesinHostingCenters.InProceedingsoftheEighteenthACMSymposiumonOperatingSystemsPrinciples(SOSP),pages103–116,October2001.[10]J.Chase,L.Grit,D.Irwin,J.Moore,andS.Sprenkle.DynamicVirtualClustersinaGridSiteManager.IntheTwelfthInternationalSymposiumonHighPerformanceDistributedComputing(HPDC-12),June2003.[11]L.CherkasovaandP.Phaal.SessionBasedAdmissionControl:aMecha-nismforImprovingPerformanceofCommercialWebSites.InProceed-ingsofSeventhInternationalWorkshoponQualityofService,IEEE/IFIPevent,London,May31-June4,1999.[12]Self-similarityinWorldWideWebtrafc:evidenceandpossiblecauses.InProceedingsofthe1996ACMSIGMETRICS,Philadelphia,PA.[13]R.Doyle,J.Chase,O.Asad,W.JinandA.Vahdat.Model-BasedRe-sourceProvisioninginaWebServiceUtilityInProceedingsofthe4thUSENIXConferenceonInternetTechnologiesandSystems(USITS'03),March2003.[14]S.Elnikety,E.Nahum,J.TraceyandW.Zwaenepoel.AMethodforTransparentAdmissionControlandRequestSchedulinginE-CommerceWebSites.Submittedforpublication.[15]J.Hellerstein,F.ZhangandP.Shahabuddin.AStatisticalApproachtoPredictiveDetection.InComputerNetworks,January2000.[16]R.Iyer,V.TewariandK.Kant.OverloadControlMechanismsforWebServers.InPerformanceandQoSofNextGenerationNetwork,Nagoya,Japan,November2000.[17]H.Jamjoom,J.ReumannandK.Shin.QGuard:ProtectingInternetServersfromOverload.TechnicalReport,DepartmentofComputerSci-enceandEngineering,UniversityofMichigan,CSE-TR-427-00,2000.[18]A.Kamra,V.MisraandE.Nahum.ControllingthePerformanceof3-TieredWebSites:Modeling,DesignandImplementation.Submittedforpublication.[19]V.KanodiaandE.Knightly.Multi-classLatency-boundedWebservices.InProceedingsofIEEE/IFIPIWQoS2000,Pittsburgh,PA,June2000.[20]L.Kleinrock.QueuingSystems,Volume2:ComputerApplications.JohnWileyandSons,Inc.,1976.[21]E.KnightlyandN.Shroff.AdmissionControlforStatisticalQoS:The-oryandPractice.InIEEENetwork,13(2):20-29,March/April1999.[22]KernelTCPVirtualServer.http://www.linuxvirtualserver.org.[23]K.LiandS.Jamin.Ameasurement-basedadmission-controlledwebserver.InProceedingsofINFOCOM2000,TelAviv,Israel,March2000.[24]J.Moore,D.Irwin,L.Grit,S.Sprenkle,andJ.Chase.ManagingMixed-UseClusterswithCluster-on-Demand.Cluster-on-DemandDraft,Inter-netSystemsandStorageGroup,DukeUniversity.[25]D.MosbergerandT.Jin.httperf—AToolforMeasuringWebServerPerformance.InProceedingsoftheSIGMETRICSWorkshoponInternetServerPerformance,June1998.[26]V.Pai,M.Aron,G.Banga,M.Svendsen,P.Druschel,W.ZwanepoelandE.Nahum.Locality-AwareRequestDistributioninCluster-basedNetworkServers.InProceedingsoftheEighthInternationalConferenceonArchitecturalSupportforProgrammingLanguagesandOperatingSystems(ASPLOS-VIII),SanJose,California,October1998.[27]S.Ranjan,J.RoliaandE.Knightly,IWQoS2002.QoS-DrivenServerMigrationforInternetDataCenters.InProceedingsofIEEEIWQoS2002.[28]B.SchroederandM.Harchol-Balter.Webserversunderoverload:Howschedulingcanhelp.In18thInternationalTeletrafcCongress,Berlin,Germany,2003.[29]TheInternetUnderCrisisConditions:LearningfromSeptember11.CommitteeontheInternetUnderCrisisConditions:LearningfromSeptember11,NRC.[30]B.UrgaonkarandP.Shenoy.Sharc:ManagingCPUandNetworkBand-widthinSharedClusters.InIEEETransactionsonParallelandDis-tributedSystems(TPDS),January2004,Vol.15,No.1.15 [31]B.Urgaonkar,P.Shenoy,andT.Roscoe.ResourceOverbookingandApplicationProlinginSharedHostingPlatforms.InProceedingsoftheFifthSymposiumonOperatingSystemsDesignandImplementation(OSDI),Boston,MA,December2002.[32]A.VermaandS.Ghosal.OnAdmissionControlforProtMaximizationofNetworkedServiceProviders.InProc.of12thInt'lWorldWideWebConf.(WWW2003),Budapest,Hungary,May2003.[33]T.Voigt,R.TewariandD.Freimuth.Kernelmechanismsforservicedifferentiationinoverloadedwebservers.InProceedingsoftheUSENIXAnnualTechnicalConference,2001.[34]WebServiceLevelAgreements(WSLA)project.http://www.research.ibm.com/wsla[35]M.WelshandD.Culler.AdaptiveOverloadControlforBusyInter-netServers.InProceedingsofthe4thUSENIXConferenceonInternetTechnologiesandSystems(USITS'03),March2003.16