Source - PDF document

Source­LevelIPPacketBursts:CausesandEffectsHaoJiangCollegeofComputingGeorgiaInstituteofTechnologyhjiang@cc.gatech.eduConstantinosDovrolisCollegeofComputingGeorgiaInstituteofTechnologydovrolis@cc.gatech.eduABSTRACTBysource-levelIPpacketburst,wemeanseveralIPpacketssentback-to-backfromthesourceofa\row.Werstiden-tifyseveralcausesofsource-levelbursts,includingTCP'sslowstart,idlerestart,windowadvancementafterlossre-covery,andsegmentationofapplicationmessagesintomulti-pleUDPpackets.Wethenshowthatthepresenceofpacketburstsinindividual\rowscanhaveamajorimpactonag-gregatetrac.Inparticular,suchburstscreatescalinginarangeoftimescaleswhichcorrespondstotheburstdura-tion.Uniform\spreading"ofburstsinthetimeaxisreducesthescalingexponentinshorttimescales(upto100-200ms)toalmostzero,meaningthattheaggregatetracbecomespracticallyuncorrelatedinthatrange.ThisresultprovidesaplausibleexplanationforthescalingbehaviorofInter-nettracinshorttimescales.Wealsoshowthatremovingpacketburstsfromindividual\rowsreducessignicantlythetailoftheaggregatemarginaldistribution,anditimprovesqueueingperformance,especiallyinmoderateutilizations(50-85%).CategoriesandSubjectDescriptors:C.2.3[NetworkOperations]:TracmodelingandanalysisGeneralTerms:Measurement,PerformanceKeywords:scaling,networktrac,TCP,packetdisper-sion,packettrains,capacityestimation,correlationstruc-ture1.INTRODUCTIONBysource-levelIPpacketburst,wemeanseveralIPpack-etssentback-to-back,i.e.,atthemaximumpossiblerate,fromthesourceofa\row.Source-levelburstsintroducestrongcorrelationsinthepacketinterarrivalsofindividual\rows.Whichprotocolmechanismscreatesuchbursts?OverThisworkwassupportedbythe\ScienticDiscoverythroughAdvancedComputing"(SciDAC)programofDOE(DE-FC02-01ER25467),andbyanequipmentdonationfromIntelCorporation.Permissiontomakedigitalorhardcopiesofallorpartofthisworkforpersonalorclassroomuseisgrantedwithoutfeeprovidedthatcopiesarenotmadeordistributedforprotorcommercialadvantageandthatcopiesbearthisnoticeandthefullcitationontherstpage.Tocopyotherwise,torepublish,topostonserversortoredistributetolists,requirespriorspecicpermissionand/orafee.IMC'03,October27–29,2003,MiamiBeach,Florida,USA.Copyright2003ACM1­58113­773­7/03/0010...$5.00.whichtimescalesdothecorrespondingcorrelationsextend?Signicantresearcheortshavefocusedrecentlyonthecor-relationstructure,orscalingbehavior,ofaggregateIPtracinshorttimescales,typicallyuptoafewhundredsofmil-liseconds[1,2,3,4].Istheshorttimescalingbehaviorofaggregatetracrelatedtothepresenceofpacketburstsinindividual\rows?Howwillthecorrelationstructureofag-gregatetracchangeif\rowsdonotincludesuchbursts?Intermsofnetworkperformance,howwillthequeueingde-laysdecreaseifweremoveburstsfromindividual\rows,andinwhatloadconditionsissuchadecreasemostimportant?Thesearesomeofthequestionsthatweinvestigateinthispaper.Backgroundonscaling.Thekeytoolthatwerelyonisthewavelet-basedmultiresolutionanalysisdevelopedin[5]andimplementedin[6].Thisstatisticaltoolallowsustoobservethescalingbehaviorofatracprocessoveracertainrangeoftimescales.ConsiderareferencetimescaleT0,andletTj=2jT0forj=1,2,:::beincreasinglycoarsertimescales.Thesetimescales,orsimplyscales,partitionatractraceinconsecutiveandnon-overlappingtimeintervals.Iftjiisthei'thtimeintervalatscalej�0,thentjiconsistsoftheintervalstj12iandtj12i+1.LetXjibetheamountoftracintji,withXji=Xj12i+Xj12i+1.TheHaarwaveletcoecientsfdjigatscalejaredenedasdji=2j=2(Xj12iXj12i+1)(1)fori=1;:::Nj,whereNjisthenumberofwaveletcoe-cientsatscalej.TheenergyfunctionEjisdenedasEj=E[(dji)2]i(dji)2Nj(2)Anenergyplot,suchasFigure1,showsthelogarithmoftheenergyEjasafunctionofthescalej.ThemagnitudeofEjincreaseswiththevariabilityofthetracprocessXj1atscalej-1.Whatismoreimportantisthescalingbehavioroftheprocess,i.e.,thevariationofEjwithj.Foranexactlyself-similarprocess,suchasfractionalBrownianmotion(fBm)withHurstparameterH(0.51),itcanbeshownthatEj=E02j(2H1),andsotheenergyplotisastraightlinewithpositiveslope2H-1.Theslopeofanen-ergyplotisreferredtoasscalingexponentandisdenotedby.ForfBm,=2H-1isconstantacrossalltimescales,andsotheprocessissaidtoshowglobalscaling.Toillustratethedetectionofscalinginatracprocess,Figure1showstheenergyplotsforthreesynthetictraces,allofwhichhavethesamemeanpacketinterarrival(50ms). 024681012-3-2-101234j = log2(scale)log2(Energy)Exponential source,Pareto source, a=0.57 (3,12) Exponential/Periodic source0.1 0.4 1.6 6.4 25.6 102.4 (sec) Figure1:Energyplotexamples.AtthetopofthegraphweshowthetimescaleTjthatcorre-spondstoscalejatthex-axis(T0=25ms).ThersttraceisaPoissonprocess.Thesignatureofuncorrelatedexponen-tialinterarrivalsintheenergyplotisahorizontalstraightline(=0).Thesecondtraceisagainarenewalprocess,butthistimetheinterarrivalsfollowtheParetodistribu-tionwithshapeparameter=1.5.Theinnitevarianceoftheinterarrivalscreatesglobalscaling.Thesignatureofsuchglobalscalingintheenergyplotisastraightlineseg-mentwithpositiveslope(=2-)acrossalltimescales.Thethirdtraceisagainbasedonexponentialinterarrivals,butthistimeweintroduceastrongperiodicityatthe400mstimescale.Specically,aftereachexponentialinterarrivalwegenerate,withprobability0.75,anotherpacket400mslater(scale4).Thisperiodicitycausesa\dip"intheenergyplotatthe800mstimescale(scale5).Thisisbecauseape-riodicityreducesthevariabilityofthetracprocessatthecorrespondingtimescale.Notethatthedipappearsatscale5,insteadof4,becausetheenergyatscalejdependsonthetracprocessvariationsinscalej-1.Inpractice,networktraccanshowdierentscalingbe-havioracrossdierenttimescales.Iftheslopeoftheenergyplotis(roughly)constantoverarangeoftimescalesjtoj+k,wesaythatthetracprocessexhibitslocalscalinginthetimescalesTjtoTj+k.ThispaperfocusesonthescalingbehaviorofIPtracinshorttimescales,typicallyextendinguptoafewhundredsofmilliseconds.Relatedwork.Ourworkismostlyrelatedtopreviousre-searchonthescalingbehaviorofInternettracinshorttimescales.OneoftherstpapersthatreportedscalinginshorttimescalesatWANtraceswas[7].In[1,8],Feldmannetal.usedthewavelet-basedmultiresolutionanalysistech-niqueof[5]todetectandcharacterizethescalingbehaviorofInternettrac.TheauthorsshowedthatscalinginshorttimescalesisrelatedtotheTCPclosed-loop\rowcontrol,andthecutobetween\short"and\long"timescalesis,roughly,theRTToftheTCPtransfers.Additionally,[1]providedempiricalevidencethatWANtraccanbemod-eledusingamultifractalmodel,similartothatdevelopedin[2].Morerecentwork,however,arguesthatthetracatatier-1ISPiswell-modeledasmonofractal,ratherthanmultifractal[3].[9]showedthatIPlayerscalingdoesnotdependontheTCP\rowarrivalprocess.Inafollow-upwork,[4]showedthatthethecorrelationstructureofaggregatetracinshorttimescalescanbecapturedbyaPoissonclusterpro-cessinwhichthepacketinterarrivalswithinindividualclus-tersfollowanoverdispersedGammadistribution.[3]intro-ducedtheconceptof\dense\rows"(i.e.,\rowswithburstsofdenselyclusteredpackets),andshowedthatitisthiskindof\rowsthatcreatescalinginshorttimescales.[10]showedthatscalinginnetimescalescanhaveasignicantimpactonqueueingperformance,especiallyinmoderateutiliza-tions,whilescalingincoarsertimescalesismoreimportantinheavyutilizations.Ourmainresult,connectingscalinginshorttimescaleswithpacketburstsfromindividual\rows,isinagreementwiththeresultsof[3,9,4,10],providingamorespecicexplanationforthenatureandcausesofscal-ingbehaviorinaggregatetrac.Thetracesthatweusedinthisstudyarepubliclyavail-ableattheNLANR-MOATsite[11].Eachtracelastsfor90seconds.ThetracesthatweincludeinthispapercomefromOC-12linksattheMerit(MRA)andIndianaUni-versity(IND)Internet2GigaPOPs.Therestofthispaperisstructuredasfollows.x2givesseveralcausesofsource-levelburstsinIPtrac.x3showsthatpacketburstscreatescalinginarangeoftimescaleswhichcorrespondstotheburstduration.x4investigatestheeectofburstsfromindividual\rowsonaggregatetracintermsofscalinginshorttimescales,marginaldistribution,andqueueingper-formance.TheAppendixdescribesapassivecapacityesti-mationmethodology,whichisrequiredforthedetectionofpacketburstsfromindividual\rowsatatrace.2.CAUSESOFSOURCE­LEVELBURSTSWehaveanalyzeddozensoftraces,attemptingtoidentifythemostcommoncausesofsource-levelbursts.Figure2showsninesuchcauses,oneforUDPandeightforTCP\rows.Unfortunately,ouranalysisisnotautomated,andsowecannotmakequantitativestatementsregardingtherelativefrequencyofeachcause.Webelieve,however,thatFigure2showsmost,orall,majorcauses.UDPmessagesegmentation.WhenaUDP-basedappli-cationsendsamessagethatislargerthanthepath'sMTU,themessageissegmentedbytheapplicationintomultipleUDPpackets,and/oritisfragmentedbytheoperatingsys-temintomultipleIPpackets.TheexampleshowninFig-ure2isfromaUDPvideo\rowwhichsendssixpacketsevery40ms.Slowstart.SlowstartincreasesthecongestionwindowbyoneMSSforeverynewACK.ThisrapidincreasecandoubletheburstlengthineachRTT.Intheexampleshown,thereceiverdoesnotuseDelayed-ACKs,andsotheburstsareonethirdlongerthannormally.LossrecoverywithFastRetransmit.TherecoveryofalostsegmentthroughFastRetransmitcanllina\hole"inthereceivingslidingwindow,andsoitcancausearapidadvancementoftheACKnumber.ThesendercanthensenduptoCW=2bytesback-to-back,whereCWisthecongestionwindowbeforetheloss.NotethattheconnectionofourexampledidnotdoFastRecovery,asnewsegmentswerenotsentinresponsetoduplicateACKsthatfollowedtheretransmission,duetocongestionwindowconstraints.1Unusedcongestionwindowincreases.Sometimes,mostlywithapplicationsthatinitiallyexchangecontrolmessages(suchasscp),thecongestionwindowincreaseswitheveryACK,butwithoutbeingusedbythesender.Then,when1FastRecoverycanreducetheburstsizeafteraretransmis-sion,ifnewsegmentscanbesentinresponsetoduplicateACKs. 0.060.080.10.120.140.160.180.20.22Time (second)0612182430Packet numberUDP packetUDP message segmentation18.218.318.418.518.618.718.818.91919.119.219.3Time (second)Sequence numberData packetACK packetSlow startSYN5.85.966.16.26.3Time (second)Sequence numberData packetACK packetLoss recovery with fast retransmissionFast retransmission0.40.60.811.21.41.6Time (second)Sequence numberData packetACK packetUnused congestion window increasesSYNTCP control segments0.60.811.21.41.61.8Time (second)0Sequence numberData packetACK packetACK compressionCompressed ACKs18.618.6518.718.7518.818.85Time (second)Sequence numberData packetACK packetCumulative or lossed ACKsAck for 6 MSS packets8.68.65Time (second)Sequence numberData packetACK packetACK reorderingReordered ACKs0.50.60.70.80.9Time (second)Sequence numberData packetACK packetBursty application234567Time (second)0Sequence numberData packetACK packetIdle restart timer bugFigure2:Majorcausesofsource-layerbursts.thesenderisreadytotransferalargemessageorle,itcansendalongbursttothenetwork.Ourexamplecomesfromthestartofanscpsession.ACKcompression.QueueinginthereversepathofaTCP\row,cancausethealmostsimultaneousarrivalofsuc-cessiveACKsatthesender.ThiscanbreakTCP'sself-clockingandcauselongbursts[12].CumulativeorlostACKs.Sometimesthereceivergen-eratesanACKformultiplereceivedsegments.Sucha\super-cumulative"ACKcantriggeraburstatthesender.ThesameeectcanoccurifoneormoreACKsarelost.Inthatcase,therstnon-lostACKwilltriggeraburst.Idlerestarttimerbug.ATCPsenderisrecommendedtouseslowstartandreturntotheinitialcongestionwindowafterithasnotsentanythingforthedurationoftheIdleRestarttimer(typicallyequaltoRTO)[13].Unfortunately,severaloperatingsystemsdonotsupport,ortheydonotimplementcorrectly,thisfeature[14].Asaresult,aTCPsendercansendalongburstafteranidletimeperiod.Burstyapplications.EveniftheIdleRestarttimerisimplementedcorrectly,itispossiblethattheapplicationitselfisbursty,meaningthatitwritesbytestotheTCPsend-socketsporadically.Ifthetimebetweenburstsissuf-cientlyshortsothatIdleRestartisnotactivated,TCP'sself-clockingbreaksandTCPcansendlongbursts.Packetreordering.ReorderingofACKsscramblesself-clockingandcantriggeraburstatthesender.Intheex-ampleshown,theout-of-orderACKacknowledgestenmoresegments,causingalargeburstatthesender.Datasegmentreorderingcanalsointerruptself-clockingandcausebursts[15].3.PACKETBURSTSANDSCALINGInthissection,weshowtheconnectionbetweensource-levelburstsandscaling,andidentifythetimescalesinwhichsuchburstscreatescalingbehavior.Considerasourcethatgeneratesasequenceofpackettrains.AtrainconsistsofNpackets,eachoflengthLbytes.IfthecapacityofthesourceisC,thedispersionofeachpacketinthetimeaxisisLC,whilethedispersionoftheentiretrainisNLC.SupposethattheinterarrivaltimeToffbetweensuccessivetrainsisexponentiallydistributed.WenextshowthatthistracprocessshowslocalscalinginthetimescalesbetweenLCandNLC.Figure3showstheautocorrelationfunctionandtheen-ergyplotforasynthetictracethatfollowsthepreviouspackettrainmodel.Inthistrace,wehavethatLC=4ms,N=16,NLC=64ms,andE[Toff]=2000ms.Considerrstthediscrete-timeprocessofpacketarrivalsinsuccessivenon-overlappingintervalsoflengthLC;thistimeseriestakesthevalues0and1.TheautocorrelationR()ofthisprocess,for=0;1;2;:::,ispositivewhen,duetothestronglycorrelatedinterarrivalswithinapackettrain.Forlargerlags&#xN000;N,R()isalmostzerobecausethecorrelationsbetweenpacketsofdierenttrainsareweak(ToffNLC),andthetimeintervalbetweensuccessivetrainsisexponentiallydis-tributed.ObservenowtheenergyplotofFigure3.Thelinearly 0246810121416-9-8-7-6-5-4-3-2-101j = log2(scale)log2(Energy)L/C = 4ms, N = 16, NL/C = 64ms2 8 32 128 512 2048 8192 (ms) 081624324048566472808896Time (ms)-0.200.20.40.60.81Autocorrelation coefficient024681012141618202224LagFigure3:Autocorrelationandenergyplotofthepackettraintracmodel.increasingsegment,betweenscales4and8,representslocalscalinginthetimescalesfrom4ms(LC)to64ms(NLC).2Thestrongpositivecorrelationsinthelagsthatcorrespondtothetrainduration(=0;:::15)arere\rectedintheenergyplotaslocalscalinginthecorrespondingtimescales.Thescalingexponentisalmostzeroinlongertimescales(higherthan64ms),duetotheexponentialtraininterarrivals.Alsonotethatthenegativedipatscale-4,whichcorrespondstothepacketspacingL=C,isduetotheperiodicarrivalofanewpacketevery4msduringpackettrains.Thepreviousmodelmayseemtooarticial,asallpack-etsappearinbursts.Source-levelburstscancreatescal-ingeveniftheyoccurlessfrequentlyhowever.Considerasourcewithtwostates:the\random"stateandthe\bursty"state.Intherandomstate,thesourcegeneratesexponen-tialinterarrivalswithameanof100ms.Intheburstystate,thesourcegeneratesasingletrainofN=16packets.Thetransitionprobabilityfromtherandomstatetotheburstystateis0.05,whilethetransitionprobabilityinthereversedirectionis1.Figure4showstheenergyplotforsuchasource,whenLC=1ms.Noticetheemergingscalingbehaviorbetweenscales2and6,whichcorrespondtothetimescales1msto16ms.Eventhoughthescalingexponentisnotcon-stantacrossthesetimescales,therangeinwhichispositivematchestheextentofpacketbursts,fromLCtoNLC.02468101214161820-7-6-5-4-3-2-101j = log2(scale)log2(Energy)Exponential OFF, a=-0.014 (10, 17)Pareto OFF, a= 0.47 (14, 20)2 8 32 128 512 2048 8192 32768 131072 (ms) Figure4:Bi-scalingbehavior.Ontheotherhand,source-levelburstsdonotcontributetothescalingbehaviorinlongtimescales.Toillustratethispoint,considertheprevioustwo-statemodel,butnowsup-posethattherandomstategeneratesParetointerarrivalswith=1.5.Figure4showstheresultingenergyplot.TheinnitevarianceoftheParetointerarrivalscreatesscalingatlargetimescales.Thescalingexponentabovescale14isestimatedas0.5,whichisconsistentwiththeshape2WeremindthereaderthattheenergyEjiscomputedbasedonthevariationsofthetracprocessatscalej-1.parameter=1.5.Thescalingbehaviorinshorttimescales,ontheotherhand,isduetopackettrains,anditremainsroughlythesameasinthecaseofexponentialinterarrivals.Thisisanexampleofbi-scalingbehavior,i.e.,dierentscal-ingexponentinshortvs.longtimescales,whichisoftenseenintheenergyplotofWANtraces[1].4.EFFECTSOFPACKETBURSTSInthissection,weshowtheeectofpacketburstsfromin-dividual\rowsinthreedierent,butrelated,characteristicsofaggregateIPtrac:scalingbehaviorinshorttimescales,marginaldistribution,andqueueingperformance.Burstidentication.First,wedescribehowtoidentifypacketburstsfromindividual\rowsinatraceofaggregatetrac.ConsideraTCP\rowfwithsourceSf.ApackettraceiscollectedattheoutputofalinkTinf'spath.Intheappendix,wegiveamethodologyfortheestimationofthepre-tracecapacity~Cfof\rowf,i.e.,theminimumlinkcapacityalongthepathbetweenSfandT.Apacketburstfrom\rowfisdenedasasequenceofpacketsfromfthatarriveatTwitharatethatisroughly~Cf.ItisimportanttonotethatwecannotdeterminewhetherthesepacketsweresentfromSfback-to-back;wecanonlydeterminewhethertheyarriveatTback-to-back.Asource-levelburstwillbedetectedasapacketburstatT,butnoteverypacketburstatTwillbeasource-levelburst.Forthisreason,thissectionreferstotheeectsofpacketbursts,asopposedtosource-levelpacketbursts,fromindividual\rows.Inpractice,theratebetweensuccessivepacketsinaburstmay\ructuateaboveorbelow~CfbecauseofcrosstracqueueingatlinksbeforeT.So,werequirethefollowing,lessrestrictive,condition:asequenceofpacketsPf(i);:::Pf(i+j)from\rowfisapacketburstoflengthj+1,ifj�0isthemaximumpositivenumberthatsatisesthefollowingtwoconditions:i+j1k=iSf(k)
f(i;j)�~Cfa(3)Sf(k)
f(k;k+1)�~Cfbforallk=i;:::j1(4)whereSf(k)isthesizeofpacketPf(k),and
f(m;n)isthedispersion(timedistance)betweenthestartofpacketsPf(m)andPf(n)atT(mn).If�a1and�b1,theseconditionsrequirethattheburst'saveragerateislargerthanafraction1=aof~Cf,andthattheratebetweensuccessivepacketsintheburstislargerthanafraction1=bof~Cf.ToillustratethefrequencyandlengthofpacketburstsinrealInternettrac,Figure5showstheCDFofburstlengthsforatracefromtheOC-12Meritlink(MRA).ThisgraphisderivedbasedonTCP\rowsforwhichwehaveapre-tracecapacityestimate(about83%oftheTCPbytesinthetrace).Weshowthreecurvesfordierentparametersaandb.Notethattheburstlengthdistributiondoesnotdependsignicantlyonthesetwoparameters;intherestofthispaperweusea=2andb=4.Alsonotethat40%ofthebytesinthistracearetransferredinburstsofatleastfourpackets,while10%ofthebytesareinburstsofmorethantwelvepackets.Burstremoval.Ifwecanidentifysource-levelbursts,wecanalsomodifyatracesothatweremovethosebursts.Weusethistechniquetoinvestigatehowwouldthestatistical 024681012141618Burst length (packets)00.10.20.30.40.50.60.70.80.91CDF (a=2, b=4)(a=3, b=5)(a=1.5, b=3)OC12 link: MRA-1028765523 (20:12 EST, 08/07/2002)Figure5:Parametersensitivityofburstidentica-tionalgorithm.proleofthetracechange,ifindividual\rowsdidnotgen-eratepacketbursts.Sucha\semi-experimental"approachhasbeenalsofollowedin[9,10].SupposethataburstBf(k)of\rowfstartsattimetf(k),whiletherstpacketoffafterthisburstappearsattimetf(k+).WeremovetheburstBf(k)byarticiallyspacingthepacketsoftheburstuniformlybetweentf(k)andtf(k+).Notethatthepacketsof\rowfremainintheiroriginalor-derafterrespacingthebursts.Alsonotethatthisburstremovalprocedurecannotbeperformedon-linebyasourceorrouter,asitrequiresknowledgeoftf(k+)whenaburststarts.Also,itisnotequivalentto\rowshapingorpac-ing;theselatterapproacheswouldtransmitthepacketsofaburstataxedrate.Werefertotheresultingtraceasmanipulated,todistinguishitfromtheoriginaltrace.Eectofbursts.Figure6comparestheoriginalandma-nipulatedtraces,fromtwoOC-12links,intermsofthreeaspects:energyplotsandscalingbehavior,taildistribution,andqueueingperformance.Attheleft,weshowtheen-ergyplotofthetracesintimescalesthatextendfromlessthanamillisecondtoafewseconds.Noticethatbothtracesshowclearbi-scalingbehavior,withascalingexponentof0:35fortheMRAtraceand0:26fortheINDtraceinshorttimescales(lessthan25200ms).Thescalingexponentatlargetimescalesis0:99and0:90,respectively,butitsesti-mationislessaccurateduetotheshortdurationofthesetraces.Thekeyobservation,however,isthedierencebe-tweentheoriginalandmanipulatedtraces:thescalingbehav-iorinshorttimescaleshasbeendramaticallyreduced,drop-pingthescalingexponenttoalmostzero.Thisimpliesthatremovingpacketburstswouldleadtoalmostuncorrelatedpacketarrivalsoverarangeofshorttimescalesthatextendsupto100-200ms.Asexpected,thescalingbehaviorinlongertimescaleshasnotbeenaected.ThemiddlegraphsofFigure6showthetaildistributionoftheamountofbytesinnon-overlapping10msintervals.Theaverageofthisdistributionis189KBfortheMRAtraceand32KBfortheINDtrace.Notethattheremovalofpacketburstsfromindividual\rowsreducessignicantlytheprob-abilityofhavingburstsintheaggregatetrace.Thiswasexpected,asmostburstsattheaggregatetraceareduetoindividual\rows,insteadofdierent\rows.Theremovalofburstsfromtheaggregatetracehintsthatthequeueingper-formancewouldalsoimprovesignicantly.Indeed,therightgraphsofFigure6showthemaximumqueuesizethatwoulddevelopatalinkthatservicestheaggregatetrac,aswevarythelink'scapacity.Thereductioninthemaximumqueuesize,afterweremovethesource-levelbursts,issig-nicantespeciallyinmoderateutilizations,between50%to85%.Thisresultagreeswiththendingsof[10].5.SUMMARYANDFUTUREWORKThispaperfocusedonthecausesandeectsofpacketburstsfromindividual\rowsinIPnetworks.Weshowedthatsuchburstscancreatescalinginshorttimescales,andin-creasedqueueingdelaysintracmultiplexers.Weidentiedseveralcausesforsource-levelbursts,investigatingthe\mi-croscopic"behavioroftheUDPandTCPprotocols.Someofthesecauses,suchastheimplementationoftheIdleRestarttimer,canbeeliminatedwithappropriatechangesintheTCPprotocolorimplementation.Someothercauses,how-ever,suchasthesegmentationofUDPmessagesinmultipleIPpackets,aremorefundamentalinnatureandtheymaynotbeavoidable.Eventhoughweidentiedaplausibleexplanationforthepresenceofscalinginshorttimescales,wedonotclaimthatsource-levelburstsaretheonlysuchexplanation.Inon-goingwork,weinvestigateotherimportantfactors,suchastheeectofTCPself-clocking.Wealsostudytheeectofper-\rowshapingandTCPpacingonthecorrelationstruc-tureandmarginaldistributionsofaggregateIPtrac.6.REFERENCES[1]A.Feldmann,A.C.Gilbert,andW.Willinger,\DataNetworksasCascades:InvestigatingtheMultifractalNatureoftheInternetWANTrac,"inProceedingsofACMSIGCOMM,1998.[2]R.Riedi,M.S.Crouse,V.Ribeiro,andR.G.Baraniuk,\AMultifractalWaveletModelwithApplicationtoNetworkTrac,"IEEETransactionsonInformationTheory,vol.45,no.3,pp.992{1019,Apr.1999.[3]Z.-L.Zhang,V.Ribeiro,S.Moon,andC.Diot,\Small-TimeScalingbehaviorsofInternetbackbonetrac:AnEmpiricalStudy,"inProceedingsofIEEEINFOCOM,Apr.2003.[4]N.Hohn,D.Veitch,andP.Abry,\ClusterProcesses,aNaturalLanguageforNetworkTrac,"IEEETransactionsonSignalProcessing,specialissueon\SignalProcessinginNetworking",2003,Acceptedforpublication.[5]P.AbryandD.Veitch,\WaveletAnalysisofLong-RangeDependentTrac,"IEEETransactionsonInformationTheory,vol.44,no.1,pp.2{15,Jan.1998.[6]D.Veitch,\CodefortheEstimationofScalingExponents,"http://www.cubinlab.ee.mu.oz.au/darryl,July2001.[7]A.Feldmann,A.C.Gilbert,W.Willinger,andT.G.Kurtz,\TheChangingNatureofNetworkTrac:ScalingPhenomena,"ACMComputerCommunicationReview,Apr.1998.[8]A.Feldmann,A.C.Gilbert,P.Huang,andW.Willinger,\DynamicsofIPTrac:AStudyoftheRoleofVariabilityandTheImpactofControl,"inProceedingsofACMSIGCOMM,1999.[9]N.Hohn,D.Veitch,andP.Abry,\DoesfractalscalingattheIPleveldependonTCP\rowarrivalprocesses?,"inProceedingsInternetMeasurementWorkshop(IMW),Nov.2002.[10]A.Erramilli,O.Narayan,A.L.Neidhardt,andI.Saniee,\PerformanceImpactsofMulti-ScalinginWide-AreaTCP/IPTrac,"inProceedingsofIEEEINFOCOM,Apr.2000.[11]NLANRMOAT,\PassiveMeasurementandAnalysis,"http://pma.nlanr.net/PMA/,May2003.[12]J.C.Mogul,\ObservingTCPdynamicsinrealnetworks,"inProceedingsofACMSIGCOMM,Aug.1992.[13]M.Allman,V.Paxson,andW.Stevens,TCPCongestionControl,Apr.1999,IETFRFC2581.[14]A.Hughes,J.Touch,andJ.Heidemann,IssuesinTCPSlow-StartRestartAfterIdle,Mar.1998,IETFInternetDraft,draft-ietf-tcpimpl-restart-00.txt(expired).[15]J.C.R.Bennett,C.Partridge,andN.Shectman,\PacketReorderingisNotPathologicalNetworkBehavior,"IEEE/ACMTransactionsonNetworking,vol.7,no.6,pp.789{798,Dec.1999. 0246810121416212223242526272829j = log2(scale)log2(Energy)Original, a=0.351 (2, 9)Manipulated, a=0.019 (2, 9)0.4 1.6 6.4 25.6 102.4 409.6 1638.4 (ms) MRA-1028765523 20406080Traffic in 10ms (KB)0.0010.010.11�P[X x]OriginalManipulatedIND-1041854717 (07:05 EST, 01/06/2003)100150200250300350Traffic in 10ms (KB)0.0010.010.11�P[X x]OriginalManipulatedMRA-1028765523 (20:12 EST, 08/07/2002)0246810121416192021222324j = log2(scale)log2(Energy)Original, a=0.262 (4, 10)Manipulated, a=0.043 (4, 10)0.4 1.6 6.4 25.6 102.4 409.6 1638.4 (ms) IND-1041854717 0.50.60.70.8Utilization050100150200Maximum queue length (KB)OriginalManipulatedIND-1041854717 (07:05 EST, 01/06/2003)0.40.50.60.70.8Utilization050100150200Maximum queue length (KB)OriginalManipulatedMRA-1028765523 (20:12 EST, 08/07/2002)Figure6:Eectofsource-levelburstsonscaling,taildistribution,andqueueingperformance.[16]C.Dovrolis,P.Ramanathan,andD.Moore,\WhatdoPacketDispersionTechniquesMeasure?,"inProceedingsofIEEEINFOCOM,Apr.2001,pp.905{914.Appendix:PassivecapacityestimationTheidenticationofpacketburstsfroma\rowfatatracepointTrequiresanestimateofthepre-tracecapacity~Cfof\rowf.Here,wesummarizeastatisticalmethodologythatestimates~CfforTCP\rows,usingthetimingofthe\row'sdatapackets.Themethodologyisbasedonthedispersion(timedistance)ofpacketpairs[16].ForaTCP\rowf,letSf(i)bethesizeofthei'thdatapacket,and
f(i)bethedispersionmeasurementbetweendatapacketsiandi+1.Whenpacketsiandi+1areofthesamesize,wecomputeabandwidthsamplebi=Sf(i)=
f(i).Packetswithdierentsizestraversethenetworkwithdif-ferentper-hoptransmissionlatencies,andsotheycannotbeusedwiththepacketpairtechnique[16].Basedonthedelayed-ACKalgorithm,TCPreceiverstypicallyacknowl-edgepairsofpackets,forcingthesendertorespondtoeveryACKwithatleasttwoback-to-backpackets.So,wecanestimatethatroughly50%ofthedatapacketsweresentback-to-back,andthustheycanbeusedforcapacityes-timation.Therestofthepacketsweresentwithalargerdispersion,andsotheywillgivelowerbandwidthmeasure-ments.Basedonthisinsight,wesortthebandwidthsamplesof\rowf,andthendropthelower50%ofthem.Toesti-matethecapacityof\rowf,weemployahistogram-basedmethodtoidentifythestrongestmodeamongtheremain-ingbandwidthsamples;thecenterofthestrongestmodegivestheestimate~Cf.Thebinwidththatweuseis!=2(IRQ)K1=3(knownas\Freedman-Diaconisrule"),whereIRQandKistheinterquartilerangeandnumber,respectively,ofbandwidthsamples.Wehaveveriedthistechniquecom-paringitsestimateswithactivemeasurements.Theresultsarequitepositive,butduetospaceconstraintswedonotincludetheminthispaper.Figure7showsthedistributionofcapacityestimatesintwotraces.NotethattheCDFisplottedintermsofTCPbytes,ratherthanTCP\rows.Inthetopgraph,weseefourdominantcapacitiesat1.5Mbps,10Mbps,40Mbps,and100Mbps.Thesevaluescorrespondtothefollowingcom-monlinkbandwidths:T1,Ethernet,T3,andFastEther-net.ThebottomgraphshowsthecapacitydistributionfortheoutbounddirectionoftheATMOC-3linkatUniver-sityofAuckland,NewZealand.Thislinkisrate-limitedto4.048Mbpsatlayer-2.Weobservetwomodes,at3.38Mbpsand3.58Mbps,atlayer-3.Theformermodecorrespondsto576BIPpackets,whilethelattermodecorrespondsto1500BIPpackets.ThedierenceisduetotheoverheadofAAL5encapsulation,whichdependsontheIPpacketsize.Wenallynotethatourcapacityestimationmethodologycannotproduceanestimateforinteractive\rows,\rowsthatconsistonlypure-ACKs,and\rowsthatcarryjustafewdatapackets.Wewereable,however,toestimatethecapacityfor83%oftheTCPbytesintheMRA-1028765523trace,92%oftheTCPbytesintheIND-1041854717trace,and82%oftheTCPbytesintheAucklandtrace.101001000100001e+051e+06Capacity (Kbps)0102030405060708090100CDF in bytes (%)OC12 link: MRA-1028765523 (20:12 EST, 08/07/2002)30003100320033003400350036003700380039004000Capacity (Kbps)0102030405060708090100CDF in bytes (%)Univ. of Auckland OC3 link (outbound rate limit = 4.048 Mbps, 2001)MSS=576MSS=1500Figure7:Capacitydistributionintermsofbytesattwolinks.