kuehlewindikrunistuttgartde Bob Briscoe BT Sirius House Adastral Park Ipswich IP5 3RE UK bobbriscoeBTcom ABSTRACT We present lessons from implementing a bandwidth probing scheme termed chirping as part of congestion control in the Linux kernel Chirpi ID: 66798
Download Pdf The PPT/PDF document "Chirping for Congestion Control Impleme..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
ChirpingforCongestionControl-ImplementationFeasibilityMirjaKühlewindInstituteofCommunicationNetworksandComputerEngineering(IKR)UniversityofStuttgart,Germanymirja.kuehlewind@ikr.uni-stuttgart.deBobBriscoeBTSiriusHouse,AdastralParkIpswich,IP53RE,UKbob.briscoe@BT.comABSTRACTWepresentlessonsfromimplementingabandwidthprobingschemetermedchirpingaspartofcongestioncontrolintheLinuxkernel.Chirpingwas rstintroducedforbandwidthestimationinthepathChirptool.Achirpisagroupofsev-eralpacketsthataresentwithdecreasinginter-packetgapsinordertocontinuouslyprobeforsparebandwidth.Cur-rentcongestioncontrolschemesareeitherslowto ndnewbandwidthortheyovershootduetolackoffastfeedbackinformation.Theattractionofusingchirpingasabuildingblockforcongestioncontrolistobeabletonon-intrusivelyprobeforavailablecapacityinaveryshorttime.Butimple-mentingchirpingischallengingbecauseitrequiresanexacttimingofeverypacketwhichisverydi erenttothetradi-tionalapproachinthenetworkstack.Astherearechangesneededatthereceiveraswell,wealsodiscussapotentialapproachfordeployment.Successindetectingfastfeedbackinformationusingchirpingopensthepossibilityoffutureworkonnewcongestioncontroldesignsthatshouldbemoreresponsivewithlessovershoot.1.INTRODUCTIONAwellknownproblemofTransmissionControlProtocol(TCP)congestioncontrolistheinabilitytoquicklyreachhighthroughputinhigh-speedorhighlydynamicbandwidthenvironments.Toovercomethisproblem,newTCPvariantsincreasetheirratemorerapidly.Butconsequentlytheysuf-ferfromovershoot,i.e.theyincreasetoofarabovethecor-rectrate,leadingtosigni cantlygreaterlossforother\rows.Apromisingnewapproachiscongestioncontrolbasedoncontinuallysendingso-calledchirpsormulti-rateprobestreamsofpackets,asproposedin[9].Achirpisagroupofseveralpacketsthataresentwithdecreasinginter-packet ThisworkispartlyfundedbyTrilogy,aresearchproject(ICT-216372)supportedbytheEuropeanCommunityunderitsSeventhFrameworkProgramme.Theviewsexpressedherearethoseoftheauthoronly.ThisworkisalsosupportedbytheGermanResearchFoundation.Permissiontomakedigitalorhardcopiesofallorpartofthisworkforpersonalorclassroomuseisgrantedwithoutfeeprovidedthatcopiesarenotmadeordistributedforprotorcommercialadvantageandthatcopiesbearthisnoticeandthefullcitationontherstpage.Tocopyotherwise,torepublish,topostonserversortoredistributetolists,requirespriorspecicpermissionand/orafee.Copyright20XXACMX-XXXXX-XX-X/XX/XX...$10.00.gapsandrespectivelyincreasingrate.Thusthepacketsofthebeginningofeverychirparesentataslowerrate,ris-ingtoafasterrateattheend.Foreverychirptheaveragesendingrateissetequaltotheratethatthesenderwantstoachieve.Thiso ersaprobingschemewithe ectiveprobingratesthatvarybyordersofmagnitudewithinonechirp,butwithoutoverloadingthenetwork.Sendingwithincreasingratewithinonechirpleadstoself-inducedcongestionforashortdurationwhentheper-packetrateishigherthantheavailablecapacity.Bymeasuringtherelativequeuingdelayswithinsuchachirp,thesendercanestimatethecurrentlyavailablebandwidth,andusethisinformationtoadapttheaveragesendingrateofsubsequentchirps.Furthermore,thisschemeismorerobusttonoisethanotherproposeddelay-basedmechanism.Rateestimationbasedonchirpingwasproposedin[14]forthepathChirpbandwidthestimationtool.The rstpro-posaltousechirpingcontinuouslyforcongestioncontrolwasTCPRAPID[9].WithRAPIDcongestioncontroltheratesofthepacketswithinachirpareselectedinawaythattheaveragerateconvergestowardstherateestimationofapre-viousstream.Thisutilizesthesparecapacityonanetworklink.ThealgorithmissummarisedinAppendixA.How-ever,[9]recognisesthatimplementationinaproductionop-eratingsystemwillbeneededtoanswerquestionsthattheirsimulationsinns-2cannotaddress.Wehavethereforeimplementedchirpingandassociatedcongestioncontrolinthenetworkstackofaproductionop-eratingsystemkernel.Thispaperpresentsthechallengesweencounteredandreasoningforthechoiceswemade.NotethatthispaperisnotanevaluationoftheRAPIDcongestioncontrolalgorithm.Wecertainlyimplementeditandwemakeafewremarksaboutit.Butwemerelyim-plementedRAPIDasaninitialplaceholderalgorithmtoex-erciseourchirpingcode.Eventuallywewouldliketobuildourowncongestioncontrolalgorithmaroundchirping,but rstwehavetocheckwhetherchirpingisfeasible.Evenifitprovesfeasibletoimplement,wethenhavetocheckwhetherchirpingcanbemadesucientlyrobustagainstnoise;butthatisbeyondourpresentscope.Thegoalofthispaperistoprovideabaseonwhichotherscanbuildresearchinthisarea.Tothisendweo erthreedi erenttypesofcontribution:Structuredtheproblemspace:Wehaveidenti edthati)rateadaptation,ii)rateestimationandiii)adaptingchirpparametersarethreeindependentsub-problems.Asolutiontoonecanbeswappedoutwithouta ectingtheothers.Identi edchallenges:Thesecanbedividedintotwosets: 1.Implementationchallenges:WecannotuseACK-clockingtodeterminewhentosendouteachpacketwithinachirp.WithACK-clockingthearrivalofeachTCPac-knowledgement(ACK)triggersmoststatetransitions,thereforeitisaconsiderablechallengetoreplaceit.In-steadeachpacketreleasehastobeseparatelytimed,whichcouldcreateaheavyinterruptprocessingbur-den.Moreover,chirpingdoesnotneedtheconceptofaCongestionWindow(CWND)either.However,ratherthanmodifyallthemechanismsthatusetheCWND,e.g.fastretransmit,wetrackanequivalentoftheCWND|acountofthepacketsallowedin-\rightinoneRTT.2.Protocoldeployabilitychallenges:Fordeployability,wewouldratheronlyhavetomodifythesender,treat-ingreceiverchangesasanoptimisation.Butwebe-lievetheprotocolweneedforcomparingone-waydelaymeasurementscannotworkunlessnewTCPreceiverbehaviourcanbenegotiated.Inventedsolutions:Forexample:i)Weproposealinearprogressiontoderivetheinter-packetgapsthatcanbeim-plementedwithintegerarithmetic,inordertolimitkernelcomplexity;ii)Wefoundaneasywaytoimprovethepre-cisionofone-waydelaymeasurementswithouttighttimingconstraintsonsentpackets;iii)Wediscussthepossibilityofholdingchirpidenti ersassoftstateinpacketheadersratherthanashardstateatthesender.Therestofthepaperisstructuredasfollows:Section2explainschirping.Section3describeschallengesintheim-plementationandinprotocolandalgorithmdesignandwhyweaddressedthemthewaywedid.Section4presentspre-liminaryresultsanddiscussestheopenissueswhenusingchirpingforcongestioncontrol.InSection6wedrawcon-clusionsandoutlinethefurtherresearchthatwillbeneeded.2.CHIRPINGASABUILDINGBLOCKFORCONGESTIONCONTROLAchirpisalogicalgroupingofNpackets.Withinachirpeachpackethasahigherratethanthepreviousone.Nor-mallythebitswithinapacketofsizeParesentatline-speed.Therefore,di erentpacketratesarerealisedbycontrollingtheinter-packettimegaps(Fig.1).Achirpisthusase-quenceofpacketswithdecreasinginter-packettimegaps.Theaveragerateravgofawholechirpisthesumofallthepacketsizeswithinit,dividedbythesumofallthegapsbetweenpacketdeparturetimes,i.e.dividedbythedurationofthewholechirp.Whenusedforcongestioncontrol,alldatapacketsarewithinchirps.Buttheaveragerateofeachchirpisarrangedtotracktheintendedsendingrateofthecongestioncontrolalgorithm.Chirpingshouldguaranteethattheimpactofprobingforhigherratesonthenetworkislimited.Packetswithlowerandhigherratesthanthechosenaverageratecanbesentoverabriefduration.Thusonesinglechirpcanprobeforawiderangeofpossiblesendingrates.Therangeofratesprobedforinacertaintimecanbecontrolledbythespreadoftheinter-packetgaps,andthenumberNofpacketinachirp.Bymonitoringtherelativequeuingdelaysofeverypacketinachirptheavailablebandwidthcanbeestimated.Thenominalqueuingdelayofpacketniscalculatedusingitssendingandreceivingtimestamps,respectivelytssndandtsrcv,asfollows:qn=tsrcv tssnd:(1)Thenominalqueuingdelayincludespropagationdelaythatstaysconstantforlongperiodsorvariesonlyslowly.Weareonlyinterestedinthegrowthinqueuingdelaybetweenpacketsqn=qn qn 1,whichcancelsoutanyconstanto setineachqn.Ifthesendingrateofasequenceofpacketsexceedstheavailablecapacityofthebottlenecklink,thepacketswillexperienceincreasingdelayasthequeueinthenetworkdevicegrows.Allsubsequentpacketsofachirpwillqueueupbehindeachother(self-congestion)andone-waydelaywillcontinuetoincrease.Fig.2showstypicalobservationsofqueuingdelaytakenfromoneofoursimulationruns.Itshowspersistentlyin-creasingvaluesattheendofachirp,startingfromacertainpacket.Toa rstapproximation,thesendingrateofthispacketrevealstheavailablecapacity.Therestofthedia-gramandabetterapproximationwillbeexplainedlater.Thustheavailablecapacityisnotonlyestimatedbasedonthequeuingdelaymeasurementofapairofpacketsbutonseveralinarow.Moreover,thosedelaysareself-inducedwhichmeansthatseveralpacketsaresentatatoohighrate,todeliberatelybuildupaqueueforashortperiodoftime.Subsequentlythischaracteristicdelaysignaturecanbecorrelatedwiththepre-setinter-packetgapssent-out.Incontrasttootherdelay-basedestimationmechanismstheuseofself-inducedcongestionshouldbemorerobusttonoise. \n \rFigure1:Multi-rateprobestreamswithN=5.Theinstantaneousavailablebandwidthcanchangeoverthedurationofachirp,ascompeting\rowsvarytheirrate.ThepathChirppaperproposesawaytoestimateaweightedaverageoftheavailablebandwidthoverthedurationofeachchirp.Instantaneouslyloweravailablecapacityisdetectedasanincreaseinqueuingdelayinthemiddleofachirp,whichthendecreasesbeforetheend(labelledexcursioninFig.2).To lteroutnoise,asuchanexcursionisincludedintheavailablebandwidthestimationifitfollowsanincreas-ingtrendforatleastLpackets.Inaddition,achangeinqueuingdelayisonlycountedaspartofanincreasingtrendifitislargerthan1=Fofthemaximumriseinqueuingde-layinthecurrentexcursion.Followingtests,thedefaultsproposedbypathChirpforthesevariablesareL=5andF=1:5.Forthe nalbandwidthestimationtheaverageofthebandwidthestimationofeachexcursionweightedbyitsdurationisused.ThepathChirppaperadmitsthatmoreso-phisticatedapproachesmaybepossibletoimproveaccuracybutinitiallywere-usetheirsimplealgorithm.3.ACHIRPINGIMPLEMENTATIONINTHELINUXKERNELInordertorealisechirpingwithinTCPcongestioncon-trol,weimplementedaRAPID-likecongestioncontrolin 140 160 180 200 220 240 260 280 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 one-way delay [ms]packet number previous chirpexcursion Figure2:Queuingdelayofa32packetchirpwithTCPcrosstrac.theLinuxkernelversion2.6.26.Thispaperfocusesontheimplementationofchirpingasabuildingblockforconges-tioncontrolbasedonper-packetsend-outtiming.WefocuslessontheRAPIDratecontrolalgorithmitself,whichre-searchersmightchoosetoswapoutfortheirownalgorithm.Thefollowingfourfunctionblocksareneededtorealisechirp-ingwithinacongestioncontrolprotocol:1.Feedbackforone-waydelaymeasurement:Theprotocolextensionsneededanddi erentsolutionfordeploymentarediscussedinthefollowingSubsection.2.Rateestimation:Thealgorithmtoevaluatethefeed-backinformationofonechirpbasedonpathChirp.PseudocodeoftheimplementedalgorithmisattachedinAppendixB.3.Rateadaption:Thechosencongestioncontrolal-gorithmevolvestheaveragesendingrateofthenextchirpbasedonitsownalgorithmusingtheavailableca-pacityestimatedbythepreviouschirp.FurthermoreCWNDshouldalsobeupdatedtoavaluethatallowstheintendednumberofpacketstobesentinoneRTT.4.Inter-packetgapcalculation:Toarriveatthecho-senaveragesendingratetheinter-packetgapsforthenextchirpneedtoberecalculated.Wedevelopedourownalgorithminordertosimplifykernelimplementa-tion(seex3.3).x3.2givesanoverviewofthetimingframeworkforsendingoutpacketsandthestructureoftheimplementationascon-gestioncontrolkernelmodulegivingtheinterdependenciesbetweenthecomponents.3.1Sender-sideDelayMeasurementbasedonTCPTimestampOptionTheprecisionofourmeasurementsdoesnotrelyonthetime-gapsbetweenpacketsconformingexactlytothepre-calculatedchirptime-gaps.Instead,wetimestampapacketwhenitisactuallysentandusethisinformationforanyfurthercalculations.Soevenifapacketexperiencesdelayelsewhereinthelowerlayersofthesender,thealgorithmisdesignedtoberobust,aslongasalltheactualsendinggapsdecreasewithinachirp.Wechosetolocatealltiminganalysiswiththetimingcon-trolthatmustnaturallysitatthesender-side.Weonlygavethereceiverthetaskoffeedingbacktimestampinformationforthedelaymeasurements.WeadoptedTCPtimestamps(TS)forthispurpose,whichareactivatedbydefaultintheLinuxkernelandinmostmodernoperatingsystems.Table1showsthestructureoftheOptionheaderwhichisaddedtotheTCPheadersineitherdirection.ThisTCPoptionpro-videsa32bitTSValue eld,whichissupposedtocarryapacketsend-outtimestamp.ThisgetsechoedbythereceiverusingtheTSEchoReply eldintheotherdirectionalongwiththesendingTSValueoftheACKitself.Bycompar-inganechoedtimestampwiththecurrentsystemtime,thesendercanestimatetheRTT.Table1:TCPTimestampOptionheader. Kind=8 10 TSValue(TSval) TSEchoReply(TSecr) 1 1 4 4 When(mis)usingtheTCPTimestampOptiontoestimatetheone-waydelay,thesend-outtimestampintheTSValue eldoftheACKiscomparedtotheechoedtimestampofthesender,asalsousedbyTCP-LP[11].ThetimegapsbetweensentpacketsarereconstructedfromtheechotimestampsofthecurrentACKandthepreviousACK.Aschirpingisonlyinterestedinchangesindelay,clocksynchronisationisirrelevant.Butthetimestampsofsenderandreceiverdoneedtousethesametimeresolution.TheTSEchoReplyisactuallyspeci edtojustechothe32bitsintheTSValuenomatterwhatthosebitsmean.Thusthesenderneedstoknowthereceiver'stimeresolution.IntheLinuxkernelTCPtimestampsnormallyhavemillisec-ondresolution.Torecognizeincreasesindelayinahigh-speedscenariothisresolutionisnotsucient.ThustousechirpingwithTCPTimestampsintheLinuxkernelahigherresolutionandsomekindofresolutionnegotiationareneeded.TheLinuxkernelprovidesso-calledhighreso-lutiontimers(hrtimers)withnanosecondresolution.Thisgivessucientlysmallinter-packetgapsfortoday'snetworkspeedsandnegotiationwouldallowscalingtohigherspeedsinfuture.Toestimateone-waydelayweneedtoassociatethesentandreceivedtimestampsofonepackettogether.Butthede-layedACKmechanism[3]confusesmatters.MissingACKsarenotaproblem|rateestimationcouldbedonewithonlytheACKsinachirpthatareavailable.TheproblemwithdelayedACK'sisthattheTSEchoReplyissupposedtore\recttheTSvalueoftheoldestpacketitacknowledges,notthemostrecent(inordertogiveaconservativeestimatefortheRetransmitTimeOut).ThentheEchoReplytimes-tampinanACKistypicallynotthesendingtimeofthepacketthattriggeredtheACK,butthatofanearlierpacketinstead.DelayedACKsareonbydefaultintheLinuxkernelandnoprotocolisavailableforaTCPsendertogetthereceivertodeactivatethem.Inourimplementationwehavemanu-allydeactivateddelayedACKsreceiver-side.Butifchirpingprovesuseful,anegotiationprotocolat\rowstartcouldturno delayedACKs.However,givendelayedACKsserveapurpose,ratherthandeactivatingthemitwouldbebettertonegotiatealterna-tivetimestampsemantics.Ideallyformorepreciseone-waydelaymeasurement,aswellasasendertimestamp,there-ceiverwouldtimestampthepacketonreceptionandechobothsentandreceivedtimestampsinanACK.Thiswould \n \r\n\n\n \n \n \n \n \n \n \n\n \n ! " \n \n \n \n \n \n\n \n # $ % & ' \n \n\n \n Figure3:ChangesintheTCPnetworkstack.intrinsicallyassociatesendandreceivetimestampsforthesamedatagraminoneACK.Experimentsusingourtestbedtoinvestigatehowmuchthiswouldimprovebandwidthes-timationarework-in-progress.Inourcurrentimplementationwerememberthestartofeverychirptokeepchirpanalysissynchronisedevenwhene.g.ACKsarelost.Tostorelessstateatthesenderinfuturewewouldliketoattachthisinformation|achirpID|tothepacketheaderitself.Onepossibilityistoexploittheopaquesemanticsofthe32bitTCPtimestampoptionTSValue eld.IfweplacedachirpIDinthis eld,itwouldbeblindlyechoedatreceiver-side.ThealternativeofanewTCPoptionwouldbecleaner,butwouldraisedeploymentissues.Twoofthesenewprotocoldesigns|timestampresolu-tionanddisablingdelayedACK's|couldbeimplementedwithnegotiationduringtheconnectionhandshake.AddingachirpIDandaddingareceivetimestamprequirefurthermoreconsideredprotocoldesign.Toenhanceaccuracy,hardwaretime-stampingcanbeused.WithHardwaretimestampingtheTSval eldoftheTCPTimestampOptionheaderisrewrittenbythenetworkinter-facedevicejustbeforesendingthepacket.Thesamemech-anismisoftenusedwithIEEE1588-2008/IEEE802.1ASTimeSynchronizationMessageswhicharealreadysupportedbyalargenumberofnetworkcards.TheLinuxkernelpro-videsaninterfacetoactivatehardwaretimestamping.3.2ImplementationStructureAsintendedbytheLinuxkerneldesignweimplementedthechirpingalgorithmsbasedonthekernelcongestioncon-trolmoduleinterface.Chirpingrequirescontrolledtimingofeachpacketsend-outtime,soweintroducedanadditionalTCPtimerbyusingthekernelTCPtimerframework.AllchangesaredisplayedinFig.3.Whennewdataareavailablethesend-outisblocked(1)untilthechirpingtimerexpires(2).Wheneverapacketissent,thenextinter-packetgapisobtainedtoresetthetimer(3).Aseverysinglepacketsent-outistriggeredbyatimerinteruptthisdesignwillleadtoanincreasednumberofcontextswitchesbetweenuserandkernelspacethatcanslowdownthesystem.Weplantofurtherinvestigatethisandtheimpactofotherprocessingdelaysinahigh-speedtestbed.Butweassumethatinfuturethesend-outtimingcanberealizedcompletelyinhardwareiftheuseofchirpingforcongestioncontrolcanbeshownasapromisingapproach.Thesimpli edpseudocodeforthethreemethodsimple-mentedinthecongestioncontrolmodulerealisingthechirp-ingalgorithmisdisplayedbelow.cong_avoidandmin_cwndaremethodsalreadyintendedforcongestioncontrolbytheLinuxkernelmoduleinterface.Withreset_timerweex-tendedthecongestioncontrolmoduleinterfacewithanewmethod.min_cwndiscalledwhenalosseventoccurs.Inthiscasewehalvetherate,recalculatetheinter-packetgapsandstartanewchirp.Themechanismtoensurethatthisisonlydoneonceforeveryrecoveryeventisnotshowninthepseudocode.cong_avoidiscalledforeveryACKpacketreceivedasitwasoriginallyintendedtoopentheCWND.Weuseittostorethechirpingfeedbackinformationandadapttheaveragesendingrateforthenextchirpwhenfeedbackinformationofawholechirpisavailable.First,thecurrentavailablerateisestimatedbyestimateRate().Thenthecongestioncon-trolalgorithmtodeterminethenewaveragesendingrateiscalled.Inathirdstepthenewinter-packetgapsarecalcu-lated.Finally,thisnewsend-outpatternwillbeadoptedinre-set_timer()inthenextchirp.reset_timer()iscalledwheneverapacketissentout.Itreturnsthenextvaluethatthechirpingtimershouldbesetto. Algorithm1Chirpingcongestioncontrolmodule. cong avoid():fcalledforeveryACKgrememberone-waydelayrememberreconstructedsend-outinter-packetgapupdateCWNDiflastpacketofchirpthengapest=estimateRate()fest.availablebandwidthggapavg=adaptRate(gapest)fnewaveragerategcalculatenewinter-packetgapswithEqn.3endifreset timer():fcalledforeverydatapacketsendgiflastpacketofchirpthenstartnewchirpremember/increasechirpidenti erendifreturnnextinter-packetgapmin cwnd():fcalledforeveryrecoveryeventafterlossggapavg=2gapavgf=halveravggrecalulateinter-packetgapswithEqn.3startnewchirp 3.3AlgorithmforInter-packetGapCalcula-tionOurimplementationisfullybasedoninter-packettimegapsinsteadofrate.Initiallyweassumeequalsizedpacketwhichisusuallythecasewhensendingastreamofdatawithoutblocking.Ifpacketscannotbesentcontinuously,be-causetheapplicationisblockingfordata,usuallytheavail-ablebandwidthcannotbefullyusedanywayandbandwidthestimationbasedonchirpingmightnotbeappropriate.TokeeptheLinuximplementationassimpleaspossible,thechirpsizeNmustbesettoavaluethatisanintegerpoweroftwo.Initiallyinourimplementationitishard-codedat32(=25),givenexperimentsin[9]recommendedN=30.Tosimplifykernelcalculatione ort,wecalculatethetimegapsizesusingaharmonicprogressionofratesofslightlywiderrangethanthegeometricprogressionof[9]. 0 1 2 3 4 5 6 0.5 1 1.5 2 2.5 3 per-packet rate [Mbps] start-up phasechirp 0 1 2 3 4 5 6 0.5 1 1.5 2 2.5 3 per-packet rate [Mbps]Time [s] Chirping at receiver-sidechirp Figure4:Per-packetrateofonecontinuousRAPIDtransferwith32pkt/chirpoveran1Mbit/slink.Thenthegapsizebeforethei-thpacketcanbecalculatedbythesimplelineardecreaseofgapi=gapi 1 gapstep=gapi 1 2gapavg N(2)Usingthiswouldcausetheithgaptobegapi=2gapavg N(N i+1)(3)withi=1:::N 1andgap0=gapavg.Withthisformulapairsofgapsfromeachendofthechirpcanberegardedasgroupedto2gapavgbesidesgap1andgap2aswedonotusethemaximumrangeforthehighrates.Thusthiscalcula-tionleadstoaslightlyloweraverageratethantheestimatedoneandtoaprobingrangefrom1 2ravgtoN 4ravg.Anex-amplefortheresultingper-packetrateisshowninFig.4.Theupperdiagramshowstheper-packetsend-outratesofchirpsofaRAPIDconnectionprobingaroundthebottle-neckcapacityof1Mbit/sonanemptylinkwithaprobingrangeof0.5{6Mbit/s.Fig.4alsoshowstheSlow-Start-likephaseproposedforRAPIDcongestioncontrol.DetailsofhowRAPIDuseschirpingduringSlowSstartarerelegatedtoAppendixA.4.PRELIMINARYRESULTSWehavebeenrunningsimulationwiththeIKRSimulationLibrary[7]andtheNetworksimulationCradle(NSC)[8].Thisenabledustousethekernelcodewithinsimulationcomponents.Today,theNSCcanhandleLinuxkernelcodeonlyuptoversion2.6.26.Weareplanningtoportourim-plementationtoanewerversionandrunreal-lifetestsinanenvironmentupto10Gigabit/s.ThecurrentsimulationresultsarelimitedinspeedduetotherestrictionsofthetimingintheNSCtoamillisecondsresolutionashrtimersarenotyetavailable.Duetothislimitedtimeresolutionourscenariosarebasedona1Mbit/sbottlenecklinkwithlargeraccessbandwidthofthesendertosendpacketswithaprobingrateupto12Mbit/s.Fig.4alsodisplaysthereceiver-sideper-packetratewhichislimitedbythebottleneckbandwidthof1Mbit/s.Notewiththissimulationsthereis100msone-waytransmissiondelayonthelink.Allpacketssentwithahigherratethanthe 0 0.2 0.4 0.6 0.8 1 0 10 20 30 40 50 60 Throughput [Mbps] RAPID TCP 0 10 20 30 40 50 60 0 10 20 30 40 50 60 Queue lengthTime [s] Figure5:ThroughputofRAPIDandTCPcrosstracstartingat10seconds.maximumbottleneckspeedbuildupaqueueintherouter.Evenwhenthenextchirpstartswithlowerper-packetsratesittakesawhileuntilthequeueemptiesagain.ThoseinitialdelayscanalsobeseeninFig.2.4.1RAPIDCongestionControlOurinvestigationoftheRAPID-likeimplementationdis-coveredseveralproblemsinthedesignofRAPIDcongestioncontrol.Thisknowledgecanbeusedtogetabetterunder-standingofhowtodesignarobustcongestioncontrolbasedonchirping.RAPIDcongestioncontroldirectlyusestheavailableband-widthestimatedbyapreviouschirpasravg.Incommonwithmanypurelydelay-basedapproaches,RAPIDcannotcompeteforanequalshareofcapacitywithloss-basedap-proaches,suchasstandardTCP.BydesignRAPIDonlyaimstoscavengeanybandwidththeyarenotalreadyusing.Nonetheless,RAPIDdoesbuildupsomeadditionalshort-timedelayintherouterqueues,whichin\ruencestheTCP\rowthroughput.Forourimplementation,weproposethatallinter-packetgapsizes,whichhavebeencalculatedwithnanosecondres-olutionasapreparationforahigh-speedtestbedusage,shouldberoundedtothenexthighervaluewhensettingamillisecondresolutiontimerforNSCusage.Thiswillslightlyunderestimatetheavailableratebutavoidsqueuebuild-up.Inthegeneralcasewhenaqueuebuildsupbecauseofpermanentoverestimationthiswillnotberecognisedbythedelaymeasurementswithinachirp,asthequeuestayscon-stantordecreaseswhilesendingpacketswithlowerrates.ThusRAPIDcongestioncontrolwillnotrecogniseitsownimpactonlong-termqueuinguntilalossoccurs.Weexpectamoresophisticatedsolutioncanbefoundheretogetclosetotherightvaluewithoutriskingoverestimation.RegardingthecaseinRAPIDdesignwheremultipleRAPID\rowsarecompeting,thereneedstobesomeadditionalmech-anismtoconvergetotheintendedbottleneckshares.With-outsuchamechanismtheRAPID\rowwhichhasalreadyachievedalargersendingratewillgetearliernoti cationifsparecapacitybecomesavailable.In[9]itisproposedtoequalisetheratestowhichRAPIDsendersconvergeby introducingaparameter ,asgiveninEquation8inAp-pendixA.Thisissupposedtoleadtoanequalrateinthesameamountoftime( ms)forcompetingRAPID\rowswithdif-ferentstartingrates.OfcourseRAPIDcanonlyconvergetoanequalratedistributionif issettothesamevalueforallcompetingRAPIDconnections.Thisisverysen-sitivetovariationsindi erentimplementations.Further-more,if islargerthanthetimeneededtosendonechirp,theproposedformulawillallowanincreaselargerthantheestimatedavailablerate.Thiswilldisturbcompeting\rowsstrongly.Consequentlyinoursimulationinalowspeedsce-nariothisapproachledtoverylargeinter-packetgapsandrespectivelyaverylowaveragerateofthecompeting\row.InfactoneRAPID\rowendedupsendingonly32packetsoverseveralsecondsasitcouldnotadaptwithinonechirp.Fig5showsthethroughputofaRAPIDconnectionandaTCPconnectioncompetingfor1Mbit/sbottleneckcapacitywithadrop-tailqueueof50packetqueuesizeandabasede-layof200msperroundtrip.TheTCPconnectionstarted10secondsaftertheRAPIDconnectionandgoesonuntil20Mbitofdatahasbeentransmitted.Duetotheintro-ductionoftheintra-protocolconvergencefactor (heresetto268msforimplementationreasons)theRAPID\rowdoesnotbacko completelybutwheneveralosseventoccursthebandwidthsharebetweentheTCPandtheRAPID\rowchanges.AfterterminationoftheTCPconnection,RAPIDconvergesslowlytothemaximumrate.Withoutsmoothingbyusingthe value,theresultsarequiteunpredictable.Werecommendthatallimplementationsshouldnotbere-quireidenticalparametervaluesifdependenciesonnetworkcharacteristicsexist,butinsteadtheparametrisationshouldbemadeadaptive.WealsoarguethatRAPIDcongestioncontrolshouldbecharacterisedasascavengingapproachemulatinglessthanbest-e ortservice,ratherthananalgo-rithmintendedtocompeteonaparwithTCPcongestioncontrol.Amorescalablecongestioncontrolshouldnotonlyrelyonchirpinginformationbutusethisfastfeedbackinfor-mationtoadapttheincreaseofthesendingratemoreappro-priatelytotheactualnetworkstate.Therateadaptionitselfstillneedstofollowaschemethatcanleadtoconvergenceincapacitysharingwithtransfersusingdi erentcongestioncontrolmechanismsordi erentparametrisations.4.2NextStepsFutureworkisstillrequiredtoinvestigatetheimpactofshorttermprobingdelaysonthequeueburstinessandthein\ruenceofalargeaggregationofprobingchirpsonthebasequeuelength.Weexpectthateverysinglechirpingtransfershouldstillbeabletocorrelateitsownsend-outpatternwiththereceiveddelaymeasurements.Inreturnthechirpinginformationcanbeusedtoreducetheconges-tioncontrolovershootandrespectivelythemaximumqueuelengthwhilestillprovidingafastaccelerationinhighspeedenvironments.Usingchirpingwithretransmissionoriftoofewpacketsareavailableneedsfurtherinvestigation,aswellasissuesonreordering,idleperiodsorincompletefeedbackinforma-tion.Furthermore,adaptationofthechirpingparametersthemselves,e.g.aselectionofthechirpsizeNbasedonthesendingrateorobservedchangesinrate,couldleadtohigheraccuracyofthechirpinginformation.5.RELATEDWORKNexttoRAPIDcongestioncontrol,NF-TCP[2]isanotherproposaltousechirpingforcongestioncontrolincombina-tionwithECN-basedcongestionavoidancetechniques.In-steadofsendingthedatapacketsgroupedinprobingstreams,NF-TCPusesseparateprobingpacketstode neanewsend-ingrate.InreturnNF-TCPbackso earlybasedonlow-priorityECN-markingstoremainTCP-friendly.Theimple-mentationofchirpingwasrealizedinuser-space.Regardingtheimplementationofrate-basedcongestioncon-trolapproaches[4]proposesthereintroductionofACK-clockingandwindow-basedrateadaptiontoovercomeimplementa-tioncomplexity.Wecannotusethisapproachasanexactper-packettimingisneeded.Asimilarapproachtoourim-plementationontimer-basedpacketspacingcanbefoundforTCPPacing[12][10][1].PSPacer[15],asanotherreferencetorealizeinter-packetspacing,followsadi erentapproach,whereso-calledPAUSEpacketsareinterleavedtoincreasethesendinggapsbetweentwodatapackets.[13]usesTCPPacingandthepacket-pairtechniquetoimprovetheTCPstart-up.PacedStart[6]introducedasent-outpacketspac-inginSlow-Startforbandwidthestimation.HybridSlowStart[5]usesthefactthatpacketburstsinSlow-StartgetpacedoutinthenetworkbymonitoringtheACKtraindu-ration.6.CONCLUSIONANDOUTLOOKInthispaperwepresentedwhatisneededtoimplementchirpingasabuildingblockforcongestioncontrolintheLinuxkernel.Thenanosecondresolutionprovidedbykernelhrtimersissucientfortoday'sspeed,butinitialnegotia-tionabouttimerresolutionandeitherreceivetimestampingornon-delayedACKingneedstobeaddedtotheprotocoldesign.Weraninitialexperimentsusingourchirpingimplementa-tiondrivenbytheRAPIDcongestioncontrolalgorithm.Wecanalreadyconcludethatchirpparametersoughttoadapttoprevailingconditionsandthattheavailablebandwidthestimatesfromchirpingshouldbeusedinadditiontoothernetworkstateinformation.WehavealsonotedthatRAPIDcongestioncontrolwouldbebettercharacterisedasascav-engerprotocol,asitisnotdesignedtotakecapacitysharefromprotocolslikeNewRenothatareloss-based.Weareplanningtofurtherstudytheimpactofchirp-ingonburstinessandqueuelengthinahigh-speedtestbed.Thiswillalsoprovideinsightintothedependencyoftheac-curacyofthechirpinginformationontimestampresolution.Aschirpingprovidesfasterfeedbackthantoday'ssolelyloss-basedmechanisms,ourgoalistousetheavailablebandwidthestimatestoenablemorescalablerateadaptionwithmini-malovershoot.7.REFERENCES[1]A.Aggarwal.Understandingtheperformanceoftcppacing.InProc.2000IEEEINFOCOMConference,pages1157{1165,2000.[2]M.Arumaithurai,X.Fu,andK.K.Ramakrishnan.NF-TCP:NetworkFriendlyTCP.In17thIEEEWorkshoponLocalandMetropolitanAreaNetworks(LANMAN2010),May2010.[3]R.Braden.Requirementsforinternethosts{communicationlayers.RFC1122,IETF,October1989.[4]S.-H.ChoiandM.Handley.DesigningTCP-FriendlyWindow-basedCongestionControlforReal-time MultimediaApplications.InInproceedingsofPFLDNeT,2009.[5]S.HaandI.Rhee.HybridSlowStartforHigh-BandwidthandLong-DistanceNetworks.InPFLDNeT'08,2008.[6]N.HuandP.Steenkiste.ImprovingTCPStartupPerformanceusingActiveMeasurements:AlgorithmandEvaluation.InProcInt'lConfonNetworkProtocols(ICNP'03).IEEE,November2003.[7]IKR,UniversityofStuttgart.IKRSimulationandEmulationLibrary,December2009.http://www.ikr.uni-stuttgart.de/Content/IKRSimLib/.[8]S.JansenandA.McGregor.SimulationwithRealWorldNetworkStacks.InProc.WinterSimulationConference,pages2454{2463,2005.[9]V.KondaandJ.Kaur.Rapid:Shrinkingthecongestion-controltimescale.InINFOCOM2009,IEEE,pages1{9,apr.2009.[10]J.Kulik,R.Coulter,D.Rockwell,andC.Partridge.PacedTCPforHighDelay-BandwidthNetworks.InIEEEWorkshoponSatelliteBasedInformationSystems,1999.[11]A.KuzmanovicandE.W.Knightly.Tcp-lp:low-priorityserviceviaend-pointcongestioncontrol.IEEE/ACMTrans.Netw.,14:739{752,August2006.[12]D.Lacamera.TCPPacing.http://danielinux.net/index.php/TCP Pacing,October2010.Website.[13]C.Partridge,D.Rockwell,M.Allman,andR.K.J.P.Sterbenz.ASwifterStartforTCP.Technicalreport,BBNTechnologies,2002.[14]V.J.Ribeiro,R.H.Riedi,R.G.Baraniuk,J.Navratil,andL.Cottrell.pathchirp:Ecientavailablebandwidthestimationfornetworkpaths.InInPassiveandActiveMeasurementWorkshop,2003.[15]R.Takano,T.Kudoh,Y.Kodama,M.Matsuda,H.Tezuka,andY.Ishikawa.Designandevaluationofprecisesoftwarepacingmechanismsforfastlong-distancenetworks.InInProceedingsofPFLDNet2005,2005.APPENDIXA.RAPIDCONGESTIONCONTROLRAPIDcongestioncontrolisdesignedtoacquire\theavailablebandwidthwithinonlyafewRTTs".Furthermore,itaimsto\achievingfairnessamongco-existingRAPIDtransfers"and\re-mainingTCP-friendly"[9].Thefollowingenumerationwillgiveanshortoverviewabouttheproposedalogorithm:1.Rate-basedpackettransmission(atthesender):Sendmulti-rateprobestreamsofNpacketswithdecreasinginter-packetgapsofti=P=ri 1witht0=tavgandravg=N 1 1 r1+1 r2+:::+1 rN 1(4)foralli�1,ri�ri 1.2.AvailableBandwidth(AB)estimationanalysis(atthereceiver):ImplementpathChirpbandwidthestima-tionbasedonself-inducedcongestion.Ifqiisthequeuingdelayexperiencedbythei-thpacketandqk=0;ifrkABqk�qk 1;otherwise;(5)theavailablebandwidthestimationABestwillbegivenbythesmallratewhereself-incudedcongestioncanbeob-served.ABestofthemostrecentp-streamwillbeforwardedtothesenderineverynextACK.3.Transmittinginanon-overloadingresponiveman-ner:Setravg=ABestfornextp-streamtonotexceedbot-tleneckcapacitybutsimultaneousprobingforde-/increase.4.Setting[r1;:::;rN 1](speedingupthesearchpro-cess):Implementmultiplicativerelationri=mi 1r1(6)with1iNandr1=mN 1 1 (N 1)(m 1)mN 2ravg(7)withN=30andm=1:07%.Thesevaluesyieldtoarangeofprobingratesfromr10:45ravgtorN 13:22ravg.5.AchievingaQuick-yet-Slow-Start:Sendonlyasinglep-streamoverthe rstfourRTT'swithN=2;4;8;16andm=2andinitializeravgto100Kbps.Thiscanprobeforupto3342Gbpsin4RTT's(andisnotmoreaggressivethanotherSlow-Startimplementations).6.Dealingwithpacketloss:Reduceravgbymultipleof0.5afterlossrecovery.7.Biasduetorate-proportionalfeedbackfrequency:EqualizetherateatwhichRAPIDsendersconvergetoABestbyravg=ravg+l (ABest ravg)(8)wherelisthedurationofthemost-recentp-stream,whichisgivenby:l=NP ravg. representsacommontimeintervaloverwhichanyRAPID\rowshouldconvergetoanincrease. isproposedtobesetbydefaultto200ms.B.AVAILABLEBANDWIDTHESTIMATIONTheimplementationofthebandwidthestimation(basedonpathChirp[14])canroughlybedescribedbythefollowingpseudocodewhereqavectorofqueuingdelayofachirpandgapavectoroftherecalculatedinter-packetsgaps:estimateRate(q,gap)fforeachpacketdoifqinotincreasingorlessthanqmax=Fthenrememberas'notanexcursion'setnewestimationgapesttogapiifpreviousexcurisonwassmallerthanLthenrememberallpacketas'notanexcursion'endifelseupdateqmaxendifendforsetgapavgtogapestforeachpacketdoifpartofexcursionthengapavg+=gapestelsegapavg+=gapiendifendforgapavg=gapavg=NgThisgoesinlinewiththemechanimsdescribedin[14]butleadstoanyeasierimplementation.