SlickPacketsGiangT.K.NguyenUniversityofIllinoisatUrbana-Champaign,USAn - PDF document

SlickPacketsGiangT.K.NguyenUniversityofIllinoisatUrbana-Champaign,USAnguyen59@illinois.eduRachitAgarwalUniversityofIllinoisatUrbana-Champaign,USAagarwa16@illinois.eduJundaLiuUniversityofCaliforniaatBerkeley,USAliujd@cs.berkeley.eduMatthewCaesarUniversityofIllinoisatUrbana-Champaign,USAcaesar@illinois.eduP.BrightenGodfreyUniversityofIllinoisatUrbana-Champaign,USApbg@illinois.eduScottShenkerUniversityofCaliforniaatBerkeley,USAshenker@cs.berkeley.eduAbstractSource-controlledroutinghasbeenproposedasawaytoimprove\rexibilityoffuturenetworkarchitectures,aswellassimplifyingthedataplane.However,ifapacketspec-iesitspath,thisprecludesfastlocalre-routingwithinthenetwork.WeproposeSlickPackets,anovelsolutionthatallowspacketstosliparoundfailuresbyspecifyingalternatepathsintheirheaders,intheformofcompactly-encodeddirectedacyclicgraphs.Weshowthatthiscanbeaccomplishedwithreasonablysmallpacketheadersforrealnetworktopologies,andresultsinresponsivenesstofailuresthatiscompetitivewithpastapproachesthatre-quiremuchmorestatewithinthenetwork.Ourapproachthusenablesfastfailureresponsewhilepreservingthebenetsofsource-controlledrouting.CategoriesandSubjectDescriptorsC.2.1[NetworkArchitectureandDesign]:Packet-switchingnetworks;C.2.2[NetworkProtocols]:Rout-ingprotocols;C.2.6[Internetworking]:RoutersGeneralTermsAlgorithms,Design,Performance,ReliabilityKeywordsReliability,failures,routing,sourcerouting,forwarding1.INTRODUCTIONTraditionalroutingprotocolsarenetwork-controlled:routesarecomputedwithinthenetwork,witheachrouterpicking,fromamongitsneighbors,thenext-hoptoeachdestination.ExamplesincludeBGPforinterdomainrout-ing,andOSPFforintradomainrouting.Analternateparadigm,source-controlledrouting(SCR),improvesPermissiontomakedigitalorhardcopiesofallorpartofthisworkforpersonalorclassroomuseisgrantedwithoutfeeprovidedthatcopiesarenotmadeordistributedforprotorcommercialadvantageandthatcopiesbearthisnoticeandthefullcitationontherstpage.Tocopyotherwise,torepublish,topostonserversortoredistributetolists,requirespriorspecicpermissionand/orafee.SIGMETRICS'11,June7–11,2011,SanJose,California,USA.Copyright2011ACM978-1-4503-0262-3/11/06...$10.00.the\rexibilityofthenetworkarchitecture.Ratherthancomputingallrouteswithinthenetwork,SCRarchitec-tures[10,20,29{31]reservesomechoiceofroutesforthesources1toselectonaper-packetbasis.TheusesofSCR'srouting\rexibilityarequitediverse.Sourcescanobserveend-to-endreliabilityproblemsandswitchtoaworkingpathwithinafewround-triptimes(RTTs);pickbetter-performingroutesbasedonobservedperformance[5,11,25];improveloadbalancesincepathselectionisner-grained[24];encouragecompetitionamongnetworkpro-viders[7];improvesecurity[28];oroptimizeforotherapplication-specicobjectives.SCRisthusapromisingapproachtoimprovethe\rexibilityofthenetworklayerinfutureInternetarchitectures.However,oneremainingproblemisthatoffastfailurereaction.Thisproblemaroseinearlynetwork-controlledrouting(NCR)protocols,whichsueredfromunrelia-bilityduringnetworkdynamics:duringthedistributedconvergenceprocess,packetscouldenter\blackholes"orloops,resultingintensofsecondsorminutesofdowntimeinInternetend-to-endpaths[14,27].Treatingthesebasicprotocolsasabaseline,twohigh-levelapproacheshavebeenproposedtoimprovefailurereaction.TherstapproachworkswithintheNCRparadigmbycomputinganalternatepathtoeachdestination(orIPprexorAS);aroutercanlocallyswitchtothealter-natepathwithoutwaitingforacontrol-planeconvergenceprocess.Packetscanthusbedeliveredcontinuously,ex-ceptfortheminimaltimeittakesforaroutertodetectfailureofoneofitsdirectlyconnectedlinksandlocallyswitchtoanalternatepath.ExamplesincludeMPLSFastReroute[23],SafeGuard[18],andFCP[17]forin-tradomainrouting,andR-BGP[16]forinterdomainrout-ing.However,thisapproachlackstherouting\rexibilityofSCR.AsecondapproachtoimprovefailurereactionistoleverageSCR'srouting\rexibility:asourcecanswitchrouteswithoutwaitingfortheInternet'scontrolplanetoreconverge.Whilethisimprovesfailurereactiontimerel-ativetothebaselineabove,thesourcestillmustwaittoreceivenoticeofthefailure.Regardlessofthemeansofnotication,thiswilltakeatleastontheorderofoneRTT,whichatInternetscaleswouldbemuchslowerthantherstapproachofusingNCRwithalternatepaths. 1Inthispaper,weuse\source"torefereithertoend-hostsortoedgeroutersactingontheirbehalf. AndintheSCRproposalsthatprovidethemost\rexibil-ity[10,30],sourcesspecifyinthepacketheaderanexplicitroute(perhapsatthelevelofautonomoussystems)ratherthanadestination,sotheNCRandSCRtechniquescan-notbeimmediatelycombined.Thegoalofthispaperistoachievethebestoftwoworlds:thefastfailurereactionofalternateroutesem-beddedwithinthenetwork,andthe\rexibilityofrouteschosenbysourcesattheedgeofthenetwork.Tomeetthisgoal,weworkwithintheSCRparadigm,butwithatwist.Insteadofspecifyingasinglepathtothedestina-tion,thepacketheadercontainsadirectedacyclicgraphthatwecalltheforwardingsubgraph(FS).Eachrouteralongthepacket'spathmaychoosetoforwarditalonganyoftheoutgoinglinksatthatrouter'snodeintheFS(optionallypreferringapathmarkedastheprimary),withnodangerofcausingaforwardingloop.Thisap-proach,whichwecallSlickPackets,allowspacketsto\slip"aroundfailuresin-\rightwhileretainingthe\rexibil-ityofsourceroutecontrol.Moreover,SlickPacketspro-videsascalabilitybenetoverNCRwithalternatepaths:ratherthanrequiringmultipleroutestoeverydestinationineveryrouter'sforwardingtable,SlickPacketsroutersneedonlylocalinformation.Ofcourse,ourapproachalsopresentsseveralchallenges.ChiefamongtheseishowtoencodeanFSwithsu-cientpathdiversityintothesmallspaceaordedbyapacketheader.WeintroducetechniquesthroughwhichtheFScanbeencodedcompactlyenoughforourmech-anismtobefeasible.Forexample,anFSprovidinganalternatepathateveryhopalongtheprimaryoccupieslessthan26bytesfor99%ofevaluatedsource-destinationpairsinanAS-levelInternetmap,andnohigherthan50bytesinallevaluatedcases.Thus,thetechniqueincursmanageableoverheadforapplicationsthatsendpacketsofmoderatetolargesize.Wealsodemonstratethroughasimulation-basedperformanceevaluationthatSlickPac-ketsachievesfailurereactionperformancethatiscom-parabletothebestofNCRarchitectures[18].Therestofthispaperproceedsasfollows.Inx2,wepresentanoverviewofSlickPacketsanditsprincipaldesignchallenges.x3givesadetailedpresentationoftheSlickPacketsdesign.Weevaluatetheperformanceofourdesignintermsofheadersizeandfailurereactioninx4.WediscussextensionsofSlickPacketsinx5andrelatedworkinx6,andconcludeinx7.2.OVERVIEWInthissection,weprovideanoverviewofSlickPac-kets,anddiscussseveralcriticaldesignchallenges.SlickPacketsisafailurereactionmechanismforSCRprotocols.IncontrasttotraditionalSCRprotocolsthatspecifyasinglepathinthepacketheader,SlickPac-ketsenablesfastrecoverywithinthenetworkbyallowingthesourcetoembedthereroutinginformationwithinthepacketheaderintheformofaforwardingsubgraph(FS).TheFSspeciesasetofpathsthatintermediaterouterscanusetoreroutepacketsincaseoffailures.Thesource,ifitdesires,candesignateoneofthesepathsastheprimarypathtobeusedintheabsenceoffailure;therestofthepathsarethentreatedasalternatepathsthatcanbeusediftheprimarypathisnotavailable.Inordertoavoidforwardingloops,SlickPacketsrequiresthattheFSbeadirectedacyclicgraph(DAG).Performingforwardinginthiswayhastwomainben-ets.First,sincethesourcespeciestheFS,ithasfullcontrolofnotonlytheprimarypath,butalsohowthenetworkforwardsthepacketwhentheprimarypathisnotavailable.Second,sincealternatepathinformationisembeddeddirectlyinthepacketheader,thenetworkcanreactimmediatelywithoutrequiringinvolvementofthesource,whichreducesthereactiontimeinpresenceoflinkfailures.Inadditiontothesetwobenets,thetaskofarouterbecomessimpler:arouterrequiresonlylocalknowledgeofitsneighbors,ratherthanneedinganalter-natepathforeverydestination(whichmayrequireinfor-mationsuchasthemulti-hominglocationsofeachhost).Insummary,SlickPacketsachieveskeybenetsofSCRarchitectures(\rexibilityinrouteselectionandscalabilityofnetworkroutingstate)whilesimultaneouslyattainingfailurereactionperformancethatiscomparabletothatofNCRarchitectureswithbackuppaths.Fig.1showsanexampletoillustratethedesignofSlickPackets.Supposethesourceswishestosendapackettoadestinationd.Thesourcehasacquired,bysomemechanismtobediscussedlater,amapofthenet-work.ItselectstheFSasshowninFig.1anddesignates(s;R1;R2;R5;d)tobetheprimarypath.NotethattheFSprovideseachnodeontheprimarypathwithsu-cientalternativessothatifalinkontheprimarypathfails,thepacketcanbereroutedtothedestination.Next,sconstructsadatapacketwiththesubgraphembeddedinthepacketheader,andforwardsitontotherst-hopR1.AtR1,thepacketisforwardedtothenext-hopontheprimarypath(R2).NowsupposethatatR2thepri-marypath'snexthopR5liesacrossafailedlink(R2;R5);thenR2forwardsthepackettoR4,thenext-hoponitsalternatepathintheFS,afterwhichthepacketcontinuestoR5andnallyd.Realizingthehighlevelideaofsource-controlledrout-ingalonganFS,however,involvesseveralkeychallenges.Weoutlinethesechallengesandoursolutionshere.Obtainingthemap.LikeotherSCRarchitecturesinwhichsourcesconstructend-to-endpaths,oursourcesre-quireamapoftheavailablelinks.WhendeployingSlick-Packetsasaninterdomainroutingprotocol,thisimme-diatelyraisesquestionsofscalabilityandpolicycompli-ance.IsitfeasibletopushamapoftheInternet,atsomelevelofgranularity,toeverysourceoratleasteveryedgerouter?Isthereanacceptablewaytobalancecontrolofnetworkresourcesbetweenthesendersandthenetworkowners?Fortunately,wecanadoptthesolutionsdevel-opedbypastwork,inparticularNIRA[30]andpath-letrouting[10],whichhaveshownhowmapsofpolicy-complianttransitservicecanbeconstructedanddissem-inatedinwaysthatcanbemuchmorescalablethantra-ditionalNCRprotocolslikeBGP.Packetheaderoverhead.ThenextchallengeistodesignanecientencodingmechanismthatembedstheFSintothepacketheaderwithminimaloverhead.Byusinglinklabelswithonlylocalsignicanceandallocat-ingeverybitcarefully,weareabletoachieveacceptablepacketheadersizesonrealisticnetworktopologies.Fastdata-planeoperations.Anotherchallengeisto sources R1 R2 R3 R4 R5destinationdPhysicalnetwork Xfailed StepI packet StepIII R1 R2 R4 R5 forwardingsubgraph(FS) StepII Figure1:OverviewoftheSlickPacketsdesign.StepI:thesourceselectsaforwardingsubgraph(FS)basedonthetopologyofthephysicalnet-work;StepII:thesourceencodesandembedstheFSinthepacketheadertoinformroutershowtoroutearoundencounteredfailures;andStepIII:routersforwardthepacketbasedontheFScon-tainedinthepacketheader.designanecientdataplaneforwardingalgorithm:theencodingandforwardingmechanismsinSlickPacketsshouldminimizenext-hoplookuptimewithoutsubstan-tiallyincreasingheaderprocessingcost,forwardingde-layand/ordesigncomplexityofmodernrouterforward-ingplanes.Fortunately,forwardingalonganFSrequiresonlylookupandpointer-incrementoperations,asinstan-dardSCRprotocols,andcanbeecientlyimplementedinpractice.Thenextsectiondiscussesourdesigninmoredetail,includingoursolutionstothesechallenges.3.SLICKPACKETSDESIGNInthissection,wepresentinmoredetailthefourmaincomponentsofSlickPackets:denitionanddissemina-tionofthenetworkmap(x3.1);selectionofaforwardingsubgraph(FS)atthesource(x3.2);encodingoftheFSintothepacketheader(x3.3);andthedataplanefor-wardingmechanismatrouters(x3.4).TheSlickPacketsapproachcouldbeappliedinmulti-plecontexts.Wedescribeherehowthedesigncanbeap-pliedtointerdomainandintradomainrouting.Thedier-encesprincipallylieinmapdisseminationanddataplaneforwarding,withthecoreapproachtakingthesameforminbothcontexts.3.1MapformatanddisseminationAsinotherSCRprotocolsinwhichthesourcecomposesend-to-endpaths[10,30],inSlickPackets,thesourcemustobtainanetwork\map"(topology)fromwhichitcanconstructpaths.Thismapisanabstractdirectedgraphinwhicheachdirecteddirectedlink(u;v)atnodenodeuisannotatedwithalabel.Thelabelisacompact,variable-lengthbitstring,whichthesourcewillusewhenencodingtheFS(x3.3)totellnodeuthatitwantsutousethelink(u;v).SimilartoanMPLSlabel,thelabelidentiesalinkonlylocallyatu,notglobally.Thus,uwillgenerallyannouncelabelsoflengthdlog2(u)ebitswhere(u)isthedegreeofu.Whatthismapcorrespondstointhephysicalnetworkandhowthemapisdisseminateddependonthedeploy-mentscenario.Inanintradomainenvironment,themapwouldcorrespondtothephysicaltopologyofroutersandlinksandcouldbedistributedviaaprotocollikeOSPForthroughacentralizedcoordinatorasin[17].Inaninterdomainenvironment,wehavetodealwiththesignicantchallengesofscalabilityandnetworkown-ers'transitpolicies.Inordertoovercomethesechallenges,webuildonsolutionsdevelopedinpastworkandbrie\rydescribethemhereforcompleteness.Basicapproach.BothNIRA[30]andpathletrout-ing[10]providesourceswithapolicy-compliantmapoftheInternet,roughlyattheautonomoussystem(AS)level.NIRA'smapassumescommoncustomer-provider-peerrelationshipsbetweenASesandallowsasubsetofvalley-freeroutes:thatis,packetstravelupachainofproviders,potentiallyacrossapeeringlink,anddownachainofcustomerstothedestination.Pathletrout-ingrepresentsthismapexplicitlyasanarbitraryvir-tualtopology,whoseedges(pathlets)representpolicy-complianttransitservice.Scalability.NIRA,whiledependentontheexistenceofatypicalASbusinesshierarchy,oerstheopportunityofvastlyimprovingBGP'scontrolplanescalability.RatherthanlearninganInternet-widetopology,eachnodelearnsits\up-graph"ofroutesthroughproviders,stoppingatthe\core"oftheInternet.Theup-graphrequiresfewerthan20entriesfor90%ofdomains[30],manyordersofmag-nitudelessthantheroughly300,000prexesthatBGPpropagatestoday.Eachdestinationstoresitsup-graphinaglobalDNS-likedatabase;toroutetoadestination,asourcequeriesthedatabaseandcombinesitsownup-graphwiththedestination'sup-graph.Thoughthere-sultingmapisasmallfractionoftheInternet,itincludesallpolicy-compliant(valley-free)routes.PathletroutingcoulduseaNIRA-styleapproachfordisseminatingthepathlettopology,oritcanbedisseminatedviaaBGP-likemechanismwithslightlymoremessagingandcontrolstate(1:7)thantraditionalBGP.SlickPacketscantakeadvantageofeitherthepathletorNIRAapproachforinterdomainmapdissemination.Thus,SlickPacketsdoesnotrequireasourcetohavecompletetopologicalknowledgeofthenetwork,butratheronlyenoughtoconstructapathandalternatepathstothedestination.WealsonotethatSlickPackets,likeotherSCRandmultipathroutingarchitectures,canbenetfromsigni-cantlyreducedrateofcontrolplaneupdates[6]comparedwithbasicsingle-pathNCRarchitectures.Thisisbecauseshort-livedfailuresneednotbedisseminatedthroughthecontrolplane,sincefailurereactionwillhappenanywayviaforwardingalongalternatepathswithoutwaitingforcontrol-planeupdates.Linklabels.Alongwiththemapitself,SlickPacketsrequireslabelsonthelinks.Routers(orASesforinterdo- main;forconveniencewe'lluse\routers"inwhatfollows)canpiggybackthisinformationwiththelinkadvertise-ments[10].Tochangealabel,arouterreadvertisesthelink.Whilereadvertisementsincreasecontroltrac,weexpectthatchangingarouter'slinklabelswillbefairlyrare,fortworeasons.First,theoperatorcouldchangeasinglelabelfromonebitsequencetoanother;however,thereshouldbelittleneedforsuchchangesbecausethela-belsarearbitraryidentierswithnosignicance.Second,theoperatormayneedtoincreasethenumberoflinksex-itingtherouter.Thismayincreasethelabellengthandrequirereadvertisementsofalloftherouter'slinklabels,creatingaperiodofinconsistencyfromwhentherouterchangesitslabellengthtowhensourcesreceivetheup-datedannouncement.However,labellengthschangeonlyonceeverytimethenumberofoutgoinglinksdoubles(orhalves)insize,whichisexpectedtobeaveryrareevent.Analternateapproachistomakelabelsself-describing:theirrstfewbitsencodethelabellength[10].Thisavoidstheneedtoreadvertiselinksafteralengthchangeandtheresultinginconsistency,butlabelsbecomeslightlylonger.SincecompactnessisimportantforSlickPac-kets,wedonotevaluatethisapproachinthispaper.Mapconsistency.Anaturalquestioniswhetherallsourcesandthenetworkmusthaveanentirelyconsistentviewofthemapatalltimes.Fortunately,thisdiculttaskisunnecessary.Therearethreepossibletypesofinconsistency.First,ifasourceusesanon-existentlabel(e.g.,thelinkhasbeenremovedoritslabelchanged),thisisequivalenttoalinkfailureandthepacketcanbere-routedalonganalternatepath.Toavoideventhisminordisturbance,routerscaninsertashortdelaybetweenannouncingalabeldeletionanditsremovalfromforwardingtables.Second,ifasourceusesalabelthathaschangedtoidentifyadierentlink,thenthepacketwillfollowanincorrectpathandwillbeunlikelytoreachitsintendeddestination.ThisissimilartoinconsistencyproblemsinbasicNCRprotocols.(UnlikeinbasicNCRprotocols,however,thepacketcannotgetintoaloopofanysigni-cantlengthbecauseonelinkintheDAGwillbeconsumedateachhop.)Toavoidlabel-changeinconsistency,routerscansimplyusenewlabelsratherthanreusingonesthathaverecentlyhadadierentmeaning.Third,asourcemightbeunawareofsomevalidlabels.Thissimplyresultsinaslightlyrestrictedsetofoptionsuntilitreceivestherelevantcontrolplaneadvertisement,asinessentiallyanyotherdistributedroutingprotocol.Thus,inallcases,inconsistencyissuescanbemitigated.3.2SelectionoftheforwardingsubgraphOnceasourcehasobtainedthenetworkmap,itse-lectsaforwardingsubgraph(FS)alongwhichitdesiresthepackettoberoutedinthenetwork.TheFSisaDAGcorrespondingtoasubsetofnodesandlinksinthenetworkmap.Thedirectededgesinformroutersofthepacket'sallowednext-hops,andacyclicityensurestherearenoforwardingloops.Additionally,foreachnodeintheFS,thesourcemaymarkoneoutgoinglinkasthepreferredprimary.Sourceshaveagreatdealof\rexibilityinhowtheychooseanFS.Forinstance,thesourcemayselectanFSthatavoidsanysinglelinkfailurealongalow-latencypri-s R1 R2 R3 R4d (a)Networkmaps R1 R01 R2 R3 R4d (b)ForwardingsubgraphFigure2:AnFSmayhavemultiplerepresenta-tionsofanetworkmapnode,toallow\backtrack-ing"withoutintroducingcyclesintheFS.marypath,avoidsnodefailures,optimizesforothermet-ricslikebandwidth,orpicksalternatepathsthatavoidsharedrisklinkgroups.Wediscusssomeoftheseusesinx5.Forconcreteness,wedescribehereandevaluateinx4howthesourcecanpickanFSthatwillminimizeprimary-pathlatencyandprovidealternatepathstoavoidanysinglelinkfailure.Asnotedbelow,accommodatingsharedrisklinkgroupsissimilar.Asources,foragivendestinationd,constructsasingle-failure-avoidingFSasfollows.First,scomputesapri-marypathPtodbyrunningashortestpathalgorithmoverthenetworkmap.Next,svisitseachlinkalongP,andcomputesthealternatepathPiitwouldpreferthepackettoberoutedalongifthatlinkweretofail.Inparticular,foreachnodeviontheprimarypath,we(a)removevi'soutgoingedgecorrespondingtoitsnexthopalongtheprimarypath;(b)computeashortestpathfromvitod,notusingtheremovedoutgoingedge;and(c)re-storetheremovededge.Incaseofanodehavingmultipleshortestpathstothedestination,thesourcemayarbitrar-ilyselectoneoftheseshortestpaths.Finally,theprimaryandthealternatepathsareassembledintotheFS.NotethattheabovealgorithmrequiresjPjrunsofDijkstra'salgorithm.Surprisingly,itispossibletoconstructapri-marypathandallthealternatepathsinasinglerunofashortest-pathalgorithm;see[12].Beyondsingle-link-failureprotection,asourcemaywanttoprotectagainstfailuresofsharedrisklinkgroups(i.e.,setsoflinksthatarelikelytohavecorrelatedfailures,suchasmultiplelogicallinksallocatedtoasinglephysi-calber).Assumingithasknowledgeofthesegroups,itcandothisbyremovingalllinksinthegroupinsubstep(a)above,andrestoringthemallin(c).Notethatthereisasubtletyinhowthetheprimaryandthealternatepathsare\assembled"intotheFS:ifwesimplytaketheunionofalltheselinksandedges,wemightcreatealoop,violatingtheacyclicityrequirement.ConsiderthenetworkmapinFig.2(a).Assumethatsdesirestouse(s;R1;R3;R4;d)astheprimarypath.Thentoescapeafailureofthelink(R3;R4),apacketlocatedatR3mustfollowthepath(R3;R1;R2;R4;d).TakingtheunionoftheseprimaryandalternatepathswouldresultinaloopR1!R3!R1.Duetosymmetry,theproblempersistsif(s;R1;R2;R4;d)istheprimarypath.Inordertoavoidsuchloops,whenaddinganalternatepathedge(u;v)totheFS,werstchecktoseeifthiswouldcausealoop.Ifso,wecreateasecondFSrep-resentationv0ofthephysicalnodev,andaddtheedge(u;v0).Thiscanbeseenas\tunneling"thepacketbackalonganalternatepath.IntheexampleofFig.2,before addingthesecondalternatepath,wecreateanewcopyR01correspondingtothenodeR1.Thealternatepaththenfollows(R3;R01;R2;R4;d),resultinginaacyclicrep-resentationoftheFSasshowninFig.2(b).3.3EncodingtheforwardingsubgraphAfterchoosinganFS,thesourcemustencodetheFSintoasequenceofbitsandplaceitinthepacketheader.SlickPacketsisagnostictotheparticularlocationthisheaderappearsinthepacket(forexample,itmayresideina\shim"headerbetweentheIPandMAClayers,inanIPoption,orinanovelheaderformatinanext-generationInternetprotocol).Therearetwokeygoalsindesigninganencodingformat:(a)minimizingthesizeofthere-sultingencoding;and(b)ensuringdataplaneforwardingoperationsaresimple.Wedesignedandevaluatedseveralencodingformatstoachievethesegoals.Inthispaperwepresenttwoencodingformats,calledDirectandDefault.Eachmayresultinasmallerencodingincertainscenariosasdiscussedbelow.Butthelatterre-sultedinsmallerencodingsizesinthenetworktopologiesweevaluatedusingthesingle-failure-avoidingFSselection(x3.2),soitisourdefault.Directformat.TheDirectformatencodestheFSdi-rectly,inthesensethattheFS'sDAGdatastructureinmemoryisessentiallydirectlyserializedintoaDAGdatastructureinthepacketheader.Theheadercontainsasequenceofnoderepresentations,eachcontainingoneormoreoutgoinglinkrepresentations;eachlinkrepresenta-tioncontainsitscorrespondinglabelandapointertoan-othernodewithintheheader,correspondingtothenodeattheotherendofthelink.Wedescribethebit-layoutofthisformatindetailin[22].Defaultformat.OnesourceofoverheadintheDirectformatistheuseofpointerswithintheheader.OurDe-faultformatavoidssomeofthatoverhead,bygroupingtogethersequencesoflabelscorrespondingtoalternatepaths,withoutneedinganexplicitrepresentationofeachnodealongthealternatepath.Thedisadvantageofthisgroupingisthatitinvolvesduplicatinglinkrepresenta-tions,similartohowadepth-rsttraversalofallpathsintheDAGcouldvisitlinksmultipletimes.Infact,thereexistDAGsthathaveexponentiallylargenumbersofpossibletraversals(thusspecifyingexponen-tiallylargenumbersofwaysthepacketcouldbeforwardedthroughthenetwork).Consequently,theDirectformatcanbeexponentiallymoreecientthantheapproachofDefaultinthemostextremecase.Ingeneral,weex-pectDirectwillbemorecompactforsituationsinwhichthealternatepathsoftensharenodeswithoneanotherorwiththeprimary.However,inthispaperwefocusontheparticularapplicationofchoosingsingle-failure-avoidingFSes.Forthatapplication,wefoundthatthesavingsfromavoidingpointersoutweighedtheduplica-tionoflinkrepresentations,sothatDefaultwassome-whatmorecompactinseveralrealisticnetworks(x4).WethereforechoosetheDefaultformatasourdefaultanddescribeitinmoredetailnow.IntheDefaultformat,theFSisrepresentedasase-quenceofsegments,oneforeachrouterontheprimarypath.Forinstance,inFig.3,theprimarypathconsistsofkhopsandS1;S2;:::;Skarethesegmentscorresponding S1 S2 ::: Sk phcodeihlengthid1:::d`Figure3:Defaultencodingformatlayout.Siisthesegmentcorrespondingtonodevionthepri-marypath.Itencodesthenode'sprimarynexthoppandalternatepath(d1;d2;:::;d`).hlengthispeciesthebit-lengthofthealternatepath,andhcodeispeciesthebit-lengthofthehlengthield.tothosekhops.Thesegmentcorrespondingtoaroutervontheprimarypathcontainsthreepiecesofinformation(seeFig.3):(a)v'snext-hopontheprimarypath;(b)thebit-lengthoftheencodingofv'salternatepath;and(c)v'salternatepath,asasequenceofnext-hoplabels.By\v'salternatepath"wemeanthealternatepathbegin-ningatvthatavoidstheprimarynext-hopfromv.(WeassumeherethattheFShastheformatofonealternatepathforeachlinkontheprimary.2)For(a),weneedtoincludetherouter'slabel(x3.1)forthegivenoutgoingedge,andsimilarlyfor(c)weincludeasequenceoflabels.Recallthattheselabelsareonlylocallyuniquetoeachnode,whichiscriticaltoachiev-ingacompactencoding,becausetheaveragenumberofneighborsofarouterinareal-worldnetworkistypicallyvastlysmallerthanthetotalnumberofroutersinthenet-work[3,13].Byexploitingthestructureofthereal-worldgraphs,weareabletoreducethesizeoftheencodingsignicantlycomparedwithglobally-uniquelabels.For(b),weusethetwoelds:hcodeiandhlengthi.Here,hlengthispeciesthetotalbit-lengthsofallthela-belsd1;:::;d`ofthealternatepath.Basedonourevalua-tion,alternatepathsareshorterthan32bitsinmostcasesandalwaysshorterthan128bits;incasesanodehasnoalternatepath,thealternatepathbit-lengthis0.Thus,forgreatercompactness,wemakethebit-lengthofthehlengthieldbevariableandstoreitinthehcodeieldusingaprex-freecode,withthehcodeibitsequences0,10,and110mappingtovaluesof5,7,and0,respectively.Theheadercontainstwoadditionalpiecesofinforma-tion.First,theSlickPacketsheaderbeginswithatwobyteeld,specifyingitsheaderlength.Second,aone-biteldon-alternate?specieswhetherthepacketistraversingalongtheprimarypathoranalternatepath,andisinitiallyfalse.Wediscussnexthowroutersusethisinformationtoforwardpackets.3.4ForwardingWenowdescribetheforwardingmechanismusedbySlickPacketsroutersfortheDefaultformat.TheinputtothismechanismistheSlickPacketsheaderdescribedinx3.3,andtheoutputistheinterfaceoutwhichthepacketwillbeforwarded. 2WhiletheDefaultformatcouldbegeneralizedtohavemultiplealternatesatarouter,orsegmentswithinseg-mentstoprovidealternatesforroutersalonganalternatepath,wedonotexplorethatgeneralizationhere;inanycase,suchapplicationscanusetheDirectformat. Uponreceivingapacket,therouterrstchecksthevalueoftheSlickPacketsheaderlength.Ifthisis0,thisrouteristhedestinationforthepacket.Ifnot,theroutercheckstheon-alternate?bittoseewhetheritisontheprimarypathoronanalternatepath.Wedescribetheforwardingoperationsforthetwocasesseparately.Routerontheprimarypath.Therouterreadstherstsegmentintheheader,whichcorrespondstoitself,andinspectstheprimarynext-hoplabelp.Ifthecorre-spondinglinkavailable,therouterdeletesthisrstseg-mentcorrespondingtoitself.Italsoupdatestheheaderlengthbysubtractingthelengthofitssegment.Thepacketisthenforwardedtothenext-hopontheprimarypathwiththenewheader.Iftheprimarynext-hoplinkisnotavailable,andthealternatepathlengthis0,thepacketisdropped.Other-wise,therouterreadsitsnext-hoplabeld1onthealter-natepath.Ifthelinkcorrespondingtod1isnotavailable,thepacketisdropped.3Ifthelinkisavailable,therouterremovesallsegmentsintheheader,replacingthembyitsremainingalternatepathlabels(d2;:::;d`).Italsoupdatestheheaderlengthappropriatelyandsetstheon-alternate?bit.Thepacketisthenforwardedtothenext-hopvialabeld1.Routeronanalternatepath.Therouterreadsitsnext-hoplabel.Ifthecorrespondinglinkisnotavailable,thepacketisdropped(or,asearlier,someotherfailurereactionmechanismisemployed).Ifthelinkisavailable,therouterdeletesitslabelfromtheheader,updatestheheaderlength,andforwardsthepackettothenext-hop.Simplifyingforwardingoperations.Theabovede-scriptioninvolvedremovingaprexoftheheader,andinthecaseofmovingtoanalternatepath,asuxaswell.Insomedataplaneimplementations,theseopera-tionsmaybecostly.Inthiscase,wecansimplyaddstartandendpointersatthefrontoftheheader,indicatingtheextentoftheremainingheader.Inanextra3bytes,wecanttwopointersthatcanpointtoindividualbitsina512-byteheader(whichisfarlargerthanweneed).Interdomainvs.intradomainissues.Inanintrado-maindeployment,wemayassumethateachrouterrunsSlickPacketsandforwardspacketsasdescribedabove.However,inaninterdomaindeploymenttheforwardingsubgraphroughlyrepresentsAS-levelpaths(asdiscussedinmoredetailin3.1).Whenthepacketisforwardedthoughanintermediatedomain,thatdomainmustfor-wardthepacketontothenextAS-levelhop.Networkoperatorsmayindependentlychoosefromavarietyofwaystodothis,forexamplebytunnelingthepacketwithMPLS,orperhapsrunningSlickPacketsinternallyaswellasinterdomain.4.EVALUATIONSlickPacketsadvocatestheideaofembeddingafor-wardingsubgraph(FS)inthepacketheader,givingroutersmultipleforwardingoptionsinordertoprovidethesourcewithsomepropertythatitdesires.WhileSlickPacketscansupport\rexibleFSselectionsthatprovidedierentguarantees,forconcreteness,thissectionevaluatestheFSselectionexempliedinx3.2,whichtargetsfastreactionin 3Oranyotherfailurereactionmechanismcanbeapplied.thepresenceofsingle-linkfailures.ThesourceconstructsaDAGcomprisedoftheshortestprimarypath,andtheshortestalternatepathforeachnodeontheprimarypathincasethatnode'soutgoinglinkalongtheprimarypathfails.Intermsofperformance,threemetricsareimpor-tant:(a)encodingsize,(b)failurereactioneectiveness,and(c)routercomplexityandpacketforwardingrates.Wepresentresultsfor(a)and(b)inthissectionanddis-cuss(c)inx7.Topologies.Weusethreenetworktopologiesinourevaluation:thelatency-annotatedtopologyfromSprintISP1239[2],with315nodesand972links;anAS-levelmapoftheInternet[13],with33;508nodesand75;001links;andthelargestcomponent,with190;914nodesand607;610links,ofarouter-levelmapoftheInternet[1].Thelattertwotopologieslacklatencyinformation;wetakealllinkstohaveequallength.WhileusingSlick-Packetsdirectlyonarouter-levelmapoftheInternetisnotalikelydeploymentscenario(duetoprivacyandscalingissues,ASesdonotpropagateinternaltopologiesgloballyintoday'sInternet),weconsiderthisextremedesignpointtoinvestigatescalingissuesofourdesign.4.1EncodingsizeSinceweencodetheFSintothepacketheader,theencodingsizedeterminesthebandwidthoverhead.WeevaluatetheresultingencodingsizesoftheDirectandDefaultencodingformatspresentedinx3.3,forFSescon-structedusingthealgorithmpresentedinx3.2.Furthermore,regardlessoftheencodingformatused,theFSsize|thenumberofedges|isafactorin\ruenc-ingtheencodingsize.Wearethusalsointerestedincom-paringthesizesofFSesconstructedbythealgorithmde-scribedinx3.2tolowerboundsonthesizesofFSesreturnedbyanyalgorithmthatprovidesshortestpathlatenciesandsingle-linkfailureprotection.Theselowerboundsimposeafundamentallimitontheencodingsize;intuitively,foragivenencodingformatthatalreadyusesoptimizedlabellengths,itishardtoreducetheencodingsizesignicantlywithoutreducingtheFSsize.Wede-scribein[22]analgorithmthatyieldsalowerboundonthesizeoftheFSforagivenprimarypathhopcount.Methodology.Weevaluateall98,910possibleorderedsource-destinationpairsoftheSprinttopology.FortheAS-androuter-leveltopologies,werandomlysampletenmillionuniqueorderedsource-destinationpairs.Foreachpair,werecordthesevalues:theDefaultandDirecten-codingsizes,thesizeoftheFSconstructedusingoural-gorithm,andthelowerboundonFSsizes.Results.Fig.4showstheencodingsizeresults.WeseethatDefaulthassomewhatsmallersizealmostalways;Directperformsnoticeablybetteronlyintheextremetailoftherouter-leveltopology.WethereforediscussDefaultinwhatfollows.FortheintradomainSprinttopology,themaximumencodingsizeis58bytes.Theplothasalongtailwith90%and99%ofthesource-destinationpairsrequiringlessthan21bytesand34bytesofencoding,respectively.FortheinterdomainAS-levelmapoftheInternet,themaximumencodingsizeis50bytes.AswiththeSprinttopology,theplothasalongtail,with90%ofthesource-destinationpairsresultinginencodingsofless 0 10 20 30 40 50 60 Encoding size (bytes) 0.0 0.2 0.4 0.6 0.8 1.0 CDF Default Direct (a)SprintTopology 0 10 20 30 40 50 60 Encoding size (bytes) 0.0 0.2 0.4 0.6 0.8 1.0 CDF Default Direct (b)AS-levelTopology 0 20 40 60 80 100 120 140 Encoding size (bytes) 0.0 0.2 0.4 0.6 0.8 1.0 CDF Default Direct (c)Router-levelTopologyFigure4:CDFofSlickPacketsencodingsizeinbytesfortheDirectandDefaultencodingformats,forhandlingsingle-linkfailures. 0 10 20 30 40 50 Number of edges 0.0 0.2 0.4 0.6 0.8 1.0 CDF SlickPackets Lower bound (a)SprintTopology 0 5 10 15 20 25 30 35 Number of edges 0.0 0.2 0.4 0.6 0.8 1.0 CDF SlickPackets Lower bound (b)AS-levelTopology 0 10 20 30 40 50 60 70 80 Number of edges 0.0 0.2 0.4 0.6 0.8 1.0 CDF SlickPackets Lower bound (c)Router-levelTopologyFigure5:CDFofSlickPacketsFSsizeandthelowerboundinnumberofedgesforhandlingsingle-linkfailures.than21bytes;99%ofthesource-destinationpairsresultinlessthan26bytes.Fortheextremecaseofrouter-leveltopology,90%ofthesource-destinationpairsresultinencodingsoflessthan43bytes;99%lessthan60bytes.Theremaininglessthan1%ofthesource-destinationpairsconstitutethelongtail,withmaximumencodingsizeof132bytes.Althoughtherouter-levelrealizationofSlickPacketsmaybeimprac-tical,theaboveresultsdemonstratethatSlickPacketscanscaleongraphsaslargeas200;000nodeswithmod-erateincreaseinthepacketheadersizes.Ifdesired,thisoverheadmaybeamortizedovermoredata(e.g.,bylever-agingIPv6jumboframes)orusingSlickPacketsonlyforapplicationdatathatismostsensitivetofailures.Fig.5showstheFSsize(innumberoflinks)andlowerbound.FortheAS-levelandrouter-leveltopologies,ourFSsizeisveryclosetothelowerbound;fortheSprinttopology,thedierenceissomewhatlarger.Overall,theresultssuggestthat,forhandlingsingle-linkfailures,oursimpleFSselectionalgorithmisrelativelyclosetooptimalintermsofminimizingthenumberoflinksintheFS.FortheSprinttopology,thereisalsoalongtailinbothourFSsizesandthelowerbounds.Thereasonisthatthereareafewsource-destinationpairsthathavelongprimarypaths,requiringalternatepathsforalargenum-berofnodes,resultinginlargernumberofedges.4.2FailurereactioneffectivenessOnemetrictoevaluatetheeectivenessofafailurere-actionmechanismisthepacketstretch,theratioofthelengthofapacket'spathtothelengthoftheshortestpossiblepath.Previousworkscalculatestretchbasedonpackets'traversedpathcostsortransittimes.However,foradelay-sensitiveapplication,weareinterestedinthetimeapacketislivefromtheapplication'sperspective|fromthetimethepacketisgeneratedbythesourceappli-cationtothetimeitisreceivedbythedestination.Thus,wedenethestretchforapacketthatdoesnotfullytra-versetheoriginalshortestpath,tobetheratioofthetimethepacketislivetothepost-link-failureshortestpathla-tency;forotherpackets|thosethattraversetheoriginalshortestpath|thestretchis1.Forbrevityoftheensuingdiscussion,l0denotesthefailedlinkontheprimarypathfromsourcestodestinationd;r0denotestherouterthatisadjacenttoandupstreamfroml0ontheprimarypath;andt0denotesthetimeoffailureofl0.Modelingdelayatnetworkdevices.Arouterinthenetwork,uponalinkfailure,hastoperformanumberoftasksbeforeithasnewvaliddefaultnexthopsforaecteddestinations.Thefourmajortasksare:(1)detectingafailedlink(iftherouterisadjacenttothefailedlink)andgeneratingacontrolplanemessage;(2)processingofre-ceivedcontrolpackets;(3)computingthenewshortestpathtree(SPT);and(4)updatingtheforwardinginfor-mationbase(FIB).Weassumethatthedelayindetectingafailedlinkiszerosinceirrespectiveoftheunderlyingroutingarchitecture,allpacketsduringthisperiodarelost;4thisdoesnotmakeadierenceinourperformancecomparisonresults.Weconsiderthethreeothermajorcontributors. 4Unlesspacketsareduplicatedalongmultiplepaths|adesignpointthatmaybereasonableforcertainkindsoftrac,butwhichwedonotconsiderinthispaper. Letdrbethetimespentbyarouterinprocessingacontrolpacket(i.e.,thetimebetweentherouter'sre-ceiptandforwardingofthepacket).dr(alongwithlinklatencies)dictatesthepropagationrateofcontrolpack-etsthroughthenetwork.Letdpbethedelaybetweenarouter'slearningofthelinkfailureandstartinganewSPTcomputation;dcbethetimetakentocomputethenewSPT;anddubethetimetakentoupdatetheFIB.Notethat,uponreceivingacontrolpacket,arouternecessarilyspendsD=(dp+dc+du)timebeforehavingnewvaliddefaultnexthopsforaecteddestinations.Thevaluesofdcanddudependontherouterarchitecture,algorithmsinuse,thetopology,andtherouter'slocation.Lackingagoodmodel,wesetthesevaluesto0inoursimulations.However,weuseD=dp=50ms[18]anddr=2ms[8,9]fortheSprinttopology.FortheAS-levelandrouter-leveltopologies,weuseD=dr=0.4.2.1FailurereactionschemesTheperformanceofsourceroutingprotocolsalsode-pendsonthecontrolplanemechanism:thetechniqueusedtoinformsourcesaboutthefailuresinthenetwork.WedescribethreevariantsofSlickPacketsdesignwithdif-ferentcontrolplanemechanisms.Wealsodescribethreeprotocols|onefromtheSCRparadigmandtwofromtheNCRparadigm|thatwecomparewithSlickPackets.Flooded-SlickPackets.Upondetectingthelinkfail-ure,r0\roodsthenetworkwithalinkstateadvertisement(LSA).ThisissimilartorunninganSCRprotocolwithanOSPF[21]stylecontrolplanemechanism.Fast-SlickPackets.Whenr0receivesapacketwhoseprimarynext-hoptraversesl0,itinformssaboutthelinkfailurebydirectlysendinganICMP-stylenotica-tionmessagetos.Therationaleisthat,toreducecontroloverhead,onlysourcesthatusel0intheirprimarypathsneedtobenotied.Intuitively,thissignicantlyreducesthecontrolplanepacketssentintothenetwork.e2e-SlickPackets.Therouterr0piggybacksthelink-failureinformationonthepacketbeingforwardedonthealternatepathtowardsd,which,uponreceivingthisin-formation,mayinformsofthelinkfailure.Thus,failureinformationissenttothesourceinanend-to-endmanner.AllSlickPacketsschemesusethesameFSselectionalgorithm(x3.2)andincurthedelayDbetweenlearningofthefailureandswitchingtonewprimarypaths.Vanillasourcerouting(VSR).Forpurposesofcom-parisonwithSlickPackets,weevaluateasimple\vanilla"sourceroutingprotocol.InVSR,eachsourcesspeciesasingleshortestpathtoitsdestinationdinthepacketheader.Forthecontrolplanemechanism,weusethe\fast"version,wherer0directlynotiess.Afterreceivingthenotication,sincursthedelayDbeforecomputinganewshortestpath.Withoutavalidpath,packetsgener-atedduringthistimearequeued.Packetsthatusel0intheirpathswillbedroppedbyr0afterthelinkfailure.However,onceshascomputedanewpath,itresendsthepacketsthatwouldhavebeendropped,i.e.,thosethatitsentinthetimeinterval[tR;t)wheretisthetimeslearnedofthefailure,andRistheRTTbetweensandr0.Notethatforsomeoftheseresentpackets,therecouldbetwoconcurrentlivecopies:theresentcopythatwillbedeliveredalongthenewpath,andtheoriginalcopythatwillbedroppedwhenitreachesr0.Thisschememaybedicultorundesirabletoimplementinpractice,butasanidealizedVSR,itisausefulcomparison.Ideal-SafeGuard.WesimulatedanidealizedversionofSafeGuard[18],anetwork-controlledroutingprotocolthatachievesfastfailurereaction.SafeGuardusesthestandardOSPFasthecontrolplanesubstrate.InSafe-Guard,r0immediatelyusespre-computedshortestalter-natepathstoquicklyredirectpacketsthatitwouldother-wiseforwardalongl0.Otherroutersrecognizeredirected(\escortmode")packetsandforwardthemalongtheirin-tendedalternatepaths;however,untiltheyhaveupdatedtheirFIBs(afterdelayDafterreceivingtheLSA),theserouterscontinuetoforward\normalmode"packetsalongtheirsub-optimalpathstowardsl0.Inpractice,the\al-ternativepathdatabases,"whicharefoundtobe2to8timeslargerthanarouter'sintradomainFIB[18],mightincreaselookuplatenciesorbeanimpracticalmemoryrequirement.However,ouridealversionofSafeGuardig-norestheseissues.Ideal-NCR.Thisrepresentsanideal(andunachievable)NCRscheme,inwhicheachrouterlearnsofalinkfailureinexactlythepropagationdelayalongtheshortestpathfromthepointoffailuretotherouter;andtherouterinstantlybeginsforwardingpacketsalongtheshortestal-ternatepath.Ideal-NCRisequivalenttoaspecialcaseofIdeal-SafeGuardwherealldelays,exceptpropagationdelay,arezero(i.e,D=dr=0).4.2.2MethodologyWewroteastaticsimulatorforourevaluationpurposes.Thesimulatorusesthepacketstretchcomputationsde-scribedin[22].Sinceweareevaluatingthereactiontosingle-linkfailures,weevaluateonly(l0;s;d)tripleswheretheprimarypathfromstodusesl0,andsanddremainconnectedafterthefailureofl0,sothatatleastoneal-ternatepathtodexistsforeachrouterupstreamfroml0.FortheSprinttopology,weevaluateall424,569possiblesuchtriples.ForeachoftheAS-androuter-leveltopolo-gies,wesample1,000randomlinksanduseasamplingalgorithm(describedin[22]duetospaceconstraints)toobtainover750,000and890,000suchtriples,respectively.Inoursimulations,theapplicationatthesourcegen-eratespacketsevery1ms,startingattimet=0ms.Forthetimeoflinkfailuret0,however,recallthatinIdeal-SafeGuard,Ideal-NCR,andFlooded-SlickPackets,r0\roodstheLSAwhenitdetectsthelinkfailure,notwhenitreceivessources'packets.Fortheseschemes,thesoonerthelinkfails,thesoonerintermediateroutersandthesourcelearnofthefailureandusebetterpaths.So,forafaircomparisonwithnon-\roodingschemes,weconsidertwoextremepoints:whent0isgreaterthanthenetworkdiameterintermsoflinklatenciesandwhent0=0.Theformercaseensuresthatbythetimet0,allsourcesinallevaluated(l0;s;d)tripleshavehadpacketsreachingr0.FortheSprinttopology,withadiameterof139ms,weuset0=150.FortheAS-androuter-leveltopologies,weassumealllinkshavelatencies1msanduset0=50.4.2.3ResultsThehigh-levelresultsrevealthatSlickPacketsschemes(particularlytheFastandFloodedvariants)achievepacketstretchcomparabletothatofNCRschemeIdeal-SafeGuard. 0 100 200 300 400 500 Packet generation time (ms) 10-5 10-4 10-3 10-2 10-1 100 101 Average stretch - 1 (a)SprintTopology 40 45 50 55 60 65 70 Packet generation time (ms) 10-5 10-4 10-3 10-2 10-1 100 Average stretch - 1 (b)AS-levelTopology 30 40 50 60 70 80 Packet generation time (ms) 10-5 10-4 10-3 10-2 10-1 100 Average stretch - 1 Fast-SlickPackets Flooded-SlickPackets e2e-SlickPackets Fast-VSR Ideal-SafeGuard Ideal-NCR (c)Router-levelTopologyFigure6:Averagepacketstretch-1vs.packetgenerationtimewhent0isgreaterthanthenetworkdiameter.They-axesareonlogscales.FortheSprinttopology,t0=150;D=50;dr=2.FortheAS-androuter-leveltopologies,t0=50;D=dr=0. 0 100 200 300 400 500 Packet generation time (ms) 10-3 10-2 10-1 100 101 102 Worst stretch - 1 (a)SprintTopology 40 45 50 55 60 65 70 Packet generation time (ms) 10-1 100 101 Worst stretch - 1 (b)AS-levelTopology 30 40 50 60 70 80 90 Packet generation time (ms) 10-2 10-1 100 101 Worst stretch - 1 Fast-SlickPackets Flooded-SlickPackets e2e-SlickPackets Fast-VSR Ideal-SafeGuard Ideal-NCR (c)Router-levelTopologyFigure7:Worstpacketstretchvs.packetgenerationtimewhent0isgreaterthanthenetworkdiameter.They-axesareonlogscales.FortheSprinttopology,t0=150;D=50;dr=2.FortheAS-androuter-leveltopologies,t0=50;D=dr=0. 0 100 200 300 400 500 Packet generation time (ms) 10-5 10-4 10-3 10-2 10-1 100 101 Average stretch - 1 (a)SprintTopology 0 5 10 15 20 Packet generation time (ms) 10-5 10-4 10-3 10-2 10-1 100 Average stretch - 1 (b)AS-levelTopology 0 5 10 15 20 25 30 35 Packet generation time (ms) 10-5 10-4 10-3 10-2 10-1 100 101 Average stretch - 1 Fast-SlickPackets Flooded-SlickPackets e2e-SlickPackets Fast-VSR Ideal-SafeGuard Ideal-NCR (c)Router-levelTopologyFigure8:Averagepacketstretch-1vs.packetgenerationtimewhent0=0.They-axesareonlogscales.FortheSprinttopology,D=50;dr=2.FortheAS-androuter-leveltopologies,D=dr=0. 0 100 200 300 400 500 Packet generation time (ms) 10-3 10-2 10-1 100 101 102 Worst stretch - 1 (a)SprintTopology 0 5 10 15 20 25 Packet generation time (ms) 10-1 100 101 Worst stretch - 1 (b)AS-levelTopology 0 5 10 15 20 25 30 35 40 45 Packet generation time (ms) 10-2 10-1 100 101 Worst stretch - 1 Fast-SlickPackets Flooded-SlickPackets e2e-SlickPackets Fast-VSR Ideal-SafeGuard Ideal-NCR (c)Router-levelTopologyFigure9:Worstpacketstretchvs.packetgenerationtimewhent0=0.They-axesareonlogscales.FortheSprinttopology,D=50;dr=2.FortheAS-androuter-leveltopologies,D=dr=0. AlthoughSlickPacketsschemestakeslightlylongertoconvergecomparedtoSafeGuard,theyavoidthehighpacketstretchofFast-VSR.Averagestretch.Fig.6showsthepacketstretchaver-agedoverallevaluated(l0;s;d)tripleswhent0isgreaterthanthenetworkdiameter.Werstconsiderfeaturescommontoallschemes.Foragivenscheme,allpacketsgeneratedearlyinthesimulationhavestretch1.Grad-ually,aspacketsgeneratedclosertot0,aswellasmoretripleswheresisclosertol0,areaectedbythefail-ure,theaveragestretchincreases.Additionally,foranytriple,allpacketsgeneratedaftert0havestretchnohigherthanthosegeneratedatt0;thisisre\rectedintheaveragestretchoveralltriples.WenowcompareNCRandSlickPacketsschemes.InNCRschemes,routersupstreamfroml0,oncetheyreceivetheLSAandupdatetheirFIBs,canredirectpacketsbe-foretheyreachl0;whileinSlickPacketsschemes,pack-etshavetoreachl0beforebeingredirected.Thisdier-encegivesNCRschemesonlyasmalladvantageforearlypackets,especiallyfortheSprinttopologyinFig.6(a),becauseupstreamroutersstillincurthedelayDbetweenreceivingtheLSAandupdatingtheirFIBs.Forlaterpackets,thisadvantagebecomesmoresignicantasmoreupstreamroutersupdatetheirFIBs.Asexpected,Ideal-NCRisthebestperformingschemeinallthreetopologies:itconverges57msbeforeIdeal-SafeGuardfortheSprinttopology(duetoD=50anddr=2)andisequivalenttoIdeal-SafeGuard(notshown)intheothertwotopologies,whereD=dr=0.ConsidertheSlickPacketsschemesinFig.6(a).Weseethatforpacketsgeneratedbetweent0=150andt0+D=200,theaveragepacketstretchis(1)con-stantwithinthesameschemeand(2)identicalacrossallschemes.RecallthatallSlickPacketsschemesusethesameFSselectionalgorithmandincurthesamedelayDbetweenlearningofthefailureandswitchingtonewpri-marypaths.Thus,theonlyfactoraectingtheirrelativeperformancesisthetimeslearnsofthefailure,whichisdeterminedbytherelativedistancesamongl0,s,anddfordierenttriplesinthesamescheme,andthedierentcon-trolschemesgiventhesametriple.So,regardlessofthe(l0;s;d)tripleorthecontrolscheme,thereisaminimumwindowofDtimewheresusesthesame(old)primarypath.Afterthiswindow,wecanseethatFast-SlickPac-ketsconvergesslightlyfasterthanFlooded-SlickPac-ketsbecausetheLSAsinFlooded-SlickPacketsincurdelaydratintermediaterouters;inFig.6(b)and(c),wheredr=0,Fast-andFlooded-SlickPacketsareiden-tical.Andbothofthemconvergesignicantlyfasterthane2e-SlickPacketsasexpected.Finally,weseethatinFast-VSR,earlypacketsexpe-riencehigherstretchthaninotherschemes.Thisisbe-causethesepacketsaredroppedandhavetoberesentbys.TheyexperienceonaverageadelayofonehalftheRTTbetweensandr0,plusthedelayDbeforebeingsentalongthenewpath,resultinginahighstretch.However,Fast-VSRcancatchuptoandovertakeFast-SlickPac-ketsfortworeasons.First,considerthepacketsent1msbeforeslearnsofthefailure:inFast-VSR,itisdelayed(1+D)msbeforebeingresentalongthenewpath;whileinFast-SlickPackets,theamountoftimethispackettraversestheoriginalprimarypathonlytoberedirectedbackwardscanbelargerthan(1+D),especiallyifboththeprimarypathandalternatepathcontainaveryhighlatencylink.Second,considerthepacketgenerated1msbeforeshasanewprimarypath:inFast-VSR,itisde-layed(queued)only1msbeforebeingsentonthenewoptimalpath;whileinFast-SlickPackets,thispacketwillbesentalongtheoriginalprimarypathandwillberedirected,experiencingahigherstretchthanitsFast-VSRcounterpart.ThesetwoeectsenableFast-VSRtonoticeablyovertakeFast-SlickPacketsinFig.6(a),butinFig.6(b)and(c),whereD=0andalllinkshavela-tencies1ms,thesetwoeectsarelesspronounced.Worststretch.Fig.7showstheworststretchofpacketsgiventheirgenerationtime,amongallevaluated(l0;s;d)triples,whent0isgreaterthanthenetworkdiameter.Notethatthesimulation-wideworststretchesforallschemesexceptFast-VSRareequal,whichare2.93,2.0,and2.2inFig.7(a),(b),and(c),respectively.Thisisbecausealltheseschemesdonotdroppackets,sotheworststretchisthatofpacketsthatr0redirects,whichisthesameforalltheseschemes.Alsonotethatforschemesthatdonotdroporqueuepackets,theworststretchoccurswhenapackettraversesthemaximumpossibledistancealongtheoriginalshortestpathwithoutreachingd,isredirectedbacktos,andtraversestheshortestalternatepath.So,3istheupper-boundstretchbecausetheshortestalternatepathcannotbeshorterthantheoriginalshortestpath.FortheSprinttopologyinFig.7(a),thesimulation-wideworststretchforFast-VSRis27.Thishappenstopacketssentrightbeforet0=150intripleswheresisclosetod,sothatthetimedurationDthatthesepacketsaredelayeddominatesthelatenciesoftheoriginalandpost-link-failureshortestpaths.IntheAS-androuter-leveltopologies,whereD=0,thesimulation-wideworststretchofFast-VSRare2.75and2.88respectively.Whent0=0.Fig.8and9showtheresultsforwhent0=0.Theoverallbehaviorofeachindividualschemeexhibitssimilarpatternstowhent0isgreaterthanthenetworkdiameter.Thedierencesarethatthepeakstretchesoccurforpacketsgeneratedatt0=0.Further-more,asexpected,\roodingschemesbenetfromtheear-liertimeoffailure:forexample,fortheSprinttopologyinFig.8(a),Ideal-NCRandIdeal-SafeGuardconvergefur-theraheadofFast-SlickPacketscomparedtoFig.6(a),andevenFlooded-SlickPacketsnowconvergesaheadofFast-SlickPackets(similarlyfortheAS-androuter-leveltopologies).Intermsofsimulation-wideworststretch,thoseofnon-\roodingschemes(Fast-ande2e-SlickPacketsaswellasFast-VSR)arethesameaswhent0isgreaterthanthenetworkdiameter.Thisisasexpectedbecausefortheseschemes,itisstillr0thatredirectspacketsand/ortriggersthenoticationofsources.For\roodingschemes,however,itcanbeexpectedthatsimulation-wideworststretchwouldbelowercomparedtowhent0isgreaterthanthenetworkdiameter.Nevertheless,theSprinttopologycon-tainstripleswhereanupstreamlinkthatisclosetor0hasveryhighlatencycomparedtothedistancebetweensandr0,sothats'srstpacketdoesnotbenetfromthe\roodedLSA:itstillhastoreachr0beforebeingredi-rected.Thisresultsinthesimulation-wideworststretchof2.93inFig.8(a). 5.DISCUSSION:FORWARDINGSUBGRAPHSELECTIONTheSlickPacketsdesignisagnostictohowthesourceselectstheforwardingsubgraph(FS).Forexample,theFSselectionmaybeguidedbydemandsoftheapplica-tionrunningatthesource(forexample,ifthesourceisanendhost)ortheperformancegoalsofanetworkop-erator(forexample,ifthesourceisanedgerouter).Inthispaper,wepresentedandevaluatedonesuchFSselec-tionalgorithm:wheretheFSallowsre-routingofpacketswithinthenetworkincaseofsingle-linkfailures.WenowdiscussalternativeFSselectionstrategies.Handlingnodefailures.FortheFStohandlenodefailures,weneedonlyasimplemodicationtothelink-failure-avoidingFSselectionofx3.2.Asources,foragivendestinationd,constructstheFSinthreesteps.First,scomputesaprimarypathPtodbyrunninganinstanceoftheshortestpathalgorithm.Next,toprotectagainstsinglenodefailures,svisitseachnodealongP,andcomputesthealternatepathPiitwouldpreferthepackettoberoutedalongifthatnodeweretofail.Inpar-ticular,foreachnodeviontheprimarypathwithnodevi+1asthenexthopalongtheprimarypath,we(a)re-movevi+1;(b)computeashortestpathfromvitod;and,(c)restorevi+1.Handlingmultiplelinkfailures.AsourcemaydesiretoconstructanFSthatprotectsagainstmultiplelinkfailures.Thismaybedonebyextendingtheschemefromx3toconstructanFSthatprotectsfrommultipleedge-failures.Forexample,itmaybesucienttohavetwostrategicallychosenalternatepathsforallnodesontheprimarypath.Theideaisthatthesourcecanchoosealternatepathsthatarenotfailure-correlatedwiththeprimarypath.Thismayallowamuchlargeramountofresiliency;althoughtheperformanceevaluationofsuchaschemeissubjecttofuturework.Congestionavoidance.Ourfocusinthisworksofarhasbeenondealingwithfailures.However,alternatepathsintheFSmayalsobeusedtoreacttocongestioninthenetwork.Forexample,intermediateroutersalongthepathmaychoosetoforwardthepacketalonganalternatepathiftheprimarypathiscongested(e.g.,iftheinter-facequeueforthecorrespondinglinkislledbeyondaparticularthreshold).UsingaFSalsoenablesthesourcetooptionallyprovidecontroloverloadbalancing,bypro-vidingfeedbackonwhichsetofpathsaretolerablefortheloadbalancingprocess.6.RELATEDWORKOurgoalsarerelatedtotwokeyareasofrelatedwork:Failurereactioninnetwork-controlledroutingpro-tocols.TherehasbeenmuchworkoncopingwithfailuresinIPnetworks.Wefocusonthemostcloselyrelatedwork:protocolsthatguaranteepacketdeliveryinthepresenceofoneormorelinkfailures.R-BGP[16]constructsinterdomainbackuppathstohandlesinglelinkfailures,givensomeassumptionsaboutroutingpolicies.SafeGuard[18]usesaremainingpathcosteldinapacketasaheuristictodeterminewhetherthepathexpectedbytheprevioushopisdierentthanthepathavailabletothecurrenthop.Inthisway,itcandecidewhentoreroutepacketsalongpre-computedbackuppaths.FCP[17]takesadierentapproachtodeterminingwhenpacketsshouldbererouted:eachpacketcarriesalistofthefailedlinksithasencountered.Thebestbackuppathsarecom-putedonthe\ryatrouters,thusallowingFCPtobero-busttomultiplelinkfailures,butrequiringfairlyheavy-weightgraphprocessinginthedataplane.MPLSFastReroute[23]reliesonprecomputationofbackuppaths.Initslocalrepairvariant,anadditionalpathisconstructedtoavoideachneighboringlinkornode,whichcanin\ratestoragerequirementsandwillnotresultinlowest-stretchbackuppaths.Asdiscussedintheintroduction,alloftheaboveapproachesareNCRprotocols,whichdonotpermitsourcecontrolofprimaryorbackuppaths.Inaddition,backuppathsarecomputedorstoredateveryrouterwithinthenetwork,sothatthereisadependencybetweeneachrouter'sforwardingtableandthetopologyoftheentirenetwork.OnewaytogetasmallamountofroutecontrolatthesourcewithinanNCRarchitectureistousemultihoming:thesourcecanthenselectbetweenseveralproviders[4].Thiscouldbeusedtoenablesomesourcecontrol,whilestillapplyingtheNCRresiliencetechniquesdescribedabove.However,thisprovidesonlyaverylimitedamountofcon-troltothesource,anddoesnotyieldthefullbenetsofsourcecontroldescribedintheintroduction.Moreover,ifmanysourcesaremultihomed,thisvastlyincreasesrout-ingstatewithinthenetwork,sinceeachrouterwouldberequiredtoknowabouteverypointofmultihomingat-tachmentifwedesiretoprovidealternatepathsthatavoidafailureofoneoftheselinks.OuruseofroutingalongFSeswasinspiredby[19],whicharguesthatadirectedacyclicgraphisabetterfor-wardingarchitecturethanthemoretraditionalshortest-pathtree.While[19]focusesonimprovingNCRschemes,wetargetachievingthebenetsofbothnetwork-andsource-controlledrouting.Additionally,while[19]willdelivereverypacketevenduringlinkfailures,itdoesnotguaranteethelatencythatthesepacketswillhave.Slick-Packetscanguaranteethatforsingle-linkfailures,pack-etswillfollowtheshortestalternatepathfromthepointoffailuretothedestination.Sourcerouting.Thereisalsoalargebodyofworkonsourcecontrolledrouting,rangingfromdynamicsourceroutinginwirelessnetworks[15]tofutureinterdomainroutingarchitectures[10,20,30,31].Twoofthese,Rout-ingDe\rections[31]andPathSplicing[20],targetfastre-routingwithinthenetwork.BothusepathlabelbitssetbythesourcetopseudorandomlyselectanexthopateachrouterorAS.In[20],pseudorandomforwardingcanleadtoforwardingloops.In[31]routersfollowcertainrulesthatensureloop-freedom,butreducepathdiversity.Therearethreeimportantdierencesbetween[20,31]andSlickPackets.First,[20,31]donotfullysupportsourcecontroloverprimaryorbackuproutes;althoughsourcescanselectamongsomesetofpaths,theycan-nottellwhichpathstheyareselecting.Second,althoughpacketscanbereroutedquicklywithinthenetworkaf-teralinkfailure,thisisnotguaranteed(packetsmaybedropped),andthebackuppathsarenotguaranteedtohaveoptimallatency.Third,[20,31]aresimilartotradi-tionalNCRschemesintermsofthestateinthenetwork;indeed,[20]increasesforwardingtablesizebecauseeach routerstoresmultiplenext-hopsforeachdestination.Incontrast,SlickPacketsenablessourcecontrol,canguar-anteeresilience5tosingle-linkfailureswithpacketssentalongtheshortestalternatepathfromthepointoffailuretothedestination,andrequiresonlylocalstateatrouters.Givingsourcescontroloverconstructingend-to-endpathsintroducesanumberofpracticalquestions,forexampleintermsofpolicycompliance,security,andscalabilityofdisseminatingtopologicalstate.Forthesequestions,werelyonpreviouswork(e.g.,[10,30],andcitationswithin),whichprovidesolutionstotheseproblems.7.CONCLUSIONInthispaper,wepresentedSlickPackets,anapproachtoroutingthatattainsfailurereaction,whilesimultane-ouslyretainingthebenetsofsourcerouting.Slick-Packetsworksbycompactlyencodingasetofalternatepathsintodatapacketheadersasadirectedacyclicgraph.Towardsthisgoal,weprovidesimplealgorithmsforcom-putingecientgraphs,andforencodingthemintopack-etsinamannerthatcanbeprocessedbyintermediateroutersinanecientmanner.OnemajorarealeftforfutureworkistoevaluatethecomplexityofimplementingSlickPacketsinproductionrouters,andachievablepacketforwardingrates;akeychallengehereisdealingwithincreasedheadersize.ApromisingavenueforevaluationistheSuperchargedPlan-etLabPlatform[26],anetworkprocessor-basedplatformonwhichJohnDeHarthasimplementedaprototypever-sionofSlickPackets.ThisworkwassupportedbyNationalScienceFounda-tiongrantCNS10-40396.8.REFERENCES[1]CAIDA'srouter-leveltopologymeasurements.http://www.caida.org/tools/measurement/skitter/router_topology/.[2]Rocketfuel:AnISPtopologymappingengine.http://www.cs.washington.edu/research/networking/rocketfuel/.[3]Y.-Y.Ahn,S.Han,H.Kwak,S.Moon,andH.Jeong.Analysisoftopologicalcharacteristicsofhugeonlinesocialnetworkingservices.InProc.ACMWWW'07,pages835{844,May2007.[4]A.Akella,J.Pang,B.Maggs,S.Seshan,andA.Shaikh.Acomparisonofoverlayroutingandmultihomingroutecontrol.ACMSIGCOMM,34(4):93{106,2004.[5]D.G.Andersen,H.Balakrishnan,M.F.Kaashoek,andR.Morris.Resilientoverlaynetworks.InProc.18thACMSOSP,October2001.[6]M.Caesar,M.Casado,T.Koponen,J.Rexford,andS.Shenker.Dynamicroutecomputationconsideredharmful.ACMSIGCOMMComputerCommunicationReview,2010.[7]D.Clark,J.Wroclawski,K.Sollins,andR.Braden.Tussleincyberspace:Deningtomorrow'sInternet.InSIGCOMM,2002.[8]P.Francois,C.Filsls,J.Evans,andO.Bonaventure.Achievingsub-secondIGPconvergenceinlargeIPnetworks.SIGCOMMComputerCommunicationsReview,35:35{44,2005.[9]J.Fu,P.Sjodin,andG.Karlsson.Intra-domainroutingconvergencewithcentralizedcontrol.ComputerNetworks,53,2009. 5Unless,ofcourse,noalternatepathexists.[10]P.B.Godfrey,I.Ganichev,S.Shenker,andI.Stoica.Pathletrouting.InACMSIGCOMM,2009.[11]K.P.Gummadi,H.V.Madhyastha,S.D.Gribble,H.M.Levy,andD.Wetherall.ImprovingthereliabilityofInternetpathswithone-hopsourcerouting.InProc.OSDI,2004.[12]J.HershbergerandS.Suri.Vickerypricesandshortestpaths:whatisanedgeworth.InIEEEFOCS,2001.[13]Y.Hyun,B.Huaker,D.Andersen,E.Aben,M.Luckie,kcclay,andC.Shannon.Theipv4routed/24aslinksdataset,November2010.http://www.caida.org/data/active/ipv4_routed_topology_aslinks_dataset.xml.[14]G.Iannaccone,C.-N.Chuah,R.Mortier,S.Bhattacharyya,andC.Diot.AnalysisoflinkfailuresinanIPbackbone.InIMC,2002.[15]D.JohnsonandD.Maltz.Dynamicsourceroutinginadhocwirelessnetworks.Mobilecomputing,pages153{181,1996.[16]N.Kushman,S.Kandula,D.Katabi,andB.Maggs.R-BGP:Stayingconnectedinaconnectedworld.InNSDI,2007.[17]K.Lakshminarayanan,M.Caesar,M.Rangan,T.Anderson,S.Shenker,andI.Stoica.Achievingconvergence-freeroutingusingfailure-carryingpackets.SIGCOMMComput.Commun.Rev.,37(4):241{252,2007.[18]A.Li,X.Yang,andD.Wetherall.Safeguard:Safeforwardingduringroutechanges.InProc.ACMCoNext,December2009.[19]J.Liu,J.Rexford,M.Schapira,S.Shenker,andJ.Naous.RoutingalongDAGs,2010.http://www.cs.berkeley.edu/liujd/RAD.pdf.[20]M.Motiwala,M.Elmore,N.Feamster,andS.Vempala.Pathsplicing.InACMSIGCOMM,2008.[21]J.Moy.OSPF:AnatomyofanInternetRoutingProtocol.1998.[22]G.T.K.Nguyen,R.Agarwal,J.Liu,M.Caesar,P.B.Godfrey,andS.Shenker.Slickpackets.TechnicalReport,UIUC,April2011.[23]P.Pan,G.Swallow,andA.Atlas.FastrerouteextensionstoRSVP-TEforLSPtunnels.InRFC4090,May2005.[24]L.Qiu,Y.R.Yang,Y.Zhang,andS.Shenker.OnselshroutinginInternet-likeenvironments.InProc.ACMSIGCOMM,pages151{162,2003.[25]S.Savage,T.Anderson,A.Aggarwal,D.Becker,N.Cardwell,A.Collins,E.Homan,J.Snell,A.Vahdat,G.Voelker,andJ.Zahorjan.Detour:InformedInternetroutingandtransport.InIEEEMicro,January1999.[26]J.Turner,P.Crowley,J.DeHart,A.Freestone,B.Heller,F.Kuhns,S.Kumar,J.Lockwood,J.Lu,M.Wilson,C.Wiesman,andD.Zar.Superchargingplanetlab:ahighperformance,multi-application,overlaynetworkplatform.ACMSIGCOMM,2007.[27]F.Wang,Z.M.Mao,J.Wang,L.Gao,andR.Bush.Ameasurementstudyontheimpactofroutingeventsonend-to-endinternetpathperformance.SIGCOMMComput.Commun.Rev.,36(4):375{386,2006.[28]D.Wendlandt,I.Avramopoulos,D.Andersen,andJ.Rexford.Don'tsecureroutingprotocols,securedatadelivery.InHOTNETS,2006.[29]W.XuandJ.Rexford.MIRO:Multi-pathInterdomainROuting.InSIGCOMM,2006.[30]X.Yang,D.Clark,andA.Berger.NIRA:anewinter-domainroutingarchitecture.IEEE/ACMTransactionsonNetworking,15(4):775{788,2007.[31]X.YangandD.Wetherall.Sourceselectablepathdiversityviaroutingde\rections.InACMSIGCOMM,2006.