48810th USENIX Symposium on Networked Systems Design and Implementatio - PDF document

48810th USENIX Symposium on Networked Sy
48810th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’13)USENIX Associationhighavailabilityandne-grainedsynchronizationtoenableInternetmeasurementexperimentation.WeusethreecasestudiestoillustratetheuniqueperspectivethataplatformlikeDasubringstoIn-ternetmeasurement.Intheprocess,wedemonstrateDasu’scapabilitiesto(i)simplifytraditionalmea-surements(e.g.,examiningroutingasymmetry),(ii)revealfundamentalshortcomingsinexistingmea-surementefforts(e.g.,mappingAS-leveltopology),and(iii)conductnovelexperimentsfororiginalsys-temevaluations(e.g.,examiningtheeffectivenessofarecently-proposedDNSextension).Therestofthispaperisstructuredasfollows.WeputourworkincontextandprovidefurthermotivationinSec.2.InSec.3and4weoutlinethedesignandimplementationofDasuandcharacterizeourcurrentdeployment.WepresentcasesstudiesthatillustratethebenetsofameasurementexperimentationplatformthatrunsattheInternet’sedgeinSec.5.Finally,wediscussfutureworkandpresentourconclusionsinSec.6.BackgroundandMotivationThelackofnetworkandgeographicdiversityincur-rentInternetexperimentationplatformsiswellaknownproblem[8,39].MostInternetmeasurementandsystemevaluationstudiesrelyondedicatedinfrastructures[3,5,7,31]whichproviderelativelycontinuousavailabilityatthecostoflimitedvantagepointdiversity(i.e.withnodesprimarilylocatedinwell-provisionedacademicorresearchnetworksthatarenotrepresentativeofthelargerSeveralresearchprojectshavepointedoutthepit-fallswhenattemptingtogeneralizeresultsofnetworkmeasurementstakenwithalimitednetworkperspective(e.g.[8,10,27,32,45]).Forexample,considerthedif-ferencesinpathsbetweenPlanetLabnodesandbetweennodesinresidentialnetworks.Thesetwosetstraversedifferentpartsofthenetwork[9],exhibitdifferentla-tencyandpacketlosscharacteristics[12,20]andresultindifferentnetworkprotocolbehaviors[16].GoalsandApproachAnexperimentalplatformfortheInternetshouldbedeployedatscaletocapturenetworkandservicediver-sity.ItshouldbehostedatthenetworkedgetoprovidevisibilityintothisopaquepartoftheInternet.Suchaplatformshouldallowdynamicextensibilityinordertoenablepurposefully-designed,controlledmeasurementexperiments,withoutcompromisingend-hostsecurity.Tosupporttime-dependentandlong-runningexperi-ments,itshouldoffer(nearly)continuousavailability.Last,itshouldfacilitatethedesignanddeploymentofexperimentsatthenetworkedgewhilecontrollingtheimpactontheresourcesofparticipatingnodesandtheunderlyingnetworkresources.Dasuisanexperimentalplatformdesignedtomatchthesegoals.TocapturethediversityofthecommercialInternet,DasusupportsbothInternetmeasurementex-perimentationandbroadbandcharacterizationandlever-agestheirsynergies.Initscurrentversion,Dasuisbuiltasanextensiontothemostpopularlarge-scalepeer-to-peersystem–BitTorrent.ThetypicalusagepatternsandcomparativelylongsessiontimesofBitTorrentusersmeansthatDasucanattainnearlycontinuousavailabilitytolaunchmeasurementexperiments.Moreimportantly,byleveragingBitTorrent’spopularity,Dasuattainsthenecessaryscaleandcoverageattheedgeofthenetwork.DasuistailoredforInternetnetworkexperimentationand,unlikegeneral-purposeInternettestbedssuchasPlanetLab,doesnotsupportthedeploymentofplanetary-scalenetworkservices.BothstrengthsandchallengesofaplatformlikeDasustemfromitsinclusionofparticipatingnodesattheInternet’sedge.Forone,theincreasednetworkcoveragefromthesehostscomesatthecostofhighervolatilityandleavestheplatformatthe“mercy”ofendusers’behavior.Thetypesofexperimentspossibleinsuchaplatformdependthusontheclients’availabilityandsessiontimessincethesepartiallydeterminethemaximumlengthoftheexperimentthatcanbesafelyassignedtoclients.Suchaplatformmustprovideascalablewaytosharemeasurementresourcesamongconcurrentexperimentswithadynamicsetofvantagepoints.Itmustalsoguar-anteethesafetyofthevolunteernodeswhereitishosted(forinstance,byrestrictingtheexecutionenvironment),andensuresecurecommunicationwithinfrastructureservers.Last,tocontroltheimpac

tthatexperimentsmayhaveonunderlyingnetwo
tthatexperimentsmayhaveonunderlyingnetworkandsystemresources,thesystemmustsupportcoordinatedmeasurementsamonglargenumbersofhostsworldwide,eachofwhichissubjecttouserinteractionandinterference.RelatedWorkDasusharesgoalswithandbuildsuponideasfromseveralpriorlarge-scaleplatformstargetingInternetex-perimentation.Mostactivemeasurementandexperi-mentationresearchreliesondedicatedinfrastructures(PlanetLab[31],Ark[7],LookingGlassservers).Suchinfrastructuresproviderelativelycontinuousavailabilityandnearlycontinuousmonitoringatthecostoflimitedvantagepointdiversity.Dasutargetstheincreasingly“invisible”portionsoftheInternet,relyingonadirectAstand-aloneversionofDasuhasbeendevelopedandweplantoreleaseitinJune2013.2USENIX Association 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’13)489incentivemodeltoensurelarge-scaleadoptionattheInternetedge.Severalrelatedprojectsusepassivemeasurementsorrestrictedactivemeasurementsfromvolunteerplatformstocapturethissameperspective(e.g.,[15,33,35,37,38,42]).Incontrast,Dasuisasoftware-basedsolutionwithamuchbroadersetofmeasurementvantagepointsthathasbeenachievedbyaltruisticandhardware-basedsystems,andsupportsaprogrammableinterfacethatenablescomplex,coordinatedmeasurementsacrosstheparticipatinghosts.Assuch,DasusharessomedesigngoalswithScriptroute[40]andSatelliteLab[15]).Un-likeScriptroute,Dasuisintendedforlargescaledeploy-mentonendusers’machines,andreliesonincentivesforuseradoptionatscale.Dasualsoenablespro-gramablemeasurementswithoutrequiringrootaccess,avoidingpotentialsecurityrisksandbarrierstoadoption.SatelliteLabadoptsaninterestingtwo-tierarchitecturethatlinksendhosts(satellites)toPlanetLabnodesandseparatestrafcforwarding(donebysatellites)fromcodeexecution.InDasu,experimentcodegeneratestrafcdirectlyfromhostsatthenetworkedge.SeveralsystemshaveproposedleveragingclientsinaP2Psystemtomeasure,diagnoseandpredicttheperformanceofend-to-endpaths(e.g.,[11,28].Dasumovesbeyondtheseefforts,exploringthechallengesandopportunitiesinsupportingprogrammableexperimenta-tionfromvolunteerendhosts.DasuDesignInthissection,weprovideanoverviewofDasu’sdesign,discussseveralsystem’scomponentsandbrieydescribetheAPIsupportingmeasurementexperiments.SystemOverviewDasuiscomposedofadistributedcollectionofclientsandasetofmanagementservices.DasuclientsprovidethedesiredcoverageandcarryonthemeasurementsneededforbroadbandcharacterizationandInternetex-perimentation.TheManagementServices,comprisingtheConguration,ExperimentAdministration,Coordi-nationandDataservices,distributeclientcongurationandexperimentsandmanagedatacollection.Figure1presentsthedifferentcomponentsandtheirinteractions.Uponinitialization,clientsusetheCongurationSer-toannouncethemselvesandobtainvariouscong-urationsettingsincludingthefrequencyanddurationofmeasurementsaswellasthelocationtowhichexperi-mentresultsshouldbereported.Dasuclientsperiod-icallycontacttheExperimentAdministrationService,whichassignsmeasurementtasks,andtheCoordinationServicetosubmitupdatesaboutcompletedprobesandretrievemeasurementlimitsforthedifferentexperimenttasks.Finally,clientsusetheDataServicetoreport\r\f\n\t\b\r\r\n\r\n\t\b\r\r\f

;
;\r\rFigure1:Dasusystemcomponents.theresultsofcompletedexperimentsastheybecomeavailable.ExperimentSpecicationDasuisadynamicallyextensibleplatformdesignedtofacilitateInternetmeasurementexperimentationwhilecontrollingtheimpactonhosts’resourcesandtheun-derlyingnetwork.Akeychallengeinthiscontextisselectingaprogramminginterfacethatisbothexible(i.e.,supportsawiderangeofexperiments)andsafe(i.e.,doesnotpermitrun-awayprograms).Werejectedseveralapproachesbasedontheseconstraintsandourplatformsgoals.Theseincludeofferingonlyasmallandxedsetofmeasurementprimitivesastheywouldlimitexibility.Wealsoavoidedprovidingarbitrarybinaryexecutionashandlingtheramicationsofsuchanapproachwouldbeneedlesslycomplex.Weoptedforarule-baseddeclarativemodelforex-perimentspecicationinDasu.Inthismodel,aruleisasimpleconstructthatspeciesthesetofactionstoexecutewhencertainactivationconditionshold.Arule’sleft-handsideistheconditionalpart)andstatestheconditionstobematched.Theright-handsideistheconsequenceoractionpartoftherule()i.e.,thelistofactionstobeexecuted.Conditionandactionstatementsarespeciedintermsofread/writeoperationsonasharedworkingmemoryandinvocationofaccessormethodsandmeasurementprimitives.Acollectionofrulesformaprogramandasetofrelatedprogramsdeneanexperiment.Therule-basedmodelprovidesacleanseparationbetweenexperimentlogicandstate.Inourexperience,thishasproventobeaexibleandlightweightapproachforspecifyingandcontrollingexperiments.Experimentlogiciscentralized,makingiteasytomaintainandextend.Also,strictconstraintscanbeimposedonrulesyntax,enablingsafetyvericationthroughsimplestaticprogramanalysis.Dasuprovidesanextensiblesetofmeasurementprimitives(modules)andaprogrammableAPItocombinethemintomeasurementexperiments.Tables1349010th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’13)USENIX AssociationMethodParams.DescriptionaddProbeTask&#xprob;&#xpara;&#xms00;&#xtime;&#xs000;[&#xwhen;[Submitmeasurementrequestofthespeciedtype.commitResult&#xrepo;&#xrt00;Submitcompletedexperimentresultstodataserver.getClientIPs[]ReturnnetworkinformationabouttheclientincludingthelistofIPaddressesassigned(bothpublicandprivate).getDnsServers[]ReturnthelistofDNSserversconguredattheclient.getEnvInfo[]Returninformationaboutthepluginandthehostnode,includingOSinformationandtypesofmeasurementprobesavailabletotheexperimenter.Table1:DasuAPI–Methods.ProbeParams.DescriptionPINGÞst;&#x-lis;&#xt-60;�(IP/name)Usethelocalhostpingimplementationtosendpacketstoahost.TRACEROUTEÞst;&#x-lis;&#xt-60;�(IP/name)Printtheroutepacketstaketoanetworkhost.NDT&#xserv;r00;[RuntheM-LabNetworkDiagnosticTool[29].DNS&#xserv;r00;[|&#xtime;&#xout0;[|&#xtcp/;&#xudp0;[|&#xopti;&#xons0;[| NS-;&#xmsg0;|Þst;&#x-lis;&#xt000;SubmitDNSresolutionrequesttoasetofservers.HTTP[server]|&#xport;[|&#xHTTP;&#x-Req;[|&#xurl-;&#xlist;SubmitHTTPrequesttoaagivenpair.Table2:DasuAPI–Measurementmodulescurrentlysupported.and2provideasummaryofthisAPIandthecurrentsetofmeasurementprimitivessupported.TheAPIincludessomebasicaccessormethods(e.g.).Themethodservestorequesttheexecutionofmeasurementsatagivenpointintime.ThemethodallowsresultsfromtheexperimenttobesubmittedtotheDataServiceafterDasuprovideslow-levelmeasurementtoolsthatcanbecombinedtobuildawiderangeofmeasurementexperiments.Currentlyavailablemeasurementprimi-tivesincludetraceroute,ping,NetworkDiagnosticTool(NDT)[29],HTTPGETandDNSresolution.Whilethissetiseasilyextensible(bytheplatformadministrators)wehavefounditsufcienttoallowcomplexexperimentstobespeciedclearlyandco

ncisely.Forinstance,theexperimentfortheR
ncisely.Forinstance,theexperimentfortheRoutingAsymmetrycasestudy(Sec.5.1)wasspeciedusingonly3differentruleswithanaverageof24linesofcodeperrule.MeasurementsprimitivesareinvokedasynchronouslybytheCoordinator,whichmultiplexesresourcesacrossexperiments.ProgressandresultsarecommunicatedthroughasharedWorkingMemory;throughthisworkingmemory,anexperimentcanalsochainrulesthatsched-ulemeasurementsandhandleresults.Inadditiontotheseactivemeasurements,Dasulever-agesthenaturally-generatedBitTorrenttrafcaspassivemeasurements(particularlyinthecontextofbroadbandcharacterization[6])bycontinuouslymonitoringtheend-hostInternetconnection.Devisinganinterfacetoexposethesepassivelycollectedmeasurementstoexperimentersispartoffuturework.ASimpleExample.Toillustratetheapplicationofrules,wewalkthroughtheexecutionofasimpleexperi-mentfordebugginghighlatencyDNSqueries.Figure2liststherulesthatimplementthisexperiment.Whenrule#1istriggered,itrequestsaDNSresolutionforadomainnameusingtheclient’sconguredDNSserver.WhentheDNSlookupcompletes,rule#2extractstheIPaddressfromtheDNSresultandschedulesapingmeasurement.Afterthepingcompletes,rule#3checksthepinglatencytotheIPaddressandschedulesatraceroutemeasurementifthisislargerthan50ms.DelegatingCodeExecutiontoClientsDasumanagesconcurrentexperiments,includingre-sourceallocation,viatheExperimentAdministrationService.Asclientsbecomeavailable,theyannouncetheirspeciccharacteristics(suchasclientIPprex,connectiontype,geographiclocationandoperatingsys-tem)andrequestnewexperimenttasks.TheAdministration(EA)Serviceassignstaskstoagivenclientbasedonexperimentrequirementsandcharacter-isticsofavailableclients(e.g.randomsampleofDSLusersinBoston).Inthesimplestofexperiments,everyDasuclientassignedtoanexperimentwillreceiveandexecutethesameexperimenttask(speciedasastand-alonerulesle).Dasualsoenablesmoresophisticatedexperimentswhereexperimentersspecifywhichclientstouseandhowtoexecutetasksbasedonclientcharacteristics.4USENIX Association 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’13)491rule"(1)ResolveIPaddressthroughlocalDNS"$fact:FactFireAction(action=="resolveIp");addProbeTask(ProbeType.DNS,"example.com");rule"(2)HandleDNSlookupresult"$dnsResult:FactDnsResult(toLookup=="example.com")Stringip=$dnsResult.getSimpleResponse();addProbeTask(ProbeType.PING,ip);rule"(3)Handlepingmeasurementresult"$pingResult:FactPingResult()if($pingResult.getRtt()�50)addProbeTask(ProbeType.TRACEROUTE,$pingResult.ip);Figure2:ExamplemeasurementexperimentfordebugginghighlatencyDNSqueries.Dasuadoptsatwo-tieredarchitecturefortheEAService,withaprimaryserver,responsibleforresourceallocation,andanumberofsecondaryserversinchargeofparticularexperiments.ThePrimaryEAserveractsasabroker,allocatingclientstoexperiments,byassigningthemtotheresponsiblesecondaryserver,basedonclients’characteristicsandresourceavailability.TheSecondaryEAserverisresponsiblefortaskparame-terizationandallocationoftaskstoclientsaccordingtotheexperiment’slogic.Whilethecustomizedtaskassignedtoaclientisgeneratedbytheexperiment’ssecondaryserver,allcommunicationwithDasuclientsismediatedbytheprimaryserverwhoisresponsibleforauthenticatinganddigitallysigningtheassignedexperiments.SubmittingExternalExperiments.Dasusupportsthird-partyexperimentsthroughthetwo-tierarchitecturedescribedabove.AuthorizedresearchgroupshosttheirownSecondaryEAserver,withsecurityandaccountabil-ityprovidedthroughthePrimaryEAserver.Inadditiontoprovidingasafeenvironmentforexecut-ingexperiments,allexperimentssubmittedtoDasuarerstcuratedandapprovedbythesystemadministratorsbeforedeployment.Thiscurationprocessservesasan-othersafetycheckandensuresthatadmittedexperimentsarealignedwiththeplatform’sstatedgoals.SecurityandSafetySafelyconductingmeasurementsisacriticalrequirementforanymeasurementplatformandparticularlyforonedeployedattheInternetedge.Wefocusontwosecurityconcerns:protectingtheho

standthenetworkwhenexecutingexperiments.
standthenetworkwhenexecutingexperiments.Weexpandontheformerhereanddiscussthelatterinthefollowingsection.Toprotectthehost,Dasuusesasandboxedenviron-mentforsafeexecutionofexternalcode,ensuressecurecommunicationwithinfrastructureservers,andcarefullylimitsresourceconsumption.ExperimentSandbox.Toensuretheexecutionsafetyofexternalexperiments,Dasuconneseachexperimenttoaseparatevirtualmachine,instantiatedwithlimitedresourcesandwithasecuritymanagerthatimplementsrestrictivesecuritypolicesakintothoseappliedtoun-signedJavaapplets.Inaddition,allDasuexperimentsarespeciedasasetofrulesthatareparsedforunsafeimportsatloadtime,restrictingthelibrariesthatcanbeimported.Dasuinspectstheexperiment’ssyntaxtreetoensurethatonlyspecicallyallowedfunctionalityisincludedandrejectsasubmittedexperimentotherwise.Securecommunication.Toensuresecurecommu-nicationbetweenparticipatinghostsandinfrastructureservers,allcongurationandexperimentrulelesservedbytheEAServicearedigitallysignedforauthenticityandallongoingcommunicationswiththeservers(e.g.forreportingresults)areestablishedoversecurechan-Limitsonresourceconsumption.Dasumustcare-fullycontroltheloaditsexperimentsimposeonthelocalhost,aswellasminimizetheimpactthatusers’interactions(i.e.,withthehostandtheapplication)canhaveonexperiments’results.Tothisend,Dasulimitsconsumptionofhosts’resourcesandrestrictsthelaunchingofexperimentstoperiodsoflowresourceutilization;themonitoredresourcesincludeCPUtime,networkbandwidth,memoryanddiskspace.TocontrolCPUutilization,DasumonitorsthefractionofCPUtimeconsumedbyeachsystemcomponent(includingthebasesystemandeachdifferentprobemodule).DasuregulatesaverageCPUutilizationbyimposingtime-delaysontheactivityofindividualprobemoduleswhenevertheir“fairshare”ofCPUtimehasbeenexceededoverthepreviousmonitoringperiod.Dasualsoemployswatchdogtimerstocontrolforlong-runningexperiments.Tocontrolbandwidthconsumption,Dasupassivelymonitorsthesystembandwidthusageandlaunchesac-tivemeasurementsonlywhenutilizationisbelowcertainthreshold(weevaluatetheimpactofthispolicyonexperimentexecutiontimeinSec.4.3).Dasuusesthe95thpercentileofclient’sthroughputratesmeasuredbyNDTtoestimatethemaximumbandwidthcapacityofthehostandcontinuoslymonitorshostnetworkactivity(usingthecommonlyavailabletool).Basedonpre-computedestimatesofapproximatebandwidthconsumptionforeachprobe,Dasulimitsprobeexecutionbyonlylaunchingthosethatwillnotexceedthepredeter-minedaveragebandwidthutilizationlimit.AdditionallyDasureliesonasetofpredenedlimitsonthenumberCurrently15%ofanymonitoredresource.549210th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’13)USENIX Associationofmeasurementprobesofeachtypethatcanbelaunchedpermonitoredinterval.Whileclientsareallowedtodispensewiththeirentirebudgetatonce,thecombinedbandwidthconsumedbyallprobemodulesmustremainbelowthespeciedlimit.Torestrictmemoryconsumption,Dasumonitorstheallocatedmemoryusedbyitsdifferentdatastructuresandlimits,forinstance,thenumberofqueuedprobe-requestsandresults.MeasurementresultsareofoadedtodiskuntiltheycansuccessfullybereportedtotheDataService.Diskspaceutilizationisalsocontrolledbylimitingthesizeofthedifferentprobe-resultlogs;olderresultsaredroppedrstwhenthepre-determinedquotalimitshavebeenreached.Inadditiontocontrollingtheloadonandguaranteeingthesafetyofvolunteerhosts,Dasumustcontroltheimpactthatmeasurementexperimentscollectivelymayhaveontheunderlyingnetworkandsystemresources.Forinstance,althoughtheindividuallaunchrateofmeasurementsislimited,alargenumberofclientsprobingthesamedestinationcanoverloadit.Tothisend,Dasuintroducestwonewconstructs-ex-perimentleaseselasticbudgets,toefcientlyallowthescalableandeffectivecoordinationofmeasurementsamongpotentiallythousandsofhosts.Inthefollowingparagraphs,wedescribebothconstructsandDasu’sapproachtocoordination.ExperimentLeases.Tosupportthenecessaryne-grainedcontrolofresourceu

sage,weintroducetheconceptofexperimentle
sage,weintroducetheconceptofexperimentleases.Ingeneral,aacontractthatgivesitsholderspeciedrightsoverasetofresourcesforalimitedperiodoftime[19].Anexperimentleasegrantstoitsholdertherighttolaunchanumberofmeasurementprobes,usingthecommoninfrastructure,from/towardaparticularnetworklocation.Originand/ortargetsfortheprobescanbespeciedasIP-prexesordomainnames(otherforms,suchasgeographiclocation,couldbeeasilyincorporated).ExperimentleasesaremanagedbytheEAService.ThePrimaryEAserverensuresthattheaggregateduseofresourcesbythedifferentexperimentsiswithinthespeciedbounds.SecondaryEAserversareresponsi-bleformanagingexperimentleasestocontroltheloadimposedbytheirparticularexperiments.TocoordinatetheuseofresourcesbytheDasuclientstakingpartinanexperiment,werelyonadistributedcoordinationservice[23].TheCoordinationServicerunsonwell-provisionedservers(PlanetLabnodes)usingreplicationforavailabilityandperformance.ClientsreceivethelistofcoordinationserversaspartoftheexperimentBeforebeginninganexperiment,clientsmustcontactacoordinatorservertoannouncetheyarejoiningtheexperimentandobtainanassociatedlease.Asprobesarelaunched,theclientssubmitperiodicupdatestothecoordinationserversaboutthedestinationsbeingprobed.TheEAServiceusesthisinformationtocom-puteestimatedaggregateloadperdestinationandtoupdatetheassociatedentriesintheexperimentlease.Beforerunningameasurement,theCoordinatorcheckswhetheritviolatestheconstraintonthenumberofprobesallowedfortheassociatedsourceanddestination,andifsodelaysit.Afteraleaseexpires,thehostmustrequestanewleaseorextendthepreviousonebeforeissuinganewmeasurement.Thechoiceoftheleasetermpresentsatrade-offbetweenminimizingoverheadontheEAServiceversusminimizingclientoverheadandmaximizingitsuse.ElasticBudget.Anexperimentleasegrantstoitsholdertherighttolaunchanumberofmeasurementprobes(i.e.,abudget)from/towardaparticularnetworklocation.Duetochurnanduser-generatedactions,thenumberofmeasurementprobesaDasuclientcanlaunchbeforeleaseexpiration(i.e.,thefractionoftheallocatedbudgetactuallyused)canvarywidely.Toaccountforthis,Dasuintroducestheideaofelasticbudgetsexpandandcontractbasedonsystemdynamics.ElasticbudgetsarecomputedbytheEAServiceandusedtoupdateboundsonexperimentleasesdistributedtoDasuclients.TheEAServicecalculatestheelasticbudgetperiodicallybasedonthecurrentnumberofclientsparticipatingintheexperiment,thenumberofmeasurementprobesallowed,assignedandcompletedbyeachclient.TheEAServiceusesthiselasticbudgettocomputemeasurementprobebudgetsforthenextleaseperiodforeachparticipatingclient.Thisapproachiswellsuitedforexperimentswheretheserverknowsaprioriwhatdestinationseachclientshouldprobe.Inthecaseofexperimentswherethedestinationstobeprobedarenotassignedbytheserver,butobtainedbytheclientsthemselves(throughaDNSresolutionforexample),thesameapproachcanbeusedifweconservativelyassumethataclientwilllaunchthemaximumnumberofprobesperunitoftimewheneveritisonline.SynchronizationDasualsoprovidessupportforInternetexperimentsthatrequiresynchronizedclientoperation(e.g.[34,41]).Forcoarse-levelsynchronization,Dasuclientsincludeacron-likeprobe-schedulerthatallowstheschedulingofmeasurementsforfutureexecution.AllDasuclientspe-riodicallysynchronizetheirclocksusingNTP.Assumingclients’clocksarecloselysynchronized,anexperimentcanrequestthe“simultaneous”launchofmeasurements6USENIX Association 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’13)493RegionPenetrationDasuDasuTotalTotalCountriesNorthAmerica78.6%21.45%60%Oceania/Australia67.5%3.82%6%Europe61.3%59.25%73%L.America/Carib.39.5%1.68%65%MiddleEast35.6%1.52%73%Asia26.2%2.59%57%Africa13.5%9.66%34%Table3:InternetpenetrationandDasucoverage(aspercentageofitstotalpopulationof90,222)byJanuary2013.byasetofclients.Wehavefoundthistobesufcienttoachievetasksynchronizationontheorderof1-3seconds.Forner-grainedsynchronization(ontheorderofmilliseconds),Dasuadoptsare

motetriggeredexecutionmodel.Allsynchroni
motetriggeredexecutionmodel.AllsynchronizedclientsmustestablishpersistentTCPconnectionswithoneofthecoordinationservers.Theseconnectionsarelaterusedtotriggerclientsactionsataprecisemoment,takingintoaccountnetworkdelaysbetweenclientsandcoordinationservers.WehaveimplementedDasuasanextensiontoapopularBitTorrentclient[43]asitoffersalargeandwidespreadclientpopulationandapowerfulinterfaceforextensions.WehavemadeDasupubliclyavailablesinceJune2010.Toparticipatingusers,DasuprovidesinformationabouttheservicetheyreceivefromtheirISP[6,36].Accesstosuchinformationhasprovensufcientincentiveforwidespreadsubscriptionwithover90Kuserswhohaveadoptedourextensionwithminimumadvertisement.ThissectiondemonstrateshowDasuclientscollec-tivelyprovidebroadnetworkcoverage,sufcientlyhighavailabilityandne-grainedsynchronizationforInternetexperimentation.DasuCoverageWeshowthecoverageofDasu’scurrentdeploymentintermsofgeographyandnetworktopology.Table3listsbroadbandpenetrationineachprimarygeographicregionandcomparesthesenumberswiththosefromourcurrentDasu’sdeployment.GiventhehighInternetpenetrationnumbersinEuropeandNorthAmerica,thedistributionofDasuclientsperregionisnotsurprising.Note,however,thepenetrationofDasuclientsperregion,measuredasthepercentageofcountriescovered.Asthetableshows,Dasupenetrationisover57%formostregionsandisparticularlyhighforLatinAmerica/Caribbean(65%)andtheMiddleEastUpondownload,usersareinformedofbothrolesofDasu.Userscan,atanypoint,opttodisableexperimentsfromrunningand/orreportingperformanceinformation,withoutlosingaccesstoDasu’sbroadbandbenchmarkinginformation.http://www.internetworldstats.comTier2&STP&Eyeball&PeerDistributionTier1&STP&Eyeball&ASDistributionFigure3:DistributionofDasupeersperAS(left).DistributionofASescoveredbyDasupeers(right).(73%),twoofthefastestgrowingInternetregions.EveninAfricaDasupenetrationreaches34%.WealsoanalyzeDasu’snetworkcoverageintermsofASeswherehostsarelocated.Withourexistinguser-baseattheendofJanuary2013,wehaveDasuclientsin1,802differentASes.WeclassifytheseASesfollowingarecentlyproposedapproach[13],asfollows:Tier-1:11knownTier-1sLTP:Large(nontier-1)transitprovidersandlarge(global)communicationsserviceprovidersSTP:Smalltransitprovidersandsmall(regional)communicationserviceprovidersEyeball:Enterprisecustomersoraccess/hostingprovidersFigure3ausesthisclassicationtoillustratewhereDasupeersaredeployed.Asthegureshows,93%ofDasupeersarelocatedinsmalltransitprovidersandeyeballASes;withonlyminimalpresenceinlargetransitandTier-1providers.Figure3bpresentsthedistributionofalltheASescoveredbyDasupeers.Thisgureshowsthat73%oftheASescoveredbyDasuareeyeballASes,highlightingtheeffectivenessofDasuasaplatformforcapturingtheviewfromthenetworkedge.DasuDynamicsInthissection,weshowthatthechurnfromDasuclientsissufcientlylowtosupportmeaningfulexperimenta-tion.ThischurnisaresultofboththevolatilityofDasu’scurrenthostingapplication(i.e.BitTorrent)andthatoftheendsystemsthemselves.Inthefollowinganalysis,wefocusonthehostingapplicationdynamics.Inparticular,weinvestigatewhatportionofclientsareonlineatanymoment,andwhethertheirsessiontimessupportcommonmeasurementdurations.First,weanalyzeDasuclients’availability,usingthepercentageofclientsonlineatanygivenhourovera31-dayperiod.Figure4plotsthisforthemonthofJanuary2013.Thefractionofavailableclientsduringtheperiodvaries,onaverage,between39%and44%ofthetotalnumberofuniqueusersseenduringaday,withatotalof1,473activeuniqueusersforthemonth.Withrespectto49410th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’13)USENIX AssociationFigure4:NumberofonlineDasuclientsovera24-hourperiod.Thefractionrangesfrom39-44%ofthetotalnumberofuniqueusers,onaverage.Figure5:SessiontimedistributionofDasuclients(timebetweentheirjoiningandleavingthesystem).theoverallstabilityoftheplatform,forthesamemonthofJanuary2013,wesawatotalof1,303installs,61useruninstallsand21userswhodisa

bledreportingwhilecontinuingtorunDasu.Ne
bledreportingwhilecontinuingtorunDasu.Next,weanalyzehowthedurationofexperimentsislimitedbyclientsessiontimes.Sessiontimeisdenedastheelapsedtimebetweenitjoiningthenetworkandsubsequentlyleavingit.Thedistributionofclients’sessiontimespartiallydeterminesthemaximumlengthofthemeasurementtasksthatcanbe“safely”assignedtoDasuclients.Figure5showsthecomplementarycumulativedistributionfunctionofsessiontimesforthestudiedperiod.Thedistributionisclearlyheavy-tailed,withamediansessiontimeforDasuclientsof178minutesorGivenanaveragesessiontime,thefractionoftasksthatareabletocompletedependsonthedurationofthe 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 10 20 30 40 50 60CCDF [Ytime (minutes)Figure6:TasktimedistributionforcompletedtasksbyDasuclients.Themediantasksuccessfullycompletesin5minutes. 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8fraction of peersfraction of probes80% upload80% download70% upload70% download60% upload60% downloaddownloaduploadFigure7:Distributionoffractionofprobesperpeerthataredelayedduetobandwidthconstraintsattheclient.Figure8:Distributionofexperimentprobesubmissionforclients.Over55%arelaunchedafterbeingscheduled.task–afunctionofthenumberofactualmeasurementsandtheloadattheclient.Figure6showsthedistributionoftaskcompletiontimesforallexperimentscompletedbyDasupeersovera3-weekperiod.AllexperimentsduringthisperiodweredoneinthecontextofthecasestudyonIXPmapping(Sec.5),wereanexperimenttaskconsistsofasetoftraceroutesissuedbyclientstodiscoverpotentialpeerings.Thegureshowsthatthemediantaskisabletosuccessfullycompleteinlessthan5minutes.Theplotalsoshowsthatnearlyalltasksareabletocompletesuccessfullyinthefaceofchurn,with70%oftasksnishinginlessthan12minutes.ControllingExperimentationLoadTominimizeDasu’simpactonhostapplicationper-formanceandtoensurethatuserinteractionsdonotinterferewithscheduledmeasurements,Dasuenforcespre-denedlimitsonthenumberofprobesexecutedperunittimeandschedulesmeasurementsduringlowutilizationperiods.Weevaluatetheimpactofoneoftheserestrictions(onbandwidthutilization)onexperi-mentexecutionbydeterminingtheportionofscheduledmeasurementsdelayed.Figure7showsaCDFofthefractionofprobesdelayedbyclientsduetodifferentbandwidthutilizationconstraints(60%,70%and80%),takenfromarandomsubsetofclientsoveratwo-weekperiod.Thedistribu-tionshows,forinstance,thatcappingatadownloaduti-lizationof80%,everyscheduledprobecanbelaunchedimmediatelyfor85%ofthepeers,andthatfor98%ofUSENIX Association 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’13)495 0 200 400 600 800 1000 1200 1400 0 5 10 15 20 25 30Request Arrival Time (milliseconds)Client Request IndexFigure9:Requestarrivaltimesatthetargetserver.Approximately80%ofrequestsarrivewithin300ms.thepeerslessthan20%oftheprobeswouldrequireanydelay.Incontrast,asmallerfractionofprobes(60%)experiencenodelaywhenan80%utilizationlimitisimposedontheuploaddirection.Thisisexpected,sincebroadbandusersareoftenallocatedloweruploadbandwidththandownload.Fig.8showsthequeueingtimeofprobesassignedtoDasuclientsforagivenexperimentovera1-weekperiod.Thegureshowsthatover55%oftheprobesarelaunchedinlessthanasecondafterbeingscheduled.ClientSynchronizationToevaluatethegranularityofDasu’sne-grainsyn-chronizationcapabilities,werunanexperimentwhereDasuclientswereinstructedtosimultaneouslylaunchanHTTPrequesttoaninstrumentedwebserver.Foraspanofveminutes,approximately30clientswererecruitedtocooperateintheexperiment.FollowingRamamurthyetal.[34],asclientsjoinedtheexperimenttheywereinstructedtomeasuretheirlatencytothetargetserveraswellastotheCoordinationServerandtoreportbacktheirndings.Attheendoftheveminutes,clientswerescheduledtolaunchtheirmeasurements(havingadjustedeachrequestbasedontheirmeasuredlatencies)whileweloggedthearrivaltimesofeachincomingHTTPrequestatthetargetserver.Werepeatedthisexper

iment10times.Figure9showsthemeanarrivalt
iment10times.Figure9showsthemeanarrivaltimeofeachrequestwithacrowdsizeof31clients.About80%oftherequestsarrivewithin300msofeachother,and91%oftherequestsarrivewithin1sofeachother.Thisresultisonparwiththesynchronizationof100sofmillisecondsreportedbyRamamurthyetal.[34]Variationsinthearrivaltimesofthetop20%ofrequestsareduetoqueuingdelaysinbroadbandnet-works[16]anderrorsinestimatingthelatencybetweenclientsandthecoordinatorserver.CaseStudiesInthissection,wepresentthreecasestudiesthatil-lustratetheuniqueperspectiveouredge-basedplatformFigure10:CCDFoffractionofDasu-PLpathhopsthatcanbedirectlymeasuredusingIPOptionsprobes.17%ofpathsreplytoprobesateachhop,meaningthatwecandeterminethecompletereversepath.bringstoInternetmeasurementandserveasexamplesofexperimentsmadepossibleusingDasu.ExtendingEarlierExperiments:RoutingAsym-Routingasymmetrycanimpacttheresultsofmeasure-menttoolssuchastraceroute.Forinstance,estimatesofdelaybetweenhostsaresubjecttoerrorsiftheforwardandreversepathdiffer.WeextendtheworkbyHeetal.[21],comparingroutingasymmetryforresearchandcommercialnetworks,byexaminingthepathsbetweenstub(Dasu)andresearch(PlanetLab)networks.Ideally,onewouldliketocontrolthehostsatbothendsofapathtodeterminetheforwardandreversepathsbetweenthem,andhavebothendpointsprobethepathconcurrentlytominimizetheimpactoffactorssuchasnetworkloadortime-of-dayonroutingdecisions.TheReverseTraceroutesystem[24]providesausefulapproachtodeterminethereversepathevenwhenonecontrolsonlyoneoftheendpoints.Theapproach,however,isnotalwayseffectiveasitcannotprobereversepathsinnetworkswhereroutersdonotreplytoIPOptionsprobes.AnumberoffeaturesofDasumakeitpossibletoconductanaccurateanalysisofpathasymmetrybetweennodeslocatedinstubnetworksvs.researchnetworksincludingtheabilitytoscheduleexperimentsandtosyn-chronizethelaunchingofmeasurementsacrossnodes.Wendthatfor28%ofthepathstestedbetweenDasuclientsandPlanetLabnodes(outof8,046)reversetraceroutewouldbeforcedtomakeanincorrectsym-metryassumptionbecauseasegmentofthereversepathtransitsatleastoneASthatdoesnotappearonthefor-wardpathandthatdoesnotrespondtoIPOptionsprobes.Figure10showsthatonly17%ofthepathsbetweenDasuandPlanetLabnodesrespondtoIPOptionsprobesateveryhop,thisincontrasttotheover40%ofpathsbetweenPlanetLabnodesreportedin[24].TostudyroutingasymmetrybetweenpairsofDasu-PlanetLab(Dasu-PL)andPlanetLab-PlanetLab(PL-PL)949610th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’13)USENIX AssociationAbsoluteAsymmetryNormalizedAsymmetryFigure11:CDFsofAS-levelasymmetryinDasu-PLandPL-PLpaths;60%ofDasu-PLpathsshowsomedegreeofasymmetry,vs.48%ofPL-PLpaths.nodes,welaunchedprobesacross8,046pathsbetweenDasuclientsandPlanetLabnodes,andacross10,067pathsbetweentwoPlanetLabnodes.Toensureaccuratemeasuresofroutingasymmetrywehadhostsatbothendpointsprobethepathconcurrently.Wemeasureroutingasymmetrybyfollowingthemethodologydescribedin[21].Thismethodmapshopsintheforwardpathtothoseofthereversepath(eitheratthelink-levelorAS-level)andassignsavalueof0ifthehopsareidenticalandavalueof1iftheyaredifferent.Throughdynamicprogramming,itthenselectsthemappingsforeachpaththatresultsintheminimaldistance.TheminimalcompositedissimilaritybetweenaforwardandreversepathisreferredtoastheAsymmetry(AA),whilethelength-basedAsymmetry(NA)isdenedasAAnormalizedbythelengthoftheround-trippath.TocomparetheasymmetryintheAS-levelpathsbetweenthetwosetsofpaths(i.e.,Dasu-PLandPL-PL),gures11aand11bshowthecumulativedistributionsoftheAS-levelAAandNAmetrics,respectively.ItcanbeobservedthattheDasu-PLpathsnotonlyhaveahigherpercentageofasymmetricroutes,butalsodisplayahighermagnitudeofasymmetrythanthePL-PLpaths.Tocomparethetwodatasetsatthelink-level,weagainfollowtheapproachdescribedinHeetal.[21]andusetheirheuristicstodetermineiftwoIPaddressescorrespondtotheinterfacesofthesamelink.TheseheuristicsconsidertwoIPaddressestobelongtothe(a)PlanetLab-PlanetLabFigure12:CDFsoflink-levelnormalizedasymmetryusingdifferen

theuristicsforIPtolinkmapping.Link-level
theuristicsforIPtolinkmapping.Link-levelNAismuchlowerforPL-PLpathsthanDasu-PLpaths.samepoint-to-pointlinkiftheybelongtothesame/30,/24,/16,orAS.Foreachofthefourheuristics,Figures12aand12bshowthecumulativedistributionsoftheresultingNAmetricforthePL-PLandDasu-PLpaths,respectively.Asnotedin[21],therst(/30)andlast(AS)heuristicsprovidetheupperandlowerbounds,ontheobservedInternetroutingasymmetryatthelinklevel.Whilegures11aand11bshowthatthetwosetsofpathsexhibitdifferencesinroutingasymmetryattheAS-level,gures12aand12bshowthesedifferencesaresignicantlymorepronouncedatthelink-levelbutdependgreatlyontheheuristicsused.QuestioningExistingExperiments:InferringAS-levelConnectivityThemodeloftheInternetasahierarchically-structuredor“tiered”networkofnetworksischanging[14,18,26].Theemergenceofnewtypesofnetworks(e.g.,contentproviders,webhostingcompanies,CDNs)andtheirresultingdemandsontheInternethaveinducedchangesinthepatternsofinterdomainconnections;however,theprecisedegreeandnatureofthesechangesremainspoorlyunderstood.InternetexchangePoints(IXPs)areanimportantpartoftherapidlydevelopingInternetecosystembecausetheyfacilitatethechanges,enablingdirectconnectionsbetweenmemberASes.ArecentstudyofalargeEuropeanIXPhasshownthatsomeofthelargestIXPs(e.g.,DEC-IXandAMS-IX)handletrafcvolumesthat10USENIX Association 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’13)49701234567891011121314151617181920212223prexes-----xxxxxxxxxxxxxx----Table4:Prex-basedpeeringatAmsterdamInternetExchange(AMS-IX)betweentwoASes.Columnsshowthehour,localtime.Legend:‘’probescrossedIXP;‘’probesdidnotcrossIXP;’-’noprobes.arecomparabletothosecarriedbysomeofthelargestglobalISPsandsupportpeeringfabricsconsistingofmorethan60%ofallpossiblepeeringsamongtheir400-500memberASes[2].However,despitetheirimportance,thereexistslittletonopubliclyavailableinformationaboutwhoispeeringwithwhomnoraboutthenatureofthesepeerings.Thesechangesinthenetwork’sstructuredemandchangestohowwehavetraditionallyconductedexper-iments.ForinstancetoquestionthestandardassumptionofhomogeneitythathasbeenmadewheninferringAS-levelconnectivityatIXPs[4,22,44]–whereasingletraceroutebetweentwoASesmembersofanIXPissufcienttodeclarethattheseASesare,asawhole,connectedintheASgraphbyapeer-peerlink–werequireanendemicpopulationofvantagepointsthatallowsforner-grainedmeasurements.Dasuprovidesanidealplatformtoexaminetheva-lidityofsuchassumptions.ItswidespreadanddiverseuserbaseprovidesvantagepointsinmultipleprexeswithinthesameASwhichallowsustoidentifyprex-specicfeaturesthatcouldnotbeidentiedfromasinglelocationinthenetwork.Additionally,Dasu’snear-continuousavailabilityofvantagepointsallowsustostudytemporaleffectsthatarecriticalfortheobservedkindofpeering.Lastly,conductingthiskindoftargetedexperimentsinvolvingspecicprexesinspecicASesatparticularIXPsreliescriticallyontheprogrammabilityofDasu.Toevaluatethevalidityofthishomogeneityassump-tion,wesetupanexperimenttolaunchmultipletracer-outeprobes,betweenthesamepairofmemberASesofagivenIXP,fromvantagepointslocatedinsidedifferentprexesofthesourceASandatdifferenthoursoftheday.Wefoundthatabout15%ofthepeeringlinksthatDasudiscoveredviolatedtheassumedhomogeneitycondition.Dependingontheprexes,theprobeseithercrossedthegivenIXPorweresentinsteadviaoneofthesourceAS’supstreamproviders.Table4showsaconcreteexampleofsuchne-grainedpeeringobservedbetweentwoASesatAMS-IX.ByprobingforpeeringsbetweenAS1andAS2repeatedlyfromdifferentprexesintheASesandseparatingtheprobesbythepeers’localtime,weobtainedaviewofthesewell-coveredpeeringsthroughouttheday.ForeachdatapointinthetablewecorroboratedtheresultacrossmultipletracerouteprobesThevariousreasonsforwhycertainASesengageinsuchn

on-traditionalpeeringarrangementsisbeyon
on-traditionalpeeringarrangementsisbeyondthescopeofthispaper.andobtainedthusanexampleofaconsistentprex-basedpeering–whileprobeslaunchedfromsourceprexAtowardsAS2areneverseencrossingtheIXP,probeslaunchedfromsourceprexBtowardsAS2seemtoalwaysgothroughtheIXP.Inshort,thediscoveryofsuchne-grainedorprex-specicpeeringarrangementsisproofthatthetraditionalviewthatasingletypeofASpeeringappliesuniformlyacrossallprexesofanASisnolongertenable.Thisndinghasclearimplicationsformeasurementandinfer-enceofAS-levelconnectivityandposesnewchallengesandrequirementsfortheplatformsandtechniquesusedforthistypeofstudies.PerformingNovelExperiments:EvaluatingaRecently-proposedDNSExtensionEDNS0extension()wasdevelopedtoaddresstheproblemsraisedbytheinter-actionbetweentheDNS-basedredirectiontechniquescommonlyemployedbyCDNsandtheincreasinguseofremoteDNSservices.CDNstypicallymapclientstoreplicasonthelocationoftheclient’slocalresolver;sincetheresolversofremoteDNSservicesmaybefarfromtheusers,thiscanresultinreducedCDNaimstoimprovethequalityofCDNredirectionsbyenablingtheDNSresolvertoprovidepartialclientlocation(i.e.client’sIPprex)directlytotheCDN’sauthoritativeDNSserver.iscurrentlybeingusedbyafewpublicDNSservices(e.g.,GoogleDNS)andCDNs(e.g.EdgeCast)andcanimproveCDNredirectionswithoutmodicationstoendhosts.Tounderstandtheperformancebenetsofthepro-extensionandcapturepotentialvariationsacrossgeographicregionswouldrequireaccesstoalargesetofvantagepoints.Thesevantagepointsshouldbelocatedinaccessnetworksaroundtheworldandallowissuingthenecessaryinterrelatedmeasurementprobes.ThesearesomeoftheuniquefeaturesthatDasuoffers.Dasu’sextensibilityallowsforthecreationandad-ditionofanewprobemoduletogenerateandparseECS-enabledDNSmessages.Additionally,Dasu’suserbaseallowsustoobtainrepresentativemeasurementsamplesfromdiverseregionsandcomparetrendsacrossgeographicareasbylookingattherelationshipsbetweenrawCDNperformance,relativeproportionsofclientsaffectedbytheextension,andthedegreeofperformanceimprovementprovidedbytheextension.ThisexperimentextendstheworkbyOttoetal.[30],whichexaminedtheimpactofvaryingtheamountof1149810th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’13)USENIX Associationinformationsharedby(i.e.prexlength)andcompareditsperformancetoaclient-basedsolution.WerstobtainCDNredirectionstoedgeserversbothwithextensionenabledanddisabled.Specically,wequeryGoogleDNS(8.8.8.8)foranEdgeCasthostname.Toobtainaredirectionwithdisabled,ourDNSprobemodulesendsaquerywiththeoptionthatspecies0bytesoftheclient’sIPprex—thiseffectivelydisablestheextension’sfunctionality.Fortheenabledquery,weprovidetheclient’s/24IPprex.AfterobtainingCDNedgeserverredirectionswithand’shelp,weconductHTTPrequeststobothsetsofCDNedgeserverstomeasuretheapplication-levelperformanceintermsoflatencytoobtaintherstbyteofcontent.Fortheresultsfromeachclient,wecomparethemedianperformancewithandwithoutbeingenabled.Weanalyzeresultsfromasubsetof1,185Dasuclientsthatconductedthisexperimentovera4monthperiodfromSeptember12th,2011toJanuary16th,Figure13showstherelationshipbetweenHTTPlatencywithdisabledandtheperformancebenets(latencysavings)withenabled.Weclassifyusersbygeographicregion;thepercentageslistedinthelegendindicatethefractionofallsampledclientsfromthatregion.Inallregions,sampledclientsarelocatedinadiversesetofnetworks;eveninOceania—theregionwithfewestclients—wecover9ISPsinAustraliaand4inNewZealand.ThegureplotsthesubsetofsamplesinwhichEDNSimpactedHTTPperformance.WhilewendclientsinalltheseregionsthatobtainedHTTPperformanceimprovementswiththesamplestendtoclusterbyregion.AlthoughclientsinNorthAmericaandWesternEuropebothtypicallyseeHTTPlatenciesbetween20and200ms,theNorthAmericanclientsgenerallyobtainhigherpercentagesav-ings.ThiswouldindicatethattheCDN’sinfrastructureinNorthAmeric

aisrelativelydenseincomparisontothatofth
aisrelativelydenseincomparisontothatofthepublicDNSservice’sdeployment.ClientsinOceaniatypicallyhaverelativelyhighHTTPlaten-ciesbetween200and1000mswithdisabled—butcommonlyrealizesavingsof70–90%withThisislikelyaresultofthespecicdeploymentsoftheCDNandDNSservices;althoughthereareactuallyCDNedgeserversneartoclientsinthisregion,itappearsthatthenearestGoogleDNSserversarefartheraway,resultinginreducedHTTPperformancewhendisabled.Finally,wecomparethenumberofclientswithbenetsfrombetweenEasternEuropeandOceania;whileclientsinOceaniaactuallycompriseaslightlysmallerfractionoftheoverallsample,thenumberofEachparticipatingclientrunstheexperimentonceoverthattime.205010020050010002000500010000HTTP latency without ECS (ms)0102030405060708090100Latency savings with ECS (%)N. America (41.9%)Asia (6.8%)E. Europe (5.2%)W. Europe (37.6%)Figure13:HTTPlatencyvs.theperformancebenetsprovidedbyECS,bygeographicregion.Percentagesinthelegendindicatethegeographiccompositionofthedataset.clientsthatactuallyobservedbetterperformanceismuchhigherthanforclientsinEasternEurope.WepresentedDasu,ameasurementexperimentationplatformfortheInternet’sedgethatsupportsandbuildsonbroadbandcharacterizationasanincentiveforadop-tion.WedescribedDasu’sdesignandimplementationandusedourcurrentdeploymenttodemonstratehowparticipatingnodescollectivelyofferbroadnetworkcov-erage,highavailabilityandne-grainedsynchronizationtoenableInternetmeasurementexperimentation.Dasurepresentsbutasinglepointinalargedesignspace.Wedescribedourrationalforourcurrentdesignchoices,butexpecttorevisitsomeofthesedecisionsaswelearnfromourownandotherexperimenters’useoftheplatform.WepresentedthreecasestudiesthatdemonstrateDasu’scapabilitiesandillustratetheuniqueperspectiveitbringstoInternetmeasurement.Aspartofongoingwork,weareexploringtheuseofnodeavailabilitypredictionforexperimentation,approachestoensuretheintegrityofexperimentalresults,andallowingne-grainedcontrolofexperimentsbyendusers.TheDasuclientisopensourceandavailablefordownloadfromAcknowledgementsWewouldliketothankLorenzoAlvisi,ourshepherdRebeccaIsaacs,andtheanonymousreviewersfortheirinvaluablefeedback.WearealwaysgratefultoPaulGardnerforhisassistancewithVuzeandtheusersofoursoftware.ThisworkwassupportedinpartbytheNationalScienceFoundationthroughAwardsCNS1218287,CNS0917233andII0855253andbyaGoogleFacultyResearchAward.ReferencesencesBroadband&IPTVprogressreport.//www.broadband-forum.org/news/12USENIX Association 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’13)499,March2012.2012.B.Ager,N.Chatzis,A.Feldmann,N.Sarrar,S.Uhlig,andW.Willinger.Anatomyofalargeeuropeanixp.InProc.ofACM,2012.2012.D.Andersen,H.Balakrishnan,F.Kaashoek,andR.Morris.Resilientoverlaynetworks.InProc.ACMSOSP,2001.2001.B.Augustin,B.Krishnamurthy,andW.Willinger.IXPs:mapped?InProc.ofIMC,2009.2009.BGP4.IPv4LookingGlassSites.Sites.Z.S.Bischof,J.S.Otto,M.A.Sanchez,J.P.Rula,D.R.Choffnes,andF.E.Bustamante.CrowdsourcingISPcharacterizationtothenetworkedge.InProc.ofW-MUST,2011.2011.Caida.Ark.Ark.M.CasadoandT.Garnkel.Opportunisticmeasurement:Spuriousnetworkeventsasalightinthedarkness.InProc.of,2005.2005.K.Chen,D.R.Choffnes,R.Potharaju,Y.Chen,F.E.Bustamante,D.Pei,andY.Zhao.Wherethesidewalkends:extendingtheInternetASgraphusingtraceroutesfromP2Pusers.InProc.ACMCoNEXT,2009.2009.D.R.ChoffnesandF.E.Bustamante.PitfallsfortestbedevaluationsofInternetsystems.SIGCOMMComput.Commun.Rev.,April2010.2010.D.R.Choffnes,F.E.Bustamante,andZ.Ge.Crowdsourcingservice-levelnetworkeventmonitoring.InProc.ofACM,2010.2010.D.R.Choffnes,M.A.Sanchez,andF.E.Bustamante.Networkpositioningfromtheedge:anempiricalstudyoftheeffectivenessofnetworkpositioninginP2Psystems.InProc.IEEE,2010.2010.A.DhamdhereandC.Dovrolis.TenyearsintheevolutionoftheInternetecosystem.InProc.ofIMC,2008.2008.A.DhamdhereandC.Dovrolis.TheInternetisat:modelingthetransitionfromatransithi

erarchytoapeeringmesh.InProc.ACMCoNEXT,2
erarchytoapeeringmesh.InProc.ACMCoNEXT,2010.2010.M.Dischinger,A.Haeberlen,I.Beschastnikh,K.P.Gummadi,andS.Saroiu.SatelliteLab:Addingheterogeneitytoplanetary-scalenetworktestbeds.InProc.ofACMSIGCOMM,2008.2008.M.Dischinger,A.Haeberlen,K.P.Gummadi,andS.Saroiu.Characterizingresidentialbroadbandnetworks.InProc.ofIMCIMCFCC.Broadbandnetworkmanagementpractices–enbancpublichearing,February2008.2008.P.Gill,M.Arlitt,Z.Li,andA.Mahanti.TheatteningInternettopology:naturalevolution,unsightlybarnaclesorcontrivedcollapse?InProc.ofPAM,2008.2008.C.G.GrayandD.R.Cheriton.Leases:Anefcientfault-tolerantmechanismfordistributedlecacheconsistency.InProc.ACM,1989.1989.K.P.Gummadi,H.V.Madhyastha,S.D.Gribble,H.M.Levy,andD.Wetherall.ImprovingthereliabilityofInternetpathswithone-hopsourcerouting.InProc.USENIXOSDI,2004.2004.Y.He,M.Faloutsos,S.Krishnamurthy,andB.Huffaker.OnroutingasymmetryintheInternet.InProceedingsofIEEE,2005.2005.Y.He,G.Siganos,M.Faloutsos,andS.Krishnamurthy.Lordofthelinks:aframeworkfordiscoveringmissinglinksintheInternettopology.IEEE/ACMTransactionsonNetworking,2009.2009.P.Hunt,M.Konar,F.P.Junqueira,andB.Reed.ZooKeeper:wait-freecoordinationforInternet-scalesystems.InProc.USENIXATC,2010.2010.E.Katz-Bassett,H.V.Madhyastha,V.K.Adhikari,C.Scott,J.Sherry,P.VanWesep,T.Anderson,andA.Krishnamurthy.Reversetraceroute.InProc.ofUSENIXNSDI,2010.2010.Keynote.Internethealthreport.report.C.Labovitz,S.Iekel-Johnson,D.McPherson,J.Oberheide,andF.Jahanian.Internetinter-domaintrafc.InProc.ofACM,2010.2010.J.Ledlie,P.Gardner,andM.Seltzer.Networkcoordinatesinthewild.InProc.ofUSENIXNSDI,2007.2007.H.M.Madhyastha,T.Isdal,M.Piatek,C.Dixon,T.Anderson,A.Krishnamurthy,andA.Venkataramani.iPlane:aninformationplanefordistributedsystems.InProc.USENIXOSDI,2006.2006.MLabs.Networkdiagnostictool.measurementlab.net/run-ndt/ndt/J.S.Otto,M.A.Sanchez,J.P.Rula,andF.E.Bustamante.ContentDeliveryandtheNaturalEvolutionofDNS:remotednstrends,performanceissuesandalternativesolutions.InProc.of,2012.2012.PlanetLab.PlanetLab.http://www.planet-lab.org/lab.org/H.Pucha,Y.C.Hu,andZ.M.Mao.Ontheimpactofresearchnetworkbasedtestbedsonwide-areaexperiments.InProc.of,2006.2006.M.Rabinovich,S.Triukose,Z.Wen,andL.Wang.Dipzoom:TheInternetmeasurementsmarketplace.InProc.IEEE,2006.2006.P.Ramamurthy,V.Sekar,A.Akella,B.Krishnamurthy,andA.Shaikh.Remoteprolingofresourceconstraintsofwebserversusingmini-ashcrowds.InProc.USENIXATC,2008.2008.RIPE.RIPEatlas.atlas.SamKnows.Accuratebroadbandinformationforconsumers,governmentsandISPs.ISPs.Y.ShavittandE.Shir.DIMES:LettheInternetmeasureitself.SIGCOMMComput.Commun.Rev.,35(5),October2005.2005.C.R.Simpson,JrandG.F.Riley.NETIhome:Adistributedapproachtocollectingend-to-endnetworkperformancemeasure-ments.InProc.ofPAM,2004.2004.N.Spring,L.Peterson,A.Bavier,andV.Pai.UsingPlanetLabfornetworkresearch:Myths,realitiesandbestpractices.ACMSIGOPSOp.Sys.Rev.,2006.2006.N.Spring,D.Wetherall,andT.Anderson.Scriptroute:apublicInternetmeasurementfacility.InProc.USENIXUSITS,2003.2003.A.-J.Su,D.Choffnes,A.Kuzmanovic,andF.Bustamante.DraftingbehindAkamai:Travelocity-baseddetouring.InProc.ofACMSIGCOMM,Sep.2006.2006.S.Sundaresan,W.deDonato,N.Feamster,R.Teixeira,S.Crawford,andA.Pescape.Broadbandinternetperformance:aviewfromthegateway.InProc.ofACMSIGCOMM,2011.2011.Vuze.Vuze.uze.K.Xu,Z.Duan,Z.-L.Zhang,andJ.Chandrashekar.OnpropertiesofInternetexchangepointsandtheirimpactonAStopologyandrelationship.InNETWORKING.2004.2004.R.Zhang,C.Tang,Y.C.Hu,S.Fahmy,andX.Lin.ImpactoftheinaccuracyofdistancepredictionalgorithmsonInternetapplications:ananalyticalandcomparativestudy.InProc.IEEE,2006.13USENIX Association 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’13)487Dasu:PushingExperimentstotheInternet’sEdgeMarioA.SJohnS.OttoZacharyS.BischofDasuisaJapanesewordthatcanmean“toreveal”or“toexpose”.Pleasesee[6]forageneraldescriptionofDasu’ssupportforbroadbandcharacteri