Nick McKeown CS244 Advanced Topics in Networking Spring 2020 Sizing Router Buffers Appenzeller et al 2004 Context Guido Appenzeller At the time CS PhD student Founded Big Switch Networks ID: 1001866
Download Presentation The PPT/PDF document "Lecture 10: Buffer Sizing" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
1. Lecture 10: Buffer SizingNick McKeownCS244Advanced Topics in NetworkingSpring 2020“Sizing Router Buffers” [Appenzeller, et al. 2004]
2. ContextGuido AppenzellerAt the time: CS PhD studentFounded Big Switch NetworksCTO at VMware for networkingSigcomm Test of Time Award, 20152At the time of writingChallenging to build ISP routers with big buffers80% of world’s SRAMs used for router and switch buffers/countersISP routers sold with ~90% profit margin
3. Why should we care?BackgroundNo universal agreement on how big a router buffer should beOr why. Yet buffers are a major cause of variation in packet delay.Big buffers require large, slow DRAM memories……which complicate the design of large routers.It would be nice if we could use single chip switches and routers
4. Simple model of FCFS router buffer4BCCCC1NQ: Why does the router have a buffer (or queue)?Q: What factors determine the buffer’s size?
5. Simple model of a routerB1N1NCC
6. Simple Internet queueing model6
7. Early Internet Models7Leonard KleinrockProfessor at UCLA1963: 1st theoretical study of packet switching using queueing theory1969: 1st message sent over Internet.UCLA IMP (Interface Message Processor)
8. 8Example model of single packet queuePoisson Traffic, fixed size packetsM/D/1Poisson arrivals, rate lBCObservations: Packet drop rate is smallIndependent of C, RTT, number of flows, etc.Define: LoadThen: drop rate loss < 1%
9. Q: How well does this model fit today’s Internet?9The model assumes traffic is generated “open loop” by the source.Today’s Internet carries mostly TCP traffic, which uses a closed loopcongestion control algorithm.
10. ABRouter bufferLink rate = CLink rate > CObservations: An arriving packet sees a usually-full queueRTT is pretty much constant (very different from single flow case)Drops are random events With multiple flows, RTT is less variable
11. tcwndBuffer occupancyand RTTOne flow vs multiple flowsBuffer occupancyand RTTt(Zoom in)cwndOne of the flowsOne flowtBufferOccupancyand RTTMultiple flows
12. A3A2A1Geometric intuition for throughput equationDroptcwnd1RTT W1W2Area But Therefore, = Expected Area, Segments sent duringThroughput: Throughput, Drop rate, Combining… segments/sec CombiningNote: To convert from packets/second to bits/second, multiply throughput by the packet size (e.g. MSS) ?
13. Interpreting the throughput equation If RTT doubles, throughput halves.If packet drop rate increases from 1% to 4%, throughput halves.
14. One reason to care about buffer sizeWith on-chip buffers we can build higher capacity switch ASICs
15. Switch Chips are Limited by Serial I/O Capacity to the outside world123N...........R e.g. 12.8 Tb/s = 128 x 100Gb/s Single chip switch ASICSmall on-chip bufferinge.g. 64 Mbytes123N/2....RSwitch ASIC with external memoryLarge off-chip bufferinge.g. 8 GbytesDRAM Switch ASIC #1I/O Capacity: C Switch ASIC #2I/O Capacity: C/2
16. Switch Chips are Limited by Serial I/O Capacity I/O Capacity: CDRAM I/O Capacity: C/2CC/2DRAM
17. CHow many switch chips with capacity C/2 do we need to make a router with capacity C?
18. We need 6 ASICs with capacity C/2 I/O Capacity:C/2C/4DRAM I/O Capacity: C/2C/4DRAM I/O Capacity: C/2C/4DRAM I/O Capacity: C/2C/4DRAM I/O Capacity: C/2DRAM I/O Capacity: C/2DRAMTotal Router Capacity: C/8C/8C/8C/8
19. It is worth understanding where and when small on-chip buffers suffice
20. A brief history of buffer size
21. 19882019 Congestion Avoidance and ControlVJ & MK1994High Performance TCP in ANSNETCV & CSABCB “Buffer size should equal the bandwidth delay product”
22. ABCB2Tcwndtime cwndtimet
23. cwndtimeABCB2T cwndtime 111111t2 1 12
24. Time Evolution of a Single TCP Flow ABCB2T
25. Single TCP New Reno flow: 100% ThroughputIf then If then If then i.e. if end host knows 2T, buffer size is independent of RTT Example: Example: Example: Example:
26. 1988199420042020 Congestion Avoidance and ControlVJ & MKHigh Performance TCP in ANSNETCV & CSSizing RouterBuffersGA, IK, NMABC B where N is the number of long-lived flows
27. 27Synchronized FlowsAggregate window has same dynamicsTherefore buffer occupancy has same dynamicsRule-of-thumb still holds.t
28. 28ProbabilityDistributionB0~ Buffer SizeMany TCP Flows
29. Many AIMD flows: 100% Throughput Example: Example: Q: Do you think will hold for othercongestion control algorithms…?
30. You saidNikhil Athreya I think the assumptions made in the original paper are beginning to change. While they acknowledge that their work is mostly based off of TCP, they argue that single-packet sources (e.g. DNS) and constant-rate UDP sources (e.g. online games) can be modeled with short flows. However, a quick Google search tells me that (https://www.caida.org/research/traffic-analysis/tcpudpratio/) that the amount of UDP flows either has surpassed TCP flows or is at least comparable to it. I’m unsure how this would affect the results in this paper; perhaps because the results for short flows shows that average queue length is only dependent on link load and flow length, this shouldn’t be a problem, but I’d be interested to hear more about it.30
31. You saidKevin Baichoo It seems like from the backbone router vendor’s will disavow this smaller buffer sizes, as otherwise they can’t justify the prices for their equipment.Esther GoldsteinDo buffer sizes today go by the rule-of-thumb equation, or has the equation been refined in a different way than what was proposed in the paper? Isabel Victoria PapadimitriouIt seems that this is working under an assumption that TCP is TCP, and router manufacturers have to deal with that regarding buffer size. Why isn’t it the other way around (“this is the buffer size, make congestion control work”), or is it possible to think about some sort of joint optimization of buffer size and paradigm? 31
32. 198819942020 Congestion Avoidance and ControlVJ & MKHigh Performance TCP in ANSNETCV & CS2004Sizing RouterBuffersGA, IK, NMABC B2006Routers with Very Small BuffersME, YG, AG, NM, TR 1. Paced Traffic 2. Link utilization < 80% AssumptionsOnly 20-50 packet buffers.Consequences
33. 198819942020 Congestion Avoidance and ControlVJ & MKHigh Performance TCP in ANSNETCV & CS2004Sizing RouterBuffersGA, IK, NMABC B2006Routers with Very Small BuffersME, YG, AG, NM, TR2008Experimental Study ofRouter BuffersNB, YG, MG, NM, GS
34. Buffer Sizing ExperimentsSmall BuffersStanford University dorm networkUniversity of WisconsinInternet2Level 3 CommunicationsTiny BuffersInternet2 Sprint Advanced Technology LabUniversity of Toronto
35. Level 3 Communications ExperimentsHigh link utilization Long duration (about two weeks)Buffer sizes 190ms (250K packets), 10ms (10K packets), 2.5ms (2500 packets), 1ms (1000 packets)Load balancing over 3 links (2.5 Gb/s each)
36. Drop vs. Load, Buffer = 190ms, 10msMax Util = 96%
37. Drop vs. Load, Buffer = 1ms
38. Internet2 Experiments38Neda Beheshti2010
39. Summary of throughput and buffer sizeThroughput100%Buffer Size~ 90% 50-100 pkts
40. End.