Distributed Rate Limiting Barath Raghavan Kashi Vishwanath Sriram Ramabhadran Kenneth Yocum and Alex C Snoeren University of California San Diego Centralized network services ID: 332176
Download Presentation The PPT/PDF document "Cloud Control with" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1Slide2
Cloud Control with
Distributed Rate Limiting
Barath
Raghavan
,
Kashi Vishwanath,Sriram Ramabhadran, Kenneth Yocum, and Alex C. SnoerenUniversity of California, San DiegoSlide3
Centralized network services
Hosting with a single physical presence
However, clients are across the InternetSlide4
Running on a cloud
Resources and clients are across the world
Services combine these distributed resources
1
GbpsSlide5
Key challenge
We want to control
distributed
resources as if they were
centralizedSlide6
Ideal: Emulate a single limiter
Make distributed
feel
centralized
Packets should experience same limiter behavior
S
S
S
D
D
D
0 ms
0 ms
0 ms
LimitersSlide7
Distributed Rate Limiting (DRL)
Achieve
functionally equivalent behavior
to a central limiter
GlobalRandom Drop
Flow
Proportional Share
Packet-level(general)
Flow-level
(TCP specific)
Global
Token Bucket
1
2
3Slide8
Distributed Rate Limiting tradeoffs
Accuracy
(how close to
K
Mbps is delivered, flow rate fairness)+
Responsiveness(how quickly demand shifts are accommodated)Vs.Communication Efficiency(how much and often rate limiters must communicate)Slide9
Limiter 1
DRL Architecture
Limiter 2
Limiter 3
Limiter 4
Gossip
Gossip
Gossip
Estimate
local demand
Estimate
interval
timer
Set allocation
Global
demand
Enforce limit
Packet
arrivalSlide10
Token Buckets
Token bucket, fill rate
K
Mbps
PacketSlide11
Demand info
(bytes/sec)
Building a Global Token Bucket
Limiter 1
Limiter 2Slide12
Baseline experiment
Limiter 1
3 TCP flows
S
D
Limiter 2
7 TCP flows
S
D
Single token bucket
10 TCP flows
S
DSlide13
Global Token Bucket (GTB)
Single token bucket
Global token bucket
7 TCP flows
3 TCP flows
10 TCP flows
Problem: GTB requires near-instantaneous arrival info
(50ms estimate interval)Slide14
Global Random Drop (GRD)
5 Mbps
(limit)
4 Mbps
(
global
arrival rate)
Case 1: Below global limit, forward packet
Limiters send, collect global rate info from othersSlide15
Global Random Drop (GRD)
5 Mbps
(limit)
6 Mbps
(
global
arrival rate)
Case 2: Above global limit, drop with probability:
Excess
Global arrival rate
Same at all limiters
1
6
=Slide16
GRD in baseline experiment
Single token bucket
Global random drop
7 TCP flows
3 TCP flows
10 TCP flows
(50ms estimate interval)
Delivers flow behavior similar to a central limiterSlide17
GRD with flow join
(
50ms
estimate interval)
Flow 1 joins at limiter 1
Flow 2 joins at limiter 2Flow 3 joins at limiter 3Slide18
Flow Proportional Share (FPS)
Limiter 1
3 TCP flows
S
D
Limiter 2
7 TCP flows
S
DSlide19
Flow Proportional Share (FPS)
Limiter 1
Limiter 2
“3 flows”
“7 flows”
Goal: Provide inter-flow fairness for TCP flows
Local token-bucket
enforcementSlide20
Estimating TCP demand
Limiter 1
D
Limiter 2
3 TCP flows
S
D
1 TCP flow
S
1 TCP flow
SSlide21
Estimating TCP demand
Local
token rate (limit) = 10 Mbps
Flow A = 5 Mbps
Flow B = 5 Mbps
Flow count = 2 flowsSlide22
Estimating TCP demand
Limiter 1
1 TCP flow
S
D
Limiter 2
3 TCP flows
S
D
S
1 TCP flowSlide23
Key insight: Use a TCP flow’s rate to infer demand
Estimating
skewed
TCP demand
Local
token rate (limit) = 10 Mbps
Flow A = 2 Mbps
Flow B = 8 Mbps
Flow count ≠ demand
Bottlenecked elsewhereSlide24
Estimating
skewed
TCP demand
Local
token rate (limit) = 10 Mbps
Flow A = 2 Mbps
Flow B = 8 Mbps
Local Limit
Largest Flow’s Rate
10
8
=
Bottlenecked elsewhere
= 1.25 flowsSlide25
2.50 flows
Limiter 2
10 Mbps x 1.25
1.25 + 2.50
Flow Proportional Share (FPS)
Global limit = 10 Mbps
1.25 flows
Limiter 1
Set local token rate =
=
3.33 Mbps
Global limit x local flow count
Total flow count
=Slide26
Under-
utilized limiters
Limiter 1
D
S
1 TCP flow
S
1 TCP flow
Set local limit equal to actual usage
Wasted rate
(limiter returns to full utilization)Slide27
Flow Proportional Share (FPS)
(
500ms
estimate interval)Slide28
Additional issues
What if a limiter has no flows and one arrives?
What about
bottlenecked traffic
?What about varied RTT flows?What about short-lived vs. long-lived flows?
Experimental evaluation in the paperEvaluated on a testbed and over PlanetlabSlide29
Cloud control on
Planetlab
Apache Web servers on 10
Planetlab
nodes5 Mbps aggregate limitShift load over time from 10 nodes to 4 nodes
5 MbpsSlide30
Static rate limiting
Demands at 10 apache servers on
Planetlab
Demand shifts to just 4 nodes
Wasted capacitySlide31
FPS
(top)
vs. Static limiting
(bottom)Slide32
Conclusions
Protocol agnostic
limiting (extra cost)
Requires shorter estimate intervals
Fine-grained packet arrival info not requiredFor TCP, flow-level granularity is sufficientMany avenues left to exploreInter-service limits, other resources (e.g. CPU)Slide33
Questions!Slide34
FPS state diagram
Case 2:
Underutilized
Set local limit to actual usage
Case 1:
Fully utilized
Flows start/end,
Network congestion,
Bottlenecked flowsSlide35
DRL cheat-
proofness
Conservation of rate among limiters for FPS
EWMA compensates for past cheating with higher drop rates in the future
To cheat GRD, forever increase the demand
Authenticated inter-limiter communication assumedDifficult to quickly move traffic demandsSlide36
DRL applications
Cloud-based services (e.g. Amazon’s EC2/S3)
Content-distribution networks (e.g.
Akamai
)
Distributed VPN limitingInternet testbeds (e.g. Planetlab)
Overlay network service limitingSlide37
DRL ≠
QoS
DRL provides:
Fixed, aggregate rate limit
No service classes
No bandwidth guaranteesNo reservationsNo explicit fairness; implicit (TCP) fairnessSlide38
DRL provisioning
Providers
could
ensure that the limit K Mbps is available at all locations; wasteful
We expect it to be like current practice: statistical multiplexing
Even today, can’t guarantee bandwidth to all destinations with a single pipeSlide39
Experimental setup
Modelnet
network emulation on
testbed
40ms inter-limiter RTT
Emulated 100 Mbps linksNo explicit inter-limiter lossRan limiters across 7
testbed machines
On Linux 2.6; packet capture via iptables
ipqSlide40
Short flows with FPS
2 limiters, 10 bulk TCP vs. Web, 10Mbps limit
Web traffic based on CAIDA OC-48 trace
Loaded via
httperf
, Poisson arrivals (μ = 15)
CTB
GRD
FPS
Bulk
rate
6900.90
7257.87
6989.76
Fairness (Jain’s)
0.971
0.997
0.962
Web rate [0, 5K)
28.17
25.84
25.71
[5K, 50K)
276.18
342.96
335.80
[50K, 500K)
472.09
612.08
571.40
[500K, ∞)
695.40
751.98
765.26Slide41
Bottlenecked flows with FPS
Baseline experiment: 3-7 flow split, 10 Mbps
At 15s, 7 flow aggregate limited to 2Mbps upstream
At 31s, 1
unbottlenecked
flow arrives joins 7 flowsSlide42
RTT heterogeneity with FPS
FPS doesn’t reproduce RTT unfairness
7 flows (10 ms RTT) vs. 3 flows (100 ms RTT)
CTB
GRD
FPS
Short RTT (Mbps)
1.41
1.35
0.92
(
stddev
)
0.16
0.71
0.15
Long RTT
(Mbps)
0.10
0.16
0.57
(
stddev
)
0.01
0.03
0.05Slide43
Scaling discussion
How do various parameters affect scaling?
Number of flows present: per-limiter smoothing
Number of limiters: ~linear overhead increase
Estimate interval: limits responsiveness
Inter-limiter latency: limits responsivenessSize of rate limit delivered: orthogonalSlide44
Comm. Fabric requirements
Up-to-date information about global rate
Many designs are possible:
Full-mesh (all-pairs)
Gossip (contact k peers per round)
Tree-based…Slide45
Gossip protocol
Based on protocol by
Kempe
et al.
(FOCS 2003)
Send packets to k peers per estimate intervalEach update contains 2 floats: a value and a weightTo send an update to k peers, divide value and weight each by (k+1), store result locally, send k updates outTo merge an update, add the update’s value and weight to the locally known value and weight