Cloud Control with - PowerPoint Presentation

Slide1
Slide2

Cloud Control with

Distributed Rate Limiting

Barath

Raghavan

,

Kashi Vishwanath,Sriram Ramabhadran, Kenneth Yocum, and Alex C. SnoerenUniversity of California, San DiegoSlide3

Centralized network services

Hosting with a single physical presence

However, clients are across the InternetSlide4

Running on a cloud

Resources and clients are across the world

Services combine these distributed resources

1

GbpsSlide5

Key challenge

We want to control

distributed

resources as if they were

centralizedSlide6

Ideal: Emulate a single limiter

Make distributed

feel

centralized

Packets should experience same limiter behavior

S

S

S

D

D

D

0 ms

0 ms

0 ms

LimitersSlide7

Distributed Rate Limiting (DRL)

Achieve

functionally equivalent behavior

to a central limiter

GlobalRandom Drop

Flow

Proportional Share

Packet-level(general)

Flow-level

(TCP specific)

Global

Token Bucket

1

2

3Slide8

Distributed Rate Limiting tradeoffs

Accuracy

(how close to

K

Mbps is delivered, flow rate fairness)+

Responsiveness(how quickly demand shifts are accommodated)Vs.Communication Efficiency(how much and often rate limiters must communicate)Slide9

Limiter 1

DRL Architecture

Limiter 2

Limiter 3

Limiter 4

Gossip

Gossip

Gossip

Estimate

local demand

Estimate

interval

timer

Set allocation

Global

demand

Enforce limit

Packet

arrivalSlide10

Token Buckets

Token bucket, fill rate

K

Mbps

PacketSlide11

Demand info

(bytes/sec)

Building a Global Token Bucket

Limiter 1

Limiter 2Slide12

Baseline experiment

Limiter 1

3 TCP flows

S

D

Limiter 2

7 TCP flows

S

D

Single token bucket

10 TCP flows

S

DSlide13

Global Token Bucket (GTB)

Single token bucket

Global token bucket

7 TCP flows

3 TCP flows

10 TCP flows

Problem: GTB requires near-instantaneous arrival info

(50ms estimate interval)Slide14

Global Random Drop (GRD)

5 Mbps

(limit)

4 Mbps

(

global

arrival rate)

Case 1: Below global limit, forward packet

Limiters send, collect global rate info from othersSlide15

Global Random Drop (GRD)

5 Mbps

(limit)

6 Mbps

(

global

arrival rate)

Case 2: Above global limit, drop with probability:

Excess

Global arrival rate

Same at all limiters

1

6

=Slide16

GRD in baseline experiment

Single token bucket

Global random drop

7 TCP flows

3 TCP flows

10 TCP flows

(50ms estimate interval)

Delivers flow behavior similar to a central limiterSlide17

GRD with flow join

(

50ms

estimate interval)

Flow 1 joins at limiter 1

Flow 2 joins at limiter 2Flow 3 joins at limiter 3Slide18

Flow Proportional Share (FPS)

Limiter 1

3 TCP flows

S

D

Limiter 2

7 TCP flows

S

DSlide19

Flow Proportional Share (FPS)

Limiter 1

Limiter 2

“3 flows”

“7 flows”

Goal: Provide inter-flow fairness for TCP flows

Local token-bucket

enforcementSlide20

Estimating TCP demand

Limiter 1

D

Limiter 2

3 TCP flows

S

D

1 TCP flow

S

1 TCP flow

SSlide21

Estimating TCP demand

Local

token rate (limit) = 10 Mbps

Flow A = 5 Mbps

Flow B = 5 Mbps

Flow count = 2 flowsSlide22

Estimating TCP demand

Limiter 1

1 TCP flow

S

D

Limiter 2

3 TCP flows

S

D

S

1 TCP flowSlide23

Key insight: Use a TCP flow’s rate to infer demand

Estimating

skewed

TCP demand

Local

token rate (limit) = 10 Mbps

Flow A = 2 Mbps

Flow B = 8 Mbps

Flow count ≠ demand

Bottlenecked elsewhereSlide24

Estimating

skewed

TCP demand

Local

token rate (limit) = 10 Mbps

Flow A = 2 Mbps

Flow B = 8 Mbps

Local Limit

Largest Flow’s Rate

10

8

=

Bottlenecked elsewhere

= 1.25 flowsSlide25

2.50 flows

Limiter 2

10 Mbps x 1.25

1.25 + 2.50

Flow Proportional Share (FPS)

Global limit = 10 Mbps

1.25 flows

Limiter 1

Set local token rate =

=

3.33 Mbps

Global limit x local flow count

Total flow count

=Slide26

Under-

utilized limiters

Limiter 1

D

S

1 TCP flow

S

1 TCP flow

Set local limit equal to actual usage

Wasted rate

(limiter returns to full utilization)Slide27

Flow Proportional Share (FPS)

(

500ms

estimate interval)Slide28

Additional issues

What if a limiter has no flows and one arrives?

What about

bottlenecked traffic

?What about varied RTT flows?What about short-lived vs. long-lived flows?

Experimental evaluation in the paperEvaluated on a testbed and over PlanetlabSlide29

Cloud control on

Planetlab

Apache Web servers on 10

Planetlab

nodes5 Mbps aggregate limitShift load over time from 10 nodes to 4 nodes

5 MbpsSlide30

Static rate limiting

Demands at 10 apache servers on

Planetlab

Demand shifts to just 4 nodes

Wasted capacitySlide31

FPS

(top)

vs. Static limiting

(bottom)Slide32

Conclusions

Protocol agnostic

limiting (extra cost)

Requires shorter estimate intervals

Fine-grained packet arrival info not requiredFor TCP, flow-level granularity is sufficientMany avenues left to exploreInter-service limits, other resources (e.g. CPU)Slide33

Questions!Slide34

FPS state diagram

Case 2:

Underutilized

Set local limit to actual usage

Case 1:

Fully utilized

Flows start/end,

Network congestion,

Bottlenecked flowsSlide35

DRL cheat-

proofness

Conservation of rate among limiters for FPS

EWMA compensates for past cheating with higher drop rates in the future

To cheat GRD, forever increase the demand

Authenticated inter-limiter communication assumedDifficult to quickly move traffic demandsSlide36

DRL applications

Cloud-based services (e.g. Amazon’s EC2/S3)

Content-distribution networks (e.g.

Akamai

)

Distributed VPN limitingInternet testbeds (e.g. Planetlab)

Overlay network service limitingSlide37

DRL ≠

QoS

DRL provides:

Fixed, aggregate rate limit

No service classes

No bandwidth guaranteesNo reservationsNo explicit fairness; implicit (TCP) fairnessSlide38

DRL provisioning

Providers

could

ensure that the limit K Mbps is available at all locations; wasteful

We expect it to be like current practice: statistical multiplexing

Even today, can’t guarantee bandwidth to all destinations with a single pipeSlide39

Experimental setup

Modelnet

network emulation on

testbed

40ms inter-limiter RTT

Emulated 100 Mbps linksNo explicit inter-limiter lossRan limiters across 7

testbed machines

On Linux 2.6; packet capture via iptables

ipqSlide40

Short flows with FPS

2 limiters, 10 bulk TCP vs. Web, 10Mbps limit

Web traffic based on CAIDA OC-48 trace

Loaded via

httperf

, Poisson arrivals (μ = 15)

CTB

GRD

FPS

Bulk

rate

6900.90

7257.87

6989.76

Fairness (Jain’s)

0.971

0.997

0.962

Web rate [0, 5K)

28.17

25.84

25.71

[5K, 50K)

276.18

342.96

335.80

[50K, 500K)

472.09

612.08

571.40

[500K, ∞)

695.40

751.98

765.26Slide41

Bottlenecked flows with FPS

Baseline experiment: 3-7 flow split, 10 Mbps

At 15s, 7 flow aggregate limited to 2Mbps upstream

At 31s, 1

unbottlenecked

flow arrives joins 7 flowsSlide42

RTT heterogeneity with FPS

FPS doesn’t reproduce RTT unfairness

7 flows (10 ms RTT) vs. 3 flows (100 ms RTT)

CTB

GRD

FPS

Short RTT (Mbps)

1.41

1.35

0.92

(

stddev

)

0.16

0.71

0.15

Long RTT

(Mbps)

0.10

0.16

0.57

(

stddev

)

0.01

0.03

0.05Slide43

Scaling discussion

How do various parameters affect scaling?

Number of flows present: per-limiter smoothing

Number of limiters: ~linear overhead increase

Estimate interval: limits responsiveness

Inter-limiter latency: limits responsivenessSize of rate limit delivered: orthogonalSlide44

Comm. Fabric requirements

Up-to-date information about global rate

Many designs are possible:

Full-mesh (all-pairs)

Gossip (contact k peers per round)

Tree-based…Slide45

Gossip protocol

Based on protocol by

Kempe

et al.

(FOCS 2003)

Send packets to k peers per estimate intervalEach update contains 2 floats: a value and a weightTo send an update to k peers, divide value and weight each by (k+1), store result locally, send k updates outTo merge an update, add the update’s value and weight to the locally known value and weight