The Access Company QoS Presented by: PowerPoint Presentation

The Access Company

QoS

Presented by:

Yaakov (J) Stein

CTO

What am I going to talk about today ?

The telecommunications service modelSLAsQoE and QoS

Soft QoS (DiffServ)

packet marking (PCP, DSCP)

PHBs – BE, EF, AF

queuing mechanisms (strict priority, WFQ)

specifying datarate, bucketing algorithms (leaky, token)

traffic policing

traffic shaping

Hard QoS (

IntServ

)

service levels – BE, CLS, GS

Network Engineering (planning) vs. Traffic Engineering (resource reservation)

RSVP

Routing protocols and RSVP-TE

thetelecommunications service model

Why do we pay for services ?

Generally good (and frequently much better than toll quality)

voice service is available free of charge (Skype,

Fring

,

Nimbuzz

, …)

So why does anyone pay for voice services ?Similarly, one can get free (WiFi) Internet accessemail boxesfile storage and sharingweb hostingsoftware servicesSo why pay for any service ?

Paying for QoS

The simple answer is that one doesn’t pay for the service

one pays for Quality of Service guarantees

In our voice model

But what does QoS mean

and why are we willing to pay for it ?

To explain, we need to review some history …

QoS

price

BE

toll quality

with mobility

Father of the telephone

Everyone knows that the father of the

telephone

was

Alexander Graham Bell

(along with his assistant Mr. Watson)

But Bell did not invent the telephone

networkBell and Watson sold pairs of phones to customersThe father of the telephone network was Theodore Vail

Father of the telephone network

Theodore Who?

Cousin of Alfred Vail (Morse’s coworker)

Ex-General Superintendent of US Railway Mail Service

First general manager of Bell Telephone

Father of the PSTN

Organized telephony as a service

(like the postal service!) *Why else is he important?Established principle of reinvestment in R&DEstablished Bell Telephones IPR divisionExecuted merger with Western Union to form AT&TSolved major technological problems

use of copper wire

use of twisted pairs

*

Vailism

is the philosophy that public services

should be run as closed centralized monopolies for the public good

What’s the difference ?

In the

Bell-Watson model

the customer pays once, but is responsible for

installation

wires

wiring

operationspowerfault repairperformance (distortion and noise)infrastructure maintenancewhile the Bell company is responsible only for providing functioning telephonesIn the Vail model

the customer pays a monthly fee

but the provider assumes responsibility for everything

including fault repair and performance maintenance

The telephone company owns the telephone sets and even the wires in the walls !

+

Service Level Agreements

In order to justify recurring payments

the provider agrees to a minimum level of service in an SLA

An SLA is a

legal

commitment

between a service provider (SP) and a customer, for example: Telco and subscriber ISP and Internet user VPN operator and enterprise cloud application provider and cloud userSLAs typically include (financial) penalties for breachesIf objectives or penalties are too low, SLA is uselessIf objectives or penalties are too high, cost will be prohibitiveBadly defined SLAs may damage operations by setting incorrect goals

SLAs and QoS parameters

SLAs should capture Quality of user Experience (QoE) but this is often hard to quantify

So SLAs usually actually detail measurable network parameters

that

influence

QoE, such as :

Connectivity parameters

availability (e.g., the famous five nines)time to repair (e.g., the famous 50 ms)Noise (error) level parametersSNRBERPacket Loss Ratiodefect densitiesInformation rate parametersbandwidth, throughput, goodputInformation latency parameters1-way delay,round trip delay performance parameters

Connectivity vs. the rest

Basic connectivity (availability) always influences QoE

The other parameters

may

influence QoE

depending on service/ application (voice, video, browsing, …)

Some

services only require basic connectivitySome also require minimum available throughputSome require delay less then some end-end (or RT) delaySome require packet loss ratio (PLR) less than some percentageNote: these parameters are not necessarily independentFor example, TCP throughput drops with PLR 1000 B packets50 ms RTT

Some rules of thumb

M

ission

C

ritical (and life critical) services require

high availability

If there are any MC services

then system traffic requires high availability tooMC services do not necessarily require strict throughput but always indirectly require a certain minimal average throughput bounded delay If the MC service uses TCP then it requires low PLRReal-time services require

sufficient throughput

but not necessarily low PLR (audio and video

codecs

have PLC)

Interactive services require

low RT delay

QoS monitoring

RECAP: SLA compliance is t

he SP’s

justification for payment

To ensure SLA compliance, the SP must :

monitor the SLA parameters

take action if parameter is dropping below compliance levels

But how does the SP verify/ensure that the SLA is being met ?Monitoring is carried out using Operations, Administration, Maintenance (OAM)The customer too may use OAM to check that the SP is compliant !

Technical note:

OAM is a

user-plane

function

but may influence control and management plane operations

for example

OAM may trigger protection switching, but doesn’t switch

OAM may detect provisioned links, but doesn’t provision them

OAM – FM and PM

The difference between connectivity and performance parameters

leads to two types of

OAM :

F

ault

M

onitoring required for maintenance of connectivity (availability)detection and reporting of anomalies, defects, and failuresOAM runs continuously/periodically at required rateused to trigger mechanisms in thecontrol plane (e.g. protection switching) and management plane (alarms)Performance Monitoring required for maintenance of all other QoS parametersmeasurement of performance criteria (delay, PDV, etc.)OAM run :

before enabling a service

on-demand or

per schedule

QoS assurance : availability

The difference between connectivity and performance parameters

leads to 2 types of QoS assurance – availability and performance

Availability is usually specified

in “nines”

In order to ensure high availability, one employs

FM OAM

Automatic P

rotection

S

witching (APS)

nines

up %

permitted down time

typical service

3 nines

99.9%

< 7

hour 18 min / month

electric power service

4 nines

99.99%

< 44 minutes / month

5 nines

99.999%

< 4 min 23 sec / month

PSTN

6

nines

99.9999%

< 26 sec / month

QoS assurance : performance

There are two main approaches to ensuring performance QoS

IntServ

(guaranteed QoS

) –

hard QoS

define traffic flows (CO approach)

guarantee QoS attributes for each flowreserve resources at each router along the flowsignaling protocol (e.g., RSVP) neededDiffServ (statistical QoS) – soft QoSretain CL paradigmno guaranteed QoS attributesno resource reservation

mark

packets (

differentiated – e.g., gold, silver, bronze)

marking can be by VLAN, P-bits, IP-

ToS

/DSCP, or general “flow”

offer special treatment (priority) relative to other packets

DiffServ is the preferred approach for Ethernet and IP

IntServ

is used in MPLS-TE

QoE

QoE and MOS

ITU-T defines QoE as

the acceptability of a service

as perceived subjectively by the end-user

A well-known QoE measure for

telephony-grade voice is Mean Opinion Score (MOS) (ITU-T P.800)MOS is measured by having a number of listeners listen and score speech on a scale from 1 (bad) to 5 (excellent) and averaging over these scores (finding the mean)Toll quality voice has MOS = 4Cellphone voice has MOS  3.5Synthetic or military voice has MOS = 2 and below

QoE and QoSTheory - QoE for a given application is a function of QoS parameters

                                 QoE = f (service; QoS1

, QoS

2

, …

QoS

n

)Researchers have found various functional forms for the dependence of QoE on a particular QoS parameter see e.g., work of Markus Fiedler (BTH, Sweden) formexpressionexamplesLinear QoE  QoSkperceived download time vs. PLRLogarithmic

QoE

 log(QoSk

)

perceived download time vs. datarate

Exponential

QoE

 exp(

QoS

k

)

VoIP MOS vs. PLR

Power Law

QoE



QoS

k

p

perceived streaming video quality vs. PDV

Absolute vs. Comparative QoE

QoE measures may be absolute determined by observing the degraded message

or

comparative

determined by comparing the degraded message to the original

Comparative measures are often more accurate

but can not be used

unintrusively on a live network scenariosAbsolute measures can be used single-ended (non-intrusively)MOS variations Absolute Category Rating (ACR) : listeners hear only the degraded speechDegradation Category Rating (DCR) : listeners hear first the original and then the degraded speech and score 1 = very annoying degradation … 5 inaudible degradationComparative Category Rating (CCR) : listeners hear the original and the degraded speech in random order and score -3 2nd is much worse than 1st ... 3 2nd is much better than 1st even simpler : AB test – simply report which sounds better

Subjective vs. Objective QoE

Direct human QoE scoring is expensive and time-consumingITU-T has defined objective measures

that can be

automated

These entail algorithms that produce scores

that

correlate well with human QoE

PSQM (ITU-T P.861) and PESQ (ITU-T P.862) are objective comparative MOS-like measures for telephone grade speechThey model the human auditory perception system (Bark scale, masking, etc.)PEAQ (ITU-R BS-1387) similarly scores wideband audioThese were selected in competitions to have highest correlation with human MOS ITU-T P.563 is a single-ended (absolute) objective MOS-like score It determines un-naturalness of telephone-grade speech sounds and the amount of non-speech-like noise

PSQM processing

E-modelThe E-model defined in ITU-T G.107 is a

planning tool It predicts a “mouth-to-ear” transmission rating factor

R

between 0 and 100

higher values signify better voice quality

should be uniquely convertible to a MOS level

R = f(QoS

1, … QoSn) and is additive in individual QoSk degradationsR starts with the basic signal to noise ratioR is reduced to account for various impairments, including simultaneous impairments (loudness, sidetone, clipping, quantization noise)delay impairments (delay, echo delay and loudness) equipment impairments (codec distortion, packet loss)R is increased when there are additional advantages such as mobility (cellphone receives A=10) R = R0 – Is – Id – Ie + A

R value meanings

R values

meaning

Equivalent MOS

90 - 100

Very satisfied

4.3-5.0

80- 90Satisfied4.0-4.370-80

Some users dissatisfied

3.6-4.0

60-70

Many users dissatisfied

3.1-3.6

50-60

Nearly all users dissatisfied

2.6-3.1

Below 50

Not

recommended

1-2.6

VQMONYears before P.563 ETSI specified

VQmonTIPHON

(Telecommunications and Internet Protocol Harmonization Over Networks)

TS 101 329-5 Annex E

VQmon

(developed by

Telchemy

) is a single-ended method for estimating the E-model factors for VoIP audio based on QoS parameters (packet loss statistics, delay)Depends on codec typeTakes human perception phenomena into account (e.g., recency effect)VQmon was later extended to audio (MOS-A)video (MOS-V)audio-video (MOS-AV)

Video qualityITU-R produced BT.500 for subjective assessment of TV quality

Similar to MOS :television sequences are shown to a group of viewerssubjective opinions are averaged

ITU-T has produced many Recommendations

for video and multimedia quality :

Subjective (P.9xx, J.140)

Objective (J.143, J.144, J.147, J.148, J.24x, J.34x)

Since 1997 the

Video Quality Experts Group (VQEG) has been producing standards and tutorialsPEVQ (J.247) is a comparative pixel by pixel objective measure that models the human visual tractand returns a 5-point MOS score and further KPIs

QoE for other applications G.1011

is a reference guide to existing standards for QoE and provides a taxonomy G.1010

discusses many applications, including

conversational voice, voice messaging, streaming audio

videophone, one-way video

web-browsing, bulk data transfer, email, e-commerce,

interactive games

SMS, instant messagingand gives performance targets for delay, PDV, and PLR G.1050 gives an IP network model for evaluating the performance of IP streams based on QoS parameters (delay, PDV, PLR). J.163 treats real-time services over cable modemsX.140 defines QoS parameters for public data networks

Network planning toolsIn addition to subjective/objective methods to quantify the QoE

of a specific (live or simulated) service instanceNetwork planners need tools to predict service quality

in order to efficiently allocate resources

G.1030

provides network planners with end-to-end (E-model-like) tools for applications over IP networks

It includes an appendix devoted to web browsing

that presents empirical perception of users to response times and proposes a MOS measure

G.1070 proposes an algorithm for network planners to estimate videophone quality

Useful background documentsISO 8402

defines quality as :

the totality of features and characteristics of a product or service that bears its ability to satisfy stated or implied needs

E.800

Definitions of terms related to QoS

G.1000

Communications quality of service: A framework and definitions

ETSI ETR 003 General aspects of QoS and Network Performance (NP)* Note – terminology in these documents is outdated

TR-126The Broadband Forum (BBF) has produced

TR-126, which includesa tutorial on QoE guidelines for QoE vs. QoS for triple play applications

TR-126 also discusses :

QoE

dimensions

: service set-up, operation, and tear-down

QoE

facets: user effort, application responsivenessinformation fidelity, security, and dependability/availability;localization of QoE contributions (access, ISP, application SPs)Guidelines are given for :video (conferencing, surveillance, streaming)voice (wired, wireless, voice messaging, IVR)best-effort Internet data (browsing, email, file transfer, VPN, P2P, ecommerce, …)

TMFThe T

eleManagement

F

orum (TMF) discusses QoE SLA management

TMF’s

Wireless Services Measurement Handbook

GB923 defines :

Key Quality Indicators (KQIs) (like QoE scores)Key Performance Indicators (KPIs)KQIs may be determined from KPIs (the mapping may be complex) KPIs are derived from QoS parameters TMF has defined a set of KQIs including :response timeservice availabilityspeech/video qualitytransaction rateoffered throughput An SLA consists of a set of KQI and KPI thresholds (see SLA Management Handbook GB917 and its Application Notes)

ApdexThe Apdex

Alliance is a consortium of companies functions as a IEEE-ISTO (Industry Standards and Technology Organization)

program

Apdex

develops open standardized methods to

report

benchmark and

track application performance. The Apdex (Application Performance Index) is a number between 0 and 1 is meant to capture user satisfaction from an application 0 means no user would be satisfied1 means that all users would be satisfied

Apdex (cont.)

To compute the Apdex N users are divided into 3 categories

satisfied (S users)

e.g., web page completely loads within 2 seconds

tolerating (T users)

e.g., web page completely loads within 8 seconds

frustrated (F users)

e.g., web page takes > 8 seconds to loadThe Apdex is given by Apdex =  ( S + T/2 ) / NApdex hierarchically deconstructs application transactions into sessions processes tasks turns protocols packets Sessions consist of the entire connect time Processes are interactions accomplishing a goal Tasks are individual interactionsThe user is mainly aware of the task response time since must wait for the task to complete before proceeding!

Behavioral QoEAll of the above subjective and objective QoE measures

are service/application-specific. But new services and applications are created every day

and different users use different features of a single application

So it is no longer feasible to study each application in depth

A new approach is

behavioral QoE estimation

the user’s satisfaction is estimated based on actions / reactions

Example : there is a high measured correlation between a user being unsatisfied with a service level his aborting the application (or at least waiting until the service level improves)Behavioral QoE can be used instead of traditional QoE measurement or to automatically find QoE(new app, QoS)

soft QoS

Queuing theoryWhy isn’t the

traffic distribution problem simple ?If we have available data rate A,

simply allow traffic from all sources that sums to



A

Wrong !

Even if the average rates sum to much below A

there may be peak rates that exceed AIn order to accommodate peaks, we insert a queueCustomers/packets/whatever wait in queue to be servicedQueue behavior can be counter-intuitive Problem 1 : two buses arrive at my bus stop every 10 minutes I take the first bus that arrives Why do I take bus A much more than bus B ? (hint : correlation)Problem 2: buses leave their first stop every 10 minutes and pick up passengers at intermediate stops Why do the buses bunch ?

S

queue

arrivals

Scheduling disciplinesWhen there is a single queue, service order

is usually First In First Out (FIFO) AKA First Come First Served but may be Last In First Out (LIFO/stack), random order, etc.

When there are multiple queues

packets belonging to a particular flow are consistently mapped to a single queue

there may be one or more flows in each queue

Service order may be :

Round Robin : each queue visited in orderStrict Priority : take from non-empty queue of highest priorityFair Queuing : preserve average datarate from each queueWeighted Fair Queuing : fair queuing with priorityhybrid : mixture of several disciplines

ErlangIn early 1900s

telcos needed to calculate the number of switches, lines, and operators they needed per call volume

too little and subscribers would be unhappy

too much wastes labor and money

Same problem for customers in stores, cars at traffic lights, manufacturing processes, call centers, etc.

Agner

Krarup Erlang developed queuing theory to solve this problem for the Copenhagen telephone exchangeThe unit of traffic use is called the Erlang in his honor1 Erlang is 1 channel being used 100%, or 2 channels used 50% when averaged over some time (generally an hour)There is also a functional programming language (used by Ericsson for telephony applications) named after him

Kendall notationLet’s call

A – the statistics of arrival times (customers / packets / whatever)B – the statistics of service times

C – the number of servers (there may be only 1 server)

K – the maximum queue length (if too many arrivals, need to drop)

Then we can describe a queuing system by

A/B/C/K

and if K=

 then we call it A/B/CImportant statistics types:Example:M/M/1 is a queue with a single server and Poisson distributed arrivals and service timesDDeterministic distribution (often fixed intervals)MMarkov process (Poisson (exponential) distribution)E(k)Erlang distribution with parameter k

Queuing theory resultsQueuing theory derives important values, such as

waiting timesnumber of customers/packets waitingnumber of customers/packets being processed

We will not develop queuing theory here

Some models are completely understood (M/M/1, M/M/K, M/D/1)

Some formulae are true in general

Little’s law

L

 = λWL : long-term average number of customers in a queue λ : long-term average arrival rateW : average time a customer (waits) in the system This law is true for any arrival distribution, service distribution, number of servers, service order, etc.

Recap: soft QoSSoft QoS does not provide any hard service level

guaranteesIt merely breaks fairness by giving priority to certain

users or applications or flows or individual packets

When there aren’t network resources to forward all packets

packets are forwarded in order of priority from highest to lowest

Low(

er

) priority packets may be delayed or discarded(when K < )In order to correctly prioritorize packets they need to be priority markedMarking is best accomplished by the originator but may need to be performed by an intermediate element based on port, or header fields, or even DPI

Marking Ethernet

DA

(6B)

SA

(

6B)ET=8100 (2B)P(3b) CFI(1b) CVID(12b)

ET=88A8

(2B)

P(3b)

DEA(1b)

SVID(12b

)

ET

(2B)

V

LAN

ID

(VID) indicates priority

In addition, for VLAN tagged frames

P

riority

C

ode

P

oint (PCP) AKA user priority field, P-bits

3 bits so takes values 0 … 7

monotonically increasing priority

can use

priority tagging

(VLAN=0) if no VLAN

P=0 means non-expedited traffic

802.1Q gives recommends mappings

Marking MPLS

Only top label is relevant

Label can indicate priority (L-LSP)

or

TC field

(previously called EXP field, previously called COS field)

(E-LSP)

3 bits so takes values 0 … 7no recognized TC value meanings Top Label (20b) TC(3b) S(1b) TTL (8b)

Label

(20b)

TC(3b)

S(1b

)

TTL

(8b)

Bottom Label

(20b)

TC

(3b

)

S

(1b

)

TTL

(8b)

Marking IPv4

The IPv4 header has a T

ype

o

f

S

ervice (

ToS) fieldRFC 2474 redefined ToS to consist of 6 bit DSCP (see also RFC 4594) 2 bit ECN (least significant bits)Guidelines for use of DSCP in many documentsVer(4b) IHL(4b) ToS(1B) Len(2B)

Source IP Address

(4B)

Destination IP Address

(4B

)

. . .

Marking IPv6

The IPv6 header has a Traffic Class

(TC) field

RFC 2460 states that it is to be used like the IPv4

ToS

field

The Flow Label

(intended to ensure flow of packets follow the same path) may be indirectly usedVer(4b) TC(1B) Flow Label (20b)Source IP Address (16B)Destination IP Address (16B)

payload

len

(2B) next

(1B) hop

(1B)

IP DiffServ methodsDiffServ was developed in IETF

after

IntServ

RFCs 2474, 2475

because

IntServ

was considered too heavy-weight for most purposesresource reservation is against IP-philosophyif not enough BW, then more democratic for all to sufferif reserve BW and don’t use, then this is simply over-provisioningDiffServ is evolutionary “coarse-grained” approach to IP QoSDiffServ divides traffic into service classesand allocates resources on a per-class basisuses 6 bits of ToS byte in IP header to mark packetsfield is renamed Differentiated Services Code Point no setup or router state requiredDSCP defines per-hop behaviors (PHB)tells router how to treat packetthree standard PHBs (BE, AF, EF) but you are free to create more

DiffServ Per Hop Behaviors (PHBs)

Best E

ffort

standard IP service

QoS depends on momentary network load

A

ssured

ForwardingAF specifies class that determines queuein addition, three drop-precedence levels (low, med, high)AF packets from a single source should not be mis-ordered even if have different drop-precedence (i.e. single queue)Expedited ForwardingEF packet should experience no queuing delaysEF packets should have low lossimplemented by dedicated EF router queueWARNING: DiffServ does not provide true assurances

What about MPLS ?MPLS DiffServ - each FEC is defined by

destination addressservice class

L-LSP

(

L

abeled-inferred

LSP)behavior based on label alonesupport different service classes by using different labelsLSP BW allocated from specific queue (class)TC field may be used for drop precedenceE-LSP (EXP-inferred LSP)behavior based on label + TC (ex-EXP) fieldTC bits in MPLS shim header are similar to DSCPs, but only three bits (like P-bits) while DiffServ ended up with 6 bitsbut 8 service classes is usually more than enough commonly 4 classes are offered (bronze, silver, gold, platinum)LSP BW allocated from link

Queuing in switches/routers

Switches and routers have queues FIFO buffers

on each output port

If there were only one queue

then traffic handling would be FCFS

To enable DiffServ prioritization

multiple queues are used

Outgoing frames are inserted into queues according to priority markingQueues are emptied according to scheduling discipline(strict priority, WFQ, etc.)

switch

fabric

input port

input port

input port

output port

output port

output port

output port

queue

queue

queue

queue

Traffic conditioning

One of the most important parts of an SLA is the

C

ommitted

I

nformation

Rate (bps)This is the datarate (bandwidth) SP tries to forwardThere may also be an Extra Information Rate (bps)

This is a datarate that the SP will forward if possible

A customer who did not send data for a while

will expect to be able to send a higher rate afterwards

Enforcement of these rates is

accomplished via

traffic conditioning

Three strategies :

policing

(rate limiting, throttling)

shaping (in ATM – GCRA)

metering

WARNING: DiffServ does not provide true commitments

Traffic policing

Traffic exceeding the committed datarate is immediately discarded

(or at least

marked as discard eligible)

CIR

time

CIR

time

policed to CIR

Traffic shaping

Traffic exceeding the committed datarate is delayed

until it can be forwarded

(placed or remains in a buffer)

(discarded only if rate exceeds CIR for an extended time)

CIR

time

CIR

time

shaped to CIR

Traffic metering

Charge extra for

traffic exceeding the committed datarate

(leads customer to self-police)

CIR

time

CIR

time

extra charge over CIR

BW specification

What does an SLA commitment of

X bps

mean ?

BW usage naturally varies for many services

Must the customer send < X bps at all times

even if he transmitted much less than X up to now ?

May the customer remain silent for 9 minutes and then send 10X bps for 1 minute ?If the measurement interval is 10 minutes then this is precisely X bps !A BW cap is only meaningful when we specify the integration timeOr, we can specify the rate and the maximum burst size (in bytes) and enforce these using a bucketing algorithmA bucketing algorithm allows bursts above X for a limited time as long as the average remains at X or below

Leaky and token buckets

Leaky bucket model water is poured into bucket as needed

water leaks out at a constant rate

if too much water poured in it overflows

Interpretation

bps of traffic are added to bucket

committed rate is continuously removed

if packet fits into bucket it is sentunused data rate is lostToken bucket model water is poured into bucket at a constant ratewater is removed as neededif too much water poured in it overflowsInterpretationtokens are added at committed rateto send traffic there have to be enough tokensunused data rate is lost

How does a token bucket work ? Part 1Let’s look at a token bucket

policer (other cases are equivalent)The bucket is configured with

height -

C

ommitted

I

nformation

Rate (CIR) filling rate - Committed Burst Size (CBS)If packets are sent at precisely the committed ratethe bucket height stays constantand all packets are forwardedIf packets arrive at less then the committed ratethe bucket height increasesall packets are forwardedexcess information rate overflows and is lostIf packets arrive at more then the committed ratethe bucket height decreaseswhen no tokens are left packets are discardedCBS

CIR

continued

Note:

Some people complicate formulas by specifying CIR in bps and CBS in Bytes

Such people should be

committed

How does a token bucket work ? Part 2If no packets have been sent for some time

and then CBS worth of packets are sentthe bucket is initially full of CBS tokensthe tokens are all removed

all packets are forwarded

If more than CBS information rate in burst

the first CBS of packets are forwarded

the rest are discarded until new tokens arrive

Note: adding of tokens can be in

discrete time - every T (e.g., 1 sec) the token are addedcontinuous time – tokens are continuously added (in practice, when new packet arrives, calculate number of tokens added since the last packet) for continuous time, the maximum burst size is larger than the configured CBSCBS

CIR

Dual token buckets

Sometimes SPs sell (and customers purchase)

Extra

traffic

This is a rate above the committed rate

that the SP will forward if it can (but doesn’t commit to forward)

Extra traffic is priced much lower than committed rate trafficTo handle Extra traffic, we use two (token or leaky) buckets, C and E the C bucket is of height CBS and is filled at rate CIR the E bucket is of height EBS and is filled at rate EIRcontinued

CBS

CIR

EBS

EIR

C

E

Dual token buckets (cont.)

Furthermore, we classify packets as

green

if passes C bucket test

(green packets are forwarded)

yellow

if fails C bucket test, but passes E bucket test

(yellow packets may be forwarded, but SLA objectives don’t apply)red if fails both bucket tests (red packets are always discarded)More precisely :if ingress traffic < number of tokens in C bucketframe is green and its length in tokens is debited from C bucketelse if ingress frame length < number of tokens in E bucketframe is yellow and its length of tokens is debited from E bucket else frame is red

More token bucket variations

As if this isn’t complicate

enough …

MEF added two more twists – coupling and sharing

Unused rate is not lost – it is coupled or shared !

coupling

sharing

coupling

priority

sharing

hard QoS

CO vs. CL networksTo guarantee QoS (and thus QoE) we need to

find path through network that can provide the needed QoSreserve resources along this path to guarantee the QoSnot accept flows for which there are insufficient resources (CAC)

optionally – optimize path placement

(to maximize number of flows that can be accommodated)

ATM and

(some)

MPLS networks are

Connection Oriented (CO), thuswe specify (or at least know) the path the packets will takewe can reserve resources at the network elements along the pathStandard IP networks are ConnectionLess (CL), thusit is hard to ensure packets will go where we want them to it is meaningless to reserve resources as we don’t know which network elements a packet will traverseThus hard QoS is not easy to add to IP which is why hard QoS is not popular in pure IP networks …

Network and Traffic Engineering

Network Engineering (planning)

putting the bandwidth where the traffic is

physical cable deployment (

thick pipes

)

over-provisioning and backup connection provisioning

does it violate provider objectives ?Traffic Engineering (TE)putting the traffic where the bandwidth isexplicit traffic routingroute optimizationcan it meet user objectives?

Traffic Engineering (TE)

TE is control of network traffic to achieve specific objectivesunfortunately users and providers have contradictory objectives

user objectives (QoS)

network availability

packet loss

end-to-end delay

round-trip delay

packet delay variation (PDV)error rateprovider objectives (CAPEX, OPEX)bandwidth utilizationresource utilizationspeed of failure recoveryease of managementmonetary outlay

Simple example - fish diagramIn the above example, there is sufficient BW for all traffic

(were these ATM switches, 1G over ACDG

,

½

over

BC

EF

GWithout TE, can’t use all the physical bandwidth!standard IP routing : minimum hop count is CDG, so all traffic flows there 1½ G over 1G link, so ½G is dropped !with administrative cost can force all traffic to go CEFG - which is worse !with ECMP half (750M) goes CDG and half (750M) goes CEFG – still drops !

A

B

C

D

E

F

G

all

links

1G

except

EF

½ G

1G

½ G

Constraint-based routing

IP uses distributed routing protocols, not centralized managementDistributed protocols are very good at finding basic connectivity

and minimizing

an additive

metric

(e.g., hop count)

but are not good at optimally utilizing network resources

or obeying constraintsCommon constraints include :explicit include/exclude links/routers (local constraint)conform to link BW constraints (local inequality constraint)meet end-end delay / PLR objectives (global inequality constraint)Routing that takes constraints into account is called constraint-based routing (CR)After CR finds an acceptable path we may need to set it up (reserve resources) using another protocol

OSPF-TE and IS-IS-TE

Constraint-based routing needs link attribute information most importantly - available BW

Link-state protocols have mechanisms to flood

link-up/link-down

to every router in the domain

We can piggyback attributes as TLVs on these messages

OSPF

add to Link-State Advertisement (LSA) (RFC 3630)IS-IS add to Link-State Packetsand the routing protocol builds an extended “TE” RIBWhen attribute information changes need to reflood informationTo decrease overhead:only flood when change passes thresholdinherent timing boundsNote: CR routing can be NP-hardStandards don’t include (proprietary) efficient algorithms

IP IntServ

IntServ is an overall QoS architecture

(not just RSVP)

initially developed for VoIP QoS

IntServ

is a radical departure from pure IP

and requires

IntServ-enabled routers IntServenables providing end-to-end QoS guaranteesdefines flows (introduces CO to IP’s CL architecture) flows are classified into three service classes (BE,CLS,GS)specifies admission control and policinglike all CO architectures, requires signaling protocol (RSVP)IntServ-enabled routersreserve needed resources along the flow’s pathmust retain stateRFCs 2205-2216, 2379-2382

IntServ CoS levels

B

est

E

ffort

standard IP service

QoS depends on momentary network load

Controlled Load Serviceservice equivalent to unloaded networklow packet lossmost packets will experience delay close to minimumno quantitative guaranteesGuaranteed Servicebounded worst case delay (no PDV guarantee)low packet loss (zero if node buffers correctly provisioned)quantitative guarantees

RSVPThe primary signaling protocol for

IntServ

is

R

esource

re

S

erVation Protocol (RSVP)RSVP protocolruns between hosts and routersruns over raw IP or UDP/IPis unidirectionaldoes not find path (fed by routing protocols)sessions identified by source and destination socket numbersrequests unidirectional QoS characteristics from networkcauses routers along path to reserve link and node resourcesnetwork responds with success/failurereservations are soft-state - time-out unless refreshedtwo main message types: PATH and RESV

RSVP

RSVP MessagesPATH

message from sender to receiver(s)carries classification info and

TSpecs

RESV

response of receiver to

PATH

message

carries session ID and RSpec specifying QoS requiredcontains the actual request for resource reservation

receivers

sender

MPLS TE

MPLS-TE protocols enable Fast

R

e

R

oute

to guarantee fault

recovery (connectivity assurance)Also, MPLS FECs can take QoS constraints into accountMPLS-TE LSPs can be setup according to constraintsinclude/exclude specific LSRs (for any reason)only include in LSP LSRs with sufficient available BWonly include in LSP LSRs that guarantee sufficiently low delayOSPF-TE or IS-IS-TE can be used to get needed network informationBut how can the path be set-up ?Vanilla LDP has no TE capabilities and its extension CR-LDP is now obsoleteThe answer is a set of extensions to RSVP called RSVP-TEUnlike RSVP, this protocol runs only between routers (LSRs)

RSVP-TERSVP-TE (RFC 3209 …) is a label distribution protocol

For downstream-on-demand bindingCreates and distributes bindings between RSVP flows and labels

Uses labels instead of source and destination socket numbers

Extends RSVP by adding new objects (e.g. label) and procedures

Allows strict/loose explicitly routed LSPs

Has peer discovery, label requests, binding messages (like LDP)

Transparent transport of QoS and traffic parameters

in TSpecs and RSpecs Although between routers - still soft state !Note: RSVP-TE is frequently used to set up FRR alongside LDP

RSVP-TE LSP setup procedure

Example setupwith explicit routes

A sends

PATH

message to B w/ explicit route BC

and resource requirements

B forwards

PATH message to C after changing explicit route to CC determines required resources, reserves, locally binds label and sends RESV to BB matches, reserves resources, remotely binds, and sends RESV to AA matches, remotely binds label and reserves resources

A

C

B

ingress

egress

PCETo optimally utilize network resources (e.g., link BW), we need

to gather all the topology and network constraints into one centralized Traffic E

ngineering

D

atabase (TED)

enough computational power to solve the complex optimization problem

to send path set-up commands to the routers to set up TE-LSP

RFC 4655 defines a Path Computation Element (PCE nicknamed Godbox) and Path Computation Clients (PCCs)The PCE may be a designated router orthe management system or a dedicated computational platformPCE is an evolutionary solution to adding computational resources

PCE

PCEP (RFC 5440)The protocol between PCE and PCCs

and between multiple PCEs (if there are)is called the Path Computation Element Protocol (PCEP)

PCEP runs over TCP with registered TCP port 4189

Messages are objects

(with common object header)

with optional TLVs

PCEP Messages :

Open between PCE and PCC to open a new session Keepalive optional heartbeat sent if no other PCEP messagesPCReq PCC → PCE to request a path computation PCRep PCE → PCC with set of computed paths or negative replyPCNtf event notification from either PCE or PCC PCErr protocol error messageClose session close

OptimizationHow does the PCE compute paths ?

The general path optimization problem is intractable (NP-hard)But there are many

combinatorial optimization

problems

with known efficient algorithms

that return

approximate solutions

For example, the (1-dimensional) bin packing problem : Given N values V1 V2 … VN between 0 and 1 how can we place them in bins of maximum size 1 using the minimum number of bins ?This problem comes up in many applications :multiprocessor schedulingstock cuttingmapping short messages into time slots of ATM cellsmapping CBR flows onto WDM wavelengthsmapping flows onto orthogonal paths

Full PCE problem is even harder !

SDNAn even more radical centralized solution is

the Software Defined

N

etwork (SDN)

Like PCE, SDN replaces distributed routing protocols

with a centralized

controller

that communicates with SDN switchesAn SDN switch is a forwarding device that can be programmed to match an arbitrary set of fields in the packet and edit / forward the packet accordinglySDN switches need not obey standard network layersThe most popular controller-switch protocol is OpenFlowGoogle uses SDN switches to optimize utilization in its inter-datacenter network