Programmable Packet Scheduling at Line Rate
Description: Programmable Packet Scheduling at Line Rate Anirudh Sivaraman, Suvinay Subramanian, Anurag Agrawal, Sharad Chole, Shang-Tse Chuang, Tom Edsall, Mohammad Alizadeh, Sachin Katti, Nick McKeown, Hari Balakrishnan Network switches over time
Related Topics
Download Presentation
"Programmable Packet Scheduling at Line Rate" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation Transcript
slide1. Programmable Packet Scheduling at Line Rate Anirudh Sivaraman, Suvinay Subramanian, Anurag Agrawal, Sharad Chole, Shang-Tse Chuang, Tom Edsall, Mohammad Alizadeh, Sachin Katti, Nick McKeown, Hari Balakrishnan<br>
slide2. Network switches over time Initially, fixed-function devices
Changing operator requirements
Tunnels
Access Control
Policy routing
New headers<br>
slide3. Alternatives to fixed-function Programmable switches
FPGAs: Arista 7124 FX, NetFPGA
NPUs: Intel IXP
CPUs: Click, RouteBricks
10—100x loss in performance
Limited adoption relative to line-rate switches<br>
slide4. Programmable scheduling at line rate Programmable: Can we express a new scheduling algorithm?
Line-rate: Highest capacity supported by a communication standard<br>
slide5. Programmability at line-rate OpenFlow: Match-Action interface, fixed fields, fixed actions
P4, RMT, FlexPipe, Xpliant: Protocol-independent match-action pipeline. Parser Ingress Pipeline Stage 1 Match Scheduler Deparser Egress Pipeline Action Match Action Match Action Stage N Match Action Match Action Match Action Stage 1 Match Action Match Action Match Action Stage N Match Action Match Action Match Action Eth IP TCP<br>
slide6. Why is scheduling hard? Scheduler touches every packet
Algorithms must meet 1 ns timing budget Classification Packet Scheduler ? DWRR
Strict priority scheduling
2-level hierarchical scheduling
Shaping<br>
slide7. Why is programmable scheduling hard? Plenty of scheduling algorithms
Yet, no consensus on the right abstractions for scheduling
In contrast to
Parse graphs for parsing
Match-Action tables for forwarding<br>
slide8. The Push-In First-Out Queue Many algorithms determine transmission order at packet arrival
Relative order of packet transmissions of packets in the queue doesn’t change with future arrivals
Examples:
SJF: Order determined by flow size
FCFS: Order determined by arrival time
Push-in first-out queues (PIFO): packets are pushed into an arbitrary location based on a priority, and dequeued from the head
First used as a proof construct by Chuang et. al<br>
slide9. A programmable scheduler Classification & Transmission Order Computation Push-In-First-Out (PIFO) Queue Scheduler Ingress Pipeline Classification & Transmission Order Computation<br>
slide10. pFabric using PIFO Push-In-First-Out (PIFO) Queue Scheduler f = flow(p)
p.prio = f.rem_size<br>
slide11. Weighted Fair Queuing f = flow(p)
p.start = T[f].finish
T[f].finish = p.start + p.len / p.w
p.prio = p.start Push-In-First-Out (PIFO) Queue Scheduler Ingress Pipeline<br>
slide12. Traffic Shaping 1. update tokens
2. p.send = now +
(p.len - tokens) / rate;
3. p.prio =p.send Push-In-First-Out (PIFO) Queue Scheduler Ingress Pipeline<br>
slide13. Composing PIFOs Hierarchical packet-fair
queueing (HPFQ) A (0.5) B (0.5) 1
(0.1) 2
(0.9) 3
(0.3) 4
(0.7) PIFO-root
(WFQ on A and B) PIFO-A
(WFQ on 1 and 2) PIFO-B
(WFQ on 3 and 4) 1 3 2 4 2 A B A B A Composing PIFOs<br>
slide14. The PIFO abstraction PIFO: A sorted array that let us insert an entry (packet or PIFO pointer) into a PIFO based on a programmable priority
Entries are always dequeued from the head
If an entry is a packet, dequeue and transmit it
If an entry is a PIFO, dequeue it, and continue recursively<br>
slide15. PIFO in hardware Meets timing at 1 GHz on a 16 nm node
5 % area overhead for 3-level hierarchy
Challenges wisdom that sorting is hard Min Max Range search CAM MiniPIFO Mini-PIFO bank 1 10 10 100 100 300 300 500 500 1000 1000 2000 1 10 10 100 100 300 300 500 1000 500 1000 2000 128 elements 1000 mini-PIFOs<br>
slide16. Closing thoughts Line-rate programmable scheduling is within reach
Two concrete benefits
Program new scheduling algorithms
Design and verify a PIFO, not many scheduling algorithms<br>
slide17. LSTF Add transmission delay to slack Push-In-First-Out (PIFO) Queue Scheduler Ingress Pipeline Decrement wait time in queue from slack Initialize slack
values<br>
slide18. Are PIFOs feasible? Could use scalable hardware heaps
Too complex for shallow-buffered chips
But, can exploit shallow buffers<br>
slide19. Packet Range search using packet priority (150) Insert into sorted array<br>
slide2. Network switches over time Initially, fixed-function devices
Changing operator requirements
Tunnels
Access Control
Policy routing
New headers<br>
slide3. Alternatives to fixed-function Programmable switches
FPGAs: Arista 7124 FX, NetFPGA
NPUs: Intel IXP
CPUs: Click, RouteBricks
10—100x loss in performance
Limited adoption relative to line-rate switches<br>
slide4. Programmable scheduling at line rate Programmable: Can we express a new scheduling algorithm?
Line-rate: Highest capacity supported by a communication standard<br>
slide5. Programmability at line-rate OpenFlow: Match-Action interface, fixed fields, fixed actions
P4, RMT, FlexPipe, Xpliant: Protocol-independent match-action pipeline. Parser Ingress Pipeline Stage 1 Match Scheduler Deparser Egress Pipeline Action Match Action Match Action Stage N Match Action Match Action Match Action Stage 1 Match Action Match Action Match Action Stage N Match Action Match Action Match Action Eth IP TCP<br>
slide6. Why is scheduling hard? Scheduler touches every packet
Algorithms must meet 1 ns timing budget Classification Packet Scheduler ? DWRR
Strict priority scheduling
2-level hierarchical scheduling
Shaping<br>
slide7. Why is programmable scheduling hard? Plenty of scheduling algorithms
Yet, no consensus on the right abstractions for scheduling
In contrast to
Parse graphs for parsing
Match-Action tables for forwarding<br>
slide8. The Push-In First-Out Queue Many algorithms determine transmission order at packet arrival
Relative order of packet transmissions of packets in the queue doesn’t change with future arrivals
Examples:
SJF: Order determined by flow size
FCFS: Order determined by arrival time
Push-in first-out queues (PIFO): packets are pushed into an arbitrary location based on a priority, and dequeued from the head
First used as a proof construct by Chuang et. al<br>
slide9. A programmable scheduler Classification & Transmission Order Computation Push-In-First-Out (PIFO) Queue Scheduler Ingress Pipeline Classification & Transmission Order Computation<br>
slide10. pFabric using PIFO Push-In-First-Out (PIFO) Queue Scheduler f = flow(p)
p.prio = f.rem_size<br>
slide11. Weighted Fair Queuing f = flow(p)
p.start = T[f].finish
T[f].finish = p.start + p.len / p.w
p.prio = p.start Push-In-First-Out (PIFO) Queue Scheduler Ingress Pipeline<br>
slide12. Traffic Shaping 1. update tokens
2. p.send = now +
(p.len - tokens) / rate;
3. p.prio =p.send Push-In-First-Out (PIFO) Queue Scheduler Ingress Pipeline<br>
slide13. Composing PIFOs Hierarchical packet-fair
queueing (HPFQ) A (0.5) B (0.5) 1
(0.1) 2
(0.9) 3
(0.3) 4
(0.7) PIFO-root
(WFQ on A and B) PIFO-A
(WFQ on 1 and 2) PIFO-B
(WFQ on 3 and 4) 1 3 2 4 2 A B A B A Composing PIFOs<br>
slide14. The PIFO abstraction PIFO: A sorted array that let us insert an entry (packet or PIFO pointer) into a PIFO based on a programmable priority
Entries are always dequeued from the head
If an entry is a packet, dequeue and transmit it
If an entry is a PIFO, dequeue it, and continue recursively<br>
slide15. PIFO in hardware Meets timing at 1 GHz on a 16 nm node
5 % area overhead for 3-level hierarchy
Challenges wisdom that sorting is hard Min Max Range search CAM MiniPIFO Mini-PIFO bank 1 10 10 100 100 300 300 500 500 1000 1000 2000 1 10 10 100 100 300 300 500 1000 500 1000 2000 128 elements 1000 mini-PIFOs<br>
slide16. Closing thoughts Line-rate programmable scheduling is within reach
Two concrete benefits
Program new scheduling algorithms
Design and verify a PIFO, not many scheduling algorithms<br>
slide17. LSTF Add transmission delay to slack Push-In-First-Out (PIFO) Queue Scheduler Ingress Pipeline Decrement wait time in queue from slack Initialize slack
values<br>
slide18. Are PIFOs feasible? Could use scalable hardware heaps
Too complex for shallow-buffered chips
But, can exploit shallow buffers<br>
slide19. Packet Range search using packet priority (150) Insert into sorted array<br>