14-760: Adv. Real-World Networking Lecture 6 *
Description: 14-760: Adv. Real-World Networking Lecture 6 Switch Internals Spring 2020 Kesden Switch Goals Allow frame to flow from input to output port With a minimum latency And with minimal involvement of or interference to stations connected
Related Topics
Download Presentation
"14-760: Adv. Real-World Networking Lecture 6 *" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation Transcript
slide1. 14-760: Adv. Real-World Networking Lecture 6 * Switch Internals * Spring 2020 * Kesden<br>
slide2. Switch Goals Allow frame to flow from input to output port
With a minimum latency
And with minimal involvement of or interference to stations connected by other ports<br>
slide3. Key Switch Considerations Line rate
Backplane rate
Memory requirements
Processing requirements
Power requirements
Etc.<br>
slide4. Base Case: Bus Architecture (“Hub”) All ports shared the same backplane without any arbitration, buffering, switching, etc.
Can be viewed as a switch with all ports connected at all times.
Efficient for broadcast.
Significant wasted network time for unicast as significant portions of the network carry messages without leading to any intended recipient(s).
Latency comes from collision prevention and recovery.
No memory required in switch – buffering on sending and receiving devices.<br>
slide5. Example BackplaNe: CrossBar Switch We can make things better by building a fabric of switched connections
We’ve already talked about this example in passing
Great in the situation where there is no concentration of multiple inputs going to a single output
But, if that happens, then what?
Collision
Queueing
And, this switches complexity grows O(N2) with N ports
Ouch!<br>
slide6. Ideal world: Output Queued (OQ) Switch: Basic Model Frames arrive at an input port
They are immediately move to the correct output port
This ensures that input ports don’t need queues
They are then queued at the output port until they are drained onto the network
This queuing is necessary because multiple input ports could simultaneously receive frames destined for the same output port, requiring that they be drained over time.<br>
slide7. Ideal world: Output Queued (OQ) Switch: Key Challenge If there are N ports the backplane bandwidth must be N*the bandwidth of each port
Why? No input queuing. Can’t delay data movement.
Is this reasonable?
As number of ports increase linearly, i.e. O(n), backplane speed needs to increase quadratically, e.g. O(N2)
This is starting to feel like a challenge<br>
slide8. Ideal world: Output Queued (OQ) Switch: Key Challenge To understand if this O(N2) connections are possible, we’ve got to understand how the connections are made.
Let’s assume they are wires and semiconductor switches
Can we clock the backplane N times faster than the ports for a large N?
Not reasonable
But, let’s assume that we could.
We’d still have to read the data from memory and write it to memory, right?
This still isn’t likely in our favor.
See next slide.<br>
slide9. Memory latency vs network Speed trend lines: See Another potential The Problem? Credit: 14/15/18-213/513/600 Credit: https://www.napatech.com/history-of-ethernet-new-rules-and-the-ongoing-evolution/<br>
slide10. Input queued Switch (IQ): Basic Model Okay. If we…
Need to queue because we could have N simultaneous inputs destined for the same port, so they can’t all be sent at the same time
And we can’t do the queueing on the output port, because that would require an unreasonable ratio of port to switch throughput
Maybe we can just queue on the input ports, and then drain to the output ports, as available?
Seems like a good idea, r-i-g-h-t?<br>
slide11. Input queued Switch (IQ): Key Challenge Packets arriving at an input port are not necessarily destined for the same output port.
What happens if the “head of the line” of the input queue can’t be dequeued, because the output port is congested?
“Head of line blocking”
Frames destined for other ports, which aren’t congested, get stuck behind it.<br>
slide12. Input queued Switch (IQ): Theoretical Results Theoretical results:
Assuming uniform traffic less than 60% throughput is achieved
Hluchyj & Karol, “Queuing in high-performance packet switching”, IEEE Journal on Selected Areas in Communications, Volume: 6 , Issue: 9 , Dec. 1988.
Graph matching algorithms can improve this to up to 100%, in theory, but at an unbearable computational complexity.<br>
slide13. Virtual Output Queues (VOQ) What if we move the output ques to the input side of the fabric?
In other words, what if each input queue is divided into virtual output queues for each output?
And we eliminate the output-side output queues
Now, each arriving frame is queued at the input port – but in a queue specific to the output port?
No more head of line blocking!<br>
slide14. Virtual Output Queues (VOQ): Ain’t Nothin’ For Free But, we now have a lot of queues – N2 queues.
That’s a lot
We now need to figure out which queue to drain to where and when
O(N2) complexity
Again, it is great if we can do it
But, going from N ports to O(N2) work may not be doable<br>
slide15. Combined Input-Output Queue (CIOQ) SWITCHING Have queues on input
Have queues on output
Use some scheduling algorithm to move inputs to outputs. Goals:
Avoid head of line blocking
Avoid running out of queue space
Avoid running out of fabric time<br>
slide16. Combined Input-Output Queue (CIOQ) Trade-Offs Good:
Avoid head of line blocking
Simpler than VOQs
Not so good:
Copying data twice, e.g. once per buffer<br>
slide17. Combined Input-Output Queue (CIOQ) Key Theoretical Findings Various algorithms are able to produce CIOQ switches that emulate OQ switches as long as they have a speed-up of 2 or more
In other words, their internal fabric can move two frame per cycle.<br>
slide18. Combined Input-Output Queue (CIOQ) Key Question How to schedule movement from inputs to outputs to…
Avoid head of line blocking
Avoid running out of queue space
Avoid running out of fabric time
All sorts of other parameters
Fabric speed
Implement priorities with separate queues?<br>
slide19. Internally Buffered Crossbar Switches (IBCS) Consider where we might put buffers in crossbar switch:
Input queues
Output queues
Each point of connection<br>
slide20. Internally Buffered Crossbar Switches (IBCS): O(N2) Again? Can buffer at each interconnect
Pairs each buffer with connectivity, no bottle neck there.
Serves as a VOQ of sorts w.r.t. hed of line blocking.
O(N2) memory better than O(N2) speed
At least for modest N<br>
slide21. Internally Buffered Crossbar Switches (IBCS): Can be combined with input buffering
Less likely to HOL block with N places to go.
Output buffering doesn’t make as much sense as input buffering
Interconnects provide an output queue<br>
slide22. Knockout Switches Concentrators “Knock out” competing inputs randomly
Prevents N-To-1 Concentration possible at a crossbar switch
Allows some L < N buffers associated with each input port vs N for a buffered crossbar.<br>
slide23. Key Parameters (For any of these) How large each buffer/queue should be
How to schedule which queue gets drained when<br>
slide24. Reflections on Buffering It takes time to copy into and out of buffers
Longer queue sizes mean the potential for greater latency and greater jitter
Of course, lower occupancy means lower latency and lower jitter, even if there is more room
Shorter buffer sizes mean greater potential for dropped frames
And, dropped frames often mean resends with associated longer delays felt by application
Impact of drops depends on application
Behavior of multiple layers of buffering is very dyamic<br>
slide25. The Real World Buffered crossbar fabrics are really common at the top level
But they are often attached to very sophisticated line cards<br>
slide26. Cisco 8-port M1-XL I/O Module Architecture https://www.cisco.com/c/en/us/td/docs/solutions/Enterprise/Data_Center/VMDC/2-6/vmdcm1f1wp.html<br>
slide2. Switch Goals Allow frame to flow from input to output port
With a minimum latency
And with minimal involvement of or interference to stations connected by other ports<br>
slide3. Key Switch Considerations Line rate
Backplane rate
Memory requirements
Processing requirements
Power requirements
Etc.<br>
slide4. Base Case: Bus Architecture (“Hub”) All ports shared the same backplane without any arbitration, buffering, switching, etc.
Can be viewed as a switch with all ports connected at all times.
Efficient for broadcast.
Significant wasted network time for unicast as significant portions of the network carry messages without leading to any intended recipient(s).
Latency comes from collision prevention and recovery.
No memory required in switch – buffering on sending and receiving devices.<br>
slide5. Example BackplaNe: CrossBar Switch We can make things better by building a fabric of switched connections
We’ve already talked about this example in passing
Great in the situation where there is no concentration of multiple inputs going to a single output
But, if that happens, then what?
Collision
Queueing
And, this switches complexity grows O(N2) with N ports
Ouch!<br>
slide6. Ideal world: Output Queued (OQ) Switch: Basic Model Frames arrive at an input port
They are immediately move to the correct output port
This ensures that input ports don’t need queues
They are then queued at the output port until they are drained onto the network
This queuing is necessary because multiple input ports could simultaneously receive frames destined for the same output port, requiring that they be drained over time.<br>
slide7. Ideal world: Output Queued (OQ) Switch: Key Challenge If there are N ports the backplane bandwidth must be N*the bandwidth of each port
Why? No input queuing. Can’t delay data movement.
Is this reasonable?
As number of ports increase linearly, i.e. O(n), backplane speed needs to increase quadratically, e.g. O(N2)
This is starting to feel like a challenge<br>
slide8. Ideal world: Output Queued (OQ) Switch: Key Challenge To understand if this O(N2) connections are possible, we’ve got to understand how the connections are made.
Let’s assume they are wires and semiconductor switches
Can we clock the backplane N times faster than the ports for a large N?
Not reasonable
But, let’s assume that we could.
We’d still have to read the data from memory and write it to memory, right?
This still isn’t likely in our favor.
See next slide.<br>
slide9. Memory latency vs network Speed trend lines: See Another potential The Problem? Credit: 14/15/18-213/513/600 Credit: https://www.napatech.com/history-of-ethernet-new-rules-and-the-ongoing-evolution/<br>
slide10. Input queued Switch (IQ): Basic Model Okay. If we…
Need to queue because we could have N simultaneous inputs destined for the same port, so they can’t all be sent at the same time
And we can’t do the queueing on the output port, because that would require an unreasonable ratio of port to switch throughput
Maybe we can just queue on the input ports, and then drain to the output ports, as available?
Seems like a good idea, r-i-g-h-t?<br>
slide11. Input queued Switch (IQ): Key Challenge Packets arriving at an input port are not necessarily destined for the same output port.
What happens if the “head of the line” of the input queue can’t be dequeued, because the output port is congested?
“Head of line blocking”
Frames destined for other ports, which aren’t congested, get stuck behind it.<br>
slide12. Input queued Switch (IQ): Theoretical Results Theoretical results:
Assuming uniform traffic less than 60% throughput is achieved
Hluchyj & Karol, “Queuing in high-performance packet switching”, IEEE Journal on Selected Areas in Communications, Volume: 6 , Issue: 9 , Dec. 1988.
Graph matching algorithms can improve this to up to 100%, in theory, but at an unbearable computational complexity.<br>
slide13. Virtual Output Queues (VOQ) What if we move the output ques to the input side of the fabric?
In other words, what if each input queue is divided into virtual output queues for each output?
And we eliminate the output-side output queues
Now, each arriving frame is queued at the input port – but in a queue specific to the output port?
No more head of line blocking!<br>
slide14. Virtual Output Queues (VOQ): Ain’t Nothin’ For Free But, we now have a lot of queues – N2 queues.
That’s a lot
We now need to figure out which queue to drain to where and when
O(N2) complexity
Again, it is great if we can do it
But, going from N ports to O(N2) work may not be doable<br>
slide15. Combined Input-Output Queue (CIOQ) SWITCHING Have queues on input
Have queues on output
Use some scheduling algorithm to move inputs to outputs. Goals:
Avoid head of line blocking
Avoid running out of queue space
Avoid running out of fabric time<br>
slide16. Combined Input-Output Queue (CIOQ) Trade-Offs Good:
Avoid head of line blocking
Simpler than VOQs
Not so good:
Copying data twice, e.g. once per buffer<br>
slide17. Combined Input-Output Queue (CIOQ) Key Theoretical Findings Various algorithms are able to produce CIOQ switches that emulate OQ switches as long as they have a speed-up of 2 or more
In other words, their internal fabric can move two frame per cycle.<br>
slide18. Combined Input-Output Queue (CIOQ) Key Question How to schedule movement from inputs to outputs to…
Avoid head of line blocking
Avoid running out of queue space
Avoid running out of fabric time
All sorts of other parameters
Fabric speed
Implement priorities with separate queues?<br>
slide19. Internally Buffered Crossbar Switches (IBCS) Consider where we might put buffers in crossbar switch:
Input queues
Output queues
Each point of connection<br>
slide20. Internally Buffered Crossbar Switches (IBCS): O(N2) Again? Can buffer at each interconnect
Pairs each buffer with connectivity, no bottle neck there.
Serves as a VOQ of sorts w.r.t. hed of line blocking.
O(N2) memory better than O(N2) speed
At least for modest N<br>
slide21. Internally Buffered Crossbar Switches (IBCS): Can be combined with input buffering
Less likely to HOL block with N places to go.
Output buffering doesn’t make as much sense as input buffering
Interconnects provide an output queue<br>
slide22. Knockout Switches Concentrators “Knock out” competing inputs randomly
Prevents N-To-1 Concentration possible at a crossbar switch
Allows some L < N buffers associated with each input port vs N for a buffered crossbar.<br>
slide23. Key Parameters (For any of these) How large each buffer/queue should be
How to schedule which queue gets drained when<br>
slide24. Reflections on Buffering It takes time to copy into and out of buffers
Longer queue sizes mean the potential for greater latency and greater jitter
Of course, lower occupancy means lower latency and lower jitter, even if there is more room
Shorter buffer sizes mean greater potential for dropped frames
And, dropped frames often mean resends with associated longer delays felt by application
Impact of drops depends on application
Behavior of multiple layers of buffering is very dyamic<br>
slide25. The Real World Buffered crossbar fabrics are really common at the top level
But they are often attached to very sophisticated line cards<br>
slide26. Cisco 8-port M1-XL I/O Module Architecture https://www.cisco.com/c/en/us/td/docs/solutions/Enterprise/Data_Center/VMDC/2-6/vmdcm1f1wp.html<br>