India DPDK Summit Bangalore March2018 Open v Switch HW offload over DPDK Virtual Switch is Overloaded as We Go Cloud Native Application OS Infrastructure App App App Guest OS Infrastructure Hypervisor ID: 769314
Download Presentation The PPT/PDF document "India DPDK Summit - Bangalore" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
India DPDK Summit - Bangalore March-2018 Open v Switch HW offload over DPDK
Virtual Switch is Overloaded as We Go Cloud Native Application OS Infrastructure App App App Guest OS Infrastructure Hypervisor App App App Infrastructure OS Hypervisor Virtualization, App Running in VM OS Resource Virtualization, App Running in Container Bare Metal Host OS OVS Guest OS Guest OS Docker Engine OVS App App App App App App Few apps per serverLow mobilityLow resource utilization 10s-100s VM per serverGood mobilityHigh resource utilization 1000s containers per server Great portability High resource utilization High performance
Telco and Cloud Applications Expose OVS Weaknesses
How do We Solve the Problem? - ASAP 2 Accelerated Switching and Packet Processing
NIC Common Operations in Networking Most network functions share some data-path operations Packet classification (into flows) Action based on the classification result Mellanox NIC has the capability to offload both the classification and the actions in hardware Packets In Processed Packets Out
Accelerated Virtual Switch (ASAP2-Flex)
Flex HW Acceleration for vSwitch / vRouter Offload some elements of the data-path to the NIC, but not the entire data-pathData will still flow via the vSwitchPara-Virtualized VM (not SR-IOV)Offloads (examples)Classification offloadApplication provide flow spec and flow ID Classification done in HW and attach a flow ID in case of matchvSwitch classify based on the flow ID rather than full flow specrte flow is used to configure the classification VxLAN Encap/decapVLAN add/removeQoS vSwitch acceleration VM ConnectX 4 eSwitch VM Hypervisor OVS PV PV TC / DPDK Offload Data Path Data Path PF
Packets flow PMD NIC Hardware User OVS DataPath OVS- vswitchD Rte_flow Flow X mark with id 0x1234mbuf ->fdir.id 0x1234 Do OVS action Y DP_IF - DPDK Config flowHW classification offload concept For every OVS flow, DP_IF should use the DPDK rte_flow to classify with Action tag (report id) or drop.When packet is received, use the tag id instead of classifying the packet againfor Example :OVS set action Y to flow X Add a rte_flow to tag with id 0x1234 for flow XConfig datapath to do action Y for mbuf->fdir.id = 0x1234OVS action drop for flow ZUse rte_flow DROP and COUNT action to drop and count flow ZUse rte_flow counter to get flow statistic
Multiple tables Programmable table size Programmable table cascadingDedicate, isolated tables for hypervisor and/or VMs Practically unlimited table size Can support million of rules/flows Flow Tables Overview
Key fields example Ethernet Layer 2Destination MAC2 outer VLANs / priority Ethertype IP (v4 /v6) Source addressDestination addressProtocol / Next headerTCP /UDPSource portDestination portFlexible fields extraction by “Flexparse”All fields mandatory by OpenFlow Flow Tables – Classification
Actions *Steering and Forwarding Drop / Allow Counter set Send to Monitor QP EncapsulationDecapsulationReport Flow IDAdditional actions in newer NICsHeader rewriteMPLS and NSH encap/decapFlexible encap/decap Hairpin mode Flow Tables – Actions * Not all combinations are supported
OVS-DPDK using Flex HW classification offload Flow id cache For every datapath rule we add a rte_flow with flow idThe flow id cache can contain flow rules in excess of 1M When packet received matches with a flow id in cache, no need to re-classify the packet to get the rule
Performance Code submitted by Yuanhan Liu. Planned to be integrated to OVS 2.10 Single core for each pmd, single queue, Case#flowsBase MPPsOffload MPPsimprovement Wire to virtio 15.88.750%Wire to wire 16.911.770%Wire to wire 5124,211,2267%
Full OVS Offload (ASAP2-Direct)
Full HW Offload for vSwitch/vRouter accelerationOffload the whole packet processing onto the embedded switch Split control plane and forwarding planeForwarding plane – use the embedded switchRemove the cost of dataplane in SW SRIOV based
Representors We use VF representors Representor ports are a netdev modeling of eSwitch portsThe VF representor supports the following operationsSend packet from the host CPU to VF (OVS Re-injection)Receive of eSwitch “miss” packets Flow configuration (add/remove)Flow statistics read for the purposes of aging and statistics The Representor devices are switchdev instances OVS
OVS-DPDK with full HW offloads OVS DPDK with direct data path to VM’s switchdev SR-IOV offloads already implemented in Kernel OVS Use DPDK ‘slow’ path for exception flows or unsupported HW features Allow DPDK to use the control and data path of embedded switch Representor ports are exposed over the PF Data Path RX & TX queues per representor Send/receive packet to/from VF is done through it’s representorACL, steering, routing encap/decap flow counters IPSec Co-exists with para- virt solutionsrte_flow API will be extended to support full HW offload in DPDK 18.05 GuestPV uplink GuestPV VF representor GuestVF VF driver virtio virtio OVS-DPDK netdev uplink Embedded Switch switchdev VF PF Rte_flow NIC
OVS DPDK with/without full HW offload Test Full HW offload Without offload Benefit1 Flow VXLAN 66M PPS7.6M PPS (VLAN)8.6X60K flowsVXLAN19.8M PPS1.9M PPS 10.4X