Making queues a first class citizen Abstracting queues in the UApi Magnus Karlsson amp Björn Töpel Intel Problem Statement 2 bindafxdpfd netdev queueid Ethtool using queueid UAPI Drivers decide what queue ids to use for what No standard ID: 764876
Download Presentation The PPT/PDF document "Making queues a first class citizen : Ab..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Making queues a first class citizen : Abstracting queues in the UApi Magnus Karlsson & Björn Töpel, Intel
Problem Statement 2 bind(af_xdp_fd, netdev, queue_id) Ethtool using queue_id UAPI Drivers decide what queue ids to use for what. No standard Only possible to use pre-configured queues in the driver. No dynamic creation XDP and AF_XDP Tx queues invisible to user space How to get rid of HW queue ids in user space and still be able to create an AF_XDP socket and direct traffic to it? Netdev queues XDP Tx queues AF_XDP Tx & Rx queues NIC specific queues
Basic Requirements Rx: Create a dedicated Rx queue and steer traffic to it Bridge the gap with a getsockopt to get queue ids for use in ethtoolTx: Multiple users (XDP Tx and AF_XDP Tx) but no traffic steeringTraffic shaping and QoS can be present though Requirements:UAPI: create an AF_XDP socket without specifying a HW queue idKernel: common module/functionality for allocating, freeing and naming/enumeration of queues Drivers: No change to drivers that do not use this featureUAPI: configuring traffic-steering and QoS to and from this socket By NIC HW steering or XDP programsWill focus on 1 – 3 only in this presentation 3
Register, alloc & free High-Level Proposal: Option 1 – Hide Queues Driver registers queues with QM (non-netdev vs all?)Components that need queues can alloc, free and lookup in QMQM uses new ndo to perform allocs and frees in the driver 4 f d = bind(af_xdp_fd, netdev) Ethtool using fd (or opaque queue_id)UAPIQueue module (QM)XDP Tx queuesAF_XDP Tx & Rx queuesXsk Ethtool Driver Alloc, free & lookup fd -> HW qid XDP
Register, alloc & free High-Level Proposal: Option 2 – Expose Queues Expose queues and required info to user space to make a decisionNew queues can be created through ethtool or devlink, if neededFeed the desired queue id into bind Flow objects could be added to control steering5 f d = bind(af_xdp_fd, netdev, queue_id)Devlink to expose and manipulate queuesUAPIQueue module (QM)XDP Tx queuesAF_XDP Tx & Rx queuesXsk Devlink Driver Alloc, free & lookup fd -> HW qid XDP
Reflections / Questions Is the queue manager uneccesary and a subdev should be used? Switchdev / tc offload to steer traffic Extend __rx and __tx structs to implement a queue manager?Socket, abstracted queue id or extended queue id in user space?Should netdev use the queue manager too?XDP Tx and AF_XDP Tx two separate paths. Unify ?AF_XDP ToDo list on xdp-project repoNeed infra in user space to list queues 6
Feedback from the Meeting Option 2 preferred Provides advanced configuration and control Simplified user interface can be provided with libbpfStart with the queue object, then the flow objectUse devlink to expose and manipulate queues and flowsKeep the current interface to bind() Queue id can be treated as an opaque7