Overview¶

AF_XDP is an address family that is optimized for high performancepacket processing.

This document assumes that the reader is familiar with BPF and XDP. Ifnot, the Cilium project has an excellent reference guide athttp://cilium.readthedocs.io/en/latest/bpf/.

Using the XDP_REDIRECT action from an XDP program, the program canredirect ingress frames to other XDP enabled netdevs, using thebpf_redirect_map() function. AF_XDP sockets enable the possibility forXDP programs to redirect frames to a memory buffer in a user-spaceapplication.

An AF_XDP socket (XSK) is created with the normal socket()syscall. Associated with each XSK are two rings: the RX ring and theTX ring. A socket can receive packets on the RX ring and it can sendpackets on the TX ring. These rings are registered and sized with thesetsockopts XDP_RX_RING and XDP_TX_RING, respectively. It is mandatoryto have at least one of these rings for each socket. An RX or TXdescriptor ring points to a data buffer in a memory area called aUMEM. RX and TX can share the same UMEM so that a packet does not haveto be copied between RX and TX. Moreover, if a packet needs to be keptfor a while due to a possible retransmit, the descriptor that pointsto that packet can be changed to point to another and reused rightaway. This again avoids copying data.

The UMEM consists of a number of equally sized chunks. A descriptor inone of the rings references a frame by referencing its addr. The addris simply an offset within the entire UMEM region. The user spaceallocates memory for this UMEM using whatever means it feels is mostappropriate (malloc, mmap, huge pages, etc). This memory area is thenregistered with the kernel using the new setsockopt XDP_UMEM_REG. TheUMEM also has two rings: the FILL ring and the COMPLETION ring. TheFILL ring is used by the application to send down addr for the kernelto fill in with RX packet data. References to these frames will thenappear in the RX ring once each packet has been received. TheCOMPLETION ring, on the other hand, contains frame addr that thekernel has transmitted completely and can now be used again by userspace, for either TX or RX. Thus, the frame addrs appearing in theCOMPLETION ring are addrs that were previously transmitted using theTX ring. In summary, the RX and FILL rings are used for the RX pathand the TX and COMPLETION rings are used for the TX path.

The socket is then finally bound with a bind() call to a device and aspecific queue id on that device, and it is not until bind iscompleted that traffic starts to flow.

The UMEM can be shared between processes, if desired. If a processwants to do this, it simply skips the registration of the UMEM and itscorresponding two rings, sets the XDP_SHARED_UMEM flag in the bindcall and submits the XSK of the process it would like to share UMEMwith as well as its own newly created XSK socket. The new process willthen receive frame addr references in its own RX ring that point tothis shared UMEM. Note that since the ring structures aresingle-consumer / single-producer (for performance reasons), the newprocess has to create its own socket with associated RX and TX rings,since it cannot share this with the other process. This is also thereason that there is only one set of FILL and COMPLETION rings perUMEM. It is the responsibility of a single process to handle the UMEM.

How is then packets distributed from an XDP program to the XSKs? Thereis a BPF map called XSKMAP (or BPF_MAP_TYPE_XSKMAP in full). Theuser-space application can place an XSK at an arbitrary place in thismap. The XDP program can then redirect a packet to a specific index inthis map and at this point XDP validates that the XSK in that map wasindeed bound to that device and ring number. If not, the packet isdropped. If the map is empty at that index, the packet is alsodropped. This also means that it is currently mandatory to have an XDPprogram loaded (and one XSK in the XSKMAP) to be able to get anytraffic to user space through the XSK.

AF_XDP can operate in two different modes: XDP_SKB and XDP_DRV. If thedriver does not have support for XDP, or XDP_SKB is explicitly chosenwhen loading the XDP program, XDP_SKB mode is employed that uses SKBstogether with the generic XDP support and copies out the data to userspace. A fallback mode that works for any network device. On the otherhand, if the driver has support for XDP, it will be used by the AF_XDPcode to provide better performance, but there is still a copy of thedata into user space.

Concepts¶

In order to use an AF_XDP socket, a number of associated objects needto be setup. These objects and their options are explained in thefollowing sections.

For an overview on how AF_XDP works, you can also take a look at theLinux Plumbers paper from 2018 on the subject:http://vger.kernel.org/lpc_net2018_talks/lpc18_paper_af_xdp_perf-v2.pdf. DoNOT consult the paper from 2017 on “AF_PACKET v4”, the first attemptat AF_XDP. Nearly everything changed since then. Jonathan Corbet hasalso written an excellent article on LWN, “Accelerating networkingwith AF_XDP”. It can be found athttps://lwn.net/Articles/750845/.

Libbpf¶

Libbpf is a helper library for eBPF and XDP that makes using thesetechnologies a lot simpler. It also contains specific helper functionsin tools/lib/bpf/xsk.h for facilitating the use of AF_XDP. Itcontains two types of functions: those that can be used to make thesetup of AF_XDP socket easier and ones that can be used in the dataplane to access the rings safely and quickly. To see an example on howto use this API, please take a look at the sample application insamples/bpf/xdpsock_usr.c which uses libbpf for both setup and dataplane operations.

We recommend that you use this library unless you have become a poweruser. It will make your program a lot simpler.

XSKMAP / BPF_MAP_TYPE_XSKMAP¶

On XDP side there is a BPF map type BPF_MAP_TYPE_XSKMAP (XSKMAP) thatis used in conjunction with bpf_redirect_map() to pass the ingressframe to a socket.

The user application inserts the socket into the map, via the bpf()system call.

Note that if an XDP program tries to redirect to a socket that doesnot match the queue configuration and netdev, the frame will bedropped. E.g. an AF_XDP socket is bound to netdev eth0 andqueue 17. Only the XDP program executing for eth0 and queue 17 willsuccessfully pass data to the socket. Please refer to the sampleapplication (samples/bpf/) in for an example.

Configuration Flags and Socket Options¶

These are the various configuration flags that can be used to controland monitor the behavior of AF_XDP sockets.

#include<linux/bpf.h>#include"bpf_helpers.h"#define MAX_SOCKS 16struct{__uint(type,BPF_MAP_TYPE_XSKMAP);__uint(max_entries,MAX_SOCKS);__uint(key_size,sizeof(int));__uint(value_size,sizeof(int));}xsks_mapSEC(".maps");staticunsignedintrr;SEC("xdp_sock")intxdp_sock_prog(structxdp_md*ctx){rr=(rr+1)&(MAX_SOCKS-1);returnbpf_redirect_map(&xsks_map,rr,XDP_DROP);}

if(xsk_ring_prod__needs_wakeup(&my_tx_ring))sendto(xsk_socket__fd(xsk_handle),NULL,0,MSG_DONTWAIT,NULL,0);

structxdp_statistics{__u64rx_dropped;/* Dropped for reasons other than invalid desc */__u64rx_invalid_descs;/* Dropped due to invalid descriptor */__u64tx_invalid_descs;/* Dropped due to invalid descriptor */};

Usage¶

In order to use AF_XDP sockets two parts are needed. Theuser-space application and the XDP program. For a complete setup andusage example, please refer to the sample application. The user-spaceside is xdpsock_user.c and the XDP side is part of libbpf.

The XDP code sample included in tools/lib/bpf/xsk.c is the following:

SEC("xdp_sock")intxdp_sock_prog(structxdp_md*ctx){intindex=ctx->rx_queue_index;// A set entry here means that the corresponding queue_id// has an active AF_XDP socket bound to it.if(bpf_map_lookup_elem(&xsks_map,&index))returnbpf_redirect_map(&xsks_map,index,0);returnXDP_PASS;}

A simple but not so performance ring dequeue and enqueue could looklike this:

// struct xdp_rxtx_ring {//  __u32 *producer;//  __u32 *consumer;//  struct xdp_desc *desc;// };// struct xdp_umem_ring {//  __u32 *producer;//  __u32 *consumer;//  __u64 *desc;// };// typedef struct xdp_rxtx_ring RING;// typedef struct xdp_umem_ring RING;// typedef struct xdp_desc RING_TYPE;// typedef __u64 RING_TYPE;intdequeue_one(RING*ring,RING_TYPE*item){__u32entries=*ring->producer-*ring->consumer;if(entries==0)return-1;// read-barrier!*item=ring->desc[*ring->consumer&(RING_SIZE-1)];(*ring->consumer)++;return0;}intenqueue_one(RING*ring,constRING_TYPE*item){u32free_entries=RING_SIZE-(*ring->producer-*ring->consumer);if(free_entries==0)return-1;ring->desc[*ring->producer&(RING_SIZE-1)]=*item;// write-barrier!(*ring->producer)++;return0;}

But please use the libbpf functions as they are optimized and ready touse. Will make your life easier.

Sample application¶

There is a xdpsock benchmarking/test application included thatdemonstrates how to use AF_XDP sockets with private UMEMs. Say thatyou would like your UDP traffic from port 4242 to end up in queue 16,that we will enable AF_XDP on. Here, we use ethtool for this:

ethtool -N p3p2 rx-flow-hash udp4 fnethtool -N p3p2 flow-type udp4 src-port 4242 dst-port 4242 \    action 16

Running the rxdrop benchmark in XDP_DRV mode can then be doneusing:

samples/bpf/xdpsock -i p3p2 -q 16 -r -N

For XDP_SKB mode, use the switch “-S” instead of “-N” and all optionscan be displayed with “-h”, as usual.

This sample application uses libbpf to make the setup and usage ofAF_XDP simpler. If you want to know how the raw uapi of AF_XDP isreally used to make something more advanced, take a look at the libbpfcode in tools/lib/bpf/xsk.[ch].

FAQ¶

Q: I am not seeing any traffic on the socket. What am I doing wrong?

A: When a netdev of a physical NIC is initialized, Linux usually

allocates one RX and TX queue pair per core. So on a 8 core system,queue ids 0 to 7 will be allocated, one per core. In the AF_XDPbind call or the xsk_socket__create libbpf function call, youspecify a specific queue id to bind to and it is only the traffictowards that queue you are going to get on you socket. So in theexample above, if you bind to queue 0, you are NOT going to get anytraffic that is distributed to queues 1 through 7. If you arelucky, you will see the traffic, but usually it will end up on oneof the queues you have not bound to.

There are a number of ways to solve the problem of getting thetraffic you want to the queue id you bound to. If you want to seeall the traffic, you can force the netdev to only have 1 queue, queueid 0, and then bind to queue 0. You can use ethtool to do this:

sudo ethtool -L <interface> combined 1

If you want to only see part of the traffic, you can program theNIC through ethtool to filter out your traffic to a single queue idthat you can bind your XDP socket to. Here is one example in whichUDP traffic to and from port 4242 are sent to queue 2:

sudo ethtool -N <interface> rx-flow-hash udp4 fnsudo ethtool -N <interface> flow-type udp4 src-port 4242 dst-port \4242 action 2

A number of other ways are possible all up to the capabilities ofthe NIC you have.

Q: Can I use the XSKMAP to implement a switch betwen different umems

in copy mode?

A: The short answer is no, that is not supported at the moment. The

XSKMAP can only be used to switch traffic coming in on queue id Xto sockets bound to the same queue id X. The XSKMAP can containsockets bound to different queue ids, for example X and Y, but onlytraffic goming in from queue id Y can be directed to sockets boundto the same queue id Y. In zero-copy mode, you should use theswitch, or other distribution mechanism, in your NIC to directtraffic to the correct queue id and socket.

Credits¶

• Björn Töpel (AF_XDP core)
• Magnus Karlsson (AF_XDP core)
• Alexander Duyck
• Alexei Starovoitov
• Daniel Borkmann
• Jesper Dangaard Brouer
• John Fastabend
• Jonathan Corbet (LWN coverage)
• Michael S. Tsirkin
• Qi Z Zhang
• Willem de Bruijn