Features¶

L2TPv2 (PPP over L2TP (UDP tunnels)).L2TPv3 ethernet pseudowires.L2TPv3 PPP pseudowires.L2TPv3 IP encapsulation.Netlink sockets for L2TPv3 configuration management.

History¶

The original pppol2tp driver was introduced in 2.6.23 and providedL2TPv2 functionality (rfc2661). L2TPv2 is used to tunnel one or more PPPsessions over a UDP tunnel.

L2TPv3 (rfc3931) changes the protocol to allow different frame typesto be passed over an L2TP tunnel by moving the PPP-specific parts ofthe protocol out of the core L2TP packet headers. Each frame type isknown as a pseudowire type. Ethernet, PPP, HDLC, Frame Relay and ATMpseudowires for L2TP are defined in separate RFC standards. Anotherchange for L2TPv3 is that it can be carried directly over IP with noUDP header (UDP is optional). It is also possible to create staticunmanaged L2TPv3 tunnels manually without a control protocol(userspace daemon) to manage them.

To support L2TPv3, the original pppol2tp driver was split up toseparate the L2TP and PPP functionality. Existing L2TPv2 userspaceapps should be unaffected as the original pppol2tp sockets API isretained. L2TPv3, however, uses netlink to manage L2TPv3 tunnels andsessions.

Design¶

The L2TP protocol separates control and data frames. The L2TP kerneldrivers handle only L2TP data frames; control frames are alwayshandled by userspace. L2TP control frames carry messages between L2TPclients/servers and are used to setup / teardown tunnels andsessions. An L2TP client or server is implemented in userspace.

Each L2TP tunnel is implemented using a UDP or L2TPIP socket; L2TPIPprovides L2TPv3 IP encapsulation (no UDP) and is implemented using anew l2tpip socket family. The tunnel socket is typically created byuserspace, though for unmanaged L2TPv3 tunnels, the socket can also becreated by the kernel. Each L2TP session (pseudowire) gets a networkinterface instance. In the case of PPP, these interfaces are createdindirectly by pppd using a pppol2tp socket. In the case of ethernet,the netdevice is created upon a netlink request to create an L2TPv3ethernet pseudowire.

For PPP, the PPPoL2TP driver, net/l2tp/l2tp_ppp.c, provides amechanism by which PPP frames carried through an L2TP session arepassed through the kernel’s PPP subsystem. The standard PPP daemon,pppd, handles all PPP interaction with the peer. PPP networkinterfaces are created for each local PPP endpoint. The kernel’s PPPsubsystem arranges for PPP control frames to be delivered to pppd,while data frames are forwarded as usual.

For ethernet, the L2TPETH driver, net/l2tp/l2tp_eth.c, implements anetdevice driver, managing virtual ethernet devices, one perpseudowire. These interfaces can be managed using standard Linux toolssuch as “ip” and “ifconfig”. If only IP frames are passed over thetunnel, the interface can be given an IP addresses of itself and itspeer. If non-IP frames are to be passed over the tunnel, the interfacecan be added to a bridge using brctl. All L2TP datapath protocolfunctions are handled by the L2TP core driver.

Each tunnel and session within a tunnel is assigned a unique tunnel_idand session_id. These ids are carried in the L2TP header of everycontrol and data packet. (Actually, in L2TPv3, the tunnel_id isn’tpresent in data frames - it is inferred from the IP connection onwhich the packet was received.) The L2TP driver uses the ids to lookupinternal tunnel and/or session contexts to determine how to handle thepacket. Zero tunnel / session ids are treated specially - zero ids arenever assigned to tunnels or sessions in the network. In the driver,the tunnel context keeps a reference to the tunnel UDP or L2TPIPsocket. The session context holds data that lets the driver interfaceto the kernel’s network frame type subsystems, i.e. PPP, ethernet.

Userspace Programming¶

For L2TPv2, there are a number of requirements on the userspace L2TPdaemon in order to use the pppol2tp driver.

1. Use a UDP socket per tunnel.
2. Create a single PPPoL2TP socket per tunnel bound to a special nullsession id. This is used only for communicating with the driver butmust remain open while the tunnel is active. Opening this tunnelmanagement socket causes the driver to mark the tunnel socket as anL2TP UDP encapsulation socket and flags it for use by thereferenced tunnel id. This hooks up the UDP receive path viaudp_encap_rcv() in net/ipv4/udp.c. PPP data frames are never passedin this special PPPoX socket.
3. Create a PPPoL2TP socket per L2TP session. This is typically doneby starting pppd with the pppol2tp plugin and appropriatearguments. A PPPoL2TP tunnel management socket (Step 2) must becreated before the first PPPoL2TP session socket is created.

When creating PPPoL2TP sockets, the application provides informationto the driver about the socket in a socket connect() call. Source anddestination tunnel and session ids are provided, as well as the filedescriptor of a UDP socket. See struct pppol2tp_addr ininclude/linux/if_pppol2tp.h. Note that zero tunnel / session ids aretreated specially. When creating the per-tunnel PPPoL2TP managementsocket in Step 2 above, zero source and destination session ids arespecified, which tells the driver to prepare the supplied UDP filedescriptor for use as an L2TP tunnel socket.

Userspace may control behavior of the tunnel or session usingsetsockopt and ioctl on the PPPoX socket. The following socketoptions are supported:-

Only the DEBUG option is supported by the special tunnel managementPPPoX socket.

In addition to the standard PPP ioctls, a PPPIOCGL2TPSTATS is providedto retrieve tunnel and session statistics from the kernel using thePPPoX socket of the appropriate tunnel or session.

For L2TPv3, userspace must use the netlink API defined ininclude/linux/l2tp.h to manage tunnel and session contexts. Thegeneral procedure to create a new L2TP tunnel with one session is:-

1. Open a GENL socket using L2TP_GENL_NAME for configuring the kernelusing netlink.
2. Create a UDP or L2TPIP socket for the tunnel.
3. Create a new L2TP tunnel using a L2TP_CMD_TUNNEL_CREATErequest. Set attributes according to desired tunnel parameters,referencing the UDP or L2TPIP socket created in the previous step.
4. Create a new L2TP session in the tunnel using aL2TP_CMD_SESSION_CREATE request.

The tunnel and all of its sessions are closed when the tunnel socketis closed. The netlink API may also be used to delete sessions andtunnels. Configuration and status info may be set or read using netlink.

The L2TP driver also supports static (unmanaged) L2TPv3 tunnels. Theseare where there is no L2TP control message exchange with the peer tosetup the tunnel; the tunnel is configured manually at each end of thetunnel. There is no need for an L2TP userspace application in thiscase – the tunnel socket is created by the kernel and configuredusing parameters sent in the L2TP_CMD_TUNNEL_CREATE netlinkrequest. The “ip” utility of iproute2 has commands for managing staticL2TPv3 tunnels; do “ip l2tp help” for more information.

Debugging¶

The driver supports a flexible debug scheme where kernel tracemessages may be optionally enabled per tunnel and per session. Care isneeded when debugging a live system since the messages are notrate-limited and a busy system could be swamped. Userspace usessetsockopt on the PPPoX socket to set a debug mask.

The following debug mask bits are available:

If enabled, files under a l2tp debugfs directory can be used to dumpkernel state about L2TP tunnels and sessions. To access it, thedebugfs filesystem must first be mounted:

# mount -t debugfs debugfs /debug

Files under the l2tp directory can then be accessed:

# cat /debug/l2tp/tunnels

The debugfs files should not be used by applications to obtain L2TPstate information because the file format is subject to change. It isimplemented to provide extra debug information to help diagnoseproblems.) Users should use the netlink API.

/proc/net/pppol2tp is also provided for backwards compatibility withthe original pppol2tp driver. It lists information about L2TPv2tunnels and sessions only. Its use is discouraged.

Unmanaged L2TPv3 Tunnels¶

Some commercial L2TP products support unmanaged L2TPv3 ethernettunnels, where there is no L2TP control protocol; tunnels areconfigured at each side manually. New commands are available iniproute2’s ip utility to support this.

To create an L2TPv3 ethernet pseudowire between local host 192.168.1.1and peer 192.168.1.2, using IP addresses 10.5.1.1 and 10.5.1.2 for thetunnel endpoints:

# ip l2tp add tunnel tunnel_id 1 peer_tunnel_id 1 udp_sport 5000 \  udp_dport 5000 encap udp local 192.168.1.1 remote 192.168.1.2# ip l2tp add session tunnel_id 1 session_id 1 peer_session_id 1# ip -s -d show dev l2tpeth0# ip addr add 10.5.1.2/32 peer 10.5.1.1/32 dev l2tpeth0# ip li set dev l2tpeth0 up

Choose IP addresses to be the address of a local IP interface and thatof the remote system. The IP addresses of the l2tpeth0 interface can beanything suitable.

Repeat the above at the peer, with ports, tunnel/session ids and IPaddresses reversed. The tunnel and session IDs can be any non-zero32-bit number, but the values must be reversed at the peer.

When done at both ends of the tunnel, it should be possible to senddata over the network. e.g.:

# ping 10.5.1.1

Sample Userspace Code¶

1. Create tunnel management PPPoX socket:

   kernel_fd = socket(AF_PPPOX, SOCK_DGRAM, PX_PROTO_OL2TP);if (kernel_fd >= 0) {        struct sockaddr_pppol2tp sax;        struct sockaddr_in const *peer_addr;        peer_addr = l2tp_tunnel_get_peer_addr(tunnel);        memset(&sax, 0, sizeof(sax));        sax.sa_family = AF_PPPOX;        sax.sa_protocol = PX_PROTO_OL2TP;        sax.pppol2tp.fd = udp_fd;       /* fd of tunnel UDP socket */        sax.pppol2tp.addr.sin_addr.s_addr = peer_addr->sin_addr.s_addr;        sax.pppol2tp.addr.sin_port = peer_addr->sin_port;        sax.pppol2tp.addr.sin_family = AF_INET;        sax.pppol2tp.s_tunnel = tunnel_id;        sax.pppol2tp.s_session = 0;     /* special case: mgmt socket */        sax.pppol2tp.d_tunnel = 0;        sax.pppol2tp.d_session = 0;     /* special case: mgmt socket */        if(connect(kernel_fd, (struct sockaddr *)&sax, sizeof(sax) ) < 0 ) {                perror("connect failed");                result = -errno;                goto err;        }}

2. Create session PPPoX data socket:

   struct sockaddr_pppol2tp sax;int fd;/* Note, the target socket must be bound already, else it will not be ready */sax.sa_family = AF_PPPOX;sax.sa_protocol = PX_PROTO_OL2TP;sax.pppol2tp.fd = tunnel_fd;sax.pppol2tp.addr.sin_addr.s_addr = addr->sin_addr.s_addr;sax.pppol2tp.addr.sin_port = addr->sin_port;sax.pppol2tp.addr.sin_family = AF_INET;sax.pppol2tp.s_tunnel  = tunnel_id;sax.pppol2tp.s_session = session_id;sax.pppol2tp.d_tunnel  = peer_tunnel_id;sax.pppol2tp.d_session = peer_session_id;/* session_fd is the fd of the session's PPPoL2TP socket. * tunnel_fd is the fd of the tunnel UDP socket. */fd = connect(session_fd, (struct sockaddr *)&sax, sizeof(sax));if (fd < 0 )    {        return -errno;}return 0;

Internal Implementation¶

The driver keeps a struct l2tp_tunnel context per L2TP tunnel and astruct l2tp_session context for each session. The l2tp_tunnel isalways associated with a UDP or L2TP/IP socket and keeps a list ofsessions in the tunnel. The l2tp_session context keeps kernel stateabout the session. It has private data which is used for data specificto the session type. With L2TPv2, the session always carried PPPtraffic. With L2TPv3, the session can also carry ethernet frames(ethernet pseudowire) or other data types such as ATM, HDLC or FrameRelay.

When a tunnel is first opened, the reference count on the socket isincreased using sock_hold(). This ensures that the kernel socketcannot be removed while L2TP’s data structures reference it.

Some L2TP sessions also have a socket (PPP pseudowires) while othersdo not (ethernet pseudowires). We can’t use the socket reference countas the reference count for session contexts. The L2TP implementationtherefore has its own internal reference counts on the sessioncontexts.

To Do¶

Add L2TP tunnel switching support. This would route tunneled trafficfrom one L2TP tunnel into another. Specified inhttp://tools.ietf.org/html/draft-ietf-l2tpext-tunnel-switching-08

Add L2TPv3 VLAN pseudowire support.

Add L2TPv3 IP pseudowire support.

Add L2TPv3 ATM pseudowire support.

Miscellaneous¶

The L2TP drivers were developed as part of the OpenL2TP project byKatalix Systems Ltd. OpenL2TP is a full-featured L2TP client / server,designed from the ground up to have the L2TP datapath in thekernel. The project also implemented the pppol2tp plugin for pppdwhich allows pppd to use the kernel driver. Details can be found athttp://www.openl2tp.org.