Overview¶

RxRPC is a two-layer protocol. There is a session layer which providesreliable virtual connections using UDP over IPv4 (or IPv6) as the transportlayer, but implements a real network protocol; and there’s the presentationlayer which renders structured data to binary blobs and back again using XDR(as does SunRPC):

+-------------+| Application |+-------------+|     XDR     |         Presentation+-------------+|    RxRPC    |         Session+-------------+|     UDP     |         Transport+-------------+

AF_RXRPC provides:

1. Part of an RxRPC facility for both kernel and userspace applications bymaking the session part of it a Linux network protocol (AF_RXRPC).

2. A two-phase protocol. The client transmits a blob (the request) and thenreceives a blob (the reply), and the server receives the request and thentransmits the reply.

3. Retention of the reusable bits of the transport system set up for one callto speed up subsequent calls.

4. A secure protocol, using the Linux kernel’s key retention facility tomanage security on the client end. The server end must of necessity bemore active in security negotiations.

AF_RXRPC does not provide XDR marshalling/presentation facilities. That isleft to the application. AF_RXRPC only deals in blobs. Even the operation IDis just the first four bytes of the request blob, and as such is beyond thekernel’s interest.

Sockets of AF_RXRPC family are:

1. created as type SOCK_DGRAM;

2. provided with a protocol of the type of underlying transport they’re goingto use - currently only PF_INET is supported.

The Andrew File System (AFS) is an example of an application that uses this andthat has both kernel (filesystem) and userspace (utility) components.

RxRPC Protocol Summary¶

An overview of the RxRPC protocol:

1. RxRPC sits on top of another networking protocol (UDP is the only optioncurrently), and uses this to provide network transport. UDP ports, forexample, provide transport endpoints.

2. RxRPC supports multiple virtual “connections” from any given transportendpoint, thus allowing the endpoints to be shared, even to the sameremote endpoint.

3. Each connection goes to a particular “service”. A connection may not goto multiple services. A service may be considered the RxRPC equivalent ofa port number. AF_RXRPC permits multiple services to share an endpoint.

4. Client-originating packets are marked, thus a transport endpoint can beshared between client and server connections (connections have adirection).

5. Up to a billion connections may be supported concurrently between onelocal transport endpoint and one service on one remote endpoint. An RxRPCconnection is described by seven numbers:

   Local address   }Local port      } Transport (UDP) addressRemote address  }Remote port     }DirectionConnection IDService ID

6. Each RxRPC operation is a “call”. A connection may make up to fourbillion calls, but only up to four calls may be in progress on aconnection at any one time.

7. Calls are two-phase and asymmetric: the client sends its request data,which the service receives; then the service transmits the reply datawhich the client receives.

8. The data blobs are of indefinite size, the end of a phase is marked with aflag in the packet. The number of packets of data making up one blob maynot exceed 4 billion, however, as this would cause the sequence number towrap.

9. The first four bytes of the request data are the service operation ID.

10. Security is negotiated on a per-connection basis. The connection isinitiated by the first data packet on it arriving. If security isrequested, the server then issues a “challenge” and then the clientreplies with a “response”. If the response is successful, the security isset for the lifetime of that connection, and all subsequent calls madeupon it use that same security. In the event that the server lets aconnection lapse before the client, the security will be renegotiated ifthe client uses the connection again.

11. Calls use ACK packets to handle reliability. Data packets are alsoexplicitly sequenced per call.

12. There are two types of positive acknowledgment: hard-ACKs and soft-ACKs.A hard-ACK indicates to the far side that all the data received to a pointhas been received and processed; a soft-ACK indicates that the data hasbeen received but may yet be discarded and re-requested. The sender maynot discard any transmittable packets until they’ve been hard-ACK’d.

13. Reception of a reply data packet implicitly hard-ACK’s all the datapackets that make up the request.

14. An call is complete when the request has been sent, the reply has beenreceived and the final hard-ACK on the last packet of the reply hasreached the server.

15. An call may be aborted by either end at any time up to its completion.

AF_RXRPC Driver Model¶

About the AF_RXRPC driver:

1. The AF_RXRPC protocol transparently uses internal sockets of the transportprotocol to represent transport endpoints.

2. AF_RXRPC sockets map onto RxRPC connection bundles. Actual RxRPCconnections are handled transparently. One client socket may be used tomake multiple simultaneous calls to the same service. One server socketmay handle calls from many clients.

3. Additional parallel client connections will be initiated to support extraconcurrent calls, up to a tunable limit.

4. Each connection is retained for a certain amount of time [tunable] afterthe last call currently using it has completed in case a new call is madethat could reuse it.

5. Each internal UDP socket is retained [tunable] for a certain amount oftime [tunable] after the last connection using it discarded, in case a newconnection is made that could use it.

6. A client-side connection is only shared between calls if they havethe same keystructdescribing their security (and assuming the callswould otherwise share the connection). Non-secured calls would also beable to share connections with each other.

7. A server-side connection is shared if the client says it is.

8. ACK’ing is handled by the protocol driver automatically, including pingreplying.

9. SO_KEEPALIVE automatically pings the other side to keep the connectionalive [TODO].

10. If an ICMP error is received, all calls affected by that error will beaborted with an appropriate network error passed throughrecvmsg().

Interaction with the user of the RxRPC socket:

1. A socket is made into a server socket by binding an address with anon-zero service ID.

2. In the client, sending a request is achieved with one or more sendmsgs,followed by the reply being received with one or more recvmsgs.

3. The first sendmsg for a request to be sent from a client contains a tag tobe used in all other sendmsgs or recvmsgs associated with that call. Thetag is carried in the control data.

4. connect() is used to supply a default destination address for a clientsocket. This may be overridden by supplying an alternate address to thefirstsendmsg() of a call (structmsghdr::msg_name).

5. Ifconnect() is called on an unbound client, a random local port willbound before the operation takes place.

6. A server socket may also be used to make client calls. To do this, thefirstsendmsg() of the call must specify the target address. The server’stransport endpoint is used to send the packets.

7. Once the application has received the last message associated with a call,the tag is guaranteed not to be seen again, and so it can be used to pinclient resources. A new call can then be initiated with the same tagwithout fear of interference.

8. In the server, a request is received with one or more recvmsgs, then thethe reply is transmitted with one or more sendmsgs, and then the final ACKis received with a last recvmsg.

9. When sending data for a call, sendmsg is given MSG_MORE if there’s moredata to come on that call.

10. When receiving data for a call, recvmsg flags MSG_MORE if there’s moredata to come for that call.

11. When receiving data or messages for a call, MSG_EOR is flagged by recvmsgto indicate the terminal message for that call.

12. A call may be aborted by adding an abort control message to the controldata. Issuing an abort terminates the kernel’s use of that call’s tag.Any messages waiting in the receive queue for that call will be discarded.

13. Aborts, busy notifications and challenge packets are delivered by recvmsg,and control data messages will be set to indicate the context. Receivingan abort or a busy message terminates the kernel’s use of that call’s tag.

14. The control data part of the msghdrstructis used for a number of things:

    1. The tag of the intended or affected call.

    2. Sending or receiving errors, aborts and busy notifications.

    3. Notifications of incoming calls.

    4. Sending debug requests and receiving debug replies [TODO].

15. When the kernel has received and set up an incoming call, it sends amessage to server application to let it know there’s a new call awaitingits acceptance [recvmsg reports a special control message]. The serverapplication then uses sendmsg to assign a tag to the new call. Once thatis done, the first part of the request data will be delivered by recvmsg.

16. The server application has to provide the server socket with a keyring ofsecret keys corresponding to the security types it permits. When a secureconnection is being set up, the kernel looks up the appropriate secret keyin the keyring and then sends a challenge packet to the client andreceives a response packet. The kernel then checks the authorisation ofthe packet and either aborts the connection or sets up the security.

17. The name of the key a client will use to secure its communications isnominated by a socket option.

Notes on sendmsg:

1. MSG_WAITALL can be set to tell sendmsg to ignore signals if the peer ismaking progress at accepting packets within a reasonable time such that wemanage to queue up all the data for transmission. This requires theclient to accept at least one packet per 2*RTT time period.

   If this isn’t set,sendmsg() will return immediately, either returningEINTR/ERESTARTSYS if nothing was consumed or returning the amount of dataconsumed.

Notes on recvmsg:

1. If there’s a sequence of data messages belonging to a particular call onthe receive queue, then recvmsg will keep working through them until:

   1. it meets the end of that call’s received data,

   2. it meets a non-data message,

   3. it meets a message belonging to a different call, or

   4. it fills the user buffer.

   If recvmsg is called in blocking mode, it will keep sleeping, awaiting thereception of further data, until one of the above four conditions is met.

2. MSG_PEEK operates similarly, but will return immediately if it has put anydata in the buffer rather than sleeping until it can fill the buffer.

3. If a data message is only partially consumed in filling a user buffer,then the remainder of that message will be left on the front of the queuefor the next taker. MSG_TRUNC will never be flagged.

4. If there is more data to be had on a call (it hasn’t copied the last byteof the last data message in that phase yet), then MSG_MORE will beflagged.

Control Messages¶

AF_RXRPC makes use of control messages insendmsg() andrecvmsg() to multiplexcalls, to invoke certain actions and to report certain conditions. These are:

MESSAGE ID	SRT	DATA	MEANING
RXRPC_USER_CALL_ID	sr-	User ID	App’s call specifier
RXRPC_ABORT	srt	Abort code	Abort code to issue/received
RXRPC_ACK	-rt	n/a	Final ACK received
RXRPC_NET_ERROR	-rt	error num	Network error on call
RXRPC_BUSY	-rt	n/a	Call rejected (server busy)
RXRPC_LOCAL_ERROR	-rt	error num	Local error encountered
RXRPC_NEW_CALL	-r-	n/a	New call received
RXRPC_ACCEPT	s--	n/a	Accept new call
RXRPC_EXCLUSIVE_CALL	s--	n/a	Make an exclusive client call
RXRPC_UPGRADE_SERVICE	s--	n/a	Client call can be upgraded
RXRPC_TX_LENGTH	s--	data len	Total length of Tx data

(SRT = usable in Sendmsg / delivered by Recvmsg / Terminal message)

1. RXRPC_USER_CALL_ID

   This is used to indicate the application’s call ID. It’s an unsigned longthat the app specifies in the client by attaching it to the first datamessage or in the server by passing it in association with an RXRPC_ACCEPTmessage.recvmsg() passes it in conjunction with all messages exceptthose of the RXRPC_NEW_CALL message.

2. RXRPC_ABORT

   This is can be used by an application to abort a call by passing it tosendmsg, or it can be delivered by recvmsg to indicate a remote abort wasreceived. Either way, it must be associated with an RXRPC_USER_CALL_ID tospecify the call affected. If an abort is being sent, then error EBADSLTwill be returned if there is no call with that user ID.

3. RXRPC_ACK

   This is delivered to a server application to indicate that the final ACKof a call was received from the client. It will be associated with anRXRPC_USER_CALL_ID to indicate the call that’s now complete.

4. RXRPC_NET_ERROR

   This is delivered to an application to indicate that an ICMP error messagewas encountered in the process of trying to talk to the peer. Anerrno-class integer value will be included in the control message dataindicating the problem, and an RXRPC_USER_CALL_ID will indicate the callaffected.

5. RXRPC_BUSY

   This is delivered to a client application to indicate that a call wasrejected by the server due to the server being busy. It will beassociated with an RXRPC_USER_CALL_ID to indicate the rejected call.

6. RXRPC_LOCAL_ERROR

   This is delivered to an application to indicate that a local error wasencountered and that a call has been aborted because of it. Anerrno-class integer value will be included in the control message dataindicating the problem, and an RXRPC_USER_CALL_ID will indicate the callaffected.

7. RXRPC_NEW_CALL

   This is delivered to indicate to a server application that a new call hasarrived and is awaiting acceptance. No user ID is associated with this,as a user ID must subsequently be assigned by doing an RXRPC_ACCEPT.

8. RXRPC_ACCEPT

   This is used by a server application to attempt to accept a call andassign it a user ID. It should be associated with an RXRPC_USER_CALL_IDto indicate the user ID to be assigned. If there is no call to beaccepted (it may have timed out, been aborted, etc.), then sendmsg willreturn error ENODATA. If the user ID is already in use by another call,then error EBADSLT will be returned.

9. RXRPC_EXCLUSIVE_CALL

   This is used to indicate that a client call should be made on a one-offconnection. The connection is discarded once the call has terminated.

10. RXRPC_UPGRADE_SERVICE

    This is used to make a client call to probe if the specified service IDmay be upgraded by the server. The caller must check msg_name returned torecvmsg() for the service ID actually in use. The operation probed mustbe one that takes the same arguments in both services.

    Once this has been used to establish the upgrade capability (or lackthereof) of the server, the service ID returned should be used for allfuture communication to that server and RXRPC_UPGRADE_SERVICE should nolonger be set.

11. RXRPC_TX_LENGTH

    This is used to inform the kernel of the total amount of data that isgoing to be transmitted by a call (whether in a client request or aservice response). If given, it allows the kernel to encrypt from theuserspace buffer directly to the packet buffers, rather than copying intothe buffer and then encrypting in place. This may only be given with thefirstsendmsg() providing data for a call. EMSGSIZE will be generated ifthe amount of data actually given is different.

    This takes a parameter of __s64 type that indicates how much will betransmitted. This may not be less than zero.

The symbol RXRPC__SUPPORTED is defined as one more than the highest controlmessage type supported. At run time this can be queried by means of theRXRPC_SUPPORTED_CMSG socket option (see below).

Socket Options¶

AF_RXRPC sockets support a few socket options at the SOL_RXRPC level:

1. RXRPC_SECURITY_KEY

   This is used to specify the description of the key to be used. The key isextracted from the calling process’s keyrings withrequest_key() andshould be of “rxrpc” type.

   The optval pointer points to the description string, and optlen indicateshow long the string is, without the NUL terminator.

2. RXRPC_SECURITY_KEYRING

   Similar to above but specifies a keyring of server secret keys to use (keytype “keyring”). See the “Security” section.

3. RXRPC_EXCLUSIVE_CONNECTION

   This is used to request that new connections should be used for each callmade subsequently on this socket. optval should be NULL and optlen 0.

4. RXRPC_MIN_SECURITY_LEVEL

   This is used to specify the minimum security level required for calls onthis socket. optval must point to an int containing one of the followingvalues:

   1. RXRPC_SECURITY_PLAIN

      Encrypted checksum only.

   2. RXRPC_SECURITY_AUTH

      Encrypted checksum plus packet padded and first eight bytes of packetencrypted - which includes the actual packet length.

   3. RXRPC_SECURITY_ENCRYPT

      Encrypted checksum plus entire packet padded and encrypted, includingactual packet length.

5. RXRPC_UPGRADEABLE_SERVICE

   This is used to indicate that a service socket with two bindings mayupgrade one bound service to the other if requested by the client. optvalmust point to an array of two unsigned short ints. The first is theservice ID to upgrade from and the second the service ID to upgrade to.

6. RXRPC_SUPPORTED_CMSG

   This is a read-only option that writes an int into the buffer indicatingthe highest control message type supported.

Security¶

Currently, only the kerberos 4 equivalent protocol has been implemented(security index 2 - rxkad). This requires the rxkad module to be loaded and,on the client, tickets of the appropriate type to be obtained from the AFSkaserver or the kerberos server and installed as “rxrpc” type keys. This isnormally done using the klog program. An example simple klog program can befound at:

http://people.redhat.com/~dhowells/rxrpc/klog.c

The payload provided toadd_key() on the client should be of the followingform:

struct rxrpc_key_sec2_v1 {        uint16_t        security_index; /* 2 */        uint16_t        ticket_length;  /* length of ticket[] */        uint32_t        expiry;         /* time at which expires */        uint8_t         kvno;           /* key version number */        uint8_t         __pad[3];        uint8_t         session_key[8]; /* DES session key */        uint8_t         ticket[0];      /* the encrypted ticket */};

Where the ticket blob is just appended to the above structure.

For the server, keys of type “rxrpc_s” must be made available to the server.They have a description of “<serviceID>:<securityIndex>” (eg: “52:2” for anrxkad key for the AFS VL service). When such a key is created, it should begiven the server’s secret key as the instantiation data (see the examplebelow).

add_key(“rxrpc_s”, “52:2”, secret_key, 8, keyring);

A keyring is passed to the server socket by naming it in a sockopt. The serversocket then looks the server secret keys up in this keyring when secureincoming connections are made. This can be seen in an example program that canbe found at:

http://people.redhat.com/~dhowells/rxrpc/listen.c

Example Client Usage¶

A client would issue an operation by:

1. An RxRPC socket is set up by:

   client = socket(AF_RXRPC, SOCK_DGRAM, PF_INET);

   Where the third parameter indicates the protocol family of the transportsocket used - usually IPv4 but it can also be IPv6 [TODO].

2. A local address can optionally be bound:

   struct sockaddr_rxrpc srx = {        .srx_family     = AF_RXRPC,        .srx_service    = 0,  /* we're a client */        .transport_type = SOCK_DGRAM,   /* type of transport socket */        .transport.sin_family   = AF_INET,        .transport.sin_port     = htons(7000), /* AFS callback */        .transport.sin_address  = 0,  /* all local interfaces */};bind(client, &srx, sizeof(srx));

   This specifies the local UDP port to be used. If not given, a randomnon-privileged port will be used. A UDP port may be shared betweenseveral unrelated RxRPC sockets. Security is handled on a basis ofper-RxRPC virtual connection.

3. The security is set:

   const char *key = "AFS:cambridge.redhat.com";setsockopt(client, SOL_RXRPC, RXRPC_SECURITY_KEY, key, strlen(key));

   This issues arequest_key() to get the key representing the securitycontext. The minimum security level can be set:

   unsigned int sec = RXRPC_SECURITY_ENCRYPT;setsockopt(client, SOL_RXRPC, RXRPC_MIN_SECURITY_LEVEL,           &sec, sizeof(sec));

4. The server to be contacted can then be specified (alternatively this canbe done through sendmsg):

   struct sockaddr_rxrpc srx = {        .srx_family     = AF_RXRPC,        .srx_service    = VL_SERVICE_ID,        .transport_type = SOCK_DGRAM,   /* type of transport socket */        .transport.sin_family   = AF_INET,        .transport.sin_port     = htons(7005), /* AFS volume manager */        .transport.sin_address  = ...,};connect(client, &srx, sizeof(srx));

5. The request data should then be posted to the server socket using a seriesofsendmsg() calls, each with the following control message attached:

   RXRPC_USER_CALL_ID

   specifies the user ID for this call

   MSG_MORE should be set in msghdr::msg_flags on all but the last part ofthe request. Multiple requests may be made simultaneously.

   An RXRPC_TX_LENGTH control message can also be specified on the firstsendmsg() call.

   If a call is intended to go to a destination other than the defaultspecified throughconnect(), then msghdr::msg_name should be set on thefirst request message of that call.

6. The reply data will then be posted to the server socket forrecvmsg() topick up. MSG_MORE will be flagged byrecvmsg() if there’s more reply datafor a particular call to be read. MSG_EOR will be set on the terminalread for a call.

   All data will be delivered with the following control message attached:

   RXRPC_USER_CALL_ID - specifies the user ID for this call

   If an abort or error occurred, this will be returned in the control databuffer instead, and MSG_EOR will be flagged to indicate the end of thatcall.

A client may ask for a service ID it knows and ask that this be upgraded to abetter service if one is available by supplying RXRPC_UPGRADE_SERVICE on thefirstsendmsg() of a call. The client should then check srx_service in themsg_name filled in byrecvmsg() when collecting the result. srx_service willhold the same value as given tosendmsg() if the upgrade request was ignored bythe service - otherwise it will be altered to indicate the service ID theserver upgraded to. Note that the upgraded service ID is chosen by the server.The caller has to wait until it sees the service ID in the reply before sendingany more calls (further calls to the same destination will be blocked until theprobe is concluded).

Example Server Usage¶

A server would be set up to accept operations in the following manner:

1. An RxRPC socket is created by:

   server = socket(AF_RXRPC, SOCK_DGRAM, PF_INET);

   Where the third parameter indicates the address type of the transportsocket used - usually IPv4.

2. Security is set up if desired by giving the socket a keyring with serversecret keys in it:

   keyring = add_key("keyring", "AFSkeys", NULL, 0,                  KEY_SPEC_PROCESS_KEYRING);const char secret_key[8] = {        0xa7, 0x83, 0x8a, 0xcb, 0xc7, 0x83, 0xec, 0x94 };add_key("rxrpc_s", "52:2", secret_key, 8, keyring);setsockopt(server, SOL_RXRPC, RXRPC_SECURITY_KEYRING, "AFSkeys", 7);

   The keyring can be manipulated after it has been given to the socket. Thispermits the server to add more keys, replace keys, etc. while it is live.

3. A local address must then be bound:

   struct sockaddr_rxrpc srx = {        .srx_family     = AF_RXRPC,        .srx_service    = VL_SERVICE_ID, /* RxRPC service ID */        .transport_type = SOCK_DGRAM,   /* type of transport socket */        .transport.sin_family   = AF_INET,        .transport.sin_port     = htons(7000), /* AFS callback */        .transport.sin_address  = 0,  /* all local interfaces */};bind(server, &srx, sizeof(srx));

   More than one service ID may be bound to a socket, provided the transportparameters are the same. The limit is currently two. To do this,bind()should be called twice.

4. If service upgrading is required, first two service IDs must have beenbound and then the following option must be set:

   unsigned short service_ids[2] = { from_ID, to_ID };setsockopt(server, SOL_RXRPC, RXRPC_UPGRADEABLE_SERVICE,           service_ids, sizeof(service_ids));

   This will automatically upgrade connections on service from_ID to serviceto_ID if they request it. This will be reflected in msg_name obtainedthroughrecvmsg() when the request data is delivered to userspace.

5. The server is then set to listen out for incoming calls:

   listen(server, 100);

6. The kernel notifies the server of pending incoming connections by sendingit a message for each. This is received withrecvmsg() on the serversocket. It has no data, and has a single dataless control messageattached:

   RXRPC_NEW_CALL

   The address that can be passed back byrecvmsg() at this point should beignored since the call for which the message was posted may have gone bythe time it is accepted - in which case the first call still on the queuewill be accepted.

7. The server then accepts the new call by issuing asendmsg() with twopieces of control data and no actual data:

   RXRPC_ACCEPT	indicate connection acceptance
   RXRPC_USER_CALL_ID	specify user ID for this call

8. The first request data packet will then be posted to the server socket forrecvmsg() to pick up. At that point, the RxRPC address for the call canbe read from the address fields in the msghdr struct.

   Subsequent request data will be posted to the server socket forrecvmsg()to collect as it arrives. All but the last piece of the request data willbe delivered with MSG_MORE flagged.

   All data will be delivered with the following control message attached:

   RXRPC_USER_CALL_ID

   specifies the user ID for this call

9. The reply data should then be posted to the server socket using a seriesofsendmsg() calls, each with the following control messages attached:

   RXRPC_USER_CALL_ID

   specifies the user ID for this call

   MSG_MORE should be set in msghdr::msg_flags on all but the last messagefor a particular call.

10. The final ACK from the client will be posted for retrieval byrecvmsg()when it is received. It will take the form of a dataless message with twocontrol messages attached:

    RXRPC_USER_CALL_ID	specifies the user ID for this call
    RXRPC_ACK	indicates final ACK (no data)

    MSG_EOR will be flagged to indicate that this is the final message forthis call.

11. Up to the point the final packet of reply data is sent, the call can beaborted by callingsendmsg() with a dataless message with the followingcontrol messages attached:

    RXRPC_USER_CALL_ID	specifies the user ID for this call
    RXRPC_ABORT	indicates abort code (4 byte data)

    Any packets waiting in the socket’s receive queue will be discarded ifthis is issued.

Note that all the communications for a particular service take place throughthe one server socket, using control messages onsendmsg() andrecvmsg() todetermine the call affected.

AF_RXRPC Kernel Interface¶

The AF_RXRPC module also provides an interface for use by in-kernel utilitiessuch as the AFS filesystem. This permits such a utility to:

1. Use different keys directly on individual client calls on one socketrather than having to open a whole slew of sockets, one for each key itmight want to use.

2. Avoid having RxRPC callrequest_key() at the point of issue of a call oropening of a socket. Instead the utility is responsible for requesting akey at the appropriate point. AFS, for instance, would do this during VFSoperations such as open() orunlink(). The key is then handed throughwhen the call is initiated.

3. Request the use of something other than GFP_KERNEL to allocate memory.

4. Avoid the overhead of using therecvmsg() call. RxRPC messages can beintercepted before they get put into the socket Rx queue and the socketbuffers manipulated directly.

To use the RxRPC facility, a kernel utility must still open an AF_RXRPC socket,bind an address as appropriate and listen if it’s to be a server socket, butthen it passes this to the kernel interface functions.

The kernel interface functions are as follows:

1. Begin a new client call:

   struct rxrpc_call *rxrpc_kernel_begin_call(struct socket *sock,                        struct sockaddr_rxrpc *srx,                        struct key *key,                        unsigned long user_call_ID,                        s64 tx_total_len,                        gfp_t gfp,                        rxrpc_notify_rx_t notify_rx,                        bool upgrade,                        bool intr,                        unsigned int debug_id);

   This allocates the infrastructure to make a new RxRPC call and assignscall and connection numbers. The call will be made on the UDP port thatthe socket is bound to. The call will go to the destination address of aconnected client socket unless an alternative is supplied (srx isnon-NULL).

   If a key is supplied then this will be used to secure the call instead ofthe key bound to the socket with the RXRPC_SECURITY_KEY sockopt. Callssecured in this way will still share connections if at all possible.

   The user_call_ID is equivalent to that supplied tosendmsg() in thecontrol data buffer. It is entirely feasible to use this to point to akernel data structure.

   tx_total_len is the amount of data the caller is intending to transmitwith this call (or -1 if unknown at this point). Setting the data sizeallows the kernel to encrypt directly to the packet buffers, therebysaving a copy. The value may not be less than -1.

   notify_rx is a pointer to a function to be called when events such asincoming data packets or remote aborts happen.

   upgrade should be set to true if a client operation should request thatthe server upgrade the service to a better one. The resultant service IDis returned byrxrpc_kernel_recv_data().

   intr should be set to true if the call should be interruptible. If thisis not set, this function may not return until a channel has beenallocated; if it is set, the function may return -ERESTARTSYS.

   debug_id is the call debugging ID to be used for tracing. This can beobtained by atomically incrementing rxrpc_debug_id.

   If this function is successful, an opaque reference to the RxRPC call isreturned. The caller now holds a reference on this and it must beproperly ended.

2. Shut down a client call:

   void rxrpc_kernel_shutdown_call(struct socket *sock,                                struct rxrpc_call *call);

   This is used to shut down a previously begun call. The user_call_ID isexpunged from AF_RXRPC’s knowledge and will not be seen again inassociation with the specified call.

3. Release the ref on a client call:

   void rxrpc_kernel_put_call(struct socket *sock,                           struct rxrpc_call *call);

   This is used to release the caller’s ref on an rxrpc call.

4. Send data through a call:

   typedef void (*rxrpc_notify_end_tx_t)(struct sock *sk,                                      unsigned long user_call_ID,                                      struct sk_buff *skb);int rxrpc_kernel_send_data(struct socket *sock,                           struct rxrpc_call *call,                           struct msghdr *msg,                           size_t len,                           rxrpc_notify_end_tx_t notify_end_rx);

   This is used to supply either the request part of a client call or thereply part of a server call. msg.msg_iovlen and msg.msg_iov specify thedata buffers to be used. msg_iov may not be NULL and must pointexclusively to in-kernel virtual addresses. msg.msg_flags may be givenMSG_MORE if there will be subsequent data sends for this call.

   The msg must not specify a destination address, control data or any flagsother than MSG_MORE. len is the total amount of data to transmit.

   notify_end_rx can be NULL or it can be used to specify a function to becalled when the call changes state to end the Tx phase. This function iscalled with a spinlock held to prevent the last DATA packet from beingtransmitted until the function returns.

5. Receive data from a call:

     int rxrpc_kernel_recv_data(struct socket *sock,                             struct rxrpc_call *call,                             void *buf,                             size_t size,                             size_t *_offset,                             bool want_more,                             u32 *_abort,                             u16 *_service)This is used to receive data from either the reply part of a client callor the request part of a service call.  buf and size specify how muchdata is desired and where to store it.  *_offset is added on to buf andsubtracted from size internally; the amount copied into the buffer isadded to *_offset before returning.want_more should be true if further data will be required after this issatisfied and false if this is the last item of the receive phase.There are three normal returns: 0 if the buffer was filled and want_morewas true; 1 if the buffer was filled, the last DATA packet has beenemptied and want_more was false; and -EAGAIN if the function needs to becalled again.If the last DATA packet is processed but the buffer contains less thanthe amount requested, EBADMSG is returned.  If want_more wasn't set, butmore data was available, EMSGSIZE is returned.If a remote ABORT is detected, the abort code received will be stored in``*_abort`` and ECONNABORTED will be returned.The service ID that the call ended up with is returned into *_service.This can be used to see if a call got a service upgrade.

6. Abort a call??

   void rxrpc_kernel_abort_call(struct socket *sock,                             struct rxrpc_call *call,                             u32 abort_code);

   This is used to abort a call if it’s still in an abortable state. Theabort code specified will be placed in the ABORT message sent.

7. Intercept received RxRPC messages:

   typedef void (*rxrpc_interceptor_t)(struct sock *sk,                                    unsigned long user_call_ID,                                    struct sk_buff *skb);voidrxrpc_kernel_intercept_rx_messages(struct socket *sock,                                   rxrpc_interceptor_t interceptor);

   This installs an interceptor function on the specified AF_RXRPC socket.All messages that would otherwise wind up in the socket’s Rx queue arethen diverted to this function. Note that care must be taken to processthe messages in the right order to maintain DATA message sequentiality.

   The interceptor function itself is provided with the address of the socketand handling the incoming message, the ID assigned by the kernel utilityto the call and the socket buffer containing the message.

   The skb->mark field indicates the type of message:

   Mark	Meaning
   RXRPC_SKB_MARK_DATA	Data message
   RXRPC_SKB_MARK_FINAL_ACK	Final ACK received for an incoming call
   RXRPC_SKB_MARK_BUSY	Client call rejected as server busy
   RXRPC_SKB_MARK_REMOTE_ABORT	Call aborted by peer
   RXRPC_SKB_MARK_NET_ERROR	Network error detected
   RXRPC_SKB_MARK_LOCAL_ERROR	Local error encountered
   RXRPC_SKB_MARK_NEW_CALL	New incoming call awaiting acceptance

   The remote abort message can be probed withrxrpc_kernel_get_abort_code().The two error messages can be probed withrxrpc_kernel_get_error_number().A new call can be accepted withrxrpc_kernel_accept_call().

   Data messages can have their contents extracted with the usual bunch ofsocket buffer manipulation functions. A data message can be determined tobe the last one in a sequence withrxrpc_kernel_is_data_last(). When adata message has been used up,rxrpc_kernel_data_consumed() should becalled on it.

   Messages should be handled torxrpc_kernel_free_skb() to dispose of. Itis possible to get extra refs on all types of message for later freeing,but this may pin the state of a call until the message is finally freed.

8. Accept an incoming call:

   struct rxrpc_call *rxrpc_kernel_accept_call(struct socket *sock,                         unsigned long user_call_ID);

   This is used to accept an incoming call and to assign it a call ID. Thisfunction is similar torxrpc_kernel_begin_call() and calls accepted mustbe ended in the same way.

   If this function is successful, an opaque reference to the RxRPC call isreturned. The caller now holds a reference on this and it must beproperly ended.

9. Reject an incoming call:

   int rxrpc_kernel_reject_call(struct socket *sock);

   This is used to reject the first incoming call on the socket’s queue witha BUSY message. -ENODATA is returned if there were no incoming calls.Other errors may be returned if the call had been aborted (-ECONNABORTED)or had timed out (-ETIME).

10. Allocate a null key for doing anonymous security:

    struct key *rxrpc_get_null_key(const char *keyname);

    This is used to allocate a null RxRPC key that can be used to indicateanonymous security for a particular domain.

11. Get the peer address of a call:

    void rxrpc_kernel_get_peer(struct socket *sock, struct rxrpc_call *call,                           struct sockaddr_rxrpc *_srx);

    This is used to find the remote peer address of a call.

12. Set the total transmit data size on a call:

    void rxrpc_kernel_set_tx_length(struct socket *sock,                                struct rxrpc_call *call,                                s64 tx_total_len);

    This sets the amount of data that the caller is intending to transmit on acall. It’s intended to be used for setting the reply size as the requestsize should be set when the call is begun. tx_total_len may not be lessthan zero.

13. Get call RTT:

    u64 rxrpc_kernel_get_rtt(struct socket *sock, struct rxrpc_call *call);

    Get the RTT time to the peer in use by a call. The value returned is innanoseconds.

14. Check call still alive:

    bool rxrpc_kernel_check_life(struct socket *sock,                             struct rxrpc_call *call,                             u32 *_life);void rxrpc_kernel_probe_life(struct socket *sock,                             struct rxrpc_call *call);

    The first function passes back in*_life a number that is updated whenACKs are received from the peer (notably including PING RESPONSE ACKswhich we can elicit by sending PING ACKs to see if the call still existson the server). The caller should compare the numbers of two calls to seeif the call is still alive after waiting for a suitable interval. It alsoreturns true as long as the call hasn’t yet reached the completed state.

    This allows the caller to work out if the server is still contactable andif the call is still alive on the server while waiting for the server toprocess a client operation.

    The second function causes a ping ACK to be transmitted to try to provokethe peer into responding, which would then cause the value returned by thefirst function to change. Note that this must be called in TASK_RUNNINGstate.

15. Apply the RXRPC_MIN_SECURITY_LEVEL sockopt to a socket from within in thekernel:

    int rxrpc_sock_set_min_security_level(struct sock *sk,                                      unsigned int val);

    This specifies the minimum security level required for calls on thissocket.

Configurable Parameters¶

The RxRPC protocol driver has a number of configurable parameters that can beadjusted through sysctls in /proc/net/rxrpc/:

1. req_ack_delay

   The amount of time in milliseconds after receiving a packet with therequest-ack flag set before we honour the flag and actually send therequested ack.

   Usually the other side won’t stop sending packets until the advertisedreception window is full (to a maximum of 255 packets), so delaying theACK permits several packets to be ACK’d in one go.

2. soft_ack_delay

   The amount of time in milliseconds after receiving a new packet before wegenerate a soft-ACK to tell the sender that it doesn’t need to resend.

3. idle_ack_delay

   The amount of time in milliseconds after all the packets currently in thereceived queue have been consumed before we generate a hard-ACK to tellthe sender it can free its buffers, assuming no other reason occurs thatwe would send an ACK.

4. resend_timeout

   The amount of time in milliseconds after transmitting a packet before wetransmit it again, assuming no ACK is received from the receiver tellingus they got it.

5. max_call_lifetime

   The maximum amount of time in seconds that a call may be in progressbefore we preemptively kill it.

6. dead_call_expiry

   The amount of time in seconds before we remove a dead call from the calllist. Dead calls are kept around for a little while for the purpose ofrepeating ACK and ABORT packets.

7. connection_expiry

   The amount of time in seconds after a connection was last used before weremove it from the connection list. While a connection is in existence,it serves as a placeholder for negotiated security; when it is deleted,the security must be renegotiated.

8. transport_expiry

   The amount of time in seconds after a transport was last used before weremove it from the transport list. While a transport is in existence, itserves to anchor the peer data and keeps the connection ID counter.

9. rxrpc_rx_window_size

   The size of the receive window in packets. This is the maximum number ofunconsumed received packets we’re willing to hold in memory for anyparticular call.

10. rxrpc_rx_mtu

    The maximum packet MTU size that we’re willing to receive in bytes. Thisindicates to the peer whether we’re willing to accept jumbo packets.

11. rxrpc_rx_jumbo_max

    The maximum number of packets that we’re willing to accept in a jumbopacket. Non-terminal packets in a jumbo packet must contain a four byteheader plus exactly 1412 bytes of data. The terminal packet must containa four byte header plus any amount of data. In any event, a jumbo packetmay not exceed rxrpc_rx_mtu in size.

API Function Reference¶

structrxrpc_peer*rxrpc_kernel_lookup_peer(structsocket*sock,structsockaddr_rxrpc*srx,gfp_tgfp)¶

Obtain remote transport endpoint for an address

structrxrpc_peer*rxrpc_kernel_get_peer(structrxrpc_peer*peer)¶

Get a reference on a peer

voidrxrpc_kernel_put_peer(structrxrpc_peer*peer)¶

Allow a kernel app to drop a peer reference

structrxrpc_call*rxrpc_kernel_begin_call(structsocket*sock,structrxrpc_peer*peer,structkey*key,unsignedlonguser_call_ID,s64tx_total_len,u32hard_timeout,gfp_tgfp,rxrpc_notify_rx_tnotify_rx,u16service_id,boolupgrade,enumrxrpc_interruptibilityinterruptibility,unsignedintdebug_id)¶

Allow a kernel service to begin a call

voidrxrpc_kernel_shutdown_call(structsocket*sock,structrxrpc_call*call)¶

Allow a kernel service to shut down a call it was using

voidrxrpc_kernel_put_call(structsocket*sock,structrxrpc_call*call)¶

Release a reference to a call

boolrxrpc_kernel_check_life(conststructsocket*sock,conststructrxrpc_call*call)¶

Check to see whether a call is still alive

voidrxrpc_kernel_set_notifications(structsocket*sock,conststructrxrpc_kernel_ops*app_ops)¶

Set table of callback operations

u8rxrpc_kernel_query_call_security(structrxrpc_call*call,u16*_service_id,u32*_enctype)¶

Query call’s security parameters

structkey*rxrpc_get_null_key(constchar*keyname)¶

Generate a null RxRPC key

enumrxrpc_oob_typerxrpc_kernel_query_oob(structsk_buff*oob,structrxrpc_peer**_peer,unsignedlong*_peer_appdata)¶

Query the parameters of an out-of-band message

structsk_buff*rxrpc_kernel_dequeue_oob(structsocket*sock,enumrxrpc_oob_type*_type)¶

Dequeue and return the front OOB message

voidrxrpc_kernel_free_oob(structsk_buff*oob)¶

Free an out-of-band message

voidrxrpc_kernel_query_challenge(structsk_buff*challenge,structrxrpc_peer**_peer,unsignedlong*_peer_appdata,u16*_service_id,u8*_security_index)¶

Query the parameters of a challenge

intrxrpc_kernel_reject_challenge(structsk_buff*challenge,u32abort_code,interror,enumrxrpc_abort_reasonwhy)¶

Allow a kernel service to reject a challenge

structrxrpc_peer*rxrpc_kernel_get_call_peer(structsocket*sock,structrxrpc_call*call)¶

Get the peer address of a call

unsignedintrxrpc_kernel_get_srtt(conststructrxrpc_peer*peer)¶

Get a call’s peer smoothed RTT

conststructsockaddr_rxrpc*rxrpc_kernel_remote_srx(conststructrxrpc_peer*peer)¶

Get the address of a peer

conststructsockaddr*rxrpc_kernel_remote_addr(conststructrxrpc_peer*peer)¶

Get the peer transport address of a call

unsignedlongrxrpc_kernel_set_peer_data(structrxrpc_peer*peer,unsignedlongapp_data)¶

Set app-specific data on a peer.

unsignedlongrxrpc_kernel_get_peer_data(conststructrxrpc_peer*peer)¶

Get app-specific data from a peer.

intrxrpc_kernel_recv_data(structsocket*sock,structrxrpc_call*call,structiov_iter*iter,size_t*_len,boolwant_more,u32*_abort,u16*_service)¶

Allow a kernel service to receive data/info

u32rxgk_kernel_query_challenge(structsk_buff*challenge)¶

Query RxGK-specific challenge parameters

intrxgk_kernel_respond_to_challenge(structsk_buff*challenge,structkrb5_buffer*appdata)¶

Respond to a challenge with appdata

intrxkad_kernel_respond_to_challenge(structsk_buff*challenge)¶

Respond to a challenge with appdata

intrxrpc_kernel_send_data(structsocket*sock,structrxrpc_call*call,structmsghdr*msg,size_tlen,rxrpc_notify_end_tx_tnotify_end_tx)¶

Allow a kernel service to send data on a call

boolrxrpc_kernel_abort_call(structsocket*sock,structrxrpc_call*call,u32abort_code,interror,enumrxrpc_abort_reasonwhy)¶

Allow a kernel service to abort a call

voidrxrpc_kernel_set_tx_length(structsocket*sock,structrxrpc_call*call,s64tx_total_len)¶

Set the total Tx length on a call

intrxrpc_sock_set_security_keyring(structsock*sk,structkey*keyring)¶

Set the security keyring for a kernel service

intrxrpc_sock_set_manage_response(structsock*sk,boolset)¶

Set the manage-response flag for a kernel service