Kernel Connector

Kernel connector - new netlink based userspace <-> kernel space easyto use communication module.

The Connector driver makes it easy to connect various agents using anetlink based network. One must register a callback and an identifier.When the driver receives a special netlink message with the appropriateidentifier, the appropriate callback will be called.

From the userspace point of view it’s quite straightforward:

  • socket();
  • bind();
  • send();
  • recv();

But if kernelspace wants to use the full power of such connections, thedriver writer must create special sockets, must know about struct sk_buffhandling, etc… The Connector driver allows any kernelspace agents to usenetlink based networking for inter-process communication in a significantlyeasier way:

int cn_add_callback(struct cb_id *id, char *name, void (*callback) (struct cn_msg *, struct netlink_skb_parms *));void cn_netlink_send_multi(struct cn_msg *msg, u16 len, u32 portid, u32 __group, int gfp_mask);void cn_netlink_send(struct cn_msg *msg, u32 portid, u32 __group, int gfp_mask);struct cb_id{      __u32                   idx;      __u32                   val;};

idx and val are unique identifiers which must be registered in theconnector.h header for in-kernel usage.void (*callback) (void *) is acallback function which will be called when a message with above idx.valis received by the connector core. The argument for that function mustbe dereferenced tostruct cn_msg *:

struct cn_msg{      struct cb_id            id;      __u32                   seq;      __u32                   ack;      __u32                   len;    /* Length of the following data */      __u8                    data[0];};

Connector interfaces

intcn_add_callback(struct cb_id * id, const char * name, void (*callback)(struct cn_msg *, struct netlink_skb_parms *))

Registers new callback with connector core.

Parameters

structcb_id*id
unique connector’s user identifier.It must be registered in connector.h for legalin-kernel users.
constchar*name
connector’s callback symbolic name.
void(*)(structcn_msg*,structnetlink_skb_parms*)callback
connector’s callback.parameters arecn_msg and the sender’s credentials
voidcn_del_callback(struct cb_id * id)

Unregisters new callback with connector core.

Parameters

structcb_id*id
unique connector’s user identifier.
intcn_netlink_send_mult(struct cn_msg * msg, u16 len, u32 portid, u32 group, gfp_t gfp_mask)

Sends message to the specified groups.

Parameters

structcn_msg*msg
message header(with attached data).
u16len
Number ofmsg to be sent.
u32portid
destination port.If non-zero the message will be sent to the given port,which should be set to the original sender.
u32group
destination group.Ifportid andgroup is zero, then appropriate group willbe searched through all registered connector users, andmessage will be delivered to the group which was createdfor user with the same ID as inmsg.Ifgroup is not zero, then message will be deliveredto the specified group.
gfp_tgfp_mask
GFP mask.

Description

It can be safely called from softirq context, but may silentlyfail under strong memory pressure.

If there are no listeners for given group-ESRCH can be returned.

intcn_netlink_send(struct cn_msg * msg, u32 portid, u32 group, gfp_t gfp_mask)

Sends message to the specified groups.

Parameters

structcn_msg*msg
message header(with attached data).
u32portid
destination port.If non-zero the message will be sent to the given port,which should be set to the original sender.
u32group
destination group.Ifportid andgroup is zero, then appropriate group willbe searched through all registered connector users, andmessage will be delivered to the group which was createdfor user with the same ID as inmsg.Ifgroup is not zero, then message will be deliveredto the specified group.
gfp_tgfp_mask
GFP mask.

Description

It can be safely called from softirq context, but may silentlyfail under strong memory pressure.

If there are no listeners for given group-ESRCH can be returned.

Note:
When registering new callback user, connector core assignsnetlink group to the user which is equal to its id.idx.

Protocol description

The current framework offers a transport layer with fixed headers. Therecommended protocol which uses such a header is as following:

msg->seq and msg->ack are used to determine message genealogy. Whensomeone sends a message, they use a locally unique sequence and randomacknowledge number. The sequence number may be copied intonlmsghdr->nlmsg_seq too.

The sequence number is incremented with each message sent.

If you expect a reply to the message, then the sequence number in thereceived message MUST be the same as in the original message, and theacknowledge number MUST be the same + 1.

If we receive a message and its sequence number is not equal to one weare expecting, then it is a new message. If we receive a message andits sequence number is the same as one we are expecting, but itsacknowledge is not equal to the sequence number in the originalmessage + 1, then it is a new message.

Obviously, the protocol header contains the above id.

The connector allows event notification in the following form: kerneldriver or userspace process can ask connector to notify it whenselected ids will be turned on or off (registered or unregistered itscallback). It is done by sending a special command to the connectordriver (it also registers itself with id={-1, -1}).

As example of this usage can be found in the cn_test.c module whichuses the connector to request notification and to send messages.

Reliability

Netlink itself is not a reliable protocol. That means that messages canbe lost due to memory pressure or process’ receiving queue overflowed,so caller is warned that it must be prepared. That is why the structcn_msg [main connector’s message header] contains u32 seq and u32 ackfields.

Userspace usage

2.6.14 has a new netlink socket implementation, which by default does notallow people to send data to netlink groups other than 1.So, if you wish to use a netlink socket (for example using connector)with a different group number, the userspace application must subscribe tothat group first. It can be achieved by the following pseudocode:

s = socket(PF_NETLINK, SOCK_DGRAM, NETLINK_CONNECTOR);l_local.nl_family = AF_NETLINK;l_local.nl_groups = 12345;l_local.nl_pid = 0;if (bind(s, (struct sockaddr *)&l_local, sizeof(struct sockaddr_nl)) == -1) {      perror("bind");      close(s);      return -1;}{      int on = l_local.nl_groups;      setsockopt(s, 270, 1, &on, sizeof(on));}

Where 270 above is SOL_NETLINK, and 1 is a NETLINK_ADD_MEMBERSHIP socketoption. To drop a multicast subscription, one should call the above socketoption with the NETLINK_DROP_MEMBERSHIP parameter which is defined as 0.

2.6.14 netlink code only allows to select a group which is less or equal tothe maximum group number, which is used at netlink_kernel_create() time.In case of connector it is CN_NETLINK_USERS + 0xf, so if you want to usegroup number 12345, you must increment CN_NETLINK_USERS to that number.Additional 0xf numbers are allocated to be used by non-in-kernel users.

Due to this limitation, group 0xffffffff does not work now, so one cannot use add/remove connector’s group notifications, but as far as I know,only cn_test.c test module used it.

Some work in netlink area is still being done, so things can be changed in2.6.15 timeframe, if it will happen, documentation will be updated for thatkernel.

Code samples

Sample code for a connector test module and user space can be foundin samples/connector/. To build this code, enable CONFIG_CONNECTORand CONFIG_SAMPLES.