The problem¶

There is a problem with active restarts in autofs (that is to sayrestarting autofs when there are busy mounts).

During normal operation autofs uses a file descriptor opened on thedirectory that is being managed in order to be able to issue controloperations. Using a file descriptor gives ioctl operations access toautofs specific information stored in the super block. The operationsare things such as setting an autofs mount catatonic, setting theexpire timeout and requesting expire checks. As is explained below,certain types of autofs triggered mounts can end up covering an autofsmount itself which prevents us being able to use open(2) to obtain afile descriptor for these operations if we don’t already have one open.

Currently autofs uses “umount -l” (lazy umount) to clear active mountsat restart. While using lazy umount works for most cases, anything thatneeds to walk back up the mount tree to construct a path, such asgetcwd(2) and the proc file system /proc/<pid>/cwd, no longer worksbecause the point from which the path is constructed has been detachedfrom the mount tree.

The actual problem with autofs is that it can’t reconnect to existingmounts. Immediately one thinks of just adding the ability to remountautofs file systems would solve it, but alas, that can’t work. This isbecause autofs direct mounts and the implementation of “on demand mountand expire” of nested mount trees have the file system mounted directlyon top of the mount trigger directory dentry.

For example, there are two types of automount maps, direct (in the kernelmodule source you will see a third type called an offset, which is justa direct mount in disguise) and indirect.

Here is a master map with direct and indirect map entries:

/-      /etc/auto.direct/test   /etc/auto.indirect

and the corresponding map files:

/etc/auto.direct:/automount/dparse/g6  budgie:/autofs/export1/automount/dparse/g1  shark:/autofs/export1and so on.

/etc/auto.indirect:

g1    shark:/autofs/export1g6    budgie:/autofs/export1and so on.

For the above indirect map an autofs file system is mounted on /test andmounts are triggered for each sub-directory key by the inode lookupoperation. So we see a mount of shark:/autofs/export1 on /test/g1, forexample.

The way that direct mounts are handled is by making an autofs mount oneach full path, such as /automount/dparse/g1, and using it as a mounttrigger. So when we walk on the path we mount shark:/autofs/export1 “ontop of this mount point”. Since these are always directories we canuse the follow_link inode operation to trigger the mount.

But, each entry in direct and indirect maps can have offsets (makingthem multi-mount map entries).

For example, an indirect mount map entry could also be:

g1  \/        shark:/autofs/export5/testing/test \/s1      shark:/autofs/export/testing/test/s1 \/s2      shark:/autofs/export5/testing/test/s2 \/s1/ss1  shark:/autofs/export1 \/s2/ss2  shark:/autofs/export2

and a similarly a direct mount map entry could also be:

/automount/dparse/g1 \    /       shark:/autofs/export5/testing/test \    /s1     shark:/autofs/export/testing/test/s1 \    /s2     shark:/autofs/export5/testing/test/s2 \    /s1/ss1 shark:/autofs/export2 \    /s2/ss2 shark:/autofs/export2

One of the issues with version 4 of autofs was that, when mounting anentry with a large number of offsets, possibly with nesting, we neededto mount and umount all of the offsets as a single unit. Not really aproblem, except for people with a large number of offsets in map entries.This mechanism is used for the well known “hosts” map and we have seencases (in 2.4) where the available number of mounts are exhausted orwhere the number of privileged ports available is exhausted.

In version 5 we mount only as we go down the tree of offsets andsimilarly for expiring them which resolves the above problem. There issomewhat more detail to the implementation but it isn’t needed for thesake of the problem explanation. The one important detail is that theseoffsets are implemented using the same mechanism as the direct mountsabove and so the mount points can be covered by a mount.

The current autofs implementation uses an ioctl file descriptor openedon the mount point for control operations. The references held by thedescriptor are accounted for in checks made to determine if a mount isin use and is also used to access autofs file system information heldin the mount super block. So the use of a file handle needs to beretained.

The Solution¶

To be able to restart autofs leaving existing direct, indirect andoffset mounts in place we need to be able to obtain a file handlefor these potentially covered autofs mount points. Rather than justimplement an isolated operation it was decided to re-implement theexisting ioctl interface and add new operations to provide thisfunctionality.

In addition, to be able to reconstruct a mount tree that has busy mounts,the uid and gid of the last user that triggered the mount needs to beavailable because these can be used as macro substitution variables inautofs maps. They are recorded at mount request time and an operationhas been added to retrieve them.

Since we’re re-implementing the control interface, a couple of otherproblems with the existing interface have been addressed. First, whena mount or expire operation completes a status is returned to thekernel by either a “send ready” or a “send fail” operation. The“send fail” operation of the ioctl interface could only ever sendENOENT so the re-implementation allows user space to send an actualstatus. Another expensive operation in user space, for those usingvery large maps, is discovering if a mount is present. Usually thisinvolves scanning /proc/mounts and since it needs to be done quiteoften it can introduce significant overhead when there are many entriesin the mount table. An operation to lookup the mount status of a mountpoint dentry (covered or not) has also been added.

Current kernel development policy recommends avoiding the use of theioctl mechanism in favor of systems such as Netlink. An implementationusing this system was attempted to evaluate its suitability and it wasfound to be inadequate, in this case. The Generic Netlink system wasused for this as raw Netlink would lead to a significant increase incomplexity. There’s no question that the Generic Netlink system is anelegant solution for common case ioctl functions but it’s not a completereplacement probably because its primary purpose in life is to be amessage bus implementation rather than specifically an ioctl replacement.While it would be possible to work around this there is one concernthat lead to the decision to not use it. This is that the autofsexpire in the daemon has become far to complex because umountcandidates are enumerated, almost for no other reason than to “count”the number of times to call the expire ioctl. This involves scanningthe mount table which has proved to be a big overhead for users withlarge maps. The best way to improve this is try and get back to theway the expire was done long ago. That is, when an expire request isissued for a mount (file handle) we should continually call back tothe daemon until we can’t umount any more mounts, then return theappropriate status to the daemon. At the moment we just expire onemount at a time. A Generic Netlink implementation would exclude thispossibility for future development due to the requirements of themessage bus architecture.

autofs Miscellaneous Device mount control interface¶

The control interface is opening a device node, typically /dev/autofs.

All the ioctls use a common structure to pass the needed parameterinformation and return operation results:

struct autofs_dev_ioctl {        __u32 ver_major;        __u32 ver_minor;        __u32 size;             /* total size of data passed in                                * including this struct */        __s32 ioctlfd;          /* automount command fd */        /* Command parameters */        union {                struct args_protover                protover;                struct args_protosubver             protosubver;                struct args_openmount               openmount;                struct args_ready           ready;                struct args_fail            fail;                struct args_setpipefd               setpipefd;                struct args_timeout         timeout;                struct args_requester               requester;                struct args_expire          expire;                struct args_askumount               askumount;                struct args_ismountpoint    ismountpoint;        };        char path[];};

The ioctlfd field is a mount point file descriptor of an autofs mountpoint. It is returned by the open call and is used by all calls exceptthe check for whether a given path is a mount point, where it mayoptionally be used to check a specific mount corresponding to a givenmount point file descriptor, and when requesting the uid and gid of thelast successful mount on a directory within the autofs file system.

Theunionis used to communicate parameters and results of calls madeas described below.

The path field is used to pass a path where it is needed and the size fieldis used account for the increased structure length when translating thestructure sent from user space.

This structure can be initialized before setting specific fields by usingthe void function call init_autofs_dev_ioctl(structautofs_dev_ioctl*).

All of the ioctls perform a copy of this structure from user space tokernel space and return -EINVAL if the size parameter is smaller thanthe structure size itself, -ENOMEM if the kernel memory allocation failsor -EFAULT if the copy itself fails. Other checks include a version checkof the compiled in user space version against the module version and amismatch results in a -EINVAL return. If the size field is greater thanthe structure size then a path is assumed to be present and is checked toensure it begins with a “/” and is NULL terminated, otherwise -EINVAL isreturned. Following these checks, for all ioctl commands exceptAUTOFS_DEV_IOCTL_VERSION_CMD, AUTOFS_DEV_IOCTL_OPENMOUNT_CMD andAUTOFS_DEV_IOCTL_CLOSEMOUNT_CMD the ioctlfd is validated and if it isnot a valid descriptor or doesn’t correspond to an autofs mount pointan error of -EBADF, -ENOTTY or -EINVAL (not an autofs descriptor) isreturned.

The ioctls¶

An example of an implementation which uses this interface can be seenin autofs version 5.0.4 and later in file lib/dev-ioctl-lib.c of thedistribution tar available for download from kernel.org in directory/pub/linux/daemons/autofs/v5.

The device node ioctl operations implemented by this interface are: