No-MMU memory mapping support

The kernel has limited support for memory mapping under no-MMU conditions, suchas are used in uClinux environments. From the userspace point of view, memorymapping is made use of in conjunction with the mmap() system call, theshmat()call and the execve() system call. From the kernel’s point of view, execve()mapping is actually performed by the binfmt drivers, which call back into themmap() routines to do the actual work.

Memory mapping behaviour also involves the way fork(),vfork(), clone() andptrace() work. Under uClinux there is no fork(), and clone() must be suppliedthe CLONE_VM flag.

The behaviour is similar between the MMU and no-MMU cases, but not identical;and it’s also much more restricted in the latter case:

  1. Anonymous mapping, MAP_PRIVATE

    In the MMU case: VM regions backed by arbitrary pages; copy-on-writeacross fork.

    In the no-MMU case: VM regions backed by arbitrary contiguous runs ofpages.

  2. Anonymous mapping, MAP_SHARED

    These behave very much like private mappings, except that they’reshared across fork() or clone() without CLONE_VM in the MMU case. Sincethe no-MMU case doesn’t support these, behaviour is identical toMAP_PRIVATE there.

  3. File, MAP_PRIVATE, PROT_READ / PROT_EXEC, !PROT_WRITE

    In the MMU case: VM regions backed by pages read from file; changes tothe underlying file are reflected in the mapping; copied across fork.

    In the no-MMU case:

    • If one exists, the kernel will re-use an existing mapping to thesame segment of the same file if that has compatible permissions,even if this was created by another process.

    • If possible, the file mapping will be directly on the backing deviceif the backing device has the NOMMU_MAP_DIRECT capability andappropriate mapping protection capabilities. Ramfs, romfs, cramfsand mtd might all permit this.

    • If the backing device can’t or won’t permit direct sharing,but does have the NOMMU_MAP_COPY capability, then a copy of theappropriate bit of the file will be read into a contiguous bit ofmemory and any extraneous space beyond the EOF will be cleared

    • Writes to the file do not affect the mapping; writes to the mappingare visible in other processes (no MMU protection), but should nothappen.

  4. File, MAP_PRIVATE, PROT_READ / PROT_EXEC, PROT_WRITE

    In the MMU case: like the non-PROT_WRITE case, except that the pages inquestion get copied before the write actually happens. From that pointon writes to the file underneath that page no longer get reflected intothe mapping’s backing pages. The page is then backed by swap instead.

    In the no-MMU case: works much like the non-PROT_WRITE case, exceptthat a copy is always taken and never shared.

  5. Regular file / blockdev, MAP_SHARED, PROT_READ / PROT_EXEC / PROT_WRITE

    In the MMU case: VM regions backed by pages read from file; changes topages written back to file; writes to file reflected into pages backingmapping; shared across fork.

    In the no-MMU case: not supported.

  6. Memory backed regular file, MAP_SHARED, PROT_READ / PROT_EXEC / PROT_WRITE

    In the MMU case: As for ordinary regular files.

    In the no-MMU case: The filesystem providing the memory-backed file(such as ramfs or tmpfs) may choose to honour an open, truncate, mmapsequence by providing a contiguous sequence of pages to map. In thatcase, a shared-writable memory mapping will be possible. It will workas for the MMU case. If the filesystem does not provide any suchsupport, then the mapping request will be denied.

  7. Memory backed blockdev, MAP_SHARED, PROT_READ / PROT_EXEC / PROT_WRITE

    In the MMU case: As for ordinary regular files.

    In the no-MMU case: As for memory backed regular files, but theblockdev must be able to provide a contiguous run of pages withouttruncate being called. The ramdisk driver could do this if it allocatedall its memory as a contiguous array upfront.

  8. Memory backed chardev, MAP_SHARED, PROT_READ / PROT_EXEC / PROT_WRITE

    In the MMU case: As for ordinary regular files.

    In the no-MMU case: The character device driver may choose to honourthe mmap() by providing direct access to the underlying device if itprovides memory or quasi-memory that can be accessed directly. Examplesof such are frame buffers and flash devices. If the driver does notprovide any such support, then the mapping request will be denied.

Further notes on no-MMU MMAP

  1. A request for a private mapping of a file may return a buffer that is notpage-aligned. This is because XIP may take place, and the data may not bepaged aligned in the backing store.

  2. A request for an anonymous mapping will always be page aligned. Ifpossible the size of the request should be a power of two otherwise someof the space may be wasted as the kernel must allocate a power-of-2granule but will only discard the excess if appropriately configured asthis has an effect on fragmentation.

  3. The memory allocated by a request for an anonymous mapping will normallybe cleared by the kernel before being returned in accordance with theLinux man pages (ver 2.22 or later).

    In the MMU case this can be achieved with reasonable performance asregions are backed by virtual pages, with the contents only being mappedto cleared physical pages when a write happens on that specific page(prior to which, the pages are effectively mapped to the global zero pagefrom which reads can take place). This spreads out the time it takes toinitialize the contents of a page - depending on the write-usage of themapping.

    In the no-MMU case, however, anonymous mappings are backed by physicalpages, and the entire map is cleared at allocation time. This can causesignificant delays during a userspacemalloc() as the C library does ananonymous mapping and the kernel then does a memset for the entire map.

    However, for memory that isn’t required to be precleared - such as thatreturned bymalloc() - mmap() can take a MAP_UNINITIALIZED flag toindicate to the kernel that it shouldn’t bother clearing the memory beforereturning it. Note that CONFIG_MMAP_ALLOW_UNINITIALIZED must be enabledto permit this, otherwise the flag will be ignored.

    uClibc uses this to speed upmalloc(), and the ELF-FDPIC binfmt uses thisto allocate the brk and stack region.

  4. A list of all the private copy and anonymous mappings on the system isvisible through /proc/maps in no-MMU mode.

  5. A list of all the mappings in use by a process is visible through/proc/<pid>/maps in no-MMU mode.

  6. Supplying MAP_FIXED or a requesting a particular mapping address willresult in an error.

  7. Files mapped privately usually have to have a read method provided by thedriver or filesystem so that the contents can be read into the memoryallocated if mmap() chooses not to map the backing device directly. Anerror will result if they don’t. This is most likely to be encounteredwith character device files, pipes, fifos and sockets.

Interprocess shared memory

Both SYSV IPC SHM shared memory and POSIX shared memory is supported in NOMMUmode. The former through the usual mechanism, the latter through files createdon ramfs or tmpfs mounts.

Futexes

Futexes are supported in NOMMU mode if the arch supports them. An error willbe given if an address passed to the futex system call lies outside themappings made by a process or if the mapping in which the address lies does notsupport futexes (such as an I/O chardev mapping).

No-MMU mremap

Themremap() function is partially supported. It may change the size of amapping, and may move it[1] if MREMAP_MAYMOVE is specified and if the new sizeof the mapping exceeds the size of the slab object currently occupied by thememory to which the mapping refers, or if a smaller slab object could be used.

MREMAP_FIXED is not supported, though it is ignored if there’s no change ofaddress and the object does not need to be moved.

Shared mappings may not be moved. Shareable mappings may not be moved either,even if they are not currently shared.

Themremap() function must be given an exact match for base address and size ofa previously mapped object. It may not be used to create holes in existingmappings, move parts of existing mappings or resize parts of mappings. It mustact on a complete mapping.

[1]

Not currently supported.

Providing shareable character device support

To provide shareable character device support, a driver must provide afile->f_op->get_unmapped_area() operation. The mmap() routines will call thisto get a proposed address for the mapping. This may return an error if itdoesn’t wish to honour the mapping because it’s too long, at a weird offset,under some unsupported combination of flags or whatever.

The driver should also provide backing device information with capabilities setto indicate the permitted types of mapping on such devices. The default isassumed to be readable and writable, not executable, and only shareabledirectly (can’t be copied).

The file->f_op->mmap() operation will be called to actually inaugurate themapping. It can be rejected at that point. Returning the ENOSYS error willcause the mapping to be copied instead if NOMMU_MAP_COPY is specified.

The vm_ops->close() routine will be invoked when the last mapping on a chardevis removed. An existing mapping will be shared, partially or not, if possiblewithout notifying the driver.

It is permitted also for the file->f_op->get_unmapped_area() operation toreturn -ENOSYS. This will be taken to mean that this operation just doesn’twant to handle it, despite the fact it’s got an operation. For instance, itmight try directing the call to a secondary driver which turns out not toimplement it. Such is the case for the framebuffer driver which attempts todirect the call to the device-specific driver. Under such circumstances, themapping request will be rejected if NOMMU_MAP_COPY is not specified, and acopy mapped otherwise.

Important

Some types of device may present a different appearance to anyonelooking at them in certain modes. Flash chips can be like this; forinstance if they’re in programming or erase mode, you might see thestatus reflected in the mapping, instead of the data.

In such a case, care must be taken lest userspace see a shared or aprivate mapping showing such information when the driver is busycontrolling the device. Remember especially: private executablemappings may still be mapped directly off the device under somecircumstances!

Providing shareable memory-backed file support

Provision of shared mappings on memory backed files is similar to the provisionof support for shared mapped character devices. The main difference is that thefilesystem providing the service will probably allocate a contiguous collectionof pages and permit mappings to be made on that.

It is recommended that a truncate operation applied to such a file thatincreases the file size, if that file is empty, be taken as a request to gatherenough pages to honour a mapping. This is required to support POSIX sharedmemory.

Memory backed devices are indicated by the mapping’s backing device info havingthe memory_backed flag set.

Providing shareable block device support

Provision of shared mappings on block device files is exactly the same as forcharacter devices. If there isn’t a real device underneath, then the drivershould allocate sufficient contiguous memory to honour any supported mapping.

Adjusting page trimming behaviour

NOMMU mmap automatically rounds up to the nearest power-of-2 number of pageswhen performing an allocation. This can have adverse effects on memoryfragmentation, and as such, is left configurable. The default behaviour is toaggressively trim allocations and discard any excess pages back in to the pageallocator. In order to retain finer-grained control over fragmentation, thisbehaviour can either be disabled completely, or bumped up to a higher pagewatermark where trimming begins.

Page trimming behaviour is configurable via the sysctlvm.nr_trim_pages.