What Is High Memory?¶

High memory (highmem) is used when the size of physical memory approaches orexceeds the maximum size of virtual memory. At that point it becomesimpossible for the kernel to keep all of the available physical memory mappedat all times. This means the kernel needs to start using temporary mappings ofthe pieces of physical memory that it wants to access.

The part of (physical) memory not covered by a permanent mapping is what werefer to as ‘highmem’. There are various architecture dependent constraints onwhere exactly that border lies.

In the i386 arch, for example, we choose to map the kernel into every process’sVM space so that we don’t have to pay the full TLB invalidation costs forkernel entry/exit. This means the available virtual memory space (4GiB oni386) has to be divided between user and kernel space.

The traditional split for architectures using this approach is 3:1, 3GiB foruserspace and the top 1GiB for kernel space:

+--------+ 0xffffffff| Kernel |+--------+ 0xc0000000|        || User   ||        |+--------+ 0x00000000

This means that the kernel can at most map 1GiB of physical memory at any onetime, but because we need virtual address space for other things - includingtemporary maps to access the rest of the physical memory - the actual directmap will typically be less (usually around ~896MiB).

Other architectures that have mm context tagged TLBs can have separate kerneland user maps. Some hardware (like some ARMs), however, have limited virtualspace when they use mm context tags.

Temporary Virtual Mappings¶

The kernel contains several ways of creating temporary mappings. The followinglist shows them in order of preference of use.

• kmap_local_page(),kmap_local_folio() - These functions are used to createshort term mappings. They can be invoked from any context (includinginterrupts) but the mappings can only be used in the context which acquiredthem. The only differences between them consist in the first taking a pointerto astructpage and the second taking a pointer tostructfolio and the byteoffset within the folio which identifies the page.

  These functions should always be used, whereaskmap_atomic() andkmap() havebeen deprecated.

  These mappings are thread-local and CPU-local, meaning that the mappingcan only be accessed from within this thread and the thread is bound to theCPU while the mapping is active. Although preemption is never disabled bythis function, the CPU can not be unplugged from the system viaCPU-hotplug until the mapping is disposed.

  It’s valid to take pagefaults in a local kmap region, unless the contextin which the local mapping is acquired does not allow it for other reasons.

  As said, pagefaults and preemption are never disabled. There is no need todisable preemption because, when context switches to a different task, themaps of the outgoing task are saved and those of the incoming one arerestored.

  kmap_local_page(), as well askmap_local_folio() always returns valid virtualkernel addresses and it is assumed thatkunmap_local() will never fail.

  On CONFIG_HIGHMEM=n kernels and for low memory pages they return thevirtual address of the direct mapping. Only real highmem pages aretemporarily mapped. Therefore, users may call a plainpage_address()for pages which are known to not come from ZONE_HIGHMEM. However, it isalways safe to use kmap_local_{page,folio}() /kunmap_local().

  While they are significantly faster thankmap(), for the highmem case theycome with restrictions about the pointers validity. Contrary tokmap()mappings, the local mappings are only valid in the context of the callerand cannot be handed to other contexts. This implies that users mustbe absolutely sure to keep the use of the return address local to thethread which mapped it.

  Most code can be designed to use thread local mappings. User shouldtherefore try to design their code to avoid the use ofkmap() by mappingpages in the same thread the address will be used and preferkmap_local_page() orkmap_local_folio().

  Nestingkmap_local_page() andkmap_atomic() mappings is allowed to a certainextent (up to KMAP_TYPE_NR) but their invocations have to be strictly orderedbecause the map implementation is stack based. Seekmap_local_page() kdocs(included in the “Functions” section) for details on how to manage nestedmappings.

• kmap_atomic(). This function has been deprecated; usekmap_local_page().

  NOTE: Conversions tokmap_local_page() must take care to follow the mappingrestrictions imposed onkmap_local_page(). Furthermore, the code betweencalls tokmap_atomic() andkunmap_atomic() may implicitly depend on the sideeffects of atomic mappings, i.e. disabling page faults or preemption, or both.In that case, explicit calls topagefault_disable() orpreempt_disable() orboth must be made in conjunction with the use ofkmap_local_page().

  [Legacy documentation]

  This permits a very short duration mapping of a single page. Since themapping is restricted to the CPU that issued it, it performs well, butthe issuing task is therefore required to stay on that CPU until it hasfinished, lest some other task displace its mappings.

  kmap_atomic() may also be used by interrupt contexts, since it does notsleep and the callers too may not sleep until afterkunmap_atomic() iscalled.

  Each call ofkmap_atomic() in the kernel creates a non-preemptible sectionand disable pagefaults. This could be a source of unwanted latency. Thereforeusers should preferkmap_local_page() instead ofkmap_atomic().

  It is assumed that k[un]map_atomic() won’t fail.

• kmap(). This function has been deprecated; usekmap_local_page().

  NOTE: Conversions tokmap_local_page() must take care to follow the mappingrestrictions imposed onkmap_local_page(). In particular, it is necessary tomake sure that the kernel virtual memory pointer is only valid in the threadthat obtained it.

  [Legacy documentation]

  This should be used to make short duration mapping of a single page with norestrictions on preemption or migration. It comes with an overhead as mappingspace is restricted and protected by a global lock for synchronization. Whenmapping is no longer needed, the address that the page was mapped to must bereleased withkunmap().

  Mapping changes must be propagated across all the CPUs.kmap() alsorequires global TLB invalidation when the kmap’s pool wraps and it mightblock when the mapping space is fully utilized until a slot becomesavailable. Therefore,kmap() is only callable from preemptible context.

  All the above work is necessary if a mapping must last for a relativelylong time but the bulk of high-memory mappings in the kernel areshort-lived and only used in one place. This means that the cost ofkmap() is mostly wasted in such cases.kmap() was not intended for longterm mappings but it has morphed in that direction and its use isstrongly discouraged in newer code and the set of the preceding functionsshould be preferred.

  On 64-bit systems, calls tokmap_local_page(),kmap_atomic() andkmap() haveno real work to do because a 64-bit address space is more than sufficient toaddress all the physical memory whose pages are permanently mapped.

• vmap(). This can be used to make a long duration mapping of multiplephysical pages into a contiguous virtual space. It needs globalsynchronization to unmap.

Cost of Temporary Mappings¶

The cost of creating temporary mappings can be quite high. The arch has tomanipulate the kernel’s page tables, the data TLB and/or the MMU’s registers.

If CONFIG_HIGHMEM is not set, then the kernel will try and create a mappingsimply with a bit of arithmetic that will convert the pagestructaddress intoa pointer to the page contents rather than juggling mappings about. In such acase, the unmap operation may be a null operation.

If CONFIG_MMU is not set, then there can be no temporary mappings and nohighmem. In such a case, the arithmetic approach will also be used.

i386 PAE¶

The i386 arch, under some circumstances, will permit you to stick up to 64GiBof RAM into your 32-bit machine. This has a number of consequences:

• Linux needs a page-frame structure for each page in the system and thepageframes need to live in the permanent mapping, which means:

• you can have 896M/sizeof(structpage) page-frames at most; withstructpage being 32-bytes that would end up being something in the order of 112Gworth of pages; the kernel, however, needs to store more than justpage-frames in that memory...

• PAE makes your page tables larger - which slows the system down as moredata has to be accessed to traverse in TLB fills and the like. Oneadvantage is that PAE has more PTE bits and can provide advanced featureslike NX and PAT.

The general recommendation is that you don’t use more than 8GiB on a 32-bitmachine - although more might work for you and your workload, you’re prettymuch on your own - don’t expect kernel developers to really care much if thingscome apart.

Functions¶

void*kmap(structpage*page)¶

Map a page for long term usage

voidkunmap(conststructpage*page)¶

Unmap the virtual address mapped bykmap()

structpage*kmap_to_page(void*addr)¶

Get the page for a kmap’ed address

voidkmap_flush_unused(void)¶

Flush all unused kmap mappings in order to remove stray mappings

void*kmap_local_page(conststructpage*page)¶

Map a page for temporary usage

void*kmap_local_folio(conststructfolio*folio,size_toffset)¶

Map a page in this folio for temporary usage

addr1 = kmap_local_folio(folio1, offset1);addr2 = kmap_local_folio(folio2, offset2);...kunmap_local(addr2);kunmap_local(addr1);

void*kmap_atomic(conststructpage*page)¶

Atomically map a page for temporary usage - Deprecated!

// Find the page of interest.struct page *page = find_get_page(mapping, offset);// Gain access to the contents of that page.void *vaddr = kmap_atomic(page);// Do something to the contents of that page.memset(vaddr, 0, PAGE_SIZE);// Unmap that page.kunmap_atomic(vaddr);

structfolio*vma_alloc_zeroed_movable_folio(structvm_area_struct*vma,unsignedlongvaddr)¶

Allocate a zeroed page for a VMA.

voidmemcpy_from_folio(char*to,structfolio*folio,size_toffset,size_tlen)¶

Copy a range of bytes from a folio.

voidmemcpy_to_folio(structfolio*folio,size_toffset,constchar*from,size_tlen)¶

Copy a range of bytes to a folio.

void*folio_zero_tail(structfolio*folio,size_toffset,void*kaddr)¶

Zero the tail of a folio.

voidfolio_fill_tail(structfolio*folio,size_toffset,constchar*from,size_tlen)¶

Copy some data to a folio and pad with zeroes.

size_tmemcpy_from_file_folio(char*to,structfolio*folio,loff_tpos,size_tlen)¶

Copy some bytes from a file folio.

voidfolio_zero_segments(structfolio*folio,size_tstart1,size_txend1,size_tstart2,size_txend2)¶

Zero two byte ranges in a folio.

voidfolio_zero_segment(structfolio*folio,size_tstart,size_txend)¶

Zero a byte range in a folio.

voidfolio_zero_range(structfolio*folio,size_tstart,size_tlength)¶

Zero a byte range in a folio.

voidfolio_release_kmap(structfolio*folio,void*addr)¶

Unmap a folio and drop a refcount.

void*kmap_high(structpage*page)¶

map a highmem page into memory

void*kmap_high_get(conststructpage*page)¶

pin a highmem page into memory

voidkunmap_high(conststructpage*page)¶

unmap a highmem page into memory

void*page_address(conststructpage*page)¶

get the mapped virtual address of a page

voidset_page_address(structpage*page,void*virtual)¶

set a page’s virtual address

kunmap_atomic¶

kunmap_atomic(__addr)

Unmap the virtual address mapped bykmap_atomic() - deprecated!

kunmap_local¶

kunmap_local(__addr)

Unmap a page mapped viakmap_local_page().