5. Kernel Entries

This file documents some of the kernel entries inarch/x86/entry/entry_64.S. A lot of this explanation is adapted froman email from Ingo Molnar:

http://lkml.kernel.org/r/<20110529191055.GC9835%40elte.hu>

The x86 architecture has quite a few different ways to jump intokernel code. Most of these entry points are registered inarch/x86/kernel/traps.c and implemented in arch/x86/entry/entry_64.Sfor 64-bit, arch/x86/entry/entry_32.S for 32-bit and finallyarch/x86/entry/entry_64_compat.S which implements the 32-bit compatibilitysyscall entry points and thus provides for 32-bit processes theability to execute syscalls when running on 64-bit kernels.

The IDT vector assignments are listed in arch/x86/include/asm/irq_vectors.h.

Some of these entries are:

  • system_call: syscall instruction from 64-bit code.
  • entry_INT80_compat: int 0x80 from 32-bit or 64-bit code; compat syscalleither way.
  • entry_INT80_compat, ia32_sysenter: syscall and sysenter from 32-bitcode
  • interrupt: An array of entries. Every IDT vector that doesn’texplicitly point somewhere else gets set to the correspondingvalue in interrupts. These point to a whole array ofmagically-generated functions that make their way to do_IRQ withthe interrupt number as a parameter.
  • APIC interrupts: Various special-purpose interrupts for thingslike TLB shootdown.
  • Architecturally-defined exceptions like divide_error.

There are a few complexities here. The different x86-64 entrieshave different calling conventions. The syscall and sysenterinstructions have their own peculiar calling conventions. Some ofthe IDT entries push an error code onto the stack; others don’t.IDT entries using the IST alternative stack mechanism need their ownmagic to get the stack frames right. (You can find somedocumentation in the AMD APM, Volume 2, Chapter 8 and the Intel SDM,Volume 3, Chapter 6.)

Dealing with the swapgs instruction is especially tricky. Swapgstoggles whether gs is the kernel gs or the user gs. The swapgsinstruction is rather fragile: it must nest perfectly and only insingle depth, it should only be used if entering from user mode tokernel mode and then when returning to user-space, and preciselyso. If we mess that up even slightly, we crash.

So when we have a secondary entry, already in kernel mode, wemustnot use SWAPGS blindly - nor must we forget doing a SWAPGS when it’snot switched/swapped yet.

Now, there’s a secondary complication: there’s a cheap way to testwhich mode the CPU is in and an expensive way.

The cheap way is to pick this info off the entry frame on the kernelstack, from the CS of the ptregs area of the kernel stack:

xorl %ebx,%ebxtestl $3,CS+8(%rsp)je error_kernelspaceSWAPGS

The expensive (paranoid) way is to read back the MSR_GS_BASE value(which is what SWAPGS modifies):

      movl $1,%ebx      movl $MSR_GS_BASE,%ecx      rdmsr      testl %edx,%edx      js 1f   /* negative -> in kernel */      SWAPGS      xorl %ebx,%ebx1:    ret

If we are at an interrupt or user-trap/gate-alike boundary then we canuse the faster check: the stack will be a reliable indicator ofwhether SWAPGS was already done: if we see that we are a secondaryentry interrupting kernel mode execution, then we know that the GSbase has already been switched. If it says that we interrupteduser-space execution then we must do the SWAPGS.

But if we are in an NMI/MCE/DEBUG/whatever super-atomic entry context,which might have triggered right after a normal entry wrote CS to thestack but before we executed SWAPGS, then the only safe way to checkfor GS is the slower method: the RDMSR.

Therefore, super-atomic entries (except NMI, which is handled separately)must use idtentry with paranoid=1 to handle gsbase correctly. Thistriggers three main behavior changes:

  • Interrupt entry will use the slower gsbase check.
  • Interrupt entry from user mode will switch off the IST stack.
  • Interrupt exit to kernel mode will not attempt to reschedule.

We try to only use IST entries and the paranoid entry code for vectorsthat absolutely need the more expensive check for the GS base - and wegenerate all ‘normal’ entry points with the regular (faster) paranoid=0variant.