15.Control-flow Enforcement Technology (CET) Shadow Stack

15.1.CET Background

Control-flow Enforcement Technology (CET) covers several related x86 processorfeatures that provide protection against control flow hijacking attacks. CETcan protect both applications and the kernel.

CET introduces shadow stack and indirect branch tracking (IBT). A shadow stackis a secondary stack allocated from memory which cannot be directly modified byapplications. When executing a CALL instruction, the processor pushes thereturn address to both the normal stack and the shadow stack. Uponfunction return, the processor pops the shadow stack copy and compares itto the normal stack copy. If the two differ, the processor raises acontrol-protection fault. IBT verifies indirect CALL/JMP targets are intendedas marked by the compiler with ‘ENDBR’ opcodes. Not all CPU’s have both ShadowStack and Indirect Branch Tracking. Today in the 64-bit kernel, only userspaceshadow stack and kernel IBT are supported.

15.2.Requirements to use Shadow Stack

To use userspace shadow stack you need HW that supports it, a kernelconfigured with it and userspace libraries compiled with it.

The kernel Kconfig option is X86_USER_SHADOW_STACK. When compiled in, shadowstacks can be disabled at runtime with the kernel parameter: nousershstk.

To build a user shadow stack enabled kernel, Binutils v2.29 or LLVM v6 or laterare required.

At run time, /proc/cpuinfo shows CET features if the processor supportsCET. “user_shstk” means that userspace shadow stack is supported on the currentkernel and HW.

15.3.Application Enabling

An application’s CET capability is marked in its ELF note and can be verifiedfrom readelf/llvm-readelf output:

readelf -n <application> | grep -a SHSTK    properties: x86 feature: SHSTK

The kernel does not process these applications markers directly. Applicationsor loaders must enable CET features using the interface described in section 4.Typically this would be done in dynamic loader or static runtime objects, as isthe case in GLIBC.

15.4.Enabling arch_prctl()’s

Elf features should be enabled by the loader using the below arch_prctl’s. Theyare only supported in 64 bit user applications. These operate on the featureson a per-thread basis. The enablement status is inherited on clone, so if thefeature is enabled on the first thread, it will propagate to all the thread’sin an app.

arch_prctl(ARCH_SHSTK_ENABLE, unsigned long feature)

Enable a single feature specified in ‘feature’. Can only operate onone feature at a time.

arch_prctl(ARCH_SHSTK_DISABLE, unsigned long feature)

Disable a single feature specified in ‘feature’. Can only operate onone feature at a time.

arch_prctl(ARCH_SHSTK_LOCK, unsigned long features)

Lock in features at their current enabled or disabled status. ‘features’is a mask of all features to lock. All bits set are processed, unset bitsare ignored. The mask is ORed with the existing value. So any feature bitsset here cannot be enabled or disabled afterwards.

arch_prctl(ARCH_SHSTK_UNLOCK, unsigned long features)

Unlock features. ‘features’ is a mask of all features to unlock. Allbits set are processed, unset bits are ignored. Only works via ptrace.

arch_prctl(ARCH_SHSTK_STATUS, unsigned long addr)

Copy the currently enabled features to the address passed in addr. Thefeatures are described using the bits passed into the others in‘features’.

The return values are as follows. On success, return 0. On error, errno canbe:

-EPERM if any of the passed feature are locked.-ENOTSUPP if the feature is not supported by the hardware or kernel.-EINVAL arguments (non existing feature, etc)-EFAULT if could not copy information back to userspace

The feature’s bits supported are:

ARCH_SHSTK_SHSTK - Shadow stackARCH_SHSTK_WRSS  - WRSS

Currently shadow stack and WRSS are supported via this interface. WRSScan only be enabled with shadow stack, and is automatically disabledif shadow stack is disabled.

15.5.Proc Status

To check if an application is actually running with shadow stack, theuser can read the /proc/$PID/status. It will report “wrss” or “shstk”depending on what is enabled. The lines look like this:

x86_Thread_features: shstk wrssx86_Thread_features_locked: shstk wrss

15.6.Implementation of the Shadow Stack

15.6.1.Shadow Stack Size

A task’s shadow stack is allocated from memory to a fixed size ofMIN(RLIMIT_STACK, 4 GB). In other words, the shadow stack is allocated tothe maximum size of the normal stack, but capped to 4 GB. In the caseof the clone3 syscall, there is a stack size passed in and shadow stackuses this instead of the rlimit.

15.6.2.Signal

The main program and its signal handlers use the same shadow stack. Becausethe shadow stack stores only return addresses, a large shadow stack coversthe condition that both the program stack and the signal alternate stack runout.

When a signal happens, the old pre-signal state is pushed on the stack. Whenshadow stack is enabled, the shadow stack specific state is pushed onto theshadow stack. Today this is only the old SSP (shadow stack pointer), pushedin a special format with bit 63 set. On sigreturn this old SSP token isverified and restored by the kernel. The kernel will also push the normalrestorer address to the shadow stack to help userspace avoid a shadow stackviolation on the sigreturn path that goes through the restorer.

So the shadow stack signal frame format is as follows:

|1...old SSP| - Pointer to old pre-signal ssp in sigframe token format                (bit 63 set to 1)|        ...| - Other state may be added in the future

32 bit ABI signals are not supported in shadow stack processes. Linux prevents32 bit execution while shadow stack is enabled by the allocating shadow stacksoutside of the 32 bit address space. When execution enters 32 bit mode, eithervia far call or returning to userspace, a #GP is generated by the hardwarewhich, will be delivered to the process as a segfault. When transitioning touserspace the register’s state will be as if the userspace ip being returned tocaused the segfault.

15.6.3.Fork

The shadow stack’s vma has VM_SHADOW_STACK flag set; its PTEs are requiredto be read-only and dirty. When a shadow stack PTE is not RO and dirty, ashadow access triggers a page fault with the shadow stack access bit setin the page fault error code.

When a task forks a child, its shadow stack PTEs are copied and both theparent’s and the child’s shadow stack PTEs are cleared of the dirty bit.Upon the next shadow stack access, the resulting shadow stack page faultis handled by page copy/re-use.

When a pthread child is created, the kernel allocates a new shadow stackfor the new thread. New shadow stack creation behaves like mmap() with respectto ASLR behavior. Similarly, on thread exit the thread’s shadow stack isdisabled.

15.6.4.Exec

On exec, shadow stack features are disabled by the kernel. At which point,userspace can choose to re-enable, or lock them.