The offline case¶

Once a CPU has been logically shutdown the teardown callbacks of registeredhotplug states will be invoked, starting withCPUHP_ONLINE and terminatingat stateCPUHP_OFFLINE. This includes:

• If tasks are frozen due to a suspend operation thencpuhp_tasks_frozenwill be set to true.

• All processes are migrated away from this outgoing CPU to new CPUs.The new CPU is chosen from each process’ current cpuset, which may bea subset of all online CPUs.

• All interrupts targeted to this CPU are migrated to a new CPU

• timers are also migrated to a new CPU

• Once all services are migrated, kernel calls an arch specific routine__cpu_disable() to perform arch specific cleanup.

CPU hotplug state machine¶

CPU hotplug uses a trivial state machine with a linear state space fromCPUHP_OFFLINE to CPUHP_ONLINE. Each state has a startup and a teardowncallback.

When a CPU is onlined, the startup callbacks are invoked sequentially untilthe state CPUHP_ONLINE is reached. They can also be invoked when thecallbacks of a state are set up or an instance is added to a multi-instancestate.

When a CPU is offlined the teardown callbacks are invoked in the reverseorder sequentially until the state CPUHP_OFFLINE is reached. They can alsobe invoked when the callbacks of a state are removed or an instance isremoved from a multi-instance state.

If a usage site requires only a callback in one direction of the hotplugoperations (CPU online or CPU offline) then the other not-required callbackcan be set to NULL when the state is set up.

The state space is divided into three sections:

• The PREPARE section

  The PREPARE section covers the state space from CPUHP_OFFLINE toCPUHP_BRINGUP_CPU.

  The startup callbacks in this section are invoked before the CPU isstarted during a CPU online operation. The teardown callbacks are invokedafter the CPU has become dysfunctional during a CPU offline operation.

  The callbacks are invoked on a control CPU as they can’t obviously run onthe hotplugged CPU which is either not yet started or has becomedysfunctional already.

  The startup callbacks are used to setup resources which are required tobring a CPU successfully online. The teardown callbacks are used to freeresources or to move pending work to an online CPU after the hotpluggedCPU became dysfunctional.

  The startup callbacks are allowed to fail. If a callback fails, the CPUonline operation is aborted and the CPU is brought down to the previousstate (usually CPUHP_OFFLINE) again.

  The teardown callbacks in this section are not allowed to fail.

• The STARTING section

  The STARTING section covers the state space between CPUHP_BRINGUP_CPU + 1and CPUHP_AP_ONLINE.

  The startup callbacks in this section are invoked on the hotplugged CPUwith interrupts disabled during a CPU online operation in the early CPUsetup code. The teardown callbacks are invoked with interrupts disabledon the hotplugged CPU during a CPU offline operation shortly before theCPU is completely shut down.

  The callbacks in this section are not allowed to fail.

  The callbacks are used for low level hardware initialization/shutdown andfor core subsystems.

• The ONLINE section

  The ONLINE section covers the state space between CPUHP_AP_ONLINE + 1 andCPUHP_ONLINE.

  The startup callbacks in this section are invoked on the hotplugged CPUduring a CPU online operation. The teardown callbacks are invoked on thehotplugged CPU during a CPU offline operation.

  The callbacks are invoked in the context of the per CPU hotplug thread,which is pinned on the hotplugged CPU. The callbacks are invoked withinterrupts and preemption enabled.

  The callbacks are allowed to fail. When a callback fails the hotplugoperation is aborted and the CPU is brought back to the previous state.

CPU online/offline operations¶

A successful online operation looks like this:

[CPUHP_OFFLINE][CPUHP_OFFLINE + 1]->startup()       -> success[CPUHP_OFFLINE + 2]->startup()       -> success[CPUHP_OFFLINE + 3]                  -> skipped because startup == NULL...[CPUHP_BRINGUP_CPU]->startup()       -> success=== End of PREPARE section[CPUHP_BRINGUP_CPU + 1]->startup()   -> success...[CPUHP_AP_ONLINE]->startup()         -> success=== End of STARTUP section[CPUHP_AP_ONLINE + 1]->startup()     -> success...[CPUHP_ONLINE - 1]->startup()        -> success[CPUHP_ONLINE]

A successful offline operation looks like this:

[CPUHP_ONLINE][CPUHP_ONLINE - 1]->teardown()       -> success...[CPUHP_AP_ONLINE + 1]->teardown()    -> success=== Start of STARTUP section[CPUHP_AP_ONLINE]->teardown()        -> success...[CPUHP_BRINGUP_ONLINE - 1]->teardown()...=== Start of PREPARE section[CPUHP_BRINGUP_CPU]->teardown()[CPUHP_OFFLINE + 3]->teardown()[CPUHP_OFFLINE + 2]                  -> skipped because teardown == NULL[CPUHP_OFFLINE + 1]->teardown()[CPUHP_OFFLINE]

A failed online operation looks like this:

[CPUHP_OFFLINE][CPUHP_OFFLINE + 1]->startup()       -> success[CPUHP_OFFLINE + 2]->startup()       -> success[CPUHP_OFFLINE + 3]                  -> skipped because startup == NULL...[CPUHP_BRINGUP_CPU]->startup()       -> success=== End of PREPARE section[CPUHP_BRINGUP_CPU + 1]->startup()   -> success...[CPUHP_AP_ONLINE]->startup()         -> success=== End of STARTUP section[CPUHP_AP_ONLINE + 1]->startup()     -> success---[CPUHP_AP_ONLINE + N]->startup()     -> fail[CPUHP_AP_ONLINE + (N - 1)]->teardown()...[CPUHP_AP_ONLINE + 1]->teardown()=== Start of STARTUP section[CPUHP_AP_ONLINE]->teardown()...[CPUHP_BRINGUP_ONLINE - 1]->teardown()...=== Start of PREPARE section[CPUHP_BRINGUP_CPU]->teardown()[CPUHP_OFFLINE + 3]->teardown()[CPUHP_OFFLINE + 2]                  -> skipped because teardown == NULL[CPUHP_OFFLINE + 1]->teardown()[CPUHP_OFFLINE]

A failed offline operation looks like this:

[CPUHP_ONLINE][CPUHP_ONLINE - 1]->teardown()       -> success...[CPUHP_ONLINE - N]->teardown()       -> fail[CPUHP_ONLINE - (N - 1)]->startup()...[CPUHP_ONLINE - 1]->startup()[CPUHP_ONLINE]

Recursive failures cannot be handled sensibly. Look at the followingexample of a recursive fail due to a failed offline operation:

[CPUHP_ONLINE][CPUHP_ONLINE - 1]->teardown()       -> success...[CPUHP_ONLINE - N]->teardown()       -> fail[CPUHP_ONLINE - (N - 1)]->startup()  -> success[CPUHP_ONLINE - (N - 2)]->startup()  -> fail

The CPU hotplug state machine stops right here and does not try to go backdown again because that would likely result in an endless loop:

[CPUHP_ONLINE - (N - 1)]->teardown() -> success[CPUHP_ONLINE - N]->teardown()       -> fail[CPUHP_ONLINE - (N - 1)]->startup()  -> success[CPUHP_ONLINE - (N - 2)]->startup()  -> fail[CPUHP_ONLINE - (N - 1)]->teardown() -> success[CPUHP_ONLINE - N]->teardown()       -> fail

Lather, rinse and repeat. In this case the CPU left in state:

[CPUHP_ONLINE - (N - 1)]

which at least lets the system make progress and gives the user a chance todebug or even resolve the situation.

Allocating a state¶

There are two ways to allocate a CPU hotplug state:

• Static allocation

  Static allocation has to be used when the subsystem or driver hasordering requirements versus other CPU hotplug states. E.g. the PERF corestartup callback has to be invoked before the PERF driver startupcallbacks during a CPU online operation. During a CPU offline operationthe driver teardown callbacks have to be invoked before the core teardowncallback. The statically allocated states are described by constants inthe cpuhp_stateenumwhich can be found in include/linux/cpuhotplug.h.

  Insert the state into theenumat the proper place so the orderingrequirements are fulfilled. The state constant has to be used for statesetup and removal.

  Static allocation is also required when the state callbacks are not setup at runtime and are part of the initializer of the CPU hotplug statearray in kernel/cpu.c.

• Dynamic allocation

  When there are no ordering requirements for the state callbacks thendynamic allocation is the preferred method. The state number is allocatedby the setup function and returned to the caller on success.

  Only the PREPARE and ONLINE sections provide a dynamic allocationrange. The STARTING section does not as most of the callbacks in thatsection have explicit ordering requirements.

Setup of a CPU hotplug state¶

The core code provides the following functions to setup a state:

• cpuhp_setup_state(state, name, startup, teardown)

• cpuhp_setup_state_nocalls(state, name, startup, teardown)

• cpuhp_setup_state_cpuslocked(state, name, startup, teardown)

• cpuhp_setup_state_nocalls_cpuslocked(state, name, startup, teardown)

For cases where a driver or a subsystem has multiple instances and the sameCPU hotplug state callbacks need to be invoked for each instance, the CPUhotplug core provides multi-instance support. The advantage over driverspecific instance lists is that the instance related functions are fullyserialized against CPU hotplug operations and provide the automaticinvocations of the state callbacks on add and removal. To set up such amulti-instance state the following function is available:

• cpuhp_setup_state_multi(state, name, startup, teardown)

The @state argument is either a statically allocated state or one of theconstants for dynamically allocated states - CPUHP_BP_PREPARE_DYN,CPUHP_AP_ONLINE_DYN - depending on the state section (PREPARE, ONLINE) forwhich a dynamic state should be allocated.

The @name argument is used for sysfs output and for instrumentation. Thenaming convention is “subsys:mode” or “subsys/driver:mode”,e.g. “perf:mode” or “perf/x86:mode”. The common mode names are:

As the @name argument is only used for sysfs and instrumentation other modedescriptors can be used as well if they describe the nature of the statebetter than the common ones.

Examples for @name arguments: “perf/online”, “perf/x86:prepare”,“RCU/tree:dying”, “sched/waitempty”

The @startup argument is a function pointer to the callback which should beinvoked during a CPU online operation. If the usage site does not require astartup callback set the pointer to NULL.

The @teardown argument is a function pointer to the callback which shouldbe invoked during a CPU offline operation. If the usage site does notrequire a teardown callback set the pointer to NULL.

The functions differ in the way how the installed callbacks are treated:

• cpuhp_setup_state_nocalls(),cpuhp_setup_state_nocalls_cpuslocked()andcpuhp_setup_state_multi() only install the callbacks

• cpuhp_setup_state() andcpuhp_setup_state_cpuslocked() install thecallbacks and invoke the @startup callback (if not NULL) for all onlineCPUs which have currently a state greater than the newly installedstate. Depending on the state section the callback is either invoked onthe current CPU (PREPARE section) or on each online CPU (ONLINEsection) in the context of the CPU’s hotplug thread.

  If a callback fails for CPU N then the teardown callback for CPU0 .. N-1 is invoked to rollback the operation. The state setup fails,the callbacks for the state are not installed and in case of dynamicallocation the allocated state is freed.

The state setup and the callback invocations are serialized against CPUhotplug operations. If the setup function has to be called from a CPUhotplug read locked region, then the_cpuslocked() variants have to beused. These functions cannot be used from within CPU hotplug callbacks.

The function return values:

0	Statically allocated state was successfully set up
>0	Dynamically allocated state was successfully set up. The returned number is the state number which was allocated. Ifthe state callbacks have to be removed later, e.g. moduleremoval, then this number has to be saved by the caller and usedas @state argument for the state remove function. Formulti-instance states the dynamically allocated state number isalso required as @state argument for the instance add/removeoperations.
<0	Operation failed

Removal of a CPU hotplug state¶

To remove a previously set up state, the following functions are provided:

• cpuhp_remove_state(state)

• cpuhp_remove_state_nocalls(state)

• cpuhp_remove_state_nocalls_cpuslocked(state)

• cpuhp_remove_multi_state(state)

The @state argument is either a statically allocated state or the statenumber which was allocated in the dynamic range by cpuhp_setup_state*(). Ifthe state is in the dynamic range, then the state number is freed andavailable for dynamic allocation again.

The functions differ in the way how the installed callbacks are treated:

• cpuhp_remove_state_nocalls(),cpuhp_remove_state_nocalls_cpuslocked()andcpuhp_remove_multi_state() only remove the callbacks.

• cpuhp_remove_state() removes the callbacks and invokes the teardowncallback (if not NULL) for all online CPUs which have currently a stategreater than the removed state. Depending on the state section thecallback is either invoked on the current CPU (PREPARE section) or oneach online CPU (ONLINE section) in the context of the CPU’s hotplugthread.

  In order to complete the removal, the teardown callback should not fail.

The state removal and the callback invocations are serialized against CPUhotplug operations. If the remove function has to be called from a CPUhotplug read locked region, then the_cpuslocked() variants have to beused. These functions cannot be used from within CPU hotplug callbacks.

If a multi-instance state is removed then the caller has to remove allinstances first.

Multi-Instance state instance management¶

Once the multi-instance state is set up, instances can be added to thestate:

• cpuhp_state_add_instance(state, node)

• cpuhp_state_add_instance_nocalls(state, node)

The @state argument is either a statically allocated state or the statenumber which was allocated in the dynamic range bycpuhp_setup_state_multi().

The @node argument is a pointer to an hlist_node which is embedded in theinstance’s data structure. The pointer is handed to the multi-instancestate callbacks and can be used by the callback to retrieve the instanceviacontainer_of().

The functions differ in the way how the installed callbacks are treated:

• cpuhp_state_add_instance_nocalls() and only adds the instance to themulti-instance state’s node list.

• cpuhp_state_add_instance() adds the instance and invokes the startupcallback (if not NULL) associated with @state for all online CPUs whichhave currently a state greater than @state. The callback is onlyinvoked for the to be added instance. Depending on the state sectionthe callback is either invoked on the current CPU (PREPARE section) oron each online CPU (ONLINE section) in the context of the CPU’s hotplugthread.

  If a callback fails for CPU N then the teardown callback for CPU0 .. N-1 is invoked to rollback the operation, the function fails andthe instance is not added to the node list of the multi-instance state.

To remove an instance from the state’s node list these functions areavailable:

• cpuhp_state_remove_instance(state, node)

• cpuhp_state_remove_instance_nocalls(state, node)

The arguments are the same as for the cpuhp_state_add_instance*()variants above.

The functions differ in the way how the installed callbacks are treated:

• cpuhp_state_remove_instance_nocalls() only removes the instance from thestate’s node list.

• cpuhp_state_remove_instance() removes the instance and invokes theteardown callback (if not NULL) associated with @state for all onlineCPUs which have currently a state greater than @state. The callback isonly invoked for the to be removed instance. Depending on the statesection the callback is either invoked on the current CPU (PREPAREsection) or on each online CPU (ONLINE section) in the context of theCPU’s hotplug thread.

  In order to complete the removal, the teardown callback should not fail.

The node list add/remove operations and the callback invocations areserialized against CPU hotplug operations. These functions cannot be usedfrom within CPU hotplug callbacks and CPU hotplug read locked regions.

Examples¶

Setup and teardown a statically allocated state in the STARTING section fornotifications on online and offline operations:

ret = cpuhp_setup_state(CPUHP_SUBSYS_STARTING, "subsys:starting", subsys_cpu_starting, subsys_cpu_dying);if (ret < 0)     return ret;....cpuhp_remove_state(CPUHP_SUBSYS_STARTING);

Setup and teardown a dynamically allocated state in the ONLINE sectionfor notifications on offline operations:

state = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "subsys:offline", NULL, subsys_cpu_offline);if (state < 0)    return state;....cpuhp_remove_state(state);

Setup and teardown a dynamically allocated state in the ONLINE sectionfor notifications on online operations without invoking the callbacks:

state = cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, "subsys:online", subsys_cpu_online, NULL);if (state < 0)    return state;....cpuhp_remove_state_nocalls(state);

Setup, use and teardown a dynamically allocated multi-instance state in theONLINE section for notifications on online and offline operation:

state = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "subsys:online", subsys_cpu_online, subsys_cpu_offline);if (state < 0)    return state;....ret = cpuhp_state_add_instance(state, &inst1->node);if (ret)     return ret;....ret = cpuhp_state_add_instance(state, &inst2->node);if (ret)     return ret;....cpuhp_remove_instance(state, &inst1->node);....cpuhp_remove_instance(state, &inst2->node);....cpuhp_remove_multi_state(state);