Never Omit Scheduling-Clock Ticks¶

Very old versions of Linux from the 1990s and the very early 2000sare incapable of omitting scheduling-clock ticks. It turns out thatthere are some situations where this old-school approach is still theright approach, for example, in heavy workloads with lots of tasksthat use short bursts of CPU, where there are very frequent idleperiods, but where these idle periods are also quite short (tens orhundreds of microseconds). For these types of workloads, schedulingclock interrupts will normally be delivered any way because therewill frequently be multiple runnable tasks per CPU. In these cases,attempting to turn off the scheduling clock interrupt will have no effectother than increasing the overhead of switching to and from idle andtransitioning between user and kernel execution.

This mode of operation can be selected using CONFIG_HZ_PERIODIC=y (orCONFIG_NO_HZ=n for older kernels).

However, if you are instead running a light workload with long idleperiods, failing to omit scheduling-clock interrupts will result inexcessive power consumption. This is especially bad on battery-powereddevices, where it results in extremely short battery lifetimes. If youare running light workloads, you should therefore read the followingsection.

In addition, if you are running either a real-time workload or an HPCworkload with short iterations, the scheduling-clock interrupts candegrade your applications performance. If this describes your workload,you should read the following two sections.

Omit Scheduling-Clock Ticks For Idle CPUs¶

If a CPU is idle, there is little point in sending it a scheduling-clockinterrupt. After all, the primary purpose of a scheduling-clock interruptis to force a busy CPU to shift its attention among multiple duties,and an idle CPU has no duties to shift its attention among.

The CONFIG_NO_HZ_IDLE=y Kconfig option causes the kernel to avoid sendingscheduling-clock interrupts to idle CPUs, which is critically importantboth to battery-powered devices and to highly virtualized mainframes.A battery-powered device running a CONFIG_HZ_PERIODIC=y kernel woulddrain its battery very quickly, easily 2-3 times as fast as would thesame device running a CONFIG_NO_HZ_IDLE=y kernel. A mainframe running1,500 OS instances might find that half of its CPU time was consumed byunnecessary scheduling-clock interrupts. In these situations, thereis strong motivation to avoid sending scheduling-clock interrupts toidle CPUs. That said, dyntick-idle mode is not free:

1. It increases the number of instructions executed on the pathto and from the idle loop.
2. On many architectures, dyntick-idle mode also increases thenumber of expensive clock-reprogramming operations.

Therefore, systems with aggressive real-time response constraints oftenrun CONFIG_HZ_PERIODIC=y kernels (or CONFIG_NO_HZ=n for older kernels)in order to avoid degrading from-idle transition latencies.

An idle CPU that is not receiving scheduling-clock interrupts is said tobe “dyntick-idle”, “in dyntick-idle mode”, “in nohz mode”, or “runningtickless”. The remainder of this document will use “dyntick-idle mode”.

There is also a boot parameter “nohz=” that can be used to disabledyntick-idle mode in CONFIG_NO_HZ_IDLE=y kernels by specifying “nohz=off”.By default, CONFIG_NO_HZ_IDLE=y kernels boot with “nohz=on”, enablingdyntick-idle mode.

Omit Scheduling-Clock Ticks For CPUs With Only One Runnable Task¶

If a CPU has only one runnable task, there is little point in sending ita scheduling-clock interrupt because there is no other task to switch to.Note that omitting scheduling-clock ticks for CPUs with only one runnabletask implies also omitting them for idle CPUs.

The CONFIG_NO_HZ_FULL=y Kconfig option causes the kernel to avoidsending scheduling-clock interrupts to CPUs with a single runnable task,and such CPUs are said to be “adaptive-ticks CPUs”. This is importantfor applications with aggressive real-time response constraints becauseit allows them to improve their worst-case response times by the maximumduration of a scheduling-clock interrupt. It is also important forcomputationally intensive short-iteration workloads: If any CPU isdelayed during a given iteration, all the other CPUs will be forced towait idle while the delayed CPU finishes. Thus, the delay is multipliedby one less than the number of CPUs. In these situations, there isagain strong motivation to avoid sending scheduling-clock interrupts.

By default, no CPU will be an adaptive-ticks CPU. The “nohz_full=”boot parameter specifies the adaptive-ticks CPUs. For example,“nohz_full=1,6-8” says that CPUs 1, 6, 7, and 8 are to be adaptive-ticksCPUs. Note that you are prohibited from marking all of the CPUs asadaptive-tick CPUs: At least one non-adaptive-tick CPU must remainonline to handle timekeeping tasks in order to ensure that systemcalls like gettimeofday() returns accurate values on adaptive-tick CPUs.(This is not an issue for CONFIG_NO_HZ_IDLE=y because there are no runninguser processes to observe slight drifts in clock rate.) Therefore, theboot CPU is prohibited from entering adaptive-ticks mode. Specifying a“nohz_full=” mask that includes the boot CPU will result in a boot-timeerror message, and the boot CPU will be removed from the mask. Note thatthis means that your system must have at least two CPUs in order forCONFIG_NO_HZ_FULL=y to do anything for you.

Finally, adaptive-ticks CPUs must have their RCU callbacks offloaded.This is covered in the “RCU IMPLICATIONS” section below.

Normally, a CPU remains in adaptive-ticks mode as long as possible.In particular, transitioning to kernel mode does not automatically changethe mode. Instead, the CPU will exit adaptive-ticks mode only if needed,for example, if that CPU enqueues an RCU callback.

Just as with dyntick-idle mode, the benefits of adaptive-tick mode donot come for free:

1. CONFIG_NO_HZ_FULL selects CONFIG_NO_HZ_COMMON, so you cannot runadaptive ticks without also running dyntick idle. This dependencyextends down into the implementation, so that all of the costsof CONFIG_NO_HZ_IDLE are also incurred by CONFIG_NO_HZ_FULL.
2. The user/kernel transitions are slightly more expensive dueto the need to inform kernel subsystems (such as RCU) aboutthe change in mode.
3. POSIX CPU timers prevent CPUs from entering adaptive-tick mode.Real-time applications needing to take actions based on CPU timeconsumption need to use other means of doing so.
4. If there are more perf events pending than the hardware canaccommodate, they are normally round-robined so as to collectall of them over time. Adaptive-tick mode may prevent thisround-robining from happening. This will likely be fixed bypreventing CPUs with large numbers of perf events pending fromentering adaptive-tick mode.
5. Scheduler statistics for adaptive-tick CPUs may be computedslightly differently than those for non-adaptive-tick CPUs.This might in turn perturb load-balancing of real-time tasks.

Although improvements are expected over time, adaptive ticks is quiteuseful for many types of real-time and compute-intensive applications.However, the drawbacks listed above mean that adaptive ticks should not(yet) be enabled by default.

RCU Implications¶

There are situations in which idle CPUs cannot be permitted toenter either dyntick-idle mode or adaptive-tick mode, the mostcommon being when that CPU has RCU callbacks pending.

The CONFIG_RCU_FAST_NO_HZ=y Kconfig option may be used to cause such CPUsto enter dyntick-idle mode or adaptive-tick mode anyway. In this case,a timer will awaken these CPUs every four jiffies in order to ensurethat the RCU callbacks are processed in a timely fashion.

Another approach is to offload RCU callback processing to “rcuo” kthreadsusing the CONFIG_RCU_NOCB_CPU=y Kconfig option. The specific CPUs tooffload may be selected using The “rcu_nocbs=” kernel boot parameter,which takes a comma-separated list of CPUs and CPU ranges, for example,“1,3-5” selects CPUs 1, 3, 4, and 5.

The offloaded CPUs will never queue RCU callbacks, and therefore RCUnever prevents offloaded CPUs from entering either dyntick-idle modeor adaptive-tick mode. That said, note that it is up to userspace topin the “rcuo” kthreads to specific CPUs if desired. Otherwise, thescheduler will decide where to run them, which might or might not bewhere you want them to run.

Testing¶

So you enable all the OS-jitter features described in this document,but do not see any change in your workload’s behavior. Is this becauseyour workload isn’t affected that much by OS jitter, or is it becausesomething else is in the way? This section helps answer this questionby providing a simple OS-jitter test suite, which is available on branchmaster of the following git archive:

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/dynticks-testing.git

Clone this archive and follow the instructions in the README file.This test procedure will produce a trace that will allow you to evaluatewhether or not you have succeeded in removing OS jitter from your system.If this trace shows that you have removed OS jitter as much as ispossible, then you can conclude that your workload is not all thatsensitive to OS jitter.

Note: this test requires that your system have at least two CPUs.We do not currently have a good way to remove OS jitter from single-CPUsystems.

Known Issues¶

• Dyntick-idle slows transitions to and from idle slightly.In practice, this has not been a problem except for the mostaggressive real-time workloads, which have the option of disablingdyntick-idle mode, an option that most of them take. However,some workloads will no doubt want to use adaptive ticks toeliminate scheduling-clock interrupt latencies. Here are someoptions for these workloads:

  1. Use PMQOS from userspace to inform the kernel of yourlatency requirements (preferred).
  2. On x86 systems, use the “idle=mwait” boot parameter.

  c. On x86 systems, use the “intel_idle.max_cstate=” to limit` the maximum C-state depth.

  4. On x86 systems, use the “idle=poll” boot parameter.However, please note that use of this parameter can causeyour CPU to overheat, which may cause thermal throttlingto degrade your latencies – and that this degradation canbe even worse than that of dyntick-idle. Furthermore,this parameter effectively disables Turbo Mode on IntelCPUs, which can significantly reduce maximum performance.

• Adaptive-ticks slows user/kernel transitions slightly.This is not expected to be a problem for computationally intensiveworkloads, which have few such transitions. Careful benchmarkingwill be required to determine whether or not other workloadsare significantly affected by this effect.

• Adaptive-ticks does not do anything unless there is only onerunnable task for a given CPU, even though there are a numberof other situations where the scheduling-clock tick is notneeded. To give but one example, consider a CPU that has onerunnable high-priority SCHED_FIFO task and an arbitrary numberof low-priority SCHED_OTHER tasks. In this case, the CPU isrequired to run the SCHED_FIFO task until it either blocks orsome other higher-priority task awakens on (or is assigned to)this CPU, so there is no point in sending a scheduling-clockinterrupt to this CPU. However, the current implementationnevertheless sends scheduling-clock interrupts to CPUs having asingle runnable SCHED_FIFO task and multiple runnable SCHED_OTHERtasks, even though these interrupts are unnecessary.

  And even when there are multiple runnable tasks on a given CPU,there is little point in interrupting that CPU until the currentrunning task’s timeslice expires, which is almost always waylonger than the time of the next scheduling-clock interrupt.

  Better handling of these sorts of situations is future work.

• A reboot is required to reconfigure both adaptive idle and RCUcallback offloading. Runtime reconfiguration could be providedif needed, however, due to the complexity of reconfiguring RCU atruntime, there would need to be an earthshakingly good reason.Especially given that you have the straightforward option ofsimply offloading RCU callbacks from all CPUs and pinning themwhere you want them whenever you want them pinned.

• Additional configuration is required to deal with other sourcesof OS jitter, including interrupts and system-utility tasksand processes. This configuration normally involves bindinginterrupts and tasks to particular CPUs.

• Some sources of OS jitter can currently be eliminated only byconstraining the workload. For example, the only way to eliminateOS jitter due to global TLB shootdowns is to avoid the unmappingoperations (such as kernel module unload operations) thatresult in these shootdowns. For another example, page faultsand TLB misses can be reduced (and in some cases eliminated) byusing huge pages and by constraining the amount of memory usedby the application. Pre-faulting the working set can also behelpful, especially when combined with the mlock() and mlockall()system calls.

• Unless all CPUs are idle, at least one CPU must keep thescheduling-clock interrupt going in order to support accuratetimekeeping.

• If there might potentially be some adaptive-ticks CPUs, therewill be at least one CPU keeping the scheduling-clock interruptgoing, even if all CPUs are otherwise idle.

  Better handling of this situation is ongoing work.

• Some process-handling operations still require the occasionalscheduling-clock tick. These operations include calculating CPUload, maintaining sched average, computing CFS entity vruntime,computing avenrun, and carrying out load balancing. They arecurrently accommodated by scheduling-clock tick every secondor so. On-going work will eliminate the need even for theseinfrequent scheduling-clock ticks.