Example 1: softirq Suicide¶

Suppose that an RCU-based algorithm scans a linked list containingelements A, B, and C in process context, and can delete elements fromthis same list in softirq context. Suppose that the process-context scanis referencing element B when it is interrupted by softirq processing,which deletes element B, and then invokescall_rcu() to free element Bafter a grace period.

Now, ifcall_rcu() were to directly invoke its arguments, then upon returnfrom softirq, the list scan would find itself referencing a newly freedelement B. This situation can greatly decrease the life expectancy ofyour kernel.

This same problem can occur ifcall_rcu() is invoked from a hardwareinterrupt handler.

Example 2: Function-Call Fatality¶

Of course, one could avert the suicide described in the preceding exampleby havingcall_rcu() directly invoke its arguments only if it was calledfrom process context. However, this can fail in a similar manner.

Suppose that an RCU-based algorithm again scans a linked list containingelements A, B, and C in process contexts, but that it invokes a functionon each element as it is scanned. Suppose further that this functiondeletes element B from the list, then passes it tocall_rcu() for deferredfreeing. This may be a bit unconventional, but it is perfectly legalRCU usage, sincecall_rcu() must wait for a grace period to elapse.Therefore, in this case, allowingcall_rcu() to immediately invokeits arguments would cause it to fail to make the fundamental guaranteeunderlying RCU, namely thatcall_rcu() defers invoking its arguments untilall RCU read-side critical sections currently executing have completed.

Quick Quiz #1:

Why is itnot legal to invokesynchronize_rcu() in this case?

Answers to Quick Quiz

Example 3: Death by Deadlock¶

Suppose thatcall_rcu() is invoked while holding a lock, and that thecallback function must acquire this same lock. In this case, ifcall_rcu() were to directly invoke the callback, the result wouldbe self-deadlock.

In some cases, it would possible to restructure to code so thatthecall_rcu() is delayed until after the lock is released. However,there are cases where this can be quite ugly:

1. If a number of items need to be passed tocall_rcu() withinthe same critical section, then the code would need to createa list of them, then traverse the list once the lock wasreleased.
2. In some cases, the lock will be held across some kernel API,so that delaying thecall_rcu() until the lock is releasedrequires that the data item be passed up via a common API.It is far better to guarantee that callbacks are invokedwith no locks held than to have to modify such APIs to allowarbitrary data items to be passed back up through them.

Ifcall_rcu() directly invokes the callback, painful locking restrictionsor API changes would be required.

Quick Quiz #2:

What locking restriction must RCU callbacks respect?

Answers to Quick Quiz

Summary¶

Permittingcall_rcu() to immediately invoke its arguments breaks RCU,even on a UP system. So do not do it! Even on a UP system, the RCUinfrastructuremust respect grace periods, andmust invoke callbacksfrom a known environment in which no locks are held.

Note that itis safe forsynchronize_rcu() to return immediately onUP systems, including PREEMPT SMP builds running on UP systems.

Quick Quiz #3:

Why can’tsynchronize_rcu() return immediately on UP systems runningpreemptable RCU?

Answer to Quick Quiz #1:

Why is itnot legal to invokesynchronize_rcu() in this case?

Because the calling function is scanning an RCU-protected linkedlist, and is therefore within an RCU read-side critical section.Therefore, the called function has been invoked within an RCUread-side critical section, and is not permitted to block.

Answer to Quick Quiz #2:

What locking restriction must RCU callbacks respect?

Any lock that is acquired within an RCU callback must be acquiredelsewhere using an _bh variant of the spinlock primitive.For example, if “mylock” is acquired by an RCU callback, thena process-context acquisition of this lock must use somethinglike spin_lock_bh() to acquire the lock. Please note thatit is also OK to use _irq variants of spinlocks, for example,spin_lock_irqsave().

If the process-context code were to simply use spin_lock(),then, since RCU callbacks can be invoked from softirq context,the callback might be called from a softirq that interruptedthe process-context critical section. This would result inself-deadlock.

This restriction might seem gratuitous, since very few RCUcallbacks acquire locks directly. However, a great many RCUcallbacks do acquire locksindirectly, for example, viathekfree() primitive.

Answer to Quick Quiz #3:

Why can’tsynchronize_rcu() return immediately on UP systemsrunning preemptable RCU?

Because some other task might have been preempted in the middleof an RCU read-side critical section. Ifsynchronize_rcu()simply immediately returned, it would prematurely signal theend of the grace period, which would come as a nasty shock tothat other thread when it started running again.