Short subjects: Realtime, Futexes, and ntfs3

Benefits for LWN subscribers

The primary benefit fromsubscribing to LWN is helping to keep us publishing, but, beyond that, subscribers get immediate access to all site content and access to a number of extra site features. Please sign up today!

Realtime

The realtime preemption story is a long one; it firstshowed up on LWN in 2004. Over the years,this work has had a significant impact on kernel development as a whole;much of what is just seen as part of the core kernel now had its origins inthe realtime tree. The code around which the realtime work was initially built — thepreemptible locking infrastructure — remains out of the mainline, though.Without the locking changes, the mainline is not able to offer the sort ofresponse-time guarantees that realtime users need.

The locking infrastructure makes almost all locks, spinlocks included, intosleeping locks; that ensures that a higher-priority task can always takeover the processor quickly. It is the sort of change that makes kerneldevelopers nervous, since mistakes in this area can lead to all sorts ofsubtle problems. For that reason, predicting when the locking code will bemerged into the mainline is a fool's game. Your editor knows this well,having confidentlypredicted that it wouldbe merged within a year — in 2007.

Still, one might be tempted to think that the end might be getting closer.Realtime developer Thomas Gleixner has brought the locking infrastructureback to the mailing lists for consideration;the fifthrevision of the 72-part patch set was posted on August 15.Normally configured kernels should behave about the same with these patchesapplied, but those configured for realtime operation will haverealtime-specific versions of mutexes, wait/wound mutexes, reader/writersemaphores, spinlocks, and reader/writer locks.

Commentary on this work has slowed; there does not appear to be much in theway of objections at this point — though it must be noted that LinusTorvalds has not yet made his feelings known on the subject. Unlesssomething surprising comes up, it might just be that the core realtime codewill finally find its way into the mainline. Your editor, however, is tooold, wise, and cowardly to venture a guess as to when that will happen.

A smaller step for futex2

Perhaps the number of comments on the realtime changes is low because mostdevelopers fear the prospect of digging into code of that complexity.There are, however, places in the kernel that are even more frightening;the futex subsystem is surely one of them. Futexes provide fast mutexesfor user space; they started out as a simple subsystem but failed to remainthat way. Over time, it has become clear that futexes could do with anumber of improvements to make them better suited for current workloadsand, at the same time, to move beyond the multiplexerfutex()system call.

For some time now, André Almeida has been pushing in that direction withthefutex2 proposal. This work would splitthe futex functionality into several single-purpose system calls, supportmultiple lock sizes, and more. While there has been interest in this work,progress has been slow (to put it charitably); it seems as if the kernel isno closer to a new futex subsystem than it was a year or two ago.

In an attempt to push this project forward, Almeida has postedanew patch set with significantly reduced ambitions. Rather thanintroduce a whole new subsystem with its own system calls, this series addsexactly one system call that works with existing futexes:

    struct futex_waitv {        uint64_t val;        void *uaddr;        unsigned int flags;    };    int futex_waitv(struct futex_waitv *waiters, unsigned int nr_futexes,                    unsigned int flags, struct timespec *timo);

This function will cause the calling process to wait on several futexessimultaneously, returning when one or more of them can be acquired (or thetimeout expires). That functionality is not supported by the current futexAPI, but it turns out to be especially useful for game engines, whichperform significantly better when using the new system call.Thisdocumentation patch describes the new API in more detail.

This patch set has drawn no comments in the week since it was posted.Assuming that silence implies a lack of objections rather than a lack ofinterest, this piece of the futex2 work might make it into a mainlinerelease before too long. Whether the rest of the futex2 work will followdepends on how strong the use cases driving it are; iffutex_waitv() solves the worst problems, there might not be muchmotivation to push the other changes.

Waiting for ntfs3

The kernel has long had an implementation of the NTFS filesystem, but ithas always suffered from performance and functionality problems; the usercommunity would gladly trade it for something better. By all accounts, thentfs3 implementation posted by Konstantin Komarov is indeed somethingbetter, but it is still not clear when it will be merged; this work wasfirstposted one year ago, andversion 27of the patch set was posted on July 29.

The delay in accepting this work is proving frustrating to users;thiscomplaint from Neal Gompa is typical:

I know that compared to all you awesome folks, I'm just a lowlyuser, but it's been frustrating to see nothing happen for monthswith something that has a seriously high impact for a lot ofpeople.

It's a shame, because the ntfs3 driver is miles better than thecurrent ntfs one, and is a solid replacement for the unmaintainedntfs-3g FUSE implementation.

Torvalds hassaidthat maybe it is time to merge this code, but that still may not happenright away.

The biggest holdup for ntfs3 at the moment would appear to be concernsabout the level of development effort behind it. From the public evidence,it seems that ntfs3 is a one-person project, and that makes otherfilesystem developers nervous. Those developers have been reporting testfailures for ntfs3 that have gone unfixed.Meanwhile,Komarov is sometimes unresponsive to questions; various comments on theversion 26posting (from early April) got no answers, for example. This sort of silencegives the impression thatntfs3 does not have a lot of effort behind it. (It's worth noting thatsome other developershave beenhappy with the level of response from Komarov).

Unsurprisingly, the filesystem developers are unenthusiastic about theprospect of taking on a new NTFS implementation that may turn out to haveserious problems and which does not come with a promise of reliablesupport. For ntfs3 to be merged, those fears will need to be addressedsomehow. One way for that to happen, assuggested by Ted Ts'o,would be for other developers,perhaps representing one or more distributors that would like to see abetter NTFS implementation in the kernel, to start contributing patches tontfs3 and commit to helping with its maintenance going forward.