Summary

Introducescoped values, which enable a method to share immutable data both with its callees within a thread, and with child threads. Scoped values are easier to reason about than thread-local variables. They also have lower space and time costs, especially when used together with virtual threads (JEP 444) and structured concurrency (JEP 480). This is apreview API.

History

The scoped values API incubated in JDK 20 viaJEP 429, became a preview API in JDK 21 viaJEP 446, and was re-previewed in JDK 22 viaJEP 464.

We here propose to re-preview the API in JDK 23 in order to gain additional experience andfeedback, with one change:

• The type of the operation parameter of theScopedValue.callWhere method is now a new functional interface which allows the Java compiler to infer whether a checked exception might be thrown. With this change, theScopedValue.getWhere method is no longer needed and is removed.

Goals

• Ease of use — It should be easy to reason about dataflow.

• Comprehensibility — The lifetime of shared data should be apparent from the syntactic structure of code.

• Robustness — Data shared by a caller should be retrievable only by legitimate callees.

• Performance — Data should be efficiently sharable across a large number of threads.

Non-Goals

• It is not a goal to change the Java programming language.

• It is not a goal to require migration away from thread-local variables, or to deprecate the existingThreadLocal API.

Motivation

Java applications and libraries are structured as collections of classes which contain methods. These methods communicate through method calls.

Most methods allow a caller to pass data to a method by passing the data as parameters. When methodA wants methodB to do some work for it, it invokesB with the appropriate parameters, andB may pass some of those parameters toC, and so forth.B may have to include in its parameter list not only the thingsB directly needs but also the thingsB has to pass toC. For example, ifB is going to set up and execute a database call, it might want a Connection passed in, even ifB is not going to use the Connection directly.

Most of the time this "pass what your indirect callees need" approach is the most effective and convenient way to share data. However, sometimes it is impractical to pass all the data that every indirect callee might need in the initial call.

An example

It is a common pattern in large Java programs to transfer control from one component (a "framework") to another ("application code") and then back. For example, a web framework could accept incoming HTTP requests and then call an application handler to handle it. The application handler may then call the framework to read data from the database or to call some other HTTP service.

@Overridepublic void handle(Request request, Response response) { // user code; called by framework    ...    var userInfo = readUserInfo();    ...}private UserInfo readUserInfo() {    return (UserInfo)framework.readKey("userInfo", context);// call framework}

The framework may maintain aFrameworkContext object, containing the authenticated user ID, the transaction ID, etc., and associate it with the current transaction. All framework operations use theFrameworkContext object, but it's unused by (and irrelevant to) user code.

In effect, the framework must be able to communicate its internal context from itsserve method (which calls the user'shandle method) to itsreadKey method:

4. Framework.readKey <--------+ use context3. Application.readUserInfo   |2. Application.handle         |1. Framework.serve  ----------+ create context

The simplest way to do this is by passing the object as an argument to all methods in the call chain:

@Overridevoid handle(Request request, Response response, FrameworkContext context) {    ...    var userInfo = readUserInfo(context);    ...}private UserInfo readUserInfo(FrameworkContext context) {    return (UserInfo)framework.readKey("userInfo", context);}

There is no way for the user code to assist in the proper handling of the context object. At worst, it could interfere by mixing up contexts; at best it is burdened with the need to add another parameter to all methods that may end up calling back into the framework. If the need to pass a context emerges during redesign of the framework, adding it requires not only the immediate clients — those user methods that directly call framework methods or those that are directly called by it — to change their signature, but all intermediate methods as well, even though the context is an internal implementation detail of the framework and user code should not interact with it.

Thread-local variables for sharing

Developers have traditionally usedthread-local variables, introduced in Java 1.2, to help share data between methods on the call stack without resorting to method parameters. A thread-local variable is a variable of typeThreadLocal. Despite looking like an ordinary variable, a thread-local variable has one current value per thread; the particular value that is used depends on which thread calls itsget orset methods to read or write its value. Typically, a thread-local variable is declared as afinalstatic field and its accessibility is set toprivate, allowing sharing to be restricted to instances of a single class or group of classes from a single code base.

Here is an example of how the two framework methods, both running in the same request-handling thread, can use a thread-local variable to share aFrameworkContext. The framework declares a thread-local variable,CONTEXT (1). WhenFramework.serve is executed in a request-handling thread, it writes a suitableFrameworkContext to the thread-local variable (2), then calls user code. If and when user code callsFramework.readKey, that method reads the thread-local variable (3) to obtain theFrameworkContext of the request-handling thread.

public class Framework {    private final Application application;    public Framework(Application app) { this.application = app; }        private final static ThreadLocal<FrameworkContext> CONTEXT                        = new ThreadLocal<>();    // (1)    void serve(Request request, Response response) {        var context = createContext(request);        CONTEXT.set(context);                    // (2)        Application.handle(request, response);    }    public PersistedObject readKey(String key) {        var context = CONTEXT.get();              // (3)        var db = getDBConnection(context);        db.readKey(key);    }}

Using a thread-local variable avoids the need to pass aFrameworkContext as a method argument when the framework calls user code, and when user code calls a framework method back. The thread-local variable serves as a hidden method parameter: A thread that callsCONTEXT.set inFramework.serve and thenCONTEXT.get inFramework.readKey will automatically see its own local copy of theCONTEXT variable. In effect, theThreadLocal field serves as a key that is used to look up aFrameworkContext value for the current thread.

WhileThreadLocals have a distinct value set in each thread, the value that is currently set in one thread can be automatically inherited by another thread that the current thread creates by using theInheritableThreadLocal class rather than theThreadLocal class.

Problems with thread-local variables

Unfortunately, thread-local variables have three inherent design flaws.

• Unconstrained mutability — Every thread-local variable is mutable: Any code that can call theget method of a thread-local variable can call theset method of that variable at any time. This is still true even if an object in a thread-local variable is immutable due to every one of its fields being declared final. TheThreadLocal API allows this in order to support a fully general model of communication, where data can flow in any direction between methods. This can lead to spaghetti-like data flow, and to programs in which it is hard to discern which method updates shared state and in what order. The more common need, shown in the example above, is a simple one-way transmission of data from one method to others.

• Unbounded lifetime — Once a thread's copy of a thread-local variable is set via theset method, the value to which it was set is retained for the lifetime of the thread, or until code in the thread calls theremove method. Unfortunately, developers often forget to callremove, so per-thread data is often retained for longer than necessary. In particular, if a thread pool is used, the value of a thread-local variable set in one task could, if not properly cleared, accidentally leak into an unrelated task, potentially leading to dangerous security vulnerabilities. In addition, for programs that rely on the unconstrained mutability of thread-local variables, there may be no clear point at which it is safe for a thread to callremove; this can cause a long-term memory leak, since per-thread data will not be garbage-collected until the thread exits. It would be better if the writing and reading of per-thread data occurred in a bounded period during execution of the thread, avoiding the possibility of leaks.

• Expensive inheritance — The overhead of thread-local variables may be worse when using large numbers of threads, because thread-local variables of a parent thread can be inherited by child threads. (A thread-local variable is not, in fact, local to one thread.) When a developer chooses to create a child thread that inherits thread-local variables, the child thread has to allocate storage for every thread-local variable previously written in the parent thread. This can add significant memory footprint. Child threads cannot share the storage used by the parent thread because theThreadLocal API requires that changing a thread's copy of the thread-local variable is not seen in other threads. This is unfortunate, because in practice child threads rarely call theset method on their inherited thread-local variables.

Toward lightweight sharing

The problems of thread-local variables have become more pressing with the availability of virtual threads (JEP 444). Virtual threads are lightweight threads implemented by the JDK. Many virtual threads share the same operating system thread, allowing for very large numbers of virtual threads. In addition to being plentiful, virtual threads are cheap enough to represent any concurrent unit of behavior. This means that a web framework can dedicate a new virtual thread to the task of handling a request and still be able to process thousands or millions of requests at once. In the ongoing example, the methodsFramework.serve,Application.handle, andFramework.readKey would all execute in a new virtual thread for each incoming request.

It would be useful for these methods to be able to share data whether they execute in virtual threads or traditional platform threads. Because virtual threads are instances ofThread, a virtual thread can have thread-local variables; in fact, the short-lived,non-pooled nature of virtual threads makes the problem of long-term memory leaks, mentioned above, less acute. (Calling a thread-local variable'sremove method is unnecessary when a thread terminates quickly, since termination automatically removes its thread-local variables.) However, if each of a million virtual threads has its own copy of thread-local variables, the memory footprint may be significant.

In summary, thread-local variables have more complexity than is usually needed for sharing data, and significant costs that cannot be avoided. The Java Platform should provide a way to maintain inheritable per-thread data for thousands or millions of virtual threads. If these per-thread variables were immutable, their data could be shared by child threads efficiently. Further, the lifetime of these per-thread variables should be bounded: Any data shared via a per-thread variable should become unusable once the method that initially shared the data is finished.

Description

Ascoped value is a container object that allows a data value to be safely and efficiently shared by a method with its direct and indirect callees within the same thread, and with child threads, without resorting to method parameters. It is a variable of typeScopedValue. It is typically declared as afinalstatic field, and its accessibility is set toprivate so that it cannot be directly accessed by code in other classes.

Like a thread-local variable, a scoped value has multiple values associated with it, one per thread. The particular value that is used depends on which thread calls its methods. Unlike a thread-local variable, a scoped value is written once, and is available only for a bounded period during execution of the thread.

A scoped value is used as shown below. Some code callsScopedValue.runWhere, presenting a scoped value and the object to which it is to be bound. The call torunWherebinds the scoped value, providing a copy that is specific to the current thread, and then executes the lambda expression passed as an argument. During the lifetime of therunWhere call, the lambda expression, or any method called directly or indirectly from that expression, can read the scoped value via the value’sget method. After therunWhere method finishes, the binding is destroyed.

final static ScopedValue<...> NAME = ScopedValue.newInstance();// In some methodScopedValue.runWhere(NAME, <value>,    () -> { ... NAME.get() ... call methods ... });// In a method called directly or indirectly from the lambda expression... NAME.get() ...

The structure of the code delineates the period of time when a thread can read its copy of a scoped value. This bounded lifetime greatly simplifies reasoning about thread behavior. The one-way transmission of data from caller to callees — both direct and indirect — is obvious at a glance. There is noset method that lets faraway code change the scoped value at any time. This also helps performance: Reading a scoped value withget is often as fast as reading a local variable, regardless of the stack distance between caller and callee.

The meaning of "scoped"

Thescope of a thing is the space in which it lives — the extent or range in which it can be used. For example, in the Java programming language, the scope of a variable declaration is the space within the program text where it is legal to refer to the variable with a simple name (JLS §6.3). This kind of scope is more accurately calledlexical scope orstatic scope, since the space where the variable is in scope can beunderstood statically by looking for{ and} characters in the program text.

Another kind of scope is calleddynamic scope. The dynamic scope of a thing refers to the parts of a program that can use the thing as the program executes. If methoda calls methodb that, in turn, calls methodc, the execution lifetime ofc is contained within the execution ofb, which is contained in that ofa, even though the three methods are distinct code units:

|  |   +–– a  |   |  |   |  +–– b  |   |  |TIME  |  |  +–– c  |   |  |  |  |   |  |  |__  |   |  |  |   |  |__  |   |  |   |__  |  v

This is the concept to whichscoped value appeals, because binding a scoped value V in arunWhere method produces a value that is accessible by certain parts of the program as it executes, namely the methods invoked directly or indirectly byrunWhere.

The unfolding execution of those methods defines a dynamic scope; the binding is in scope during the execution of those methods, and nowhere else.

Web framework example with scoped values

The framework code shown earlier can easily be rewritten to use a scoped value instead of a thread-local variable. At (1), the framework declares a scoped value instead of a thread-local variable. At (2), the serve method callsScopedValue.runWhere instead of a thread-local variable'sset method.

class Framework {    private final static ScopedValue<FrameworkContext> CONTEXT                        = ScopedValue.newInstance();    // (1)    void serve(Request request, Response response) {        var context = createContext(request);        ScopedValue.runWhere(CONTEXT, context,          // (2)                   () -> Application.handle(request, response));    }        public PersistedObject readKey(String key) {        var context = CONTEXT.get();                    // (3)        var db = getDBConnection(context);        db.readKey(key);    }}

TherunWhere method provides one-way sharing of data from theserve method to thereadKey method. The scoped value passed torunWhere is bound to the corresponding object for the lifetime of therunWhere call, soCONTEXT.get() in any method called fromrunWhere will read that value. Accordingly, whenFramework.serve calls user code, and user code callsFramework.readKey, the value read from the scoped value (3) is the value written byFramework.serve earlier in the thread.

The binding established byrunWhere is usable only in code called fromrunWhere. IfCONTEXT.get() appeared inFramework.serve after the call torunWhere, an exception would be thrown becauseCONTEXT is no longer bound in the thread.

As before, the framework relies on Java's access control to restrict access to its internal data: TheCONTEXT field has private access, which allows the framework to share information internally between its two methods. That information is inaccessible to, and hidden from, user code. We say that theScopedValue object is acapability object that gives code with permissions to access it the ability to bind or read the value. Often theScopedValue will haveprivate access, but sometimes it may haveprotected or package access to allow multiple cooperating classes to read and bind the value.

Rebinding scoped values

That scoped values have noset method means that a caller can use a scoped value to reliably communicate a value to its callees in the same thread. However, there are occasions when one of its callees might need to use the same scoped value to communicate a different value to its own callees. TheScopedValue API allows a new nested binding to be established for subsequent calls:

private static final ScopedValue<String> X = ScopedValue.newInstance();void foo() {    ScopedValue.runWhere(X, "hello", () -> bar());}void bar() {    System.out.println(X.get()); // prints hello    ScopedValue.runWhere(X, "goodbye", () -> baz());    System.out.println(X.get()); // prints hello}void baz() {    System.out.println(X.get()); // prints goodbye}

bar reads the value ofX to be"hello", as that is the binding in the scope established infoo. But thenbar establishes a nested scope to runbaz whereX is bound togoodbye.

Notice how the"goodbye" binding is in effect only inside the nested scope. Oncebaz returns, the value ofX insidebar reverts to '"hello"'. The body ofbar cannot change the binding seen by that method itself but can change the binding seen by its callees. Afterfoo exits,X reverts to being unbound. This nesting guarantees a bounded lifetime for sharing of the new value.

Inheriting scoped values

The web framework example dedicates a thread to handling each request, so the same thread executes some framework code, then user code from the application developer, then more framework code to access the database. However, user code can exploit the lightweight nature of virtual threads by creating its own virtual threads and running its own code in them. These virtual threads will be child threads of the request-handling thread.

Context data shared by a code running in the request-handling thread needs to be available to code running in child threads. Otherwise, when user code running in a child thread calls a framework method it will be unable to access theFrameworkContext created by the framework code running in the request-handling thread. To enable cross-thread sharing, scoped values can be inherited by child threads.

The preferred mechanism for user code to create virtual threads is the Structured Concurrency API (JEP 480), specifically the classStructuredTaskScope. Scoped values in the parent thread are automatically inherited by child threads created withStructuredTaskScope. Code in a child thread can use bindings established for a scoped value in the parent thread with minimal overhead. Unlike with thread-local variables, there is no copying of a parent thread's scoped value bindings to the child thread.

Here is an example of scoped value inheritance occurring behind the scenes in user code. TheServer.serve method bindsCONTEXT and callsApplication.handle just as before. However, the user code inApplication.handle calls run thereadUserInfo andfetchOffers methods concurrently, each in its own virtual threads, usingStructuredTaskScope.fork (1, 2). Each method may useFramework.readKey which, as before, consults the scoped valueCONTEXT (4). Further details of the user code are not discussed here; seeJEP 480 for further information.

@Overridepublic Response handle(Request request, Response response) {      try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {          Supplier<UserInfo>    user   = scope.fork(() -> readUserInfo());  // (1)          Supplier<List<Offer>> offers = scope.fork(() -> fetchOffers());   // (2)          scope.join().throwIfFailed();  // Wait for both forks          return new Response(user.get(), order.get());      } catch (Exception ex) {          reportError(response, ex);      }}

StructuredTaskScope.fork ensures that the binding of the scoped valueCONTEXT made in the request-handling thread —in Framework.serve — is read byCONTEXT.get in the child thread. The following diagram shows how the dynamic scope of the binding is extended to all methods executed in the child thread:

Thread 1                        Thread 2--------                        --------                                   5. Framework.readKey <----------+                                                                |                                                              CONTEXT                                4. Application.readUserInfo     |3. StructuredTaskScope.fork                                     |2. Application.handle                                           |1. Server.serve     --------------------------------------------+

The fork/join model offered byStructuredTaskScope means that the dynamic scope of the binding is still bounded by the lifetime of the call toScopedValue.runWhere. ThePrincipal will remain in scope while the child thread is running, andscope.join ensures that child threads terminate beforerunWhere can return, destroying the binding. This avoids the problem of unbounded lifetimes seen when using thread-local variables. Legacy thread management classes such asForkJoinPool do not support inheritance of scoped values because they cannot guarantee that a child thread forked from some parent thread scope will exit before the parent leaves that scope.

Migrating to scoped values

Scoped values are likely to be useful and preferable in many scenarios where thread-local variables are used today. Beyond serving as hidden method parameters, scoped values may assist with:

• Re-entrant code — Sometimes it is desirable to detect recursion, perhaps because a framework is not re-entrant or because recursion must be limited in some way. A scoped value provides a way to do this: Set it up as usual, withScopedValue.runWhere, and then deep in the call stack, callScopedValue.isBound to check if it has a binding for the current thread. More elaborately, the scoped value can model a recursion counter by being repeatedly rebound.

• Nested transactions — Detecting recursion can also be useful in the case of flattened transactions: Any transaction started while a transaction is in progress becomes part of the outermost transaction.

• Graphics contexts — Another example occurs in graphics, where there is often a drawing context to be shared between parts of the program. Scoped values, because of their automatic cleanup and re-entrancy, are better suited to this than thread-local variables.

In general, we advise migration to scoped values when the purpose of a thread-local variable aligns with the goal of a scoped value: one-way transmission of unchanging data. If a codebase uses thread-local variables in a two-way fashion — where a callee deep in the call stack transmits data to a faraway caller viaThreadLocal.set — or in a completely unstructured fashion, then migration is not an option.

There are a few scenarios that favor thread-local variables. An example is caching objects that are expensive to create and use.java.text.SimpleDateFormat objects, for example, are expensive to create and, notoriously, also mutable, so they cannot be shared between threads without synchronization. Thus giving each thread its ownSimpleDateFormat object, via a thread-local variable that persists for the lifetime of the thread, has often been a practical approach. (Today, though, any code caching aSimpleDateFormat object could move to using the newerjava.util.time.DateTimeFormatter, which can be stored in astatic final field and shared between threads.)

TheScopedValue API

The fullScopedValue API is richer than the small subset described above. While here we only present examples that useScopedValue<V>.runWhere(V, <value>, aRunnable), there are more ways to bind a scoped value. For example, the API also provides a version which returns a value and may also throw anException:

try {        var result = ScopedValue.callWhere(X, "hello", () -> bar());        catch (Exception e) {            handleFailure(e);        }        ...

Additionally, there are versions of the binding methods that can bind multiple scoped values at a call site.

The following example runs an operation with k1 bound (or rebound) to v1, and k2 bound (or rebound) to v2:

ScopedValue.where(k1, v1).where(k2, v2).run(        () -> ... );

This is both more efficient and much easier to read than nested invocations ofScopedValue.runWhere.

The full scoped value API can be foundhere.

Alternatives

It is possible to emulate many of the features of scoped values with thread-local variables, albeit at some cost in memory footprint, security, and performance.

We experimented with a modified version ofThreadLocal that supports some of the characteristics of scoped values. However, carrying the additional baggage of thread-local variables results in an implementation that is unduly burdensome, or an API that returnsUnsupportedOperationException for much of its core functionality, or both. It is better, therefore, not to modifyThreadLocal but to introduce scoped values as an entirely separate concept.

Scoped values were inspired by the way that many Lisp dialects provide support for dynamically scoped free variables; in particular, how such variables behave in a deep-bound, multi-threaded runtime such asInterlisp-D. Scoped values improve on Lisp's free variables by adding type safety, immutability, encapsulation, and efficient access within and across threads.