Summary

Extend the JAR file format to allow multiple, Java-release-specificversions of class files to coexist in a single archive.

Goals

1. Enhance the Java Archive Tool (jar) so that it can createmulti-release JAR files.

2. Implement multi-release JAR files in the JRE, including supportin the standard class loaders andJarFile API.

3. Enhance other critical tools (e.g.,javac,javap,jdeps,etc.) to interpret multi-release JAR files.

4. Support multi-releasemodular JAR files for goals 1 to 3.

5. Preserve performance: The performance of tools and components thatuse multi-release JAR files must not be significantly impacted. Inparticular, performance when accessing ordinary (i.e., notmulti-release) JAR files must not be degraded.

Motivation

Third party libraries and frameworks typically support a range of Javaplatform versions, generally going several versions back. As aconsequence they often do not take advantage of language or API featuresavailable in newer releases since it is difficult to express conditionalplatform dependencies, which generally involves reflection, or todistribute different library artifacts for different platform versions.

This creates a disincentive for libraries and frameworks to use newfeatures, that in turn creates a disincentive for users to upgrade tonew JDK versions---a vicious circle that impedes adoption, to everyone'sdetriment.

Some libraries and frameworks, furthermore, use internal APIs of the JDKthat will be made inaccessible in Java 9 when module boundaries arestrictly enforced. This also creates a disincentive to support newplatform versions when there are public, supported API replacements forsuch internal APIs.

Description

A JAR file has a content root, which contains classes and resources, aswell as aMETA-INF directory which contains metadata about the JAR.By adding some versioning metadata to specific groups of files the JARformat can encode, in a compatible way, multiple versions of a libraryfor different target Javaplatform releases.

A multi-release JAR ("MRJAR") will contain the main attribute:

Multi-Release: true

declared in the main section of the JARMANIFEST.MF. The attributename is also declared as a constantjava.util.jar.Attributes.MULTI_RELEASE.Like other main attributes the name declared in theMANIFEST.MF iscase insensitive. The value is also case-insensitive, but there must beno preceding or trailing white space (such a restriction helps ensurethe performance goal is met).

A multi-release JAR ("MRJAR") will contain additional directories forclasses and resources specific to particular Java platform releases. AJAR for a typical library might look like this:

jar root  - A.class  - B.class  - C.class  - D.class

Suppose there are alternate versions of A and B that can take advantageof Java 9 features. We can bundle them into a single JAR as follows:

jar root  - A.class  - B.class  - C.class  - D.class  - META-INF     - versions        - 9           - A.class           - B.class

In a JDK that does not support MRJARs, only the classes and resources inthe root directory will be visible, and the two packagings will beindistinguishable. In a JDK that does support MRJARs, the directoriescorresponding to any later Java platform release would be ignored;it would search for classes and resources first in the Javaplatform-specific directory corresponding to the currently-running majorJava platform release version, then search those for lower versions, andfinally the JAR root. On a Java 9 JDK, it would be as if there were aJAR-specific class path containing first the version 9 files, and thenthe JAR root; on a Java 8 JDK, this class path would contain only theJAR root.

Suppose later on in the future Java 10 is released and A is updated totake advantage of Java 10 features. The MRJAR may then look like this:

jar root  - A.class  - B.class  - C.class  - D.class  - META-INF     - versions        - 9           - A.class           - B.class        - 10           - A.class

By this scheme, it is possible for versions of a class designed for alater Java platform release to override the version of that same classdesigned for an earlier Java platform release. In the example above,when running on an MRJAR-aware Java 9 JDK, it would see the 9-specificversions of A and B and the general versions of C and D; on a futureMRJAR-aware Java 10 JDK, it would see the 10-specific version of A andthe 9-specific version of B; on older or non-MRJAR-aware JDKs it wouldonly see the root versions of all.

JAR metadata, such as that found in theMANIFEST.MF file and theMETA-INF/services directory, need not be versioned. An MRJAR isessentially one unit of release, so it has just one release version(which is no different from a normal JAR, distributed say via MavenCentral), even though internally it contains multiple versions of alibrary implementation for use on different Java platform releases.Every version of the library should offer the same API; investigation isrequired to determine whether this should be strict backwardscompatibility where the API is exactly the same (byte code signatureequality), or whether this can be relaxed to some degree withoutnecessarily enabling the introduction of new enhancements that wouldblur the notion of one unit of release. This may imply, at a minimum,that a public class present in a release-specific directory should alsobe present in the root, though it need not be present in an earlierrelease directory. The run-time system will not verify this property,but tooling can and should detect such API compatibility issues, and alibrary method may also be provided to perform such varification (forexample onjava.util.jar.JarFile).

Ultimately, this mechanism enables library and framework developers todecouple the use of APIs in a specific Java platform release versionfrom the requirement that all their users migrate to that version.Library and framework maintainers can gradually migrate to and supportnew features while still carrying around support for the old features,breaking the chicken-and-egg cycle so that a library can be"Java 9-ready" without actually requiring Java 9.

Details

The following components of the JDK will be modified in order to supportmulti-release JAR files.

• The JAR-basedURLClassLoader must read selected versions of classfiles as indicated by the running Java platform version. Themodule-based class loader introduced with Project Jigsaw willrequire similar modifications.

• The protocol handler for thejar URL scheme andjava.util.jar.JarFile class must select the appropriate version ofa class from a multi-release JAR.

• The Java compiler (javac), via the underlyingJavacFileManagerandZipFileSystem APIs, must read selected versions of classfiles as specified by the-target and-release command-lineoptions. The toolsjavah,schemagen, andwsgen will leveragethe underlying changes toJavacFileManager andZipFileSystem.

• The Java Archive tool (jar) will be enhanced so that it cancreate multi-release JAR files.

• The JAR packing tool (pack200/unpack200) must be updated (seeJDK-8066272).

• Thejavap tool must be updated to enable selection of versionedclass files.

• Thejdeps tool will require modifications to display versioninformation and follow version specific class file dependencies.

• The JAR specification must be revised to describe the multi-releaseJAR file format and any related changes (e.g., possible additions tothe manifest).

Compatibility

By default the behaviour ofjava.util.jar.JarFile and thejar schemeprotocol handlers will remain the same. It is necessary to opt-in toconstruct aJarFile pointing to a MRJAR for version selection ofentries. Likewise it is necessary to opt-in forjar URLs (see nextsection for details).

JarFile instances created by the runtime for class loading will opt-inand create instances that are configured to select entries according tothe version of the running Java platform. Such asJarFile instanceis referred to as being runtime versioned.

Class loader resources

A resource URL, produced by a class loader, identifying a resource in aMRJAR will refer directly to a versioned entry (if present). Forexample for a versioned resource,foo/baz/resource.txt:

URL r = loader.getResource("foo/baz/resource.txt");

the URL ‘r’ may be:

jar:file:/mrjar.jar!/META-INF/versions/9/foo/baz/resource.txt

rather than:

jar:file:/mrjar.jar!/foo/baz/resource.txt

This approach is considered the least disruptive option. Changing thestructure of resources URLs is not without risk (e.g. a new scheme or anappended fragment). Legacy code may process URL characters directly,rather than parsing the URL and correctly extracting the components.While such URL process is incorrect it was considered preferable to notbreaking such code.

Modular multi-release JAR files

A modular multi-release JAR file is a multi-release JAR file that hasa module descriptor,module-info.class, in the root at the top, justlike a modular JAR file (see thePackaging: Modular JARsection of JEP 261). In addition modular descriptors may be presentin versioned areas. Such versioned descriptors must be identical to theroot module descriptor, with two exceptions:

• A versioned descriptor can have different non-transitiverequiresclauses ofjava.* andjdk.* modules; and

• A versioned descriptor can have differentuses clauses, even ofservice types defined outside ofjava.* andjdk.* modules.

The reasoning here is that these are implementation details rather thanparts of a module's API surface, and that one may well want to changethem as the JDK itself evolves. Changes to non-publicrequires ofnon-JDK modules are not allowed. If that is necessary then new versionof the module is required (at least increasing it's version number) andthis is a different kind of compatibility problem, and one that's beyondthe scope of MRJARs.

A multi-release modular need not have a module descriptor at the locatedroot. In this respect a module descriptor would be treated no differently toany other class or resource file. This can ensure that, for example, onlyJava 8 versioned classes are present in the root area while Java 9 versionedclasses (including the module descriptor) are present in the 9 versioned area.

Classpath and modulepath

A modular JAR can be constructed such that it works correctly on theclasspath of a Java 8 runtime, the classpath of a Java 9 runtime, orthe modulepath of a Java 9 runtime. The situation is the same for amodular multi-release JAR file (which in addition to themodule-info.class other classes may be compiled for the Java 9platform).

If a module descriptor does not declare some packages as exported, andtherefore public classes in those packages are private to the module,then when the corresponding JAR file is placed on the module path theclasses will not be accessible. However, if the JAR file is placed onthe classpath then those classes will be accessible. This is anunfortunate consequence of supporting the classpath and modulepath.

As a consequence the public API for multi-release JAR file may bedifferent when placed on the classpath compared to when placed on themodule path. Ordinarily the jar tool when constructing a multi-releaseJAR file will, on a best effort basis, fail if any observed differencesin the public API are detected. However, when constructing a modularmulti-release JAR file it is proposed the jar tool output a warning ifpublic API differences are a result of module private classes beingaccessible when the JAR file is placed on the classpath.

Multi-release jars and the boot loader

Multi-release JARs are not supported by the boot loader (for example, when amulti-release JAR file is declared with the -Xbootclasspath/a option). Suchsupport would complicate the boot loader implementation for what isconsidered a rare use-case.

Alternatives

A common approach is to use a static reflective check to determine if anAPI feature is present or not and accordingly select an appropriateclass that respectively depends on that feature or not. The reflectivecost is incurred at class initialization and not every time thedependent feature is used. A Java platform release is selected forcompilation with the source and target flags set to a lower release togenerate class files compatible with that lower release. This approachis often augmented with tools such asAnimal Snifferto check for API incompatibilities, where in addition to enforcing APIcompatibility code can be annotated to state whether it depends on alater Java platform release. There are a number of limitations withthis approach:

1. The reflective checks need to be carefully maintained.

2. It is not possible to utilize newer language features.

3. If a platform release API feature is removed (perhaps an internalAPI) then dependent code will fail to compile.

"Fat" class files were considered, where a class may have one or moremethods targeted to different Java platform versions. This was deemedtoo complicated in terms of the language and runtime features requiredto support such method declarations and dynamic selection.

Method handles (invokedynamic) cannot be used because of the need tomaintain binary compatibility.

Risks and Assumptions

It is anticipated that the production of MRJARs is primarily compatiblewith existing popular build tools and therefore IDEs that support suchtools, but the developer experience could be improved with enhancements.

The source layout and building of an MRJAR file can be supported byMaven using a multi-module project. For example, seethisexample Maven project that can produce a, currently rudimentary, MRJARfile. There would be a sub-project for the root and specific Javaplatform releases, and a sub-project to assemble the aforementionedsub-projects into an MVJAR. The assembly process could be enhanced,perhaps using a specific Maven plugin, leveraging the same features asthejar tool to enforce backwards compatibility.

The design and implementation of the runtime processing of MRJARscurrently assumes that a runtime uses the URL class loader or a customclass loader leveragesJarFile to obtain platform-specific classfiles. Runtimes whose class loaders useZipFile to load classes willnot be MRJAR aware. Popular application frameworks and tools, such asJetty, Tomcat, and Maven, etc., need to be checked for compatibility.

Dependences

Theenhanced JAR-file format under consideration for theJava Platform Module System will need to take multi-releaseJAR metadata into account.

JEP 247 (Compile for Older Platform Versions), which supportscompiling against older versions of the platform libraries, may aidbuild tools in the production of multi-release JAR files.