J2CL is a tool by Google that transpiles Java code to Closure-compatible JavaScript.It’s been started in 2015 to eventually replace GWT at Google,for various reasons explained in theproject README,and will be used as the basis for GWT 3.As a Google project, it’s deeply integrated with their build tool:Bazel.We’ll look at how it all works,with the goal to create an equivalentGradle plugin.

This reverse-engineering work has already been done several times,I had done it a couple times a few years ago,but I don’t think it’s ever be documented though,so here it is.

Starting points

From a user point of view,the main starting points are the Bazel rulesj2cl_library andj2cl_application.

Thej2cl_application rule is actually just a macro around therules_closure rules.It takes as inputclosure_js_library andj2cl_library dependencies,and entry point Closure namespaces (that must be present in the dependencies),and produces optimized JavaScript through aclosure_js_binary rule,with a handful configuration options.It also generates aweb_library rule for running your code during development,we’ll come back to it later.

As we just saw, aj2cl_application doesn’t directly have sources,but takes them as dependencies.All the code of your application would thus actually be in aj2cl_library.

Aj2cl_library rule is what actually translates your Java source to JS.It takes Java source files as inputand produces a JAR of compiled Java classes,and a ZIP of the transpiled JS (known as a JSZip).

Finally, there are rules for tests(we’ll get to them later on),and for importing third-party libraries,including from Maven repositories.

j2cl_library

Let’s look closer at what aj2cl_library does.

Aj2cl_library is somehow both ajava_library and aclosure_js_library.It takes as input a set of Java and JS source files,along with dependencies that can be eitherclosure_js_library or otherj2cl_library rules.Note that in Bazel, Java code that’s also used in the server for example willalso be used by ajava_library.

Aj2cl_library will start by removing all the code from the Java source files that’s annotated with@GwtIncompatible(technically, it replaces that code with spaces, such that line numbers are preserved).This stripped code will then be compiled withjavac;this step could also produce new Java sources through annotation processing(technically, it could even generate JS files).The Java and JS source files, and the ones generated during compilation,will then be passed to J2CL to generate a Closure JS library.

To transpile Java code,J2CL needs the dependencies as compiled Java classes to resolve the Java APIs(specifically the method overloads, implicit casts, or even type inference through Java 10’svar keyword);those need to be thestripped variants of the dependencies,which is whyj2cl_library rules depend on otherj2cl_library rules,and not onstandardjava_library rules.J2CL also resolves.native.js files sibling to the.java files,that generally contains JS code implementingnative methods (somehow equivalent to GWT’s JSNI),and concatenates their content into the generated.js files.It should be noted that thejavac step replaces the bootstrap classpathwith the Java Runtime Emulation library(which is almost-100% shared with GWT).

So, aj2cl_library outputs both a JAR like ajava_library,except the sources have been stripped of any@GwtIncompatible code first,and a ZIP of JS files like aclosure_js_library.

Technically, in Bazel, aj2cl_library, just like aclosure_js_library,also type-checks the JS (this is probably a bit redundant in J2CL’s case, though thereare the.native.js files too),and outputs metadata files about the libraryto help speed up downstream Closure JS libraries(by not rebuilding / re-type-checking them when their upstream API hasn’t changed;this is similar to how Bazel, and Gradle, extract an API-only JAR/info from Java classes)and pass diagnostic suppressions down to the Closure compilation.This is quite specific to Bazel and therules_closure,though could possibly be implemented in Gradle as well(the way Gradle extracts and checks the API-only info from Java classesis entirely different from how Bazel does it though:Gradle apparently fingerprinting a whole classpath of a Java compilation,whereas Bazel generates an iJAR for eachjava_library and then compares those files checksums as any other input).

As we saw, aj2cl_library leverages three tools:

• GwtIncompatibleStripper
• javac
• and finallyJ2clTranspiler

And last, but not least, those tools are all run in persistent worker processes,so you don’t have to spawn a JVM every time(which with Bazel’s design of having rules almost for each Java package,would mean alot of times).

Importing JARs

There are three Bazel rules for importing JARs in J2CL.

Thej2cl_import rule is a simplebridge for when you need aj2cl_library to depend on ajava_library;this should only be used for annotation-only JARs though(J2CL doesn’t need the sources for annotations, and they don’t generate JS code;they’re only useful to trigger annotation processors,or configure static analysis tools, such asErrorProne).

Thej2cl_import_external rule takes a set of alternative URLs to a JAR (and its SHA256 checksum),and generates either aj2cl_import for annotation-only JARs,or aj2cl_library.In the latter case, the JAR should then contain Java sources.It is actually expected to be GWT library,with super-sources insuper subpackages.Thej2cl_library will then use the super-sources(and ignore the equivalent super-sourced files, that is then GWT-incompatible),and also ignore any*_CustomFieldSerializer* file(as Google doesn’t use GWT-RPC anymore, and thus didn’t port it to J2CL).

Finally, thej2cl_maven_import_external rule is a wrapper aroundj2cl_import_externalsimply generating the URLs from Maven coordinates and a set of Maven repository URLs.It should be noted that this macro uses the sources JAR,i.e. it replaces the classifier and packaging withsources andjar respectively.

Yes, you understand it right:those last two rules will only consider source files in the JARs(unless they are annotation-only JARs)and will therefore strip their@GwtIncompatible codeand (re)compile them withjavac,before finally translating them to JS.It is thus expected that the JARs either include all the source code(most importantly including the sources generated by annotation processing),or that, possibly, their dependencies are configured in such a way thatthejavac step will be able to process the annotations and generate the missing files.GWT libraries would fall in the former bucket,but their sources JAR as deployed in Maven repositories might not,so when reproducing this outside Bazel,we’d possibly make different choices.

Actually, if you look at the J2CL repository,you’ll see that it has to use a different technique when importing the jbox2d library,as that one puts super-sources in agwtemul subpackage(and includes GWT-incompatible code in another package).In this specific case, it grabs the sources from GitHub,filters out the super-sourced or J2CL-incompatible files,and then declare aj2cl_library for those source files(therefore really rebuilding the library from sources).

j2cl_application

As we briefly saw above, this macro is actually only a (rather simple)helper to generate Closure rules,and you could actually just use the Closure rules directly.This means that this rule is actually not concerned at all with J2CL,except for the few configuration options it passes to the Closure compiler.

The inputs to the Closure compiler are the transitive closure of all theclosure_js_libraryand the JS output of thej2cl_library dependencies.This is the only place where the JS output of thej2cl_library rules are used;when aj2cl_library depends on anotherj2cl_library, as we saw above,it only uses its JAR output.

It’s interesting to look at how users woud run and debug their code though:thej2cl_application will generate a JS non-optimized version(mostly as aclosure_js_binary with different configuration),and an HTML file that to load it,and will launch a development server to serve them all.The HTML page and dev server are alsoibazel and livereload aware,such that if run throughibazel the page willlivereload whenever a source file is changed.

There are notes in the code about applications withcustom dev servers,one could imagine servers that guard the page behind authentication,or need to somehow inject dynamic things into the page.This is left undocumented for now though.In any case, for those who knowhow GWT’s super dev mode works,this is much different, much more likeparcel serve thanparcel watch for those who know JS development withParcel, and without hot module replacement (HMR);though we do not really know how GWT development works at Google with Bazel,therules_gwt being developped and maintained by a Googler whose not in the GWT team.

Tests

Tests are defined byj2cl_test rules,whose implementation has not actually been open sourced (yet).Those rules can be generated through agen_j2cl_tests rule,where we can learn a bit more about it,though we’ll actually learn more by looking at how it’s used in J2CL’s own tests.

Eachj2cl_test rule corresponds to only one test class,which can be test case or a test suite.It should be noted that J2CL supports both JUnit 3 and JUnit 4 tests cases,but only supports JUnit 4 test suites.

Tests can be run in two flavors:compiled or not;Isuppose it means whether to use optimized or unoptimized Closure output.And they can be run in several specific browsers,or, I suppose, against a set of globally-defined ones.

In the J2CL Git repository, we can also find an unusedj2cl_generate_jsunit_suite macro,which is probably used internally at Google by thej2cl_test rule.It takes as input a test class name,generates a dummy Java file that references it in a@J2clTestInput annotation,and compiles it with an annotation processor that will generate support files.So, the dummy Java file isonly used to trigger the annotation processor,but is otherwise completely useless.The processor will generateatest_summary.json file describing the tests,a JS file for each test case,and a Java file that needs to be processed by J2CL and is used by the JS files.Compilation is done through aj2cl_library,so the JS files actually have a.testsuite extensionsuch that they’re not picked up by J2CL.Technically, the dummy Java file could be processed usingjavac -proc:only,and then the generate Java file processed by J2CL without needing to be compiled to a Java class first.In the Bazel macro, the output of thej2cl_library is then processedto generate a ZIP containing only JS (with.testsuite renamed to.js) and thetest_summary.json.Thejsunit_test referenced in the macrois probably similar to theclosure_js_test rule fromrules_closure,but we can’t really know.

To actually learn more about how testing works with J2CL,we have to look at the ongoingj2cl-maven-plugin effort,whose dev team asked Google how to actually do it.

In the Maven plugin,test cases (or test suites) are expected to be annotated with@J2clTestInputdirectly referencing the annotated class(whereas in Bazel, the annotation is actually entirely kept as an implementation detail).The plugin will itself compile the classes(despite them already having been compiled by themaven-compiler-plugin),because is will also, as Bazel does, first strip@GwtIncompatible code,and add the annotation processor to the compilation process.It will then read the generatedtest_summary.json and, for each test,copy the generated.testsuite file to.js,run the Closure compiler on it,similar to aj2cl_application with that script as the entry point,generate a simple HTML file loading the resulting script,and finally load that page in a browser to actually run the tests.To detect that the tests have finished running,the plugin will poll the page for specific JS state;this is actually the same as thephantomjs_test plumbing inrules_closure.AFAICT, the main difference fromrules_closure is thatthe HTML page is loaded as afile:// URL in the Maven plugin,whereas it’s actually served through HTTP in therules_closure test harness,and this can actually have real consequences when dealing with cookies, resources, or HTTP requests.

Closing thoughts

Gradle should have all that’s needed to make it performant to builda similar tooling as the J2CL–Bazel integration described here:built-in up-to-date checks for performant incremental builds,variants (JS vs Java classes for a J2CL library),worker processes for best performance,artifact transforms for J2CL’ing external dependencies(with built-in caching),continuous builds for rerunning the whole build pipeline on file change,etc.

I’ll try to design such a Gradle plugin in a following post. Stay tuned!