
Over thepastthreeposts, we have explored classpaths and class loading on the JVM and Android. Till now, we have focused on the build: we've learned how to inspect, influence, and even manipulate build classpaths to achieve our software goals.
In this post, we're going to take this a step further and look at the classpaths involved in compiling and running our applications themselves. These are distinct from the classpaths that influence our build environment, and are critical for understanding our apps. Knowing how they work will make you a more effective JVM/ART (Android Runtime) programmer; it will help you solve problems and avoid the hyperbolic curve of desperation that is reading ancient answers on Stack Overflow.
By the end of this post, the next time you see aNoClassDefFoundError
, you will feel like a superhero rather than an innocent bystander.
Foundations
Let's start with a simple Kotlin program, which we'll use to probe a bit what we mean when we talk about compile-time vs runtime classpaths.
In these examples, we'll be using
kotlinc
(for compilation of Kotlin source), andkotlin
(for running Kotlin programs). (official docs)
// Main.ktimportlib.Libraryfunmain(){Library().truth()}// lib/Library.ktpackagelibclassLibrary{funtruth()=println("All billionaires are bad.")}
In order to compile this program, we'll need to ensure all the classes involved are on the compile classpath: bothMain
and our library,Library
. That looks like the following:
$ cd lib && kotlinc Library.kt && cd ..
This produces a class file,lib/Library.class
(the path is important). We need this for the next step:
$ kotlinc -classpath lib Main.kt -d main.jar
This compilesMain.kt
. The programkotlinc
will look for a class file atlib/Library.class
(based on the import statement,import lib.Library
inMain.kt
!). Including-classpath lib
ensures that the library class file is available for compilingMain.kt
.
Finally, we have-d main.jar
, which tellskotlinc
to bundle its output into a jar with that name. It is illuminating to inspect the jar that results from this command:
$ jar tf main.jarMETA-INF/MANIFEST.MFMETA-INF/main.kotlin_moduleMainKt.class
We see the usual jar cruft (manifest, kotlin_module); more importantly, we see our compiled class,MainKt.class
. Note that one thing wedon't see isLibrary.class
. Wecompiled against it, but it does not get bundled into our jar file. When we run our program, we'll have to provide that class file another way, on theruntime classpath. Let's try to figure out how to do that. We'll start naively:
$ kotlin MainKterror: could not find or load main class MainKt
Whoops, that didn't work. Let's try again:
$ kotlin -classpath main.jar MainKtException in thread "main" java.lang.NoClassDefFoundError: lib/Library...
Arrgh. One last time:
$ kotlin -classpath lib:main.jar MainKtAll billionaires are bad.
What's going on here? Well, the first command didn't work becauseMainKt
was not on the classpath for ourkotlin
invocation (kotlin
could not find the classMainKt
, which we told it to run). We fixed that by tellingkotlin
aboutmain.jar
, but the command still failed, this time with aNoClassDefFoundError
forlib/Library
. We then fixedthat by updating our classpath to include the directory that contains the library class file (class files and jars are specified on a:
-delimited list). This last command finally succeeded, and in an even more positive turn of events, taught us an important truth!
We now have a better grasp of the difference between the compile-time and runtime classpaths. In particular, we know that specifying the compile-time classpath (as we did withkotlinc
) is not sufficient to alsorun our program (withkotlin
). For that, we need to provide a separate (but possibly identical)runtime classpath.
The runtime would like a word
What about when the runtime differs from the compile-time? Let's explore.
// Main.kt same as before// lib/Library.kt same as before// lib-alt/Library.ktpackagelibclassLibrary{funtruth()=println("Some billionaires are very fine people.")}
We've already compiledMain.kt
andlib/Library.kt
. Let's compilelib-alt/Library.kt
.
$ cd lib-alt && kotlinc Library.kt && cd ..
And now run our program again, but with a different runtime classpath!
$ kotlin -classpath lib-alt:main.jar MainKtSome billionaires are very fine people.
We didnot recompileMain.kt
. We simply ran it in another context — a different classpath that provided a different implementation of thetruth()
function (lib-alt/Library
andlib/Library
are binary-compatible). We also learned another important truth, albeit one in a meta-ironic mode.
On the perils of being too clever
I want to really hammer home the power of the runtime, so let's work through one more example. We'll start by updating our alternative library:
// lib-alt/Library.ktpackagelibclassLibrary{funtruth()=println("Some billionaires are very fine people.")funlie()=println("Actually, billionaires earned their wealth.")}
Sneaky! I wonder what that's for.
// Main.ktimportlib.Libraryfunmain(args:Array<String>){vallib=Library()valarg=args.firstOrNull()?:"truth"lib.javaClass.getDeclaredMethod(arg).invoke(lib)}
Huh... where's this going? Note that we must recompileMain.kt
because we have changed it this time, unlike in our past examples. We also use a new flag,-include-runtime
, which bundles the Kotlin stdlib (the "runtime") into our jar, so we can use the stdlib functionfirstOrNull()
without having to manually put it on the classpath later.
$ kotlinc -classpath lib Main.kt -include-runtime -d main.jar$ cd lib-alt && kotlinc Library.kt && cd ..
Note that the order of compilations above does not matter, nor the fact that we first compileMain.kt
againstlib/Library
, and later run it againstlib-alt/Library
. The Kotlin compiler only needs some way to resolve all the symbols, and in this case, that is onlyimport lib.Library
.
Ok! Time to run our new program.
$ kotlin -classpath lib-alt:main.jar MainKt lieActually, billionaires earned their wealth.
See, this is why reflection is bad.
Connecting this to Gradle
Since we don't generally use the SDK tools directly, but rather via a build tool such as Gradle, let's take a moment to translate the above into Gradle terms. Imagine a project with a structure like the following:
.├── app│ ├── build.gradle│ └── src│ └── main│ └── Main.kt├── lib│ ├── build.gradle│ └── src│ └── main│ └── lib│ └── Library.kt├── lib-alt│ ├── build.gradle│ └── src│ └── main│ └── lib│ └── Library.kt└── settings.gradle
You already know what the source files look like.settings.gradle
is simple:
include':app',':lib',':lib-alt'
The only other interesting file isapp/build.gradle
:
plugins{id'org.jetbrains.kotlin.jvm'version'1.5.10'}dependencies{// Two choices:// 1. :lib is on the compile and runtime classpaths of :appimplementationproject(':lib')// 2. :lib is on the compile classpath, while :lib-alt is// on the runtime classpath.compileOnlyproject(':lib')runtimeOnlyproject(':lib-alt')}
Gradle's docs have a nice graphic describing the relationship between the various "configurations"here.
Use-cases
Writing maximally-compatible Gradle plugins
Let's say you want to write a Gradle plugin that is compatible across a range of Android Gradle Plugin (AGP) versions, say 4.2 through 7.0. There are at least two things you should do:
- Compile your plugin against theminimum version you intend to support. In this case, 4.2.0.
- Declare this dependency as a
compileOnly
dependency,not asimplementation
orapi
.
Here's what that looks like:
dependencies{// 4.2.0 is the minimum version we supportcompileOnly'com.android.tools.build:gradle-api:4.2.0'}
Doing this ensures that the dependency is available at compile-time, and also that it isnot exposed to users of your plugin at runtime. (For an excellent discussion of Gradle "configurations" like compileOnly, etc., seethis post by Martin Bonnin.) Because the dependency is available at compile-time, you can see all its types in your IDE, enabling a pleasant development experience. And because the dependency is not automatically supplied at runtime, you're requiring your users to ensure it's on their build-runtime classpath — which they normally will have anyway, if they're using a plugin (yours) that is meant to enhance AGP in some way.
MysteriousRuntimeException
s: Stub!
At work, we were seeing unit test failures, butonly in the IDE. The command line, and CI, worked fine. (In fact, this is how the IDE error got distributed to the whole company: green build on CI.)
Caused by: java.lang.RuntimeException: Stub! at android.os.Looper.getMainLooper(Looper.java:7) at flow.Preconditions.getMainThread(Preconditions.java:55) at flow.Preconditions.<clinit>(Preconditions.java:22)
Naturally, we were suspicious that this was an IDE bug (spoiler alert: it was), but since app devs generally prefer to run tests from their IDEs, we couldn't just ignore it. After all, my team is named "Mobile Developer Experience," not "It's Green on CI."
If you have a bit of experience with Android, you may already be thinking that we've somehow messed up and are trying to unit test something that ought to be instrument-tested instead. If so, you'd bekind of on to something. We had recently added acompileOnly
(!) dependency on'com.google.android:android:4.1.1.4'
— these are the maven coordinates for "android.jar," that is, the Android runtime. We wanted to compile against them because some code touched theParcelable
class, but didn't actuallyuse it. Therefore, we knew we'd be safe at runtime because we'd never try to invoke anything relating to aParcelable
— it just had to be available at compile-time.
To reiterate, this worked as expected from the command line, but when invoking tests from Android Studio, they'd always fail (at least it was deterministic). Turns out the reason was, and I quote (emphasis mine) "there areclasspath issues running the tests inside the IDE because the whole setup is a mess," and "unit tests have some many issues before [Arctic Fox], it is surprising they worked at all." Basically, Android Studio before Arctic Fox is using thewrong runtime classpath, leading to runtime failures.
The fix for this turned out to be very simple, once the problem was understood. First, we removedcompileOnly 'com.google.android:android:4.1.1.4'
. We added a new module which had hand-written stubs forParcelable
andParcel
, and we putthat module on the compile classpath instead. Now all our tests pass, from both CLI and (old, broken) Android Studio!
UnderstandingNoClassDefFoundError
s
Maybe you've seen something like the following in your career as an Android or JVM dev:
Instrumentation test failed due to uncaught exception in thread [main]:
java.lang.NoClassDefFoundError
: Failed resolution of:Lcom/example/ThisClassShouldDefinitelyBeHere;
"Huh," you think. "ThisClassShouldDefinitelyBeHere should definitely be there. What's going on?"
What's going on is a runtime failure because the ART/JVM tried to resolve a class and couldn't find it anywhere: it was not available on the runtime classpath.
"But my code compiled fine!" you say. "That class is in dependency 'foo', which is declared on the 'implementation' configuration. Why isn't it available at runtime?"
These are excellent questions. You've already identified that the classshould be there: you declared it on the right configuration,implementation
, and you know that such dependencies are — or ought to be — available atboth compile-time and runtime. So what you've found, actually, is a bug in one of your build tools. I recently encounteredjust such an issue with AGP 4.2.0, which occurs with project dependencies declared onandroidTestImplementation
and which contain mixed Java/Kotlin source. The bug is already fixed and will be released as 4.2.2, but we'vemonkey patched it locally while we wait for the official release.
The takeaway here is that, once you have a sufficient understanding of what can cause problems likeNoClassDefFoundError
, you can already eliminate a whole class of possible causes and narrow your focus considerably.
Well, that's it for the crash course in classpaths. I hope you've found it interesting and useful. Happy coding!
Thanks once again toCésar Puerta for his typically thorough reviews. If at any point you find this post confusing, know that it means I ignored César's advice. Thanks as well toNate Ebel for his early encouragement.
Top comments(0)
For further actions, you may consider blocking this person and/orreporting abuse