Crash with JUnit 5.14 due to TestPlan internal API breaking change

[TWiStErRob]

I see you've just updated to JUnit 5.14, but haven't really changed anything other than the dep.

I'm getting this crash on JUnit 5.13 to 5.14 update (keeping your lib 1.8):

java.lang.NoSuchMethodError: No direct method (ZLorg/junit/platform/engine/ConfigurationParameters;Lorg/junit/platform/engine/reporting/OutputDirectoryProvider;)V in class Lorg/junit/platform/launcher/TestPlan; or its super classes (declaration of 'org.junit.platform.launcher.TestPlan' appears in /data/app/net.twisterrob.colorfilters.android.base.test-12STHJQGEsmSwg_I_QSkWg==/base.apk!classes5.dex)

java.lang.NoSuchMethodError: No direct method <init>(ZLorg/junit/platform/engine/ConfigurationParameters;Lorg/junit/platform/engine/reporting/OutputDirectoryProvider;)V in class Lorg/junit/platform/launcher/TestPlan; or its super classes (declaration of 'org.junit.platform.launcher.TestPlan' appears in /data/app/net.twisterrob.colorfilters.android.base.test-12STHJQGEsmSwg_I_QSkWg==/base.apk!classes5.dex)
at de.mannodermaus.junit5.internal.runners.AndroidJUnitPlatformTestTree$ModifiedTestPlan.<init>(AndroidJUnitPlatformTestTree.kt:184)
at de.mannodermaus.junit5.internal.runners.AndroidJUnitPlatformTestTree.<init>(AndroidJUnitPlatformTestTree.kt:33)
at de.mannodermaus.junit5.internal.runners.AndroidJUnit5.generateTestTree(AndroidJUnit5.kt:74)
at de.mannodermaus.junit5.internal.runners.AndroidJUnit5.testTree_delegate$lambda$0(AndroidJUnit5.kt:30)
at de.mannodermaus.junit5.internal.runners.AndroidJUnit5.$r8$lambda$KSfagmfVkPVu2wrnBs3DKovzKK4(Unknown Source:0)
at de.mannodermaus.junit5.internal.runners.AndroidJUnit5$$ExternalSyntheticLambda0.invoke(D8$$SyntheticClass:0)
at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:86)
at de.mannodermaus.junit5.internal.runners.AndroidJUnit5.getTestTree(AndroidJUnit5.kt:30)
at de.mannodermaus.junit5.internal.runners.AndroidJUnit5.getDescription(AndroidJUnit5.kt:32)
at de.mannodermaus.junit5.internal.runners.JUnit5RunnerFactoryKt.hasExecutableTests(JUnit5RunnerFactory.kt:32)
at de.mannodermaus.junit5.internal.runners.JUnit5RunnerFactoryKt.tryCreateJUnit5Runner(JUnit5RunnerFactory.kt:28)
at de.mannodermaus.junit5.AndroidJUnit5Builder.runnerForClass(AndroidJUnit5Builder.kt:70)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
at androidx.test.internal.runner.AndroidRunnerBuilder.runnerForClass(AndroidRunnerBuilder.java:146)
at androidx.test.internal.runner.ScanningTestLoader.doCreateRunner(ScanningTestLoader.java:50)
at androidx.test.internal.runner.TestLoader.getRunnersFor(TestLoader.java:64)
at androidx.test.internal.runner.TestRequestBuilder.build(TestRequestBuilder.java:842)
at androidx.test.runner.AndroidJUnitRunner.buildRequest(AndroidJUnitRunner.java:672)
at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:453)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2189)

Version Catalog: https://github.com/TWiStErRob/net.twisterrob.colorfilters/blob/5bf271cc162aa69dc1ccf65a2d17acc23d0455fd/gradle/libs.versions.toml
Reference: https://github.com/TWiStErRob/net.twisterrob.colorfilters/blob/5bf271cc162aa69dc1ccf65a2d17acc23d0455fd/component/test-base-ui/build.gradle#L40
Failed CI: https://github.com/TWiStErRob/net.twisterrob.colorfilters/actions/runs/18140714173/job/51630654198?pr=474
Test report: https://github.com/TWiStErRob/net.twisterrob.colorfilters/pull/474/checks?check_run_id=51631156586
Build scan: https://scans.gradle.com/s/krkvqoazfjik2

I wonder if I'm missing something, or there's an actual breakage your CI didn't pick up?

Crash pointing to

android-junit5/instrumentation/runner/src/main/kotlin/de/mannodermaus/junit5/internal/runners/AndroidJUnitPlatformTestTree.kt

Lines 183 to 188 in c7efb72

	private class ModifiedTestPlan(val delegate: TestPlan) :
	TestPlan(
	/* containsTests = */ delegate.containsTests(),
	/* configurationParameters = */ delegate.configurationParameters,
	/* outputDirectoryProvider = */ delegate.outputDirectoryProvider
	) {

Breakage caused by

junit-team/junit-framework@d1335cd#diff-9f26cdfdb6e74804dd2eddb0eb902ed4b0391e67b972c1dde3e630b8ff41e293R101 (TestPlan.java#101)

My guess is that a recompilation on your side will make android-junit5 1.9.0 incompatible with JUnit <5.14, because

-public interface OutputDirectoryProvider {
+public interface OutputDirectoryProvider extends OutputDirectoryCreator {

means that the new bytecode will bind to the new constructor and a similar (but opposite) exception to the above will throw on earlier versions.

[mannodermaus]

Hi, thanks for reporting. I'm surprised that CI did not catch this, and in fact I cannot seem to make it happen locally just yet either. I can see the deprecation of OutputDirectoryProvider in favor of OutputDirectoryCreator and you are correct that this would cause a binary incompatibility. The best bet is probably to update the instrumentation to the new type and instruct people to double-check that they will use JUnit 5.14.x together with android-junit5 1.14.0.x.

[TWiStErRob]

To repro, I would imagine what's missing for you is to separate what version is used for compiling the lib and what is used for testing in androidTest.

[mannodermaus]

Yeah, that probably makes sense. Honestly, I'm not sure I want to be compatible with whichever version of JUnit 5, though - that's why the plugin version scheme follows its cadence so closely. I will prioritize releasing a proper 1.14.0.0 with compatibility for JUnit 5.14.0 (accompanied by instrumentation 1.9.0 under the hood), but users of an older version would have to stick to the 1.13.x.y line. 🤔

[TWiStErRob]

Makes sense, I (Renovate) would be happy to update two libs together.

At the same time it's a bit annoying not being able to use any version. Is there a particular reason for not using public APIs?

[mannodermaus]

I'll need to review again, but when this was originally written, IIRC the only way to create a customized TestPlan was to go through this API. Unfortunately, we do need a few unorthodox hooks into the Jupiter system to connect the Android instrumentation correctly...

[TWiStErRob]

I think all this needs then is a new release (compiler will take care of the rest) and the only supported version will be 5.14.x.

#391 is not even necessary for 5.14, but it is for 6.0 support.