Infrastructure worth keeping — and kept patched.

Skip tests that passed: Gradle and Maven build cache

Rerunning tests that already passed wastes build time. FosterStack Cache lets Gradle and Maven take a passing test result from the cache when nothing it depends on changed.

Does Gradle skip tests whose code didn't change?

We did not test tests that read the clock, the network or files outside the project. Gradle does not see those inputs, so a test result saved in the cache could be reused when it should not. That is about your tests, not about the cache.

Our advice, not a test result: if the test task does not come from the cache, work through why the hit rate is low. For a multi-module build, see Gradle build cache for a monorepo.

Does Maven skip tests that already passed?

We did not test a project with several modules here. For that, see Maven build cache extension in a multi-module project. We also did not test tests that read the clock, the network or files outside the project: Maven does not see those inputs, so a test result restored from the cache could stand when it should not. That is about your tests, not about the cache.

Our advice, not a test result: to run the tests every time, turn the cache off or skip the lookup for that build. The switches, and what each did in a real run, are on enable or disable the Maven build cache extension. If nothing is restored, see Maven build cache not restoring. The setup is on Maven remote build cache setup.

How we checked

Gradle. A small Java project with one class and two JUnit tests. Every run used a fresh copy of the project on the same Mac, with the same Java and Gradle, and the local build cache was switched off. The runs shared the cache server, and one Gradle home folder so the test libraries were downloaded once. A different machine or Java version was not tested. All the runs used gradle test --console=plain --no-daemon.

RuncompileJavacompileTestJavatest
1. First build, as CI would do itranranran
2. Fresh copy, same codeFROM-CACHEFROM-CACHEFROM-CACHE
3. Change the main coderanFROM-CACHEran
4. Add a testFROM-CACHEranran
5. Add a comment on the same line in the main coderanFROM-CACHEFROM-CACHE
6. A test that failsFROM-CACHEranFAILED
7. The same failing test, fresh copyFROM-CACHEFROM-CACHEFAILED

Runs 1 to 5 ended with BUILD SUCCESSFUL. Runs 6 and 7 ended with BUILD FAILED, as intended. This is the line Gradle run 2 printed:

> Task :test FROM-CACHE

What about time? (Gradle)

We do not quote times for these runs. They ran on a Mac that was busy with other jobs, so the times were not a fair measure, and a project whose tests are quick tells you nothing. What we can say is what ran: in runs 2 and 5 the :test task was not executed, so those builds did not spend time running the tests. How much time that is depends on how slow your tests are, so measure your own build.

Gradle files. build.gradle.kts:

plugins { java }

repositories { mavenCentral() }

dependencies {
    testImplementation(platform("org.junit:junit-bom:5.11.0"))
    testImplementation("org.junit.jupiter:junit-jupiter")
    testRuntimeOnly("org.junit.platform:junit-platform-launcher")
}

tasks.test { useJUnitPlatform() }

gradle.properties has one line, org.gradle.caching=true. settings.gradle.kts is the same cache setup as on enable or disable the Gradle build cache, with a different port. We used 127.0.0.1 over plain HTTP because the test ran on one machine; use HTTPS across a network.

Maven. A one-module Maven project with one class and two JUnit tests, run with the Surefire plugin (3.5.2). Every build used a fresh copy of the project on the same Mac and an empty local build cache. The builds shared the cache server, and one Maven repository folder so the test libraries were downloaded once. All runs used mvn verify, with extra options for the cache location and the shared repository. The cache setup is the one on Maven remote build cache setup. We counted the Tests run: line that Maven prints.

RunResultTests run
1. First build, as CI would do itBuilt, BUILD SUCCESS2
2. Fresh copy, same codeRestored from the cache, BUILD SUCCESSnone
3. Change the main codeBuilt, BUILD SUCCESS2
4. Add a testBuilt, BUILD SUCCESS3
5. A test that failsBuilt, BUILD FAILURE2, with 1 failure
6. The same failing test, fresh copyBuilt again, BUILD FAILURE2, with 1 failure

In Maven run 2 it restored the module and then listed the steps it skipped, the tests included:

[INFO] Found cached build, restoring demo:mtest from cache by checksum ...
[INFO] Skipping plugin execution (cached): resources:resources
[INFO] Skipping plugin execution (cached): compiler:compile
[INFO] Skipping plugin execution (cached): resources:testResources
[INFO] Skipping plugin execution (cached): compiler:testCompile
[INFO] Skipping plugin execution (cached): surefire:test
[INFO] Skipping plugin execution (cached): jar:jar

Gradle run on Gradle 9.8.0 with FosterStack Cache 0.2.1, macOS (arm64), Java 27, JUnit 5.11.0. Maven run on Maven 3.10.0 with maven-build-cache-extension 1.2.3, Surefire 3.5.2, JUnit 5.11.0 and FosterStack Cache 0.2.1, macOS (arm64), Java 27.

Second run: Linux, 0.2.2

The runs above were on a Mac with 0.2.1 and Java 27. We ran both tables again on a GitHub-hosted Linux machine with FosterStack Cache 0.2.2, Gradle 9.8.0, Maven 3.10.0 and Java 21. We rebuilt the same small test project (one class, two JUnit tests). Each run started from a fresh copy of the same base project, with at most one change.

The run and its log are public.