How big a server does a build cache need? CPU, RAM, disk and load

You are sizing a disk, not a compute tier. The server is one small program that idles in tens of megabytes, so the question is how much cache you want to keep.

The starting sizes, what to do and what to watch are our advice, not a test result. Our measurements say "in our runs" or "in our tests" and are explained in How we checked.

Starting sizes

These are starting points from our deployment guide. Your build graph decides the rest. It is hard to be limited by CPU, because the busy path is reading and writing stored files, not computing.

Set a size cap

Choose how much cache you want, set it with FSCACHE_MAX_BYTES, and make the disk about 20% bigger. When the cache reaches the cap it removes the least recently used entries right after each upload until it is back under the cap. The extra room covers file system overhead and uploads in progress.

The value is in bytes. It is easy to be wrong by a factor of 1024, so copy one:

Two mistakes that look like "the cache doesn't help"

• A cap that is far too small, such as 20 MiB when you meant 20 GiB, makes it evict constantly, so the hit rate never climbs. The status page at /statusz shows the cap it read next to current usage.
• Leaving the cap at its default of 0, which means no limit, on a small disk is the one setup that will fill the disk.

What makes the disk grow

The cache grows only when a build makes an output the cache does not already have. A build that changes nothing adds nothing. In our runs, an edit inside one method added one small entry per build, about 2.5 KB. A new public method added four entries, about 11.5 KB, because the other three modules depend on it, directly or through another module, so all four were recompiled.

• To size your own: run your normal builds for a day, then run du -sk on your FSCACHE_DATA_DIR folder. It also counts a 128 KB index file and rounds each small entry up, so on a small project it reads higher than the entries alone.
• To limit growth: set a size cap (see Set a size cap above). The oldest-unused entries go first; see build cache eviction and the size limit.

Put it near the runners

On bigger setups the network matters more than the processor. A small server in the same region, ideally the same network, as your CI runners beats a large one far away.

Can I run the build cache on a small VM or laptop?

In our runs the FosterStack Cache server, which you run yourself, stayed under 19 MiB of memory and used under half a second of CPU in total across 31 builds of the small project. It also kept working inside a container limited to 64 MiB.

Our advice, not a test result:

• Memory was not the constraint in our tests. A 1 GB machine, the minimum in the starting sizes above, leaves plenty of room for loads like these. Plan RAM for the whole machine, not just the server.
• Disk is the one to plan. Decide how much cache you want to keep, and set a size cap (see Set a size cap).
• A laptop is fine for a trial. For a team, use a machine that stays on and that CI and everyone can reach. See your first 15 minutes with a self-hosted build cache.

How many builds can use one cache server at once?

In our runs, one FosterStack Cache server handled 8 builds at once, the most we tried, and every build succeeded. It refuses an upload only when 32 uploads are already in progress, which is the default limit.

• Up to 8 builds at once worked. All 39 builds in our main run ended with BUILD SUCCESSFUL. None printed a cache warning.
• The server stayed small. Its memory peaked at about 16 MiB under the builds. It peaked at about 23 MiB under a burst of 100 uploads of 1 MiB each.
• Uploads have a limit. By default the server takes 32 uploads at a time. An upload past the limit gets a 429 with Retry-After: 1, and nothing is stored.
• A refused upload turns the remote cache off for that build only. In our run, Gradle printed one warning, switched the remote cache off for the rest of that build and still ended with BUILD SUCCESSFUL. The next build was normal.
• Reads and repeat uploads behaved correctly in our runs. No read failed. Two uploads of the same entry at once left one complete entry. A reader got 404 while a slow upload was running, never half an entry.

Our advice, not a test result, on what to watch:

• Watch the refused-upload counter. It is fscache_http_requests_total{method="PUT",status="429"}. In our main build run it never appeared, because a counter that never fires is absent. If it climbs, uploads are being refused.
• If it climbs, count how many jobs upload at the same moment. Many CI jobs that finish together and upload large outputs are the likely cause. Our builds uploaded small entries, 13 of them in about 35 KiB, and none was refused even with the limit set to 1.
• The limit is a setting. It is FSCACHE_MAX_CONCURRENT_UPLOADS, and 0 turns it off. We tested 1, 8, the default of 32 and 0. We did not test what 0 does on a small machine.
• Keep watching memory and server errors too. See is my build cache server healthy and how do I know my build cache hit rate.

How we checked

Every build in every run below used a fresh copy of the project and a new Gradle home, with Gradle's local cache off, against FosterStack Cache 0.2.1.

Disk growth

Three series of eleven real Gradle builds on the four-module project from Gradle build cache for a monorepo. Each series used a new empty FosterStack Cache 0.2.1 server and a new Gradle home for every build. Each edit was made to a clean copy, not on top of the last. We counted the stored entries (the files in the server's blobs folder) after each build. The project is tiny, so the sizes are small; the pattern is what carries over. Not tested: a real-sized project, tests, Maven, or the cap switched on.

Memory and CPU on a small machine

We ran the real server and measured the server process only, under Gradle builds of a small and a larger project. Memory was 9 to 16 MiB while idle at startup, and the highest we saw was 18.2 MiB. CPU was 0.46 seconds in total for one cold build, 20 builds restoring everything and 10 builds at once, on the small project. Ten builds at once all succeeded, both restoring and storing. In containers limited to 128 MiB and to 64 MiB, every build succeeded, and the container was never killed for memory or restarted. Their peaks were 16.0 MiB and 13.9 MiB.

What ran: FosterStack Cache 0.2.1 on a Mac (arm64), Gradle 9.8.0, Java 21.0.12.1, compile tasks only. Every build used a new empty Gradle home and no long-lived Gradle daemon. Server memory is the resident size (RSS) read with ps once a second. CPU is the server process's own CPU time, not wall-clock time. We make no claim about build times.

Projects: the small one has 4 modules of 12 small classes. The large one is generated: 8 modules of 300 classes each, and its 17 entries are about 531 KB. Both are small next to a real codebase, so the disk numbers will be bigger for you. The memory and CPU figures are for these loads only.

Containers: the 0.2.1 image with a hard limit of 128 MiB and then 64 MiB, no swap, with Docker on the same Mac. Each ran a cold build and a few warm builds, then 10 builds at once, for the small and the large project. The container reported no memory kill and no restart, and answered its health check afterwards. Peak memory is from docker stats about every 3 seconds. Not tested: a real-sized project, a long-running server, a smaller limit than 64 MiB, Maven, or a slow disk.

Builds at once and the upload limit

Builds at once. We started the server empty and ran 1, 2, 4 and 8 identical builds at the same moment. Then we ran the same number again against the filled server.

With identical builds, several of them uploaded the same entry. The server took every upload and kept one copy of each entry. Every warm build restored all four of its compile tasks from the cache.

Changed and unchanged builds together. We filled the server with one build, then ran 4 builds with changed code and 4 with unchanged code at the same moment. All 8 succeeded. The server held 13 entries before and 29 after, which is 4 new entries for each changed build. Each unchanged build restored all four compile tasks. Peak memory was 15.8 MiB.

Direct requests. To test the server without Gradle in the way, we sent uploads and reads with curl, with random contents, once for each setting of the upload limit.

Reads were never refused. A read returned 200 for every entry that was stored and 404 for every entry that was not, including those whose upload had been refused.

A refused upload in a Gradle build. We set the limit to 1 and held the only upload slot open with a slow upload. A build that stored entries during that time printed one warning, Could not store entry ... response status 429: Too Many Requests, then The remote build cache was disabled during the build due to errors. It ended with BUILD SUCCESSFUL, and Gradle did not retry. The build after it, with the slot free, printed no warning. We also ran 8 builds that all stored at the same moment with the limit at 1. All succeeded and none was refused.

What: FosterStack Cache 0.2.1 on a Mac (arm64, 10 CPUs, 16 GiB), no password. Gradle 9.8.0 with Java 21, a four-module project, compile tasks only. Builds did not reuse a Gradle daemon. Memory is the server process only, sampled every second for builds and every 0.2 seconds for the direct requests.

Limits: the Mac was heavily loaded by other work, with a load average from about 8 to 160 during these runs. So we report counts and outcomes, not times, and nothing here says how your hardware will behave. One run for each row. Not tested: Maven, more than 8 builds at once, uploads larger than 8 MiB, Linux, Docker and Kubernetes.

Related: Gradle and Maven setup and Jenkins and CI build cache. Full detail is in the Docker deploy guide.