Caffeine From First Principles: The Guarantee getIfPresent Doesn’t Make

Here’s a cache read. Three lines, no locks, nothing clever. It has a concurrency bug, and so does the identical code inside a library you may already depend on.

Flags flags = cache.getIfPresent(key);
if (flags == null) {
    flags = callTheApi();
    cache.put(key, flags);
}

This article is about why that’s broken, what the fix actually guarantees, and the three clocks that decide how a cached value ages. It’s a reference piece rather than a story: mechanism first, with the timelines that make each behaviour obvious.

Caffeine 3.2.4, which is what Spring Boot 3.5 resolves to. Every contract quoted here is from that version’s own Javadoc.

1. What is a cache, mechanically?

Strip away the vocabulary and a Caffeine cache is two things:

• A ConcurrentHashMap holding your entries.
• A policy deciding which entries to throw away, and when.

That’s the whole idea. Nearly everything written about caching is about the second part, sizing, eviction, hit ratios. Almost nothing is written about the first, and the first is where the bug lives.

The bug isn’t in how entries leave the cache. It’s in how you read one that isn’t there.

2. What’s actually wrong with those three lines?

Read them again as three separate operations, because that’s what they are:

Flags flags = cache.getIfPresent(key);   // 1. is it in the box?
if (flags == null) {                     // 2. no?
    flags = callTheApi();                //    fetch it
    cache.put(key, flags);               // 3. put it in the box
}

Nothing prevents two threads from sitting between step 1 and step 3 at the same time. Each one checks, finds the box empty, and independently concludes that fetching is its job.

Twenty requests for the same key arriving just after it expires:

t=60.000   entry expires, the box is empty
t=60.001   thread A:  getIfPresent -> null  ->  starts HTTP call
t=60.002   thread B:  getIfPresent -> null  ->  starts HTTP call
t=60.003   thread C:  getIfPresent -> null  ->  starts HTTP call
   ...     seventeen more threads, all identical
t=60.180   A returns  -> put
t=60.185   B returns  -> put   (same value, overwrites A)
   ...     eighteen more pointless writes

Twenty HTTP calls for one piece of data. No thread did anything wrong. The problem is that no thread knew the other nineteen existed.

This has a name. Cache stampede, or thundering herd. And it’s a specific instance of a much older shape:

check-then-act: you check a condition, then act on it, and something changes in between.

Once you can see that shape, you find it everywhere: if (!map.containsKey(k)) map.put(k, v), if (!file.exists()) file.create(), if (count < limit) count++. Each is two operations pretending to be one.

3. What does get(key, loader) do differently?

Flags flags = cache.get(key, k -> callTheApi());

One call instead of three steps. You hand the cache two things: the key, and the recipe for producing the value if it’s missing.

What is that second argument, exactly?

k -> callTheApi() is not a call. It’s an object, of type Function<String, Flags>, and writing it does not run it. Lambdas make this easy to miss, so here’s the identical code without one:

Function<String, Flags> recipe = new Function<>() {
    @Override
    public Flags apply(String k) {
        return callTheApi();
    }
};

Flags flags = cache.get(key, recipe);

Now it’s obvious that you’re passing something. Four consequences follow, and they’re the whole reason this works:

• You don’t decide when it runs. Caffeine does. That inversion is what lets it wrap the call in a lock.
• It runs only if the key is missing. On a hit, apply is never invoked, so a slow recipe costs nothing on the common path.
• It runs at most once per key, no matter how many threads asked.
• It runs on your thread, not a background one. Your request still pays for the load; it just isn’t doing it nineteen times over.

The k parameter is the key handed back to you. Most people ignore it and close over the outer variable, which is why you see k -> callTheApi() rather than k -> callTheApi(k). Using k is the better habit, since it’s the key Caffeine is actually loading.

Why handing it over changes anything

In the broken version you asked a question, got an answer, and then acted on it. Between the answer and the action, the world moved.

YOU CONTROL THE SEQUENCE          THE LIBRARY CONTROLS THE SEQUENCE

  ask: is it there?                 here is the key, and here is
  <- no                             what to do if it is missing
  (gap: anything can happen)
  fetch it                          <- the value
  put it back                       (no gap for you to fall into)

You can’t be atomic across four statements you wrote yourself. You can be atomic inside one method call the library controls. That’s the entire trade.

Straight from the Javadoc, then:

The entire method invocation is performed atomically, so the function is applied at most once per key.

“Atomically” means no other thread can observe a half-finished state. “At most once per key” means exactly what you want:

t=60.000   entry expires
t=60.001   thread A:  get() -> missing -> A runs the recipe
t=60.002   thread B:  get() -> A is already loading this key -> B waits
t=60.003   thread C:  get() -> waits too
   ...     seventeen more wait
t=60.180   A's call returns
t=60.180   all twenty threads receive that same value

One HTTP call. Twenty threads served.

Two things worth being precise about.

“Per key” is a real limit. A thread asking for a different key isn’t held up. The waiting only happens between threads that want the same missing entry, which is exactly the group that would otherwise have stampeded.

It isn’t completely free of collateral, and the Javadoc says so rather than hiding it:

Some attempted update operations on this cache by other threads may be blocked while the computation is in progress.

The underlying map locks a bucket, not a single key, so a long-running load can briefly hold up writes for unrelated keys that hash nearby. In practice this is a non-issue for loads measured in milliseconds and a real one for loads measured in seconds, which is the first argument for putting a timeout on whatever your recipe calls.

4. What is a ConcurrentHashMap, and where does computeIfAbsent fit?

Worth answering properly, because the relationship between these and Caffeine is layered rather than a choice between them.

A ConcurrentHashMap is a HashMap that several threads can use at once without corrupting it. Every single operation on it, get, put, remove, is atomic: no thread ever sees a half-written state.

And here is the sentence that explains the entire bug in this article:

Thread-safe operations do not add up to a thread-safe sequence.

get is safe. put is safe. get followed by put is not safe, because between the two, the map is unlocked and anyone can do anything. Every thread in section 2 used a thread-safe cache correctly and still produced twenty HTTP calls. The data structure was never the problem. The gap between two correct calls was.

computeIfAbsent closes the gap by collapsing those two operations into one:

Map<String, Flags> map = new ConcurrentHashMap<>();

Flags flags = map.computeIfAbsent(key, k -> callTheApi());

Same idea as cache.get(key, loader): you pass the recipe instead of running it yourself, and the map guarantees it runs at most once per key while other threads asking for that key wait.

So are they interchangeable?

No. They stack.

  your code            cache.get(key, loader)
                                |
  Caffeine                      v
                       eviction, expiry, refresh, stats
                                |
                                v
  java.util.concurrent   ConcurrentHashMap.compute(key, fn)
                                |
                                v
                       locks that key's bucket, runs fn once

A Caffeine cache is a ConcurrentHashMap with a policy bolted on. That’s not an analogy, it’s the class comment in Caffeine’s own source:

This class performs a best-effort bounding of a ConcurrentHashMap using a page-replacement algorithm to determine which entries to evict when the capacity is exceeded.

The field is final ConcurrentHashMap<Object, Node<K, V>> data, and the loading path ends in data.compute(...), which is the sibling of computeIfAbsent and carries the same atomicity guarantee. So when you call cache.get(key, loader), the promise you’re relying on is ConcurrentHashMap’s promise, with Caffeine’s bookkeeping wrapped around it.

Then when would you use a plain map instead?

When you never need to forget anything. That’s the one thing a ConcurrentHashMap cannot do for you:

For a small fixed set of keys computed once at startup, a plain map is the right answer and a cache library is overkill. The moment keys are unbounded (one per user, one per tenant) or values go out of date, a plain map is a memory leak that also serves stale data forever, and you want the policy layer.

5. How do I prove this to myself?

Two tests, and the difference between them is the whole article. The trick is a starting gate: without one the threads stagger, the first finishes before the second starts, and the bug hides.

private static final int THREADS = 20;

private int countLoads(Consumer<AtomicInteger> readOnce) throws Exception {
    AtomicInteger loads = new AtomicInteger();
    CountDownLatch gate = new CountDownLatch(1);      // the starting gate
    var pool = Executors.newFixedThreadPool(THREADS);

    var done = IntStream.range(0, THREADS)
        .mapToObj(i -> pool.submit(() -> {
            gate.await();                             // everybody waits here
            readOnce.accept(loads);                   // then goes at once
            return null;
        }))
        .toList();

    gate.countDown();                                 // go
    for (var f : done) f.get();
    pool.shutdown();
    return loads.get();
}

Now the two readers:

@Test
void getIfPresent_lets_every_thread_load() throws Exception {
    Cache<String, String> cache = Caffeine.newBuilder().build();

    int loads = countLoads(counter -> {
        String v = cache.getIfPresent("k");
        if (v == null) {
            counter.incrementAndGet();
            v = slowLoad();                 // pretend HTTP, ~100ms
            cache.put("k", v);
        }
    });

    assertThat(loads).isEqualTo(20);        // one load per thread
}

@Test
void get_with_loader_loads_once() throws Exception {
    Cache<String, String> cache = Caffeine.newBuilder().build();

    int loads = countLoads(counter ->
        cache.get("k", k -> {
            counter.incrementAndGet();
            return slowLoad();
        }));

    assertThat(loads).isEqualTo(1);         // one load, full stop
}

20 against 1, same cache library, same twenty threads, same key. The only difference is which method reads it.

Two things this test teaches beyond the headline:

• Remove the latch and the first test starts passing. Not because the code got safer, but because the threads stopped overlapping. Concurrency bugs that only appear under simultaneity are invisible to tests that don’t force it.
• Make slowLoad() fast and the gap narrows. The window for a stampede is exactly as wide as your load is slow, which is why this hurts most against a slow dependency and barely shows against a local one.

6. So what’s a LoadingCache then?

The same guarantee with the recipe attached once, at build time, instead of at every call site.

LoadingCache<String, Flags> cache = Caffeine.newBuilder()
    .maximumSize(10_000)
    .build(key -> callTheApi(key));      // the recipe lives here

Flags flags = cache.get(key);            // no recipe needed here

Compare the two:

The last row is the one that matters most, and refreshing is why: it happens with no caller present, so the cache has to already know how to produce a value on its own.

Use LoadingCache when every reader loads the same way, which is nearly always. Use get(key, loader) when the recipe genuinely varies per call.

7. Why does the fixed version still make people wait?

Look at that second timeline again. One HTTP call, which is the win. But nineteen threads sat waiting 180ms for it, and each of those was somebody’s request getting slower.

The stampede is gone. The latency isn’t.

Fixing that means changing not how the value is loaded but when, which is what the expiry settings control, and there are more of them than most people realise.

8. What are the three clocks?

Caffeine gives you three independent timers. They’re often described as variations on “how long to keep things”, which is why people pick one at random. They do genuinely different jobs.

expireAfterWrite is a hard ceiling on staleness. The Javadoc measures it from “the entry’s creation, or the most recent replacement of its value”, so reading it a thousand times doesn’t extend its life. This is what you want when the value has a correctness deadline.

expireAfterAccess is about memory, not freshness. Each read pushes the deadline back, so popular entries live forever and forgotten ones die. Reach for this when your worry is the cache filling up with things nobody wants.

refreshAfterWrite is the interesting one, and it’s the answer to the latency problem from the previous section.

9. How does refreshAfterWrite avoid the wait?

When the timer fires, the value isn’t deleted. It’s marked stale and kept. The next reader gets the stale value straight away, and the reload happens in the background.

The Javadoc is precise about the mechanics:

Automatic refreshes are performed when the first stale request for an entry occurs. The request triggering the refresh will make a synchronous call to asyncReload to obtain a future of the new value. If the returned future is already complete, it is returned immediately. Otherwise, the old value is returned.

In a timeline:

t=60.001   thread A:  get() -> value is 60s old
                             -> A starts a background reload
                             -> A gets the OLD value immediately
t=60.002   thread B:  get() -> reload already in flight
                             -> B gets the OLD value too
   ...     everyone gets the old value, nobody waits
t=60.180   the reload lands, cache now holds the new value
t=60.181   thread U:  get() -> gets the NEW value

Between 60.001 and 60.180, callers see data up to 60.18 seconds old instead of 60 seconds old. You traded 180 milliseconds of extra staleness for zero requests blocked on the network, and it’s still one call, not twenty.

Whether that trade is right depends entirely on what you’re caching. For a feature flag, an exchange rate, a config value, it’s free money. For anything where a stale read is a correctness problem, it isn’t a trade you can make.

10. Why would you set two clocks at once?

Caffeine.newBuilder()
    .maximumSize(10_000)
    .refreshAfterWrite(Duration.ofSeconds(60))
    .expireAfterWrite(Duration.ofMinutes(10))
    .executor(cacheRefreshExecutor)
    .build(this::load);

Because refreshAfterWrite on its own has a nasty failure mode, and it’s documented in one sentence on CacheLoader.reload:

Note: all exceptions thrown by this method will be logged and then swallowed.

So if the API you’re loading from goes down, every reload fails quietly and the cache carries on serving the last good value. Not for sixty seconds. Indefinitely. Your dashboards look fine, your latency looks fine, and you’re serving data from last Tuesday.

expireAfterWrite(10 min) is the backstop. After ten minutes with no successful reload, the entry really is removed, and the next read fails honestly instead of lying quietly.

Short refresh window, long expiry window. The refresh clock is the normal path; the expiry clock is the seatbelt.

11. What else bites?

Five things, in rough order of how often I’ve seen them cause trouble.

• The default executor is ForkJoinPool.commonPool(). That pool is sized roughly to your core count, built for CPU-bound work, and shared with every parallel stream in the JVM. Background reloads doing blocking HTTP will squat on it. Give the cache its own small pool via .executor(...).
• A loader with no timeout can hold a lock. Section 3 covered why: the atomic computation blocks other updates while it runs. Whatever your recipe calls needs a connect and read timeout, always.
• Null means “not cached”. Caffeine won’t store a null value. If your loader returns null, nothing is cached and get hands back null, so a lookup that legitimately finds nothing will re-query every single time. Cache an empty value, not a null one.
• Expiry is lazy. “Expired” means “won’t be returned”, not “already freed”. Caffeine does its cleanup during other cache operations, so an untouched cache can hold expired entries in memory for a while. Fine for correctness, surprising when you’re reading a heap dump.
• Refresh only happens if somebody reads. No traffic, no refresh. An idle entry goes stale, then eventually expires, and the next reader pays full price.

12. What this doesn’t solve

All of the above is per JVM. Each process has its own ConcurrentHashMap and its own locks. Twenty instances with perfect single-flight caching still make twenty calls, because none of them can see the others.

You can reach for a shared cache with a distributed lock, and sometimes that’s right. Often it isn’t: you’d be adding a network round trip and a new shared failure mode to a request path, in order to save a network round trip. Do the arithmetic before assuming it’s a win.

The better question is usually whether you should be caching answers at all, or whether you can fetch the rules once and compute answers locally. That’s a bigger change than a cache setting, and it’s the only one that makes the per-process multiplier disappear.

13. The shape worth remembering

The specific fix is small: pass the loader, don’t check-then-act. The shape underneath it is the part that transfers.

Every one of these is two operations wearing one operation’s clothes:

if (cache.getIfPresent(k) == null)  { ... cache.put(k, v); }
if (!map.containsKey(k))            { map.put(k, v); }
if (!set.contains(x))               { set.add(x); }
if (counter < limit)                { counter++; }

And in each case the library already has the single-operation version: cache.get(k, loader), map.computeIfAbsent(k, fn), set.add(x) returning a boolean, AtomicInteger with a compare-and-set. The atomic version usually looks less explicit than the check-then-act one, which is exactly why people reach past it for the version they can read at a glance.

If you want to see this cost real money rather than just being theoretically wrong: a feature flag SDK I looked at recently reads its cache with getIfPresent, falls through to an HTTP call, and puts the result back. Three steps, no coordination, on a metered API. That story is over here, and the cache read is only the third-most expensive thing in it.

For neighbouring mechanisms: threads, @Async and @Transactional covers what’s holding a connection while your loader blocks, and JPA fetch types covers a different invisible fetch, the one hiding behind a getter.