Your Feature Flag Bill Is a Cache Key Problem

A client exceeded their feature flag vendor’s monthly request quota by 100%.

The two fixes everyone proposed were delete the dead flags and stop calling flags inside loops. Both are sensible. Both save exactly zero requests.

What actually drives the meter isn’t in any of the code anyone was looking at. It’s the product of three numbers, and not one of them appears at a call site.

The vendor here is Flagsmith, and the specific mechanics are theirs, but the failure mode belongs to any metered API you’ve put a cache in front of.

Java SDK 7.4.3, JavaScript SDK 9.1.0, Spring Boot 3.5. Client details are anonymised throughout; the SDK behaviour is all from published sources.

1. Both obvious fixes save nothing

When a usage graph goes red, the instinct is to reduce the thing you can see. You can see flags in the console and flag checks in the code, so you reduce those. Neither one is what the vendor counts.

“Delete the dead flags.” Flagsmith meters HTTP requests: /flags, /identities, /traits, /environment-document. It does not meter how many flags exist.

• One request returns the whole flag set, so 47 flags and 20 flags cost the same.
• In this codebase it’s worse than neutral. The flag enum feeds a generated API schema, so deleting constants means regenerating an SDK and touching a public OpenAPI file.
• All that work, zero requests saved.

“Stop checking flags inside loops.” The Java SDK checks a local Caffeine cache before every HTTP call, in FlagsmithApiWrapper.identifyUserWithTraits. Five hundred iterations over the same user cost one request, not five hundred.

That instinct isn’t worthless, mind you. Those loops were genuinely wasteful, just not in the way anyone thought: the flag service logs at INFO on entry and again on every evaluation, so a 500-iteration loop emitted around a thousand log lines. Real money, wrong vendor.

2. What the vendor is actually counting

Here’s the shape of it, once you read the pricing page instead of the code.

The bolded rows are the bill:

requests  =  distinct_identities  x  (3600 / cache_ttl_seconds)  x  processes

Every term is a configuration decision or an accident of architecture, and two of the three were set once during setup and never revisited.

What follows is the four things that were wrong, in the order the expression makes them matter.

3. Finding one: the cache key is the user’s email

The mechanism. Here’s the call every developer on the team writes:

if (featureFlagService.isFeatureEnabled(Feature.NEW_MATCHING_ENGINE)) {

That asks nothing about any user. It’s a question about the deployment. And here’s what it becomes, inside a wrapper that reaches for the request-scoped principal:

return featureFlagService.isFeatureEnabled(
    feature, principal.getEmail(), tenantSlug);

The SDK’s cache key is "identity" + identifier, with traits deliberately excluded. So the identifier is the user’s email, and the cache holds one entry per active user.

Cache entries, one tenant with three users

TODAY                              KEYED BY TENANT

"identity" + amir@example.com      "identity" + acme
"identity" + sarah@example.com  →  (same entry)
"identity" + paul@example.com      (same entry)

3 entries, 3 requests              1 entry, 1 request

Cardinality equals active users. The question had cardinality one.

That’s the generalisable bug and it has nothing to do with feature flags: a cache key carrying more entropy than the question it answers. Every extra bit in the key divides your hit rate and multiplies your calls to whatever sits behind it.

The fix is one line in one file, and the codebase had already voted for it. Fifteen production call sites passed a null email with only the tenant, using an existing synthetic-identity convention. Exactly one passed a real email, and that was a pass-through of a nullable parameter. Tenant-keyed was the house pattern; the wrapper was the outlier, silently overriding it for about a hundred and twenty call sites.

What it buys: divides backend requests by the average active users per tenant, and removes a database lookup per flag check. It also multiplies with finding two.

What to check first, because this makes flag values tenant-wide, so three things break if they exist:

• a segment rule targeting the email trait,
• a per-identity override on a specific user,
• a percentage rollout, which is the nasty one. The engine splits by hashing the identity, so “10% of users” quietly becomes “10% of tenants, all-or-nothing inside each”.

None of that is answerable from the repository. It’s a console question, and it blocks the change.

4. Why nobody noticed the wrapper

A short detour, because the reason this went unseen for years is a Spring lesson rather than a Flagsmith one.

Nobody injects the wrapper. It has two production references. All ninety-four injection points across the JVM are spelled identically:

private final FeatureFlagService featureFlagService;

Against the interface. So no developer writing a flag check has any reason to know the wrapper exists, let alone that it rewrites their one-argument call into a three-argument one.

And here’s the part that should make you slightly uncomfortable. There are three beans implementing FeatureFlagService in that context, and not one is annotated @Primary. By type, the injection is ambiguous. It should fail.

It doesn’t fail, because Spring has a fallback most people never think about:

1. Match by type. Three candidates, so no.
2. Look for a qualifier. None.
3. Look for @Primary. None.
4. Match the candidate bean names against the injection point’s field name. The bean is registered as featureFlagService. The field is called featureFlagService. Ninety-four times.

Rename one of those fields to flags and the context fails to start with NoUniqueBeanDefinitionException.

The wiring is correct, and it’s correct by coincidence. An unwritten naming convention is the only thing selecting which implementation of a critical interface gets used. Worth knowing generally: multiple beans of one interface with no @Primary isn’t a latent bug, it’s a live one that hasn’t been triggered yet. The trigger is a rename.

5. Finding two: the TTL is ten seconds

The mechanism. The SDK’s own default expiry is five minutes. FlagsmithCacheConfig.DEFAULT_EXPIRE_AFTER_WRITE is 5, TimeUnit.MINUTES. This codebase set ten seconds: thirty times more aggressive, for data that changes when a human clicks a toggle in a web UI.

The cache uses expireAfterWrite, so an entry dies a fixed time after it’s written no matter how often it’s read. Refresh cadence is 3600 / TTL per identity, per JVM, and it’s completely insensitive to load.

Refreshes per identity, per hour, per JVM
(assuming continuous activity across the full hour)

10s   ████████████████████████████████████  360
60s   ██████                                 60      -83%
5min  █                                      12

The fix is a default and two properties:

flagsmith.cache.ttl-millis=${FLAGSMITH_CACHE_TTL_MILLIS:60000}
flagsmith.cache.max-size=${FLAGSMITH_CACHE_MAX_SIZE:1000}

Neither key existed anywhere in the repository, and neither was set as a config var in production. Worth verifying rather than assuming, because it’s the difference between “the repo defaults to ten seconds” and “production runs ten seconds”. Once the knob is declared, moving to five minutes later is config, not a deploy.

What it buys: up to 6x fewer backend requests, no change to flag semantics.

Two ways that graph flatters the change, and both are worth saying out loud:

• It’s a ceiling, not a measurement. It assumes someone hammering the system for a full hour. Ten minutes of activity gives you 6x, thirty seconds gives you 3x, a single request gives you nothing. Real savings track session shape, which you don’t control.
• There’s no load-locking. The cache does a plain getIfPresent, so when an entry expires, every concurrent thread wanting that identity misses and every one of them fires a request.

And the propagation delay is real. My first pass at this claimed the codebase had no kill-switch-shaped flags. That was wrong, and it had several:

• one choosing between a background system authentication token and requiring the user’s own two-factor step,
• three guarding whether events get published into a matching queue,
• one selecting between a new read path and a legacy one.

Turning any of those off is an operational action where somebody is watching a graph and wants it to stop. Sixty seconds is still defensible for all of them, since none is a security control whose failure mode is measured in seconds and nothing toggles flags programmatically. But you argue that on the call sites rather than by claiming the category doesn’t exist.

6. Finding three: the browser refetches everything on every click

Check the dashboard split by SDK key before you believe any of the backend arithmetic. The browser uses a separate key. If it carries the volume, findings one, two and four do nothing and this one is the whole story.

The mechanism. In the vendor-facing portal, every client-side navigation cost one POST /identities/. Same user, same flags, same answer.

click "Invoices"   ->  1 request   <- full re-init, refetches every flag
click "Profile"    ->  1 request   <- full re-init
click "Invoices"   ->  1 request   <- full re-init
browser back       ->  1 request   <- full re-init

The waste isn’t how many flags a page reads. One request returns the whole set, so reading flags after initialisation is free. The waste is the client being rebuilt on every screen, and the cause is referential equality: the JavaScript version of comparing two Java objects with != instead of .equals().

The provider re-runs its init effect whenever its dependencies change, comparing them by reference. One dependency was an array built inline:

vendorTenantSlugs={context.vendorInfos.map((v) => v.tenantSlug)}

The chain from there:

• .map() returns a new array every time it runs.
• The provider above it calls usePathname(), so it re-renders on every navigation.
• New array, identical contents, different reference, so the comparison says “changed”.
• The whole client gets torn down and rebuilt with a fresh identity call.
• Client-side flag caching defaults to off, so there’s no local fallback to soften it.

The fix is to stop passing an array:

vendorTenants={context.vendorInfos.map((v) => v.tenantSlug).join(",")}

Strings compare by value, so "acme,bob" equals "acme,bob" and the re-initialisation stops. No useMemo, which matters: a memo would work here and would also be exactly the kind of defensive memoisation that draws a review comment asking whether it’s needed. Passing a value type instead of a reference type is the smaller idea and the better one.

What it buys: one request removed per vendor-portal navigation.

The detail that made this satisfying: the SDK was already joining that array into a comma-separated string internally before sending it, so the fix just builds the same string one level earlier, where the comparison can see it. And the main tenant app, refactored at the same time, passes plain strings for its equivalent inputs and was never affected. Same refactor, two apps, one of them quietly paying per click.

7. Finding four: the number still scales with the business

Everything above lowers a number that keeps climbing. More tenants, more hires, more dynos, and it’s back. Exactly one change alters what the number depends on.

The mechanism. Local evaluation inverts the protocol. Each process downloads the entire environment document once (every flag, every segment rule, every override, one payload), then answers checks in memory and re-downloads on a fixed interval.

REMOTE (today)                      LOCAL EVALUATION

app --"flag X for user Y?"-->       vendor --whole rulebook--> app
    <--------"yes"---------                                     |
                                    app answers in memory <-----+
cost scales with users
                                    cost = 1 download / interval / process

The fix is four lines of builder:

FlagsmithClient.newBuilder()
    .setApiKey(secretKey)
    .withConfiguration(
        FlagsmithConfig.newBuilder()
            .withLocalEvaluation(true)
            .withEnvironmentRefreshIntervalSeconds(60)
            .build())
    .build();

What it buys: backend cost becomes processes x 1440 requests/day, flat, whatever the user count. The environment document is itself metered, so this makes the bill predictable rather than free, which is the property you actually wanted. It’s the only one of the four that stops the problem recurring, and it makes findings one and two irrelevant for billing.

Three traps, and every one of them lives in the SDK’s source rather than its documentation.

Trap A: it refuses to start without a server-side key

Flagsmith has two key types. The client-side one is public and can only ask questions. Only the server-side key, prefixed ser., may download the rulebook, and FlagsmithClient.Builder.build() throws rather than degrading when local evaluation is on without one.

• In production: fine, if the key is already a ser. one. Verify it on every app, not just the main one.
• In tests: a dozen modules configure fake keys like flagsmith or test. Every Spring context that loads the client dies on startup.
• In local dev: an empty key is allowed today, logging “all features disabled” and booting anyway. That becomes a boot failure.

The fix: enable local evaluation only when the key actually starts with ser., otherwise build the remote client you have now. One condition, one place. Tests and local dev keep working, no properties files change.

Trap B: the first download blocks startup, and its failure is swallowed

This is the one worth the price of admission.

The polling manager fetches the rulebook in its constructor, not in a start() method, and build() constructs it. So creating the client bean performs a synchronous HTTP round trip to the vendor during Spring context refresh. Then updateEnvironment catches RuntimeException and logs it.

Put those together:

• The vendor is briefly unreachable while your app boots.
• The app starts successfully. Nothing alerts.
• The environment stays null, lookups throw, the service layer catches and returns false.
• Every gated feature reads false across every process until the next poll.

Deploy time is exactly when this bites, because that’s when every dyno cold-starts at once against the same endpoint. You’ve turned a soft dependency into a boot-time one and made it fail closed, silently.

The reassuring half: the exposure is first boot only. Once a rulebook has loaded, the SDK never discards it, and a later refresh that fails keeps the copy it already has.

The fix: hand the client a set of default flag values, so a missing rulebook falls back to known answers instead of false. That’s real work rather than a config line, because those defaults have to be maintained against the flag enum, and the file that usually already exists for this has drifted out of sync with it.

Trap C: you’re using traits you never sent

The subtle one, and the one I’d have missed by reading only the code.

Under remote evaluation, the vendor’s server remembers traits between calls. Your backend sends two, your frontend sends four, and both write to the same identity. So the server has been merging them, and any segment rule keyed on a frontend-only attribute has been quietly working for backend evaluations too.

Local evaluation removes the vendor from the loop. The SDK builds the identity from only the traits passed at that call.

• Traits you send from both sides (email, tenant): no change.
• Traits only the frontend sends (role): the backend stops seeing them.

That’s a dependency on state you never wrote and can’t see in your own repository. It appears in no diff. The only way to find it is to ask which traits your segments reference, then compare against what each client sends.

Two things that turn out to be fine, so you scope the check correctly:

• Per-identity console overrides survive. They travel inside the environment document.
• The vendor-portal trait is usually moot, because the surface that sets it never evaluates on the server anyway.

8. What order to do it in

Ship the two unblocked ones. Read the dashboard, split by key. Then choose between the last two.

They overlap, which is the argument for measuring between steps rather than shipping all four. Under local evaluation the Caffeine cache is never consulted, so the TTL change stops mattering for billing and the tenant-keying change survives only as a cleanup that removes a database read per flag check.

9. The part that isn’t about feature flags

Three separate mistakes, and none is a bug in the ordinary sense. Every line does what it was written to do.

• A wrapper adds the current user to a call that never asked about a user, because somebody wanted per-user targeting and this was the tidiest place to put it.
• A ten-second expiry gets chosen during setup, when correctness feels more pressing than cost, and never revisited because it never breaks.
• A .map() in a prop is the most natural thing to write in React, and costs nothing until something downstream compares it by reference.

What connects them is that each one is invisible from where you’d look. The cache key isn’t at the call site. The bean selection isn’t at the injection point. The array identity isn’t in the component that renders.

And underneath all of it is the thing that made both original instructions wrong: nobody had checked what the vendor counts. Not flags, not checks, but distinct identities divided by cache lifetime, multiplied by process count.

That question generalises further than I’d like. Any metered API you’ve put a cache in front of has this same expression behind it, and most teams can’t tell you any of the three numbers off the top of their head. If your monthly bill for something is a surprise, the surprise is usually in the cache key.

For the other half of the caching story, the JPA fetch types piece covers what happens when a cache key you can’t see multiplies queries instead of API calls. And for a different flavour of “the fix made it worse”, there’s the connection pool ceiling incident.