Your Message Broker Has a Schema: Freezing RabbitMQ Topology in a Build-Time Manifest
Table of Contents
Every backend engineer knows you don’t change a column type by editing the entity and shipping. You write a migration, it gets reviewed as a migration, and Flyway refuses to let history rewrite itself.
Then the same engineer changes an exchange from topic to direct in a @Bean method, and nothing anywhere blinks.
That one-line change took down our production and staging simultaneously — a 406 PRECONDITION_FAILED crash-loop at boot. This article is the systemic fix: the claim that broker topology is a schema, the procedure for changing it safely, and the build guard we wrote so the next breaking topology change fails in CI instead of in production. The guard is small — one utility class, one test, one checked-in JSON file — and by the end you’ll have all three.
The claim: topology is schema
A RabbitMQ exchange type. A queue’s x-dead-letter-exchange argument. Durability flags. These have every property of database schema:
- They’re structural, not behavioural — they define the shape messages flow through, not what handlers do.
- They’re shared — every service that touches the broker depends on them being what it expects.
- They’re immutable in place — RabbitMQ will never change the type of an existing exchange or the arguments of an existing queue. There is no
ALTER EXCHANGE. The only “migration” is delete-and-recreate or create-under-a-new-name. - Getting them wrong is fatal at boot, not degraded at runtime.
And yet, where database schema gets migration files, review ceremony, and a tool whose whole job is refusing unsafe history, broker topology lives in ordinary Spring beans:
@Bean
Exchange entityEventsExchange() {
return new DirectExchange("api.entity-events"); // was TopicExchange yesterday
}
No reviewer reads that diff as DDL. No tool compares it against what’s running. That asymmetry — schema-grade consequences with config-grade ceremony — is the entire root cause class.
Declaration semantics: assert-or-fail
The mechanics that make this dangerous, compressed:
When Spring AMQP declares an exchange or queue on the broker, the broker does one of three things: creates it (didn’t exist), silently no-ops (exists with identical properties), or refuses with 406 PRECONDITION_FAILED and closes the channel (exists with different properties). A declaration is an assertion about the broker’s state, re-checked every time it runs.
Which raises the question: when does it run? Two moments matter:
RabbitAdmin.initialize()— declares everything the context knows about, typically wired at application startup. If any assertion fails, the app doesn’t boot.- Listener startup — a listener container declares the queues it consumes from before subscribing.
If neither runs eagerly, a code-vs-broker mismatch can sit silent for months — until some innocent change (adding initialize(), adding a listener, a container restart policy) forces the comparison, and a long-dead diff detonates. That’s exactly the two-innocent-changes shape of our outage: the mismatch and the assertion arrived in the same deploy.
Changing topology safely: expand/contract, broker edition
Since the broker can’t alter in place, an intentional topology change has exactly two safe shapes:
Expand/contract (zero downtime). Create the new object under a new name. Move consumers to it first, then producers, then delete the old object once it’s drained. Same discipline as renaming a database column with a view over the old name — boring, incremental, reversible at every step.
Coordinated recreate (downtime window). Delete the object and let the app recreate it with the new properties at boot. Only acceptable when losing the messages in flight (or having none) is explicitly OK, and it has to be coordinated — the delete and the deploy are one operation, not two hopeful ones.
What is never safe: editing the declaration and shipping, which is an uncoordinated recreate minus the recreate.
The first layer of defence is simply writing this down where the change gets made — a javadoc warning on the topology config classes (“changing the type or arguments of an existing exchange/queue is a breaking change; see the runbook”) and a short runbook next to the code with the two procedures above. Put the warning where people edit, not in a wiki nobody opens mid-PR.
But documentation is advisory. The layer that actually bites is the build.
The build guard: a checked-in topology manifest
The idea, in one sentence: extract the topology your code declares, freeze it into a JSON file in the repo, and fail the build whenever an existing object’s definition drifts from it.
Three properties make it work:
- No broker needed. The test loads only the topology-declaring beans from the Spring context, with a mocked
ConnectionFactory. We’re checking what the code declares, not what the broker contains — the declaration is the source of truth the broker will be asserted against. - Additions are cheap, mutations are loud. A new exchange or queue just requires regenerating the manifest (one flag), which shows up as a reviewable JSON diff in the PR. Changing an existing object’s type or arguments fails the build with a message explaining why the broker can’t apply it and pointing at the expand/contract runbook.
- The manifest is deterministic. Sorted maps, stable serialization — so regeneration produces minimal diffs and review stays honest.
Here’s the utility. It extracts the topology model from a Spring context (including exchanges and queues wrapped inside Declarables beans), serializes it, and diffs a manifest against the live declarations:
package com.example.queue.config;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.TreeMap;
import java.util.function.Function;
import org.springframework.amqp.core.Declarables;
import org.springframework.amqp.core.Exchange;
import org.springframework.amqp.core.Queue;
import org.springframework.context.ApplicationContext;
/**
* Extracts the declared RabbitMQ topology (exchanges and queues) from a Spring
* context, serializes it to and from a deterministic JSON manifest, and diffs a
* manifest against the current declarations. Bindings are out of scope: the
* breaking dimensions are exchange type and queue arguments.
*/
public final class RabbitTopologyManifest {
public static final String UPDATE_FLAG = "rabbitmq.topology.manifest.update";
private static final String DOC = "doc/backend/rabbitmq-topology-changes.md";
private static final String REGEN_HINT =
"re-run with -D" + UPDATE_FLAG + "=true to regenerate the manifest";
private static final ObjectMapper MAPPER =
new ObjectMapper()
.enable(SerializationFeature.INDENT_OUTPUT)
.enable(SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS);
private RabbitTopologyManifest() {}
/** Builds the topology model from the context's exchange, queue and {@link Declarables} beans. */
public static Model extractFrom(ApplicationContext context) {
Map<String, ExchangeSpec> exchanges = new TreeMap<>();
Map<String, QueueSpec> queues = new TreeMap<>();
context.getBeansOfType(Exchange.class).values().forEach(e -> addExchange(exchanges, e));
context.getBeansOfType(Queue.class).values().forEach(q -> addQueue(queues, q));
// Some configs wrap exchanges/queues inside a Declarables bean rather than
// exposing them as individual beans, so unpack those too.
for (Declarables declarables : context.getBeansOfType(Declarables.class).values()) {
for (Object declarable : declarables.getDeclarables()) {
if (declarable instanceof Exchange exchange) {
addExchange(exchanges, exchange);
} else if (declarable instanceof Queue queue) {
addQueue(queues, queue);
}
}
}
return new Model(exchanges, queues);
}
public static Model read(InputStream in) throws IOException {
return MAPPER.readValue(in, Model.class);
}
public static void write(Model model, File file) throws IOException {
java.nio.file.Files.createDirectories(file.getParentFile().toPath());
MAPPER.writeValue(file, model);
}
/** One human-readable message per breaking change, addition or removal; empty when ok. */
public static List<String> diff(Model expected, Model actual) {
List<String> diffs = new ArrayList<>();
diffObjects("exchange", expected.exchanges(), actual.exchanges(), ExchangeSpec::fields, diffs);
diffObjects("queue", expected.queues(), actual.queues(), QueueSpec::fields, diffs);
return diffs;
}
private static void addExchange(Map<String, ExchangeSpec> exchanges, Exchange exchange) {
ExchangeSpec spec =
new ExchangeSpec(exchange.getType(), exchange.isDurable(), exchange.isAutoDelete());
ExchangeSpec existing = exchanges.putIfAbsent(exchange.getName(), spec);
if (existing != null && !existing.equals(spec)) {
throw new IllegalStateException(
"Exchange '" + exchange.getName() + "' declared twice with different definitions");
}
}
private static void addQueue(Map<String, QueueSpec> queues, Queue queue) {
QueueSpec spec =
new QueueSpec(
queue.isDurable(),
queue.isExclusive(),
queue.isAutoDelete(),
stringifyArguments(queue.getArguments()));
QueueSpec existing = queues.putIfAbsent(queue.getName(), spec);
if (existing != null && !existing.equals(spec)) {
throw new IllegalStateException(
"Queue '" + queue.getName() + "' declared twice with different definitions");
}
}
/** Arguments as a sorted string-valued map so serialization is stable and diff-friendly. */
private static Map<String, String> stringifyArguments(Map<String, Object> arguments) {
Map<String, String> result = new TreeMap<>();
if (arguments != null) {
arguments.forEach((k, v) -> result.put(k, String.valueOf(v)));
}
return result;
}
private static <T> void diffObjects(
String kind,
Map<String, T> expected,
Map<String, T> actual,
Function<T, Map<String, String>> toFields,
List<String> diffs) {
for (Map.Entry<String, T> entry : actual.entrySet()) {
String name = entry.getKey();
T expectedSpec = expected.get(name);
if (expectedSpec == null) {
diffs.add(
"New topology object '" + name + "' (" + kind + "). "
+ "Add it to the manifest (" + REGEN_HINT + ").");
continue;
}
Map<String, String> oldFields = toFields.apply(expectedSpec);
Map<String, String> newFields = toFields.apply(entry.getValue());
for (String field : union(oldFields.keySet(), newFields.keySet())) {
String oldValue = oldFields.get(field);
String newValue = newFields.get(field);
if (!Objects.equals(oldValue, newValue)) {
diffs.add(
"BREAKING topology change: " + kind + " '" + name + "' " + field
+ " changed '" + oldValue + "' -> '" + newValue
+ "'. RabbitMQ can't apply this in place, use expand/contract, see " + DOC);
}
}
}
for (String name : expected.keySet()) {
if (!actual.containsKey(name)) {
diffs.add(
"Topology object '" + name + "' (" + kind + ") was removed. "
+ "Update the manifest (" + REGEN_HINT + ").");
}
}
}
private static List<String> union(Iterable<String> a, Iterable<String> b) {
List<String> keys = new ArrayList<>();
a.forEach(keys::add);
b.forEach(
k -> {
if (!keys.contains(k)) {
keys.add(k);
}
});
return keys;
}
/** Root of the JSON manifest: exchanges and queues keyed by name. */
public record Model(Map<String, ExchangeSpec> exchanges, Map<String, QueueSpec> queues) {}
/** An exchange definition. {@code type} is the load-bearing, non-mutable-in-place field. */
public record ExchangeSpec(String type, boolean durable, boolean autoDelete) {
Map<String, String> fields() {
Map<String, String> fields = new LinkedHashMap<>();
fields.put("type", type);
fields.put("durable", String.valueOf(durable));
fields.put("autoDelete", String.valueOf(autoDelete));
return fields;
}
}
/** A queue definition. {@code arguments} is the breaking dimension RabbitMQ can't change. */
public record QueueSpec(
boolean durable, boolean exclusive, boolean autoDelete, Map<String, String> arguments) {
Map<String, String> fields() {
Map<String, String> fields = new LinkedHashMap<>();
fields.put("durable", String.valueOf(durable));
fields.put("exclusive", String.valueOf(exclusive));
fields.put("autoDelete", String.valueOf(autoDelete));
fields.put("arguments", String.valueOf(new TreeMap<>(arguments)));
return fields;
}
}
}
Design notes worth pausing on:
ExchangeSpec.typeandQueueSpec.argumentsare the load-bearing fields — they’re the two dimensions RabbitMQ physically cannot change in place. The rest (durability, exclusivity, auto-delete) ride along because they’re equally frozen at creation.- The duplicate-declaration check (
putIfAbsent+IllegalStateException) catches a subtler bug for free: two configs declaring the same queue with different definitions — which would otherwise be a nondeterministic race over whose declaration runs first. - Everything stringifies through sorted
TreeMaps so the JSON is byte-stable across regenerations. A manifest that reorders itself on every run trains reviewers to skim its diffs; a stable one keeps them honest.
And the test that turns the model into a build gate:
package com.example.queue.config;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.fail;
import static org.mockito.Mockito.mock;
import com.example.queue.config.RabbitTopologyManifest.Model;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.util.List;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.springframework.amqp.rabbit.connection.ConnectionFactory;
import org.springframework.amqp.support.converter.Jackson2JsonMessageConverter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.ComponentScan.Filter;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.FilterType;
import org.springframework.test.context.junit.jupiter.SpringJUnitConfig;
/**
* Build guard against breaking RabbitMQ topology changes.
*
* <p>Freezes the declared topology into a checked-in JSON manifest and fails the
* build when an existing exchange changes type or an existing queue changes its
* arguments. Adding new objects is allowed but must be reflected in the manifest
* so the change is reviewed.
*/
@SpringJUnitConfig(RabbitTopologyManifestITest.TopologyOnlyConfig.class)
class RabbitTopologyManifestITest {
private static final String MANIFEST_RESOURCE = "/rabbitmq-topology.json";
private static final String MANIFEST_SOURCE_PATH = "src/test/resources/rabbitmq-topology.json";
@Autowired private ApplicationContext context;
/**
* Loads only the topology-declaring beans from the queue-config package. The
* broker-facing infrastructure living in the same package — message senders,
* listeners, listener-container-factory configs — is excluded so no real broker
* connection is ever attempted. Configs that mix topology with a listener
* factory can stay: their factory beans instantiate against the mocked
* {@link ConnectionFactory} without connecting.
*/
@Configuration
@ComponentScan(
basePackageClasses = DlqConfig.class,
excludeFilters =
@Filter(
type = FilterType.ASSIGNABLE_TYPE,
classes = {
RabbitMessageSender.class,
RabbitMqListener.class,
// ...the module's listener-factory configs go here too
}))
static class TopologyOnlyConfig {
@Bean
ConnectionFactory connectionFactory() {
return mock(ConnectionFactory.class);
}
@Bean
Jackson2JsonMessageConverter jackson2JsonMessageConverter() {
return new Jackson2JsonMessageConverter();
}
}
@Test
@DisplayName("Declared RabbitMQ topology matches the checked-in manifest (no breaking changes)")
void topologyMatchesManifest() throws IOException {
Model actual = RabbitTopologyManifest.extractFrom(context);
if (Boolean.getBoolean(RabbitTopologyManifest.UPDATE_FLAG)) {
RabbitTopologyManifest.write(actual, new File(MANIFEST_SOURCE_PATH));
return;
}
Model expected = readManifest();
List<String> diffs = RabbitTopologyManifest.diff(expected, actual);
if (!diffs.isEmpty()) {
fail(
"RabbitMQ topology has diverged from " + MANIFEST_SOURCE_PATH
+ " (" + diffs.size() + " issue(s)):\n - "
+ String.join("\n - ", diffs)
+ "\n\nIf the change is intentional and safe, re-run with -D"
+ RabbitTopologyManifest.UPDATE_FLAG + "=true and review the diff.");
}
}
private Model readManifest() throws IOException {
try (InputStream in = getClass().getResourceAsStream(MANIFEST_RESOURCE)) {
assertThat(in)
.as(
"Manifest %s is missing; generate it with -D%s=true",
MANIFEST_SOURCE_PATH, RabbitTopologyManifest.UPDATE_FLAG)
.isNotNull();
return RabbitTopologyManifest.read(in);
}
}
}
The test-side tricks that took the longest to get right:
- The topology-only context. The queue-config package mixes topology declarations with broker-facing infrastructure. Component-scan the package, exclude the senders/listeners/factory configs by type, and hand the rest a Mockito-mocked
ConnectionFactory. Configs that declare topology and a listener factory can stay — a listener container factory happily instantiates against a mock as long as nothing starts it. Iterate on the exclusion list until the context boots with zero connection attempts. - The regeneration flag doubles as the workflow. After an intentional, safe change:
mvn test -Dtest=RabbitTopologyManifestITest -Drabbitmq.topology.manifest.update=true, and the manifest rewrites itself. The PR then carries a JSON diff a reviewer can actually read — which is the whole point. The manifest isn’t just a tripwire; it’s the topology change made visible. - The failure message teaches. “BREAKING topology change: exchange ‘api.entity-events’ type changed ’topic’ -> ‘direct’. RabbitMQ can’t apply this in place, use expand/contract, see doc/…” — the dev who trips the guard at 4pm gets the diagnosis, the reason, and the procedure in one string. Guards that just say “assertion failed” get deleted within a year.
And one verification step that belongs in the definition of done: prove the guard bites. Flip an exchange type locally, run the test, watch it fail with the BREAKING message, revert. A guard you’ve never seen fail is a guard you hope works.
Scaling it across a multi-module build
The first question anyone with a multi-module monorepo asks: our shared queue library declares common topology, but three services declare their own on top — do they share a manifest? Doesn’t the regeneration overwrite it?
The distinction that resolves everything: the class is not the manifest. Share the tool, never the data — a shared recipe, but each module cooks its own dish.
| Element | Shared or per-module? | Why |
|---|---|---|
| The manifest utility (extract / read / write / diff) | Shared | Pure functions, no data. Nothing to overwrite. |
rabbitmq-topology.json | One per module | Each module’s declared topology, in its own src/test/resources. |
| The guard test | One per module | Each loads its own configs and calls the shared logic. |
Nothing collides, for two mechanical reasons: Maven resolves the write path src/test/resources/rabbitmq-topology.json against the basedir of the module whose test is running, so each regeneration lands in the right module; and getResourceAsStream reads from the running module’s own test classpath, so each test reads its own manifest.
There’s one genuine decision in the scaling step: the utility starts life in the shared library’s test sources, and Maven doesn’t share test code between modules by default. The honest options:
| Option | What it is | Verdict |
|---|---|---|
| Test-jar | publish the library’s test classes as a separate jar; modules depend on it in test scope | Idiomatic for sharing test code; stays test-only; a little pom plumbing |
Move it to main | relocate the utility into the library’s main sources, make it public | Dead simple, zero plumbing — at the cost of a small “test infra in prod sources” smell |
| Dedicated test-support module | a module containing only the utility | Clean but a whole new module for one class |
| Duplicate it | copy the class into each module | Four copies to maintain — kills the point |
We went with move it to main. The class is tiny, its dependencies (Jackson, Spring AMQP) are already main-scope in the library, and zero pom plumbing beats architectural purity for a single utility. Test-jar is the defensible “clean” answer if test infrastructure leaking into main genuinely bothers you — both end in the same place: one shared class, one manifest per module.
The per-module gotcha worth printing in bold: scope each module’s guard to its own topology beans. If a service’s test component-scans broadly enough to drag in the shared library’s configs, you’re re-guarding the library’s topology in every service — four manifests disagreeing about the same exchanges. Register the module’s own config classes explicitly rather than scanning, and let the library’s own test guard the library.
Defense in depth: what each layer catches
The guard is the strongest layer, but it’s one of four that came out of the same incident — each catching what the others structurally can’t:
| Layer | Mechanism | What it catches |
|---|---|---|
| Editing | javadoc warning on the config classes + runbook | the engineer about to make a breaking change, before any tooling runs |
| Build | the manifest test | the breaking change that got written anyway — before merge |
| Review | CI flags PRs touching exchange/queue definitions with a comment | reviewer attention on the diffs that look like config but are DDL |
| Deploy | knowing preboot’s real semantics (timer-based, no health check, web-only) | false confidence — nothing at this layer rescues a bad topology change |
The last row is deliberately an anti-layer. Part of defense in depth is knowing which defenses you don’t have: our deploy stack does not protect against boot crashes, which is precisely why the build layer has to.
The closing thought
The uncomfortable takeaway from the outage wasn’t that RabbitMQ is finicky. It’s that we’d been running production for years with a second schema — one with all of the blast radius of the database’s and none of its ceremony. The broker’s topology was load-bearing, shared, immutable-in-place, and completely invisible to every process we had.
The manifest doesn’t make topology changes safe. Expand/contract makes them safe. What the manifest does is make them deliberate — one JSON diff at a time, reviewed by someone who can see exactly what’s about to be asserted against a broker that never forgets.
For the incident that motivated all of this — two innocent changes, a 406 crash-loop across two environments, and the rollback ledger that confused everyone mid-incident — read the war story: The deploy that took down prod and staging at once.
