We Didn't Need a Message Queue — We Needed an Event Bus
Buzzmi platform (client) · Lead Software Engineer, T & A Technologies · migrating a distributed monolith to Kafka
The decision in one line: the platform had microservices in name only — separately deployed modules calling each other synchronously over RPC, with no event bus. A failure mid-operation meant manually unwinding earlier calls, and the code had no answer for a rollback that itself failed. The fix wasn’t a better message queue. It was an event bus — Kafka.
A note on this write-up: the decisions and engineering are mine; I used AI to turn my notes into readable prose. I’m a software engineer, not a writer, and I’d rather be upfront about that.
What I walked into
The brief I got on day one came from the PM, in the language of symptoms: the backend was slow, threw errors constantly, was a pain to develop against, and had too many services wired together with everything depending on everything else. Nobody framed it as an architecture problem yet — it just felt broken and fragile.
When I dug in, the cause was structural. The backend was NestJS with RabbitMQ, used purely for RPC. On paper it was microservices — several independently deployed modules. In practice it was a distributed monolith: the modules called each other synchronously, request/response, as if they were function calls that happened to cross a network. That’s why it was slow (every operation waited on a chain of network calls), why errors cascaded (one service hiccups, the whole chain fails), and why the developer experience was miserable (you couldn’t touch one service without understanding all the ones it called).
That works until an operation spans more than one service. Say a single user action needs to touch three services. The code called them in sequence:
callServiceA() // ✅
callServiceB() // ✅
callServiceC() // ❌ fails
// now manually undo B, then undo A...
When something failed halfway, the code manually rolled back the calls that had already succeeded. And here’s the part that stayed with me: what happens if a rollback call fails? I went looking in the code. There was no answer. No compensation, no retry, no record that the system was now in an inconsistent state. It just… hoped.
flowchart TD
U[User action] --> A[Service A ✅]
A --> B[Service B ✅]
B --> C[Service C ❌ fails]
C --> R1[Undo B]
R1 --> R2[Undo A]
R2 --> Q{"Undo fails?"}
Q -->|no answer in code| X[Inconsistent state 💥]
style C fill:#8a1f1f,stroke:#8a1f1f,color:#fff
style X fill:#8a1f1f,stroke:#8a1f1f,color:#fffThe reasoning
1 · Name the real problem. This wasn’t a messaging problem — RabbitMQ was doing its job fine as an RPC transport. The problem was architectural coupling. Synchronous RPC chains mean every service in an operation has to be up, fast, and successful at the same instant, and there’s no clean way to make a multi-service change atomic. Manual rollback is a distributed transaction implemented by hand — and it fails exactly when you most need it to work.
2 · Identify what was actually needed. The services didn’t need to ask each other questions synchronously. They needed to react to things that happened. “A user subscribed.” “A post was created.” “A payment succeeded.” Those are events — facts about the past that many services care about independently. That’s not a message queue’s job; that’s an event bus: durable, replayable, one event fanned out to many consumers, producers and consumers fully decoupled.
3 · Make the call — Kafka. Of the event-bus options, I chose Kafka, for reasons that were about the future as much as the present:
- Durability and replay — events are a persistent log, not a transient message that’s gone once delivered. A new or recovering consumer can replay history.
- Battle-tested at scale — it’s the boring, proven backbone; I wasn’t going to bet the platform on something exotic.
- It unlocked what came next — an event log is the foundation for event sourcing and for CDC (Debezium), both of which were on the roadmap. Kafka wasn’t just solving today’s coupling; it was the substrate for the platform’s next two years.
4 · Decouple, don’t just re-route. The point wasn’t “swap RabbitMQ for Kafka.” It was to flip the model: services stop calling each other and start publishing events and subscribing to the ones they care about. A failure in one consumer no longer unwinds three others — it retries its own work, independently, from the log.
flowchart LR P[Producer service] -->|publishes event| K[(Kafka event bus)] K --> C1[Consumer A] K --> C2[Consumer B] K --> C3[Consumer C] style K fill:#4a9eff,stroke:#4a9eff,color:#fff
What it changed
- The platform became genuinely event-driven — services decoupled, no more hand-written cross-service rollback, no more “hope the undo works.”
- Failures became local and retryable — a consumer that fails reprocesses from the log instead of corrupting a multi-service operation.
- It became the foundation for event sourcing and CDC — the architectural decisions that followed all stood on this one.
The honest costs
I won’t pretend an event bus is free:
- Eventual consistency — you trade synchronous “it’s done now” for “it will be done shortly.” That’s a real mental shift for the team and the product.
- New problems to own — ordering, idempotency, and duplicate handling become your responsibility (at-least-once delivery means consumers must be idempotent).
- Operational weight — Kafka is more to run and reason about than an RPC call.
- Migration cost — you don’t flip a distributed monolith to events overnight; it’s incremental, and some flows genuinely are better synchronous.
When I’d keep synchronous RPC: a read that needs an immediate answer, or a truly single-service operation, doesn’t need an event bus. The win is specifically for multi-service, state-changing operations — which is exactly where the manual-rollback pain lived.