The Outbox Pattern
Save to the database and publish to Kafka without losing either
What you'll learn
- Why writing to a database and a broker can't be one transaction
- The outbox table and the relay that publishes from it
- Polling vs change data capture
- Why consumers must handle duplicate events
In this video, we'll look at what happens when your service saves an order to the database but crashes before publishing the event to Kafka. We'll use that example to explain the dual write problem, and how the transactional outbox pattern solves it.
We'll also cover the outbox table, the relay that publishes events, polling vs reading the change log, and at-least-once delivery.
By the end, you'll understand how the outbox pattern works, why duplicate events can happen, and how idempotent consumers handle them.
