A service that updates its database and publishes an event to a message broker crosses two independent commit boundaries. If the database commit succeeds and the broker publish fails, durable state exists without its corresponding message. Reversing the order only reverses the failure: a consumer can observe a message for a state change that never commits.
A transactional outbox narrows this gap by placing the application write and a durable message record in the same local database transaction. Publication moves to a separate relay. The pattern does not make the database and broker one atomic system; it changes the boundary so message intent becomes part of the database commit.
The dual write has no shared commit point
Consider an order service that marks an order as paid and emits OrderPaid. A direct implementation has two durable side effects:
UPDATE orders SET status = 'paid' WHERE id = 42;
PUBLISH OrderPaid(order_id=42);No ordering of these operations removes the failure interval. Database-first can leave a committed order without an event. Broker-first can expose an event before the database transaction commits, and a later rollback leaves consumers acting on a state transition that did not survive.
Retries do not create atomicity. Retrying the database operation can repeat application effects unless the write is safe to repeat. Retrying publication can create duplicate messages unless the broker and downstream processing provide stronger guarantees. The central issue is not retry policy but the absence of one commit decision covering both resources.
Distributed transaction protocols can provide a coordinated commit when every participating system and deployment model supports the required protocol. A transactional outbox addresses a different operating model: the database remains the authority for the local state transition, and the broker receives a later projection of a message record already committed there.
Message intent becomes database state
The outbox table is written in the same transaction as the domain data. A minimal schema might contain an identifier, message type, payload, and publication state:
CREATE TABLE outbox (
id uuid PRIMARY KEY,
message_type text NOT NULL,
payload jsonb NOT NULL,
created_at timestamptz NOT NULL,
published_at timestamptz
);The application transaction now has one local commit boundary:
BEGIN;
UPDATE orders
SET status = 'paid'
WHERE id = 42;
INSERT INTO outbox (id, message_type, payload, created_at)
VALUES (:id, 'OrderPaid', :payload, CURRENT_TIMESTAMP);
COMMIT;If the transaction rolls back, neither the order change nor the outbox row remains. If it commits, both remain. That is the atomic property the pattern relies on.
The outbox row is not the broker message itself. It is durable evidence that publication is required. The relay may be delayed, restarted, or temporarily disconnected from the broker without erasing that intent.
This separation also makes transaction isolation relevant. The relay must only process committed outbox rows. Normal database visibility rules provide that boundary when the relay reads through ordinary transactional queries or a change stream with documented commit semantics.
The relay introduces an at-least-once edge
A relay reads pending rows, publishes them, then records completion. Those actions again cross the database-broker boundary:
read pending row
publish message
mark row as publishedSuppose the broker accepts a message and the relay crashes before published_at is stored. After restart, the same outbox row still appears pending and can be published again. Marking the row first is not a safe inversion: a crash after the mark but before publication would lose the message.
For this reason, a basic transactional outbox normally produces at-least-once publication behavior at the relay boundary. Duplicate delivery is not an exceptional corner case; it follows directly from the location of the crash interval.
A stable outbox identifier gives consumers a useful deduplication key. A consumer can record processed message IDs together with its own state change when both fit inside one local transaction. That does not make every downstream side effect automatically idempotent. An email provider, payment API, or another external system has its own commit boundary and requires its own duplicate-control contract.
Broker features can alter parts of this behavior, but their guarantees must be read at the exact interface in use. A broker-side producer identity, transaction, or deduplication window does not automatically extend into the service database or into arbitrary consumer side effects.
Ordering depends on the relay contract
An outbox preserves the database transaction that created each message intent, but it does not by itself define global publication order. Multiple application transactions can commit concurrently, multiple relay workers can claim rows concurrently, and the broker can partition messages across independent streams.
If consumers require per-aggregate order, the design needs an explicit ordering key or sequence. For example, events for one order can carry a monotonically increasing aggregate version. A consumer can then reject, buffer, or otherwise handle a version that does not follow the contract it expects.
A timestamp is usually a poor substitute for a sequencing contract. Clock values can collide, application clocks can differ, and row creation time does not necessarily equal broker observation order. Database-generated sequence values can impose an order, but that order must still survive the relay’s concurrency and broker partitioning rules if consumers are expected to observe it.
Global ordering is often stronger than the application needs. Encoding the narrower requirement, such as order within one account or aggregate, allows relay concurrency without pretending that unrelated transactions form one meaningful total sequence.
Claiming rows is a concurrency problem
A polling relay commonly runs more than one worker. Without coordination, two workers can select the same pending row and both publish it. Row locking can reduce concurrent claims inside the database:
SELECT id, message_type, payload
FROM outbox
WHERE published_at IS NULL
ORDER BY created_at, id
FOR UPDATE SKIP LOCKED
LIMIT 100;The exact syntax and lock behavior are database-specific. SKIP LOCKED, for example, is an implementation feature whose semantics must be checked for the selected database and isolation level.
Holding a database transaction open during network publication can also be undesirable. It extends lock lifetime and couples broker latency to database resource occupancy. Designs often claim rows briefly, publish outside that claim transaction, and keep enough state to retry abandoned claims. That reduces lock duration but adds lease, timeout, or recovery semantics.
None of these claim schemes removes the publish-then-record crash interval. They control concurrent relay work; they do not turn the broker acknowledgement and database update into one commit.
Change-data capture moves the relay boundary
Polling is not the only way to observe outbox rows. A change-data-capture system can read committed database log records and turn inserts into broker messages. This can avoid repeated table scans and can inherit ordering properties from the database log within the limits of the capture system.
The core invariant remains the same: the application transaction writes domain state and message intent together. Change-data capture changes the transport from committed outbox state to the broker. Its recovery checkpoints, log retention, transaction boundaries, schema evolution behavior, and duplicate semantics become part of the delivery contract.
Using change-data capture does not justify writing only the domain table and inferring every event later. That is a separate design. An explicit outbox row can carry the event type, stable identifier, payload version, and data selected at transaction time, making message intent a first-class part of the committed state.
Cleanup is part of the storage model
Published rows accumulate unless the system removes or archives them. Cleanup must not race with publication state. A retention process can delete rows that have been durably marked as published and are older than a chosen recovery horizon, subject to the relay’s actual retry and audit requirements.
The outbox is therefore not merely a transient queue implemented in SQL. It participates in database growth, indexes, vacuum or compaction behavior, backup volume, and operational recovery. A high message rate can make an unbounded outbox table a material storage workload.
Partitioning by time, archiving, or bounded retention can keep that workload controlled, but each choice changes recovery options. Deleting a published row removes the local evidence that the message was intended and later marked complete. Systems that require a longer audit trail need to retain that evidence somewhere appropriate rather than treating cleanup as an invisible implementation detail.
The guarantee stops at durable intent
The strongest property of the transactional outbox is narrow and useful: a local state change cannot commit without its associated message intent when both writes are in the same successful database transaction. The pattern does not guarantee exactly-once effects across the broker and every consumer.
Publication still needs retry behavior. Consumers still need a duplicate policy. Ordering still needs an explicit scope. External side effects still cross their own commit boundaries. Those are not defects hidden by the pattern; they are the remaining boundaries after the original dual write has been reduced to one local transaction plus a recoverable relay.
That distinction keeps the design precise. The database commit records both what changed and the obligation to publish. Everything after that commit is delivery work whose failure and repetition semantics must be stated separately.