A service often needs one operation to change database state and emit an event. An order may move to paid while OrderPaid must reach a message broker. Those two writes cross different systems, so a normal database transaction cannot make both commits atomic.
Writing the database first leaves a gap: the process can stop after commit but before publishing. Publishing first creates the opposite gap: consumers can observe an event for a database change that later fails.
The transactional outbox pattern removes that split decision. The service writes the business change and a durable outbox record in the same local database transaction. A separate relay publishes committed outbox records to the broker.
One transaction records state and intent
The outbox row represents an obligation to publish, not proof that publication already happened.
A simplified transaction can look like this:
BEGIN;
UPDATE orders
SET status = 'paid'
WHERE id = 'ord_123';
INSERT INTO outbox_events (
event_id,
aggregate_id,
event_type,
payload,
created_at
) VALUES (
'evt_456',
'ord_123',
'OrderPaid',
'{"order_id":"ord_123"}',
CURRENT_TIMESTAMP
);
COMMIT;If the transaction rolls back, neither record becomes visible. If it commits, both become durable together. The application no longer has a crash window between the business commit and creation of the publication intent.
This guarantee depends on both rows participating in the same transactional resource. Placing the outbox in another database restores the distributed commit problem the pattern is meant to avoid.
The relay owns delivery after commit
Once the transaction commits, a relay finds unpublished outbox rows and sends them to the broker. The relay can be a polling worker, a process driven by change data capture, or another mechanism that observes committed rows without joining the original request transaction.
A polling relay might use a flow such as:
repeat:
rows = claim_next_batch()
for row in rows:
publish(row.event_id, row.event_type, row.payload)
mark_published(row.event_id)The request path only needs the local transaction to succeed. Broker latency or a temporary broker outage does not have to extend the database transaction. Pending rows remain durable until the relay can resume.
The relay should use bounded batches and a claim strategy suited to concurrent workers. Holding a database transaction open during a slow network publish can increase lock duration and couple database health to broker latency, so implementations commonly separate claiming from external I/O with explicit recovery semantics.
Publication is normally at least once
A relay cannot generally make publish() and mark_published() atomic across a broker and a database.
Consider this sequence:
1. relay publishes evt_456
2. broker accepts evt_456
3. relay stops before marking the row published
4. relay restarts
5. evt_456 is published againThe outbox closes the missing-event gap, but duplicate publication remains possible. Treating the pattern as exactly-once delivery hides this boundary.
Each event therefore needs a stable identifier. Consumers that cannot safely repeat their side effect can record processed event IDs, enforce a unique constraint on an inbox table, or use an operation-specific idempotency mechanism.
Broker features may reduce duplicates within their own boundary, but they do not automatically make a consumer’s database mutation atomic with message acknowledgement.
Ordering requires an explicit scope
A single global event order is rarely free. Multiple relay workers, partitions, retries, and broker routing can all affect observation order.
Many systems only need ordering per aggregate. Events for one order, account, or document can carry a monotonically increasing sequence number generated with the state change. Consumers can then reject stale versions, buffer gaps when appropriate, or route the same aggregate key to one ordered broker partition.
A timestamp alone is a weak ordering contract. Clock precision, concurrent transactions, and clock differences can produce ties or misleading order. A database sequence or aggregate version usually states the intended relation more directly.
If no ordering guarantee is required, documenting that fact keeps consumers from depending on accidental relay behavior.
Payload design controls coupling
An outbox can store a complete event payload or enough fields for the relay to construct one. Storing the final payload inside the business transaction freezes the event representation at the same point as the state change. A later deployment cannot silently publish old rows using new serialization logic.
That stability has a cost: payloads consume database space and schema evolution needs deliberate handling. Event metadata commonly includes an event ID, event type, schema version, aggregate ID, creation time, and trace or correlation identifiers when those are part of the service contract.
Reading current business rows later to reconstruct an old event can produce a different fact from the one that triggered the event. If exact historical values matter, persist them with the outbox record.
Sensitive data also deserves the same retention review as any other durable database field. Copying secrets or unnecessary personal data into an outbox expands the places that must be protected and deleted according to policy.
Cleanup must not race with delivery
Outbox tables grow continuously unless published rows are archived or removed. Cleanup needs a state boundary that cannot delete work still eligible for delivery.
A practical design may retain published rows for a fixed interval, move them to cheaper storage, or delete them in small batches. The retention period can support operational investigation and replay procedures, but replay should be an explicit operation rather than an accidental consequence of resetting a flag.
Indexes should match the relay query. An index on publication state plus an ordering or creation column can keep batch selection efficient, while excessive indexes add cost to every business transaction that inserts an outbox row.
Large backlogs need separate attention. A relay recovering from an outage should not consume all database I/O while catching up. Batch size, polling interval, worker count, and broker throughput form one capacity path.
Observability follows the durable boundary
The most useful signals describe the distance between committed intent and completed publication. Operators can track pending row count, age of the oldest pending row, publish attempts, publish failures, duplicate detections, relay throughput, and cleanup progress.
Oldest-pending age is especially useful because a small queue can still contain one event that has been stuck for a long time. Queue depth alone can hide that condition.
Tracing can carry the request correlation identifier into the outbox row and then into broker headers. The relay runs later, so this explicit context preserves the connection between the original transaction and asynchronous delivery.
The pattern narrows the atomicity boundary
Transactional outbox does not turn a database and broker into one transaction. It changes the problem into two stages with a durable handoff between them.
The first stage atomically commits business state plus publication intent. The second stage delivers that intent with retry and duplicate handling. This division gives each failure a recoverable state: before commit there is nothing to publish; after commit there is a durable row the relay can continue processing.
That boundary is the core value of the pattern. The system accepts at-least-once publication and consumer idempotency in exchange for removing the fragile interval where committed business state can permanently lose its corresponding event.