Saga Compensation Is Not Transaction Rollback
A multi-service operation can cross inventory, payments, shipping, and other independently committed systems. Once one service commits its step, a later failure cannot make that earlier commit disappear through an ordinary database rollback.
A saga handles this boundary by pairing forward actions with explicit recovery actions. If a later step fails, the coordinator invokes compensations for earlier completed steps where the business process permits them.
Compensation is a new operation. It is not time travel.
Local commits remain real
Consider a checkout flow:
1. reserve inventory
2. charge payment
3. create shipmentEach service owns its own durable state. Suppose inventory reservation commits, then the payment service declines the charge. The inventory transaction has already ended. No transaction coordinator can issue ROLLBACK against that completed local transaction unless the architecture uses a separate distributed transaction protocol from the start.
The saga instead sends another command:
reserve inventory -> committed
charge payment -> declined
release inventory -> compensationThe final business state may resemble the state before reservation, but the history is different. A reservation existed for some interval. Logs, metrics, audit records, notifications, caches, or other observers may already have seen it.
That distinction matters whenever a forward action has effects beyond one mutable row.
A compensation needs domain semantics
A generic inverse such as “subtract what was added” is often insufficient. The correct compensation depends on the business state at the time recovery runs.
A hotel booking may be cancellable before check-in but subject to a fee later. A shipment may be cancellable before carrier pickup but require a return process afterward. A payment authorization may be voided before capture, while a captured payment may require a refund.
These are different domain operations, not mechanical inverses.
The saga definition therefore needs to state which forward steps are compensatable, which compensation applies in each state, and which steps become irreversible after a boundary.
authorize payment -> void authorization
capture payment -> refund payment
ship parcel -> return workflow, not "unship"Naming compensation after the real business action keeps this distinction visible in code and operations.
Compensation can fail too
Recovery code runs across the same unreliable network and service boundaries as forward code. A timeout, service outage, concurrency conflict, or policy rejection can make compensation fail.
A saga coordinator must persist progress so recovery can continue after process restarts. Keeping the entire saga state only in memory turns coordinator failure into lost recovery work.
A minimal state model might record:
saga_id: checkout-918
step: payment
status: compensating
completed:
- reserve_inventory
pending_compensation:
- release_inventoryThe exact representation varies, but the coordinator needs enough durable information to decide which action is next without reconstructing intent from logs.
Retries also need stable command identity. If a timeout leaves the coordinator uncertain whether release_inventory succeeded, sending the same logical compensation again should not release inventory twice. Idempotent handlers or deduplication keyed by a stable command identifier are common safeguards.
Timeouts create uncertainty, not proof of failure
A network timeout does not establish that the remote operation failed. The remote service may have committed and lost the response, or the request may never have arrived.
This ambiguity affects both forward steps and compensations.
coordinator -> charge payment
X response timeout
possible remote states:
- charge never started
- charge failed
- charge committedImmediately compensating on the assumption that the charge failed can race with a successful late result. The protocol needs a way to resolve uncertain outcomes, such as an idempotent retry, a status query keyed by operation ID, or a service contract that returns the prior result for repeated commands.
Timeout policy is therefore part of saga correctness, not merely a latency setting.
Concurrent changes can invalidate a simple inverse
State may continue changing while a saga is active. A compensation written as a blind update can erase legitimate work performed after the original step.
Suppose a saga reserves five units, another process adjusts inventory, then compensation writes an old absolute quantity back. That replacement can overwrite the later adjustment.
Operations that express intent are safer:
forward: create reservation R for 5 units
compensate: cancel reservation RThe compensation refers to the artifact created by the forward step instead of reconstructing a previous global value.
Conditional writes, version checks, and domain invariants can also prevent a compensation from applying against state that no longer permits it.
Ordering compensation in reverse is common, not universal
For a linear saga, compensating completed steps in reverse order is a useful default:
A -> B -> C -> D fails
|
compensate C -> B -> ADependencies often make reverse order natural. A resource created by A may still be required while compensating B.
Real workflows can form a graph rather than a simple chain. Independent branches may compensate in parallel, while some actions have dependency constraints that require a specific sequence. The saga model should encode those dependencies instead of assuming stack order always matches business requirements.
A step can also be a pivot: after it succeeds, the workflow moves into a phase that cannot be compensated in the original sense. Later failures then require forward recovery, manual intervention, or another business process.
Orchestration and choreography place state in different locations
An orchestrated saga uses a coordinator that sends commands and records progress. This makes the control flow and recovery state explicit in one component.
A choreographed saga lets services react to events and emit subsequent events. It can reduce central coordination, but the overall workflow becomes distributed across event handlers. Recovery still needs explicit semantics; removing a central coordinator does not remove compensation, deduplication, ordering, or observability concerns.
The choice changes where control state lives, not the fundamental commit boundary. Every participating service still commits locally.
For long or high-value workflows, an explicit saga identifier carried through commands, events, logs, and traces helps operators reconstruct progress across services.
Side effects need classification before implementation
Some effects are reversible, some are compensatable only through a different action, and some cannot be retracted.
An internal reservation can usually be cancelled. An email already delivered cannot be withdrawn from a recipient’s mailbox. A physical package already handed to a carrier cannot be made “unshipped.” A third-party API may offer cancellation only during a limited window.
A useful design review classifies each step before the workflow is implemented:
- local atomic change;
- retryable remote action;
- compensatable action;
- irreversible or externally visible action;
- action requiring human resolution after a boundary.
Irreversible actions are often placed late in the saga, after failure-prone reversible work has completed. That reduces the number of states requiring exceptional recovery, though it cannot eliminate every failure mode.
Observability should show forward and recovery progress
A saga that is merely marked failed hides the operational state that matters. Failure before any durable step differs sharply from a saga with three committed steps and one compensation still retrying.
Useful signals include current saga phase, age, last successful step, pending compensation, retry count, uncertain remote outcomes, and terminal manual-review state. Metrics should use bounded labels; individual saga identifiers belong in traces or logs rather than high-cardinality metric labels.
Operators also need a safe way to resume or resolve stuck sagas without repeating completed effects. Administrative actions should use the same durable state and idempotency rules as automatic recovery.
The guarantee is coordinated recovery
A saga does not provide isolation across services, and it does not make a sequence of local commits equivalent to one ACID transaction. Other actors can observe intermediate state, and compensation itself may take time.
Its useful guarantee is narrower: the system records workflow progress and applies defined recovery actions when the forward path cannot continue.
That contract becomes reliable only when compensations have real domain semantics, ambiguous outcomes can be reconciled, commands tolerate retry, concurrency is guarded, and irreversible boundaries are explicit. Treating compensation as ordinary rollback conceals each of those engineering obligations.