A queue consumer receives a message, rejects it, and receives the same message again. That cycle is useful when the rejection came from a transient condition. It is structurally different when the payload can never be processed by the current consumer. The broker can keep honoring redelivery semantics while the application makes no forward progress on that message.
Such a message is commonly called a poison message. The important property is not that it contains malformed bytes. A syntactically valid message can be permanently unprocessable because its schema is unsupported, a required invariant is violated, referenced data can never exist, or application logic deterministically rejects its state.
Once a failure is permanent, retry policy stops being only a recovery mechanism. It becomes a decision about how long the system will retain failed work in the active delivery path, how that work is isolated, and what evidence remains for later inspection.
Redelivery cannot distinguish failure classes by itself
A broker generally observes delivery outcomes, not the full semantics of application failure. If a consumer fails before acknowledging a message, redelivery can preserve at-least-once processing behavior. That is valuable for failures such as a temporary database outage or a process termination between receipt and acknowledgement.
The same mechanism can repeatedly present a deterministic failure. Suppose a consumer accepts events with schema versions 2 and 3, while one queued event declares version 7. If the consumer rejects version 7 on every attempt, additional deliveries do not alter the compatibility relation.
The distinction is semantic:
transient failure: same input + changed environment -> success may become possible
permanent failure: same input + relevant state unchanged -> rejection remains expectedA retry counter does not prove permanence. A dependency can remain unavailable for many attempts and later recover. Conversely, a validation error can be permanent on its first occurrence. Delivery count is therefore an operational signal, not a complete classifier.
This matters because unlimited redelivery gives a permanent failure an unlimited claim on active queue capacity. The exact effect depends on broker ordering, partitioning, prefetch, concurrency, and acknowledgement rules. In a strictly ordered partition, one repeatedly failing record may prevent later records in that partition from advancing. In a system that permits concurrent or out-of-order handling, later messages may continue while the failed record consumes repeated delivery attempts.
A delivery limit creates a state transition
A bounded retry policy changes the model. After some configured condition, the message leaves ordinary delivery and enters another state. That state may be represented by a dead-letter queue, a quarantine topic, a failed-message table, or broker-specific dead-letter routing.
The transition is more significant than its storage location. Before the limit, the system still treats the message as candidate work for the normal consumer. After the limit, the system records that normal delivery has been suspended under the current policy.
That boundary can be expressed as a small state machine:
active -> retryable -> active
|
+---- delivery limit ----> quarantinedThe limit may be based on attempt count, elapsed time, explicit error classification, or a combination. Each choice encodes different assumptions. Attempt limits are simple but treat rapid and slow failures alike. Time limits express a retention window but require suitable time semantics. Error classification can separate known permanent conditions early, yet only when the consumer can classify them accurately.
Quarantine does not make the message correct. It prevents one failed unit of work from remaining indefinitely in the ordinary delivery loop and preserves a durable representation of the unresolved condition.
A dead-letter queue is not an error log
Treating dead-letter storage as a log loses an important property: the stored item is still application data with delivery history and recovery implications.
Useful quarantine records commonly need enough context to distinguish the original message from the failure observations around it. Depending on the transport, that can include the original payload, message identifier, source destination, schema or event type, delivery count, failure category, and timestamps recorded by the systems that own those values.
The error text from the final attempt is not necessarily a complete diagnosis. Earlier attempts may have failed for different reasons. A consumer might first encounter a network timeout and later reject the same message after a schema check. If only the last exception survives, the record describes the terminal observation rather than the entire retry history.
There is also a data-governance consequence. Copying messages into another durable destination duplicates whatever sensitive or regulated fields those messages contain. Retention, access control, encryption, and deletion rules therefore apply to quarantine storage according to the data it actually holds. A dead-letter destination is not exempt merely because its purpose is operational.
Replay changes the duplicate boundary
Quarantined messages often need a path back into processing after code, configuration, or referenced state changes. Replay is not equivalent to first delivery. The message has already crossed part of the system and may have produced side effects before a later operation failed.
Consider a consumer that performs two effects and acknowledges only after both complete:
write database row
call external service
acknowledge messageIf the database write commits and the external call fails, the message can be retried or quarantined after a partial effect. Replaying it later repeats the consumer from a state that differs from the original attempt.
This is the same fundamental pressure that at-least-once delivery places on consumer design: acknowledgement boundaries do not automatically make side effects atomic. Idempotent operations, deduplication state, transactional boundaries, or compensating semantics may be needed according to the effects involved.
A replay tool that simply copies payloads back to the source queue can also erase useful identity. If it assigns a new application message identifier, deduplication keyed by the original identifier may no longer recognize the operation. If it preserves the identifier, downstream components must tolerate seeing that identifier again. Neither choice is universally correct; identity semantics need to match the system’s duplicate-detection model.
Ordering and quarantine pull in opposite directions
Moving a failed message aside improves progress only if the application permits later messages to overtake it. Some streams encode ordering as part of their correctness model.
If event B is meaningful only after event A has been applied, quarantining A and processing B may exchange a liveness problem for an invalid state transition. The broker can deliver B successfully while the application-level sequence is incomplete.
Ordering scope is therefore part of poison-message handling. A queue with independent jobs can often isolate one failed job without affecting unrelated work. A partitioned event stream may require pausing only the affected key or partition. A globally ordered sequence has fewer isolation options because skipping an item changes the sequence observed by consumers.
The right boundary follows the dependency relation among messages, not the mere existence of a dead-letter feature in the broker.
Failure classification belongs near application semantics
Infrastructure can count attempts and route messages after a threshold, but it cannot reliably infer every permanent application failure. The consumer often has stronger evidence.
A parser can identify invalid encoding. A schema boundary can reject an unsupported version. Domain validation can identify a state that violates an invariant. A downstream timeout, in contrast, may provide no basis for declaring the message permanently invalid.
This suggests a separation of responsibilities. The application can expose failure categories when it has semantic evidence, while transport policy can bound cases that remain ambiguous. Known permanent failures can move directly to quarantine. Unknown failures can receive a finite retry budget rather than being classified as permanent merely because they happened repeatedly.
The categories should remain small enough to carry operational meaning. Encoding every exception class into routing policy couples the broker configuration to implementation details that may change without altering the actual recovery semantics.
Quarantine makes unresolved work explicit
A poison message exposes a boundary that ordinary retry logic can hide. Redelivery says that an unacknowledged unit of work remains eligible for another attempt. It does not say that another attempt has a credible path to success.
A bounded transition to quarantine makes that distinction durable. The active queue contains work still admitted for normal processing; the quarantine destination contains work whose normal delivery policy has been exhausted or explicitly rejected. Replay then becomes a separate operation with its own identity, ordering, and side-effect constraints.
That separation does not eliminate failure. It prevents permanent rejection from masquerading indefinitely as transient recovery, and it gives unresolved messages a state that the rest of the system can reason about.