Most calls to a dependency may finish quickly while a small fraction take much longer. A request that depends on one of those slow calls inherits the delay even when another healthy instance could have answered sooner.

Increasing the timeout does not solve this problem. Retrying only after the timeout may also be too late: by then, the caller has already spent most of its latency budget.

A hedged request is a deliberately delayed duplicate of an operation that is still in progress. The original request starts normally. If it has not completed after a chosen delay, the caller sends one additional equivalent request, usually to another eligible instance. The first acceptable result wins, and the remaining work is cancelled or ignored.

The idea is simple, but the engineering decision is not. Hedging trades additional work for a chance to avoid unusually slow responses. This article explains that trade-off, shows how to reason about the delay, and identifies the cases where hedging can make a system worse.

Start with the tail, not the average

Suppose a service calls a dependency whose typical response takes about 40 milliseconds. Occasionally, a call takes 500 milliseconds because one instance is busy, a connection is delayed, or some other transient condition affects that particular attempt.

Averages can hide this problem. If most calls are fast, the average may still look healthy while users experience occasional long waits.

The useful mental model is to separate typical latency from tail latency. Tail latency describes the slower end of the latency distribution: the relatively uncommon requests that take much longer than most others.

Hedging targets that tail. It does not make the normal 40-millisecond call faster. It gives an unusually slow call another path to completion before the caller’s deadline expires.

The smallest useful hedge

Consider a read operation:

result = fetchProfile("user-42")

Without hedging, the caller waits for that one attempt until it succeeds, fails, or reaches its deadline.

With a simple hedge, the control flow becomes:

start attempt A

if A has not finished after hedgeDelay:
    start attempt B

return the first acceptable result
cancel or ignore the other attempt

If attempt A finishes before hedgeDelay, attempt B never starts. That detail is essential. Sending two requests immediately for every operation is replication, not delayed hedging, and it doubles request work even when the first attempt is healthy.

Suppose hedgeDelay is 100 milliseconds. A normal 40-millisecond call produces no extra request. If attempt A is still running at 100 milliseconds, attempt B gets a chance to finish first.

For example:

0 ms      attempt A starts
100 ms    A is still running, so B starts
145 ms    B succeeds
145 ms    caller returns B's result
later     A is cancelled or its result is ignored

The caller finishes at roughly 145 milliseconds instead of waiting for the unusually slow first attempt. The improvement came from spending extra capacity only after the original call looked suspiciously slow.

This example is intentionally simplified. Production code also needs deadlines, cancellation, result validation, routing, observability, and limits on how many hedges may be created.

A hedge is different from a normal retry

Retries and hedges both create another attempt, but they respond to different situations.

A normal retry usually starts after an earlier attempt has failed or timed out. At most one attempt may be active at a time.

A hedge starts while the original attempt is still active because slowness itself is treated as a reason to try another path.

That distinction matters for both latency and load:

sequential retry:
A -------- fails
             B -------- succeeds

hedged request:
A -------------------------
       B ------ succeeds

The hedge can finish sooner because B overlaps A. The cost is that A and B consume resources at the same time.

This also means retry policies should not be copied directly into hedge policies. Three sequential retries may be tolerable in one system, while three concurrent hedges could multiply load at exactly the wrong moment.

Choose the delay from observed latency

A hedge delay should distinguish an unusually slow attempt from an ordinary one. A fixed number chosen without measurements cannot reliably make that distinction.

If the delay is too short, many healthy operations trigger unnecessary duplicates. The system pays extra network, CPU, connection, and dependency cost for little latency benefit.

If the delay is too long, the hedge starts too late to help before the caller’s deadline.

A practical starting point is a high percentile of recent successful latency for the operation, adjusted for the caller’s remaining time budget. The exact percentile is a policy choice, not a universal constant. A service with abundant spare capacity and strict latency goals may hedge more aggressively than a service whose dependency is already expensive or close to saturation.

The important relationship is:

hedgeDelay < remainingDeadline

But that condition alone is not enough. The second attempt also needs enough remaining time to do useful work. Starting a hedge 5 milliseconds before a deadline when the dependency normally needs 40 milliseconds only creates extra load before both attempts are cancelled.

A better decision asks two questions:

  1. Has the first attempt become unusually slow compared with normal successful calls?
  2. Is there enough time and capacity left for another attempt to have a realistic chance of helping?

If either answer is no, do not hedge.

Prefer a different failure path

A duplicate request is most useful when its fate is not strongly tied to the original attempt.

If both attempts go to the same overloaded process over the same constrained connection, the second attempt may reproduce the same delay. Worse, it adds work to the bottleneck.

When the system has multiple equivalent instances or routes, a hedge should usually avoid the instance already serving the original request when the routing layer can do so safely. The goal is not merely to duplicate the call; it is to give the operation another plausible path to completion.

This does not make the attempts independent. They may still share a database, network link, queue, or other downstream resource. Hedging therefore cannot guarantee lower latency. It only helps when the causes of slow attempts have enough variation that another attempt can sometimes escape the original delay.

Only hedge operations whose duplicate execution is acceptable

The most important correctness question is not latency. It is what happens if both attempts execute.

For a pure read, duplicate execution may only consume extra resources. For a command such as chargeCard, reserveSeat, or sendEmail, running the operation twice can change externally visible state twice.

Cancellation does not remove this risk. By the time the caller cancels the losing attempt, that attempt may already have committed its side effect. Cancellation is generally a request to stop work, not proof that no work occurred.

For that reason, hedging is easiest to justify for read-only operations or operations with a well-defined idempotency mechanism. An idempotency key can allow a server to recognize multiple attempts as the same logical operation, but only if the server’s implementation actually guarantees the required duplicate-handling semantics.

Do not infer safety from an HTTP method name, function name, or client-side cancellation alone. Verify the operation’s real effects.

Bound the extra work

Hedging consumes spare capacity to improve latency. It becomes dangerous when spare capacity disappears.

Imagine a dependency slowing down because it is overloaded. More original requests cross the hedge threshold, so callers create more duplicates. Those duplicates increase the dependency’s load, which makes more requests slow, which creates still more hedges.

That is a positive feedback loop:

higher load
   -> more slow requests
   -> more hedges
   -> higher load

A robust hedge policy therefore needs limits. Common controls include allowing at most one hedge per logical operation, limiting the fraction of calls that may hedge, applying a global or per-dependency hedge budget, and suppressing hedges when overload signals show that the dependency lacks spare capacity.

The exact mechanism depends on the system, but the principle is stable: a latency optimization must not have permission to create unbounded work.

If a dependency is routinely saturated, fix the capacity, queueing, or workload problem first. Hedging is not a substitute for adequate capacity.

Treat the attempts as one logical operation

Once two attempts exist, observability can become misleading unless they are linked.

If metrics count each attempt as an independent user request, a 1% hedge rate can look like unexplained traffic growth. If the losing attempt is cancelled and recorded as an ordinary failure, error rates can also become misleading.

Track at least two levels:

logical operation
    attempt A
    attempt B  <- hedge

At the logical-operation level, record whether the operation succeeded and its user-visible latency. At the attempt level, record which attempt was hedged, why it started, which attempt won, whether cancellation succeeded, and how much duplicate work was performed.

Useful operational questions include:

  • What fraction of operations start a hedge?
  • How often does the hedge win?
  • How much does hedging change tail latency?
  • How much extra attempt traffic does it create?
  • Are hedges concentrated on particular instances or routes?
  • Does hedge activity increase during overload?

A hedge policy that reduces latency in a benchmark but cannot be explained in production is difficult to operate safely.

Handle success and failure deliberately

“First response wins” is often too simplistic. The caller usually wants the first acceptable result.

Suppose attempt A quickly returns a transient transport error while attempt B is still running and may succeed. Returning A’s failure immediately defeats the purpose of the hedge.

A more useful policy can distinguish terminal results from results for which another active attempt should still be allowed to finish. The exact rules depend on the operation’s contract.

For example:

A returns success        -> return A, cancel B
A returns terminal error -> return A, cancel B
A returns retryable error while B runs
                         -> keep waiting for B within deadline

Do not make the rule so complicated that the caller silently changes application semantics. Authentication failures, validation failures, not-found results, and transport failures may have very different meanings. Define which outcomes are equivalent across attempts and which are final.

Respect one end-to-end deadline

Each hedge should inherit the logical operation’s existing deadline rather than receiving a fresh full timeout.

Suppose the caller has a 300-millisecond deadline and starts a hedge after 120 milliseconds. Giving the hedge a new 300-millisecond timeout could allow the logical operation to run for about 420 milliseconds, violating the caller’s budget.

Instead, both attempts operate inside the same remaining deadline:

logical deadline: 300 ms

0 ms      A starts
120 ms    B starts
300 ms    both must be finished or cancelled

The hedge changes how the remaining time is spent. It should not silently extend the amount of time the caller agreed to wait.

Common mistakes

Hedging every request immediately

This spends roughly twice the attempt capacity even when the original call would have completed normally. Delay the hedge so normal requests remain single-attempt operations.

Hedging writes without duplicate protection

The losing attempt can still commit. Use hedging only when duplicate execution is acceptable or the operation provides a verified idempotency guarantee.

Starting unlimited hedges

A slow dependency can trigger a traffic multiplier. Bound hedges per operation and across the dependency.

Using hedging to hide permanent slowness

If most calls are slow, the problem is no longer an occasional tail event. Duplicating most calls adds cost without addressing the underlying bottleneck.

Ignoring shared bottlenecks

Two attempts that share the same saturated resource may fail together. Prefer a meaningfully different route when possible, and measure whether hedges actually escape slow paths.

Resetting the timeout for the hedge

A new full timeout lets a secondary attempt outlive the caller’s original latency budget. Propagate the existing deadline instead.

When hedging is a reasonable tool

Hedging is most promising when several conditions hold together: most operations are fast, a small tail is much slower, equivalent alternative execution paths exist, duplicate execution is safe, the system has spare capacity, and user-visible latency matters enough to justify extra work.

It is a poor fit when the dependency is generally overloaded, operations have unsafe side effects, every attempt shares the same bottleneck, request cost is high, or the latency objective does not justify additional capacity.

A simpler timeout and retry policy is often preferable when failures are clear and waiting for an attempt to fail does not consume an important part of the latency budget. Simpler policies are easier to reason about and create less concurrent work.

Conclusion

A hedged request is not “retry sooner.” It is a controlled decision to spend additional capacity while an original attempt is still running, hoping that another path can avoid an unusually slow result.

Use that trade deliberately. Start with observed tail latency, delay the duplicate so normal calls stay cheap, keep every attempt inside one deadline, verify that duplicate execution is safe, route the hedge through a meaningfully different path when possible, and bound the amount of extra work.

The practical test is straightforward: hedging should reduce user-visible tail latency enough to justify the additional attempt traffic without destabilizing the dependency. If measurements do not show that trade working in your system, the simpler design is the better one.

Software Engineering Reliability Performance Distributed Systems