A service can receive many requests for the same expensive result at almost the same time. If every request starts identical work, a brief traffic burst can become a much larger burst against a database, remote API, filesystem, or CPU-heavy computation.

Caching can help after a result exists. It does not necessarily help when the result is missing and many callers discover that miss together.

Request coalescing solves this narrower problem. While an operation for a particular key is already running, later callers for the same key join that operation instead of starting another one. When it finishes, the waiting callers receive the same outcome.

This article develops the mental model, shows the coordination required to make it correct, and explains where coalescing helps and where it creates new risks.

Start with the duplicate-work problem

Suppose an application builds a product summary by loading several records and performing an expensive calculation:

getSummary("P42")
    -> load data
    -> calculate summary
    -> return result

One call is fine. Now imagine 100 requests for product P42 arrive during the 200 milliseconds needed to build the summary.

Without coordination, the application may start 100 copies of the same operation:

caller 1  -> build P42
caller 2  -> build P42
caller 3  -> build P42
...
caller 100 -> build P42

The requests are not merely concurrent. They are asking for work that is equivalent for the purpose of this operation.

If each build performs four database reads, the burst can create roughly 400 reads. The exact cost depends on the implementation, but the amplification mechanism is simple: more callers create more copies of work that could have been shared.

With coalescing, the shape changes:

caller 1 ----\
caller 2 -----+--> one build for P42 --> shared outcome
caller 3 ----/

The first caller starts the work. Later callers find that work already in progress and wait for it. The important boundary is the in-flight lifetime: sharing lasts only while that operation is running.

Think in terms of a keyed in-flight registry

The core data structure is a registry from a work key to an operation that has not finished yet:

in_flight = {
    "P42": operation A,
    "P99": operation B
}

A caller requesting P42 follows this decision:

if an operation for P42 is already in flight:
    join it
else:
    create one
    register it under P42
    run it

When the operation finishes, its entry is removed.

This differs from a normal result cache:

cache:      key -> completed value kept for some lifetime
coalescing: key -> unfinished operation kept until completion

A system can use both. For example, it may first check a cache, coalesce concurrent misses, then place the completed result in the cache. But the two mechanisms solve different timing problems and should be reasoned about separately.

The key defines which work is safe to share

The most important design decision is not the lock or promise type. It is the key.

Two callers may share an operation only when one execution can legitimately satisfy both requests.

Suppose a price calculation depends on product, currency, and customer tier:

price(product_id, currency, customer_tier)

Using only product_id as the coalescing key would be wrong:

bad key:  "P42"

A request for P42 in EUR could accidentally join a request for P42 in USD. The system would save work by returning the wrong result.

The key must include every input that can change the outcome and that is relevant to sharing:

better key: (product_id, currency, customer_tier)

Authentication, permissions, locale, feature configuration, consistency requirements, and request options can also matter. They do not all belong in every key. The rule is more precise:

If two requests can require different outcomes, they must not be coalesced merely because some of their inputs match.

When defining a key, write down what makes two executions equivalent. If that equivalence is difficult to state, coalescing may be premature.

Coordination must make creation atomic

A naive implementation can still start duplicate work:

if key not in in_flight:
    in_flight[key] = start_work(key)

return wait_for(in_flight[key])

Two callers can both observe that the key is absent before either inserts its operation:

caller A: checks P42 -> absent
caller B: checks P42 -> absent
caller A: starts operation A
caller B: starts operation B

The check-and-create step therefore needs synchronization appropriate to the runtime: a lock, an atomic map operation, an actor or event-loop owner, or another mechanism that guarantees only one operation becomes the registered operation for a key.

In language-neutral pseudocode, the intent looks like this:

function get_or_join(key):
    lock registry

    if key exists:
        operation = registry[key]
        unlock registry
        return wait(operation)

    operation = new_operation()
    registry[key] = operation
    unlock registry

    run operation
    return wait(operation)

The expensive work should not normally run while the registry lock is held. The lock protects coordination, not the entire operation. Holding it during I/O or expensive computation would serialize unrelated keys and turn a small critical section into a bottleneck.

Real implementations also need careful cleanup, which is where many subtle bugs appear.

Share the outcome, including failure

Suppose the single operation for P42 fails because its dependency times out. What should the joined callers receive?

Usually, they should observe that same failure. If every waiter responds to the shared failure by independently starting replacement work immediately, the system has recreated the original amplification problem.

The shared object therefore needs to represent an outcome, not only a successful value:

operation completes
    |
    +-- success -> wake waiters with value
    |
    +-- failure -> wake waiters with error

After completion, remove the operation from the in-flight registry so a later request can make a fresh attempt according to the application’s retry and backoff policy.

Cleanup must happen on every completion path. If failed or cancelled operations remain registered forever, future callers may keep joining an operation that can no longer produce useful work. A finally-style cleanup path is a common way to express this requirement in runtimes that support it.

There is one race to handle deliberately: cleanup for an old operation must not remove a newer operation registered under the same key. One safe design removes an entry only if the registry still points to the operation being completed.

Conceptually:

if registry[key] is this_operation:
    remove registry[key]

That identity check prevents an old completion from deleting newer in-flight state.

Decide what cancellation means

Coalescing changes cancellation semantics because several callers now depend on one operation.

If caller A starts the work and caller B joins it, cancelling A should not automatically destroy work that B still needs. Treating the first caller as the permanent owner makes the shared operation accidentally depend on one participant’s lifetime.

A useful mental model separates caller cancellation from shared-work cancellation:

caller cancellation -> this caller stops waiting
shared cancellation -> underlying operation stops

The underlying operation can be cancelled when the application knows no interested callers remain, when its own deadline expires, or when a higher-level shutdown requires it. The exact policy depends on the concurrency model.

This is also why blindly reusing one caller’s timeout for everyone can be wrong. If caller A has 50 milliseconds left and caller B can wait 500 milliseconds, tying the shared operation to A’s deadline may cause B to fail unnecessarily.

There is no universal deadline policy. Common choices include giving the shared work its own bounded deadline, tracking waiter lifetimes, or declining to coalesce requests whose execution constraints differ materially. What matters is that the policy is explicit rather than inherited accidentally from whichever caller arrived first.

Coalescing controls concurrency, not total demand

Request coalescing can dramatically reduce simultaneous duplicate work, but its guarantee is limited.

If 100 callers for P42 overlap while one build is running, one build may serve all 100. If those callers arrive one after another with no overlap, coalescing may save nothing:

request -> build finishes
request -> build finishes
request -> build finishes

A cache can reuse a completed value across that time gap. Coalescing cannot unless it is combined with caching.

Likewise, coalescing does not help much when every request has a different key:

P1, P2, P3, P4, P5, ...

That is a general load problem rather than a duplicate-work problem. Concurrency limits, queues, backpressure, caching, capacity changes, or a cheaper algorithm may be more appropriate.

This distinction prevents a common mistake: adding coalescing because a service is overloaded without first establishing that overlapping equivalent work is a meaningful part of the load.

Watch for hot keys

Coalescing reduces work for a hot key, but it also concentrates many callers on one outcome.

If 10,000 callers join one slow operation, the dependency may see only one request, which is useful. The application still has 10,000 callers waiting. They consume whatever resources the runtime associates with waiting requests, and they may all wake at nearly the same time.

That creates several operational questions:

  • How many waiters can join one key?
  • What happens when that limit is reached?
  • How long may the shared operation run?
  • Does releasing many waiters create a burst of follow-up work?
  • Can one pathological key consume too much memory?

Coalescing is therefore compatible with, rather than a replacement for, bounded queues, deadlines, admission control, and resource isolation.

A practical implementation should expose at least enough telemetry to answer whether it is helping. Useful measurements include the number of operations started, the number of callers that joined existing operations, operation duration, waiter counts, and shared failures. These measurements reveal both savings and hot-key pressure.

Do not coalesce work with unsafe side effects

Sharing is easiest to reason about for read-like or computation-like operations whose result depends only on the key and relevant context.

Side-effecting operations require more care.

Consider:

charge_card(order_id)

Two concurrent calls with the same order_id might be accidental duplicates, or they might represent distinct attempts whose semantics are governed by an idempotency contract. Simply making one caller wait for the other’s in-process operation does not provide durable duplicate protection. If the process crashes after the charge succeeds but before the result reaches callers, an in-memory coalescing registry disappears.

For durable side effects, use the mechanism required by the operation’s correctness model, such as an idempotency key backed by durable state or a transactional constraint. Coalescing may still reduce simultaneous work as an optimization, but it should not be mistaken for a correctness guarantee across crashes or multiple processes.

The same boundary applies to a multi-instance service. An in-memory registry coordinates callers that reach one process. Two requests routed to different instances can still start duplicate work. Cross-instance coalescing requires distributed coordination, which adds latency, failure modes, and operational complexity. Sometimes duplicate work is cheaper and safer than that coordination.

Prefer the simpler approach when duplication is cheap

Coalescing adds shared mutable state, synchronization, cleanup logic, cancellation policy, and new observability needs. Those costs are justified only when duplicate in-flight work is expensive enough to matter.

A direct call is often better when:

  • the operation is cheap;
  • duplicate calls are rare;
  • requests usually have distinct keys;
  • caller-specific deadlines or permissions make sharing complicated; or
  • coordination would be more expensive than repeating the work.

Start by measuring or otherwise establishing the duplicate-work problem. Then choose the smallest sharing boundary that addresses it. A process-local registry is often enough when bursts are concentrated within individual instances. Distributed coordination should be a response to a demonstrated cross-instance problem, not the default design.

A practical design checklist

Before adding coalescing to an operation, answer these questions in order:

  1. What overlapping work is actually duplicated?
  2. What exact inputs make two executions equivalent?
  3. How is check-and-create made atomic?
  4. How are success, failure, and cleanup shared?
  5. What does one caller cancelling mean for the shared operation?
  6. What bounds the operation lifetime and number of waiters?
  7. Is process-local coordination sufficient?
  8. Which metrics will show whether duplicate work decreased?

If any answer is vague, the implementation probably has an undefined edge case.

Conclusion

Request coalescing is a focused form of concurrency control: concurrent callers asking for equivalent work share one in-flight operation instead of multiplying it.

The useful mental model is a keyed registry of unfinished work. Correctness depends on choosing an equivalence key that includes every result-changing input, creating entries atomically, sharing failures as well as successes, cleaning up safely, and defining cancellation and deadline behavior explicitly.

Use coalescing when overlapping duplicate work is a real source of load. Keep caching, durable idempotency, and general overload control conceptually separate. They can complement coalescing, but each solves a different problem.

Software Engineering Reliability Concurrency Performance