A cache entry expires at a single instant, but requests for that entry do not necessarily arrive one at a time. If twenty callers observe the same miss before any caller has repopulated the cache, a conventional cache-aside path can send twenty equivalent reads to the origin. The cache is functioning according to its contract; the concurrency around the miss is creating duplicated work.

Request coalescing changes that boundary. Instead of treating each miss as permission to start an origin operation, callers for the same key can share one in-flight operation. One caller becomes the producer of the pending result. Other callers wait for that result rather than starting equivalent work.

The mechanism is small, but its semantics are more specific than a generic lock. Coalescing groups work by identity, shares completion, and defines what happens to errors, cancellation, and late arrivals. Those details determine whether the mechanism actually bounds duplicate work or merely moves contention to another place.

A miss is not yet an origin request

A basic cache-aside read often has this shape:

value = cache.get(key)
if value exists:
    return value

value = origin.get(key)
cache.set(key, value)
return value

Nothing in that protocol prevents two callers from passing the miss check concurrently. With an empty cache and three overlapping callers, the execution can become:

A: miss ---- origin read ---------------- set
B:   miss ---- origin read -------------- set
C:     miss ---- origin read ------------ set

If the three origin reads are equivalent, the additional operations do not provide additional information. They exist because the cache records completed values but does not represent work currently in progress.

Coalescing introduces that missing state:

cache state:     absent -> loading -> present
                           ^   |
                           |   +-- shared completion
                           +------ one producer

The loading state need not live inside the cache itself. It can be an in-process map from keys to promises, futures, channels, or another completion primitive. The important property is that concurrent callers can discover an existing operation for the same logical key.

This makes coalescing distinct from caching. A cache shares completed results across time. Coalescing shares an unfinished computation across overlapping callers. A system can use either mechanism independently, although they often complement each other.

The key defines the equivalence class

Coalescing is safe only when the grouping key captures every input that can change the result.

Suppose a service fetches a rendered document by document_id, but the result also depends on locale and authorization scope. Grouping only by document_id can make callers with materially different requests share one computation. The concurrency mechanism would then alter application semantics rather than merely eliminate duplicate work.

A suitable key represents the equivalence relation the origin operation already assumes. If two calls with the same key are allowed to share one result, then the inputs omitted from that key must either be irrelevant, constant within the scope of the coalescer, or incorporated into the operation by another safe boundary.

This concern also appears with mutable request options. A timeout, consistency level, projection, or feature flag can affect the operation even when the resource identifier is identical. Coalescing by resource name alone is valid only if those differences do not change the result that callers are entitled to receive.

The same precision applies to cache keys. Coalescing does not repair an underspecified cache identity; it can amplify the consequence by sharing the mismatched computation before any value reaches the cache.

Completion has to fan out both values and errors

The producer does not always return a value. It can fail before reading the origin, receive an origin error, or discover that the requested object does not exist. Waiting callers need a defined outcome for each case.

A common model is to share the producer’s exact completion with all current waiters. If the origin read succeeds, they receive the same logical value. If it fails, they observe the same failure. The in-flight entry is then removed so a later caller can attempt a fresh operation.

That removal point matters. Removing the entry before publishing completion can open a small window in which a new caller starts a second origin request while existing waiters are still being released. Removing it after completion can briefly allow a late caller to attach to an operation that has already finished. The latter is often harmless when the completion primitive retains its result, but the desired behavior should be explicit.

Errors also interact with retry policy. If every waiter independently retries immediately after a shared failure, coalescing can produce repeated waves of synchronized work:

shared attempt 1 -> error
many waiters retry
shared attempt 2 -> error
many waiters retry

The coalescer has reduced each wave to one origin operation, but it has not introduced spacing between waves. Retry delay, jitter, attempt budgets, or a short-lived negative result address a different part of the behavior.

Cancellation exposes two lifetimes

A shared operation has at least two relevant lifetimes: each caller’s willingness to wait and the producer computation itself.

Those lifetimes should not be conflated. If caller A starts an origin read and caller B joins it, cancellation by B does not automatically imply that the origin read has become useless. A may still need the result. Conversely, keeping an expensive origin operation alive after every waiter has departed may waste resources when the operation supports safe cancellation.

This creates several valid policies.

A simple policy lets each waiter cancel only its own wait while the shared computation continues. This avoids one caller terminating work needed by others. It also means the producer can outlive all current callers.

A reference-counted policy can cancel the producer after the final waiter leaves, provided the underlying operation supports cancellation and no other subsystem depends on its completion. That requires careful synchronization because a new waiter can arrive near the transition to zero.

Another policy gives the producer an independent deadline derived from service-level constraints rather than from any individual caller. Waiters may have shorter deadlines, while the shared operation has enough time to populate the cache for later traffic.

No single policy fits every operation. The key point is that caller cancellation and shared-work cancellation are separate state transitions. Treating the first cancellation signal as authority to abort the shared operation gives an arbitrary waiter control over other callers.

Coalescing bounds duplication, not latency

When ten callers share one slow origin read, the origin sees one operation instead of ten, but all ten callers can still wait for the slow result. Coalescing therefore changes load amplification without making the origin operation itself faster.

It can also concentrate latency. Callers that might have issued independent requests to replicated or variable-latency backends now share the fate of one selected attempt. If independent attempts have meaningful hedging semantics, unconditional coalescing can remove that diversity.

The trade is easiest to see as a concurrency decision rather than a cache optimization. Coalescing says that equivalent overlapping work should have one producer. That is valuable when duplicate work is costly or can overload a dependency. It is less attractive when independent attempts intentionally provide redundancy and the dependency can absorb them.

A coalescer also needs bounded internal state. If requests continually arrive for unique keys, a per-key in-flight map can grow with the number of concurrent unique operations. Entries should disappear on every completion path, including errors and cancellation. Global concurrency limits may still be necessary because coalescing only merges calls that share a key.

For example, one thousand simultaneous misses for one key can become one origin request. One thousand simultaneous misses for one thousand keys remain one thousand distinct operations unless another capacity control intervenes.

Hot keys and expiration boundaries

Expiration makes coalescing especially visible because many callers can cross from a cached value to a miss at nearly the same time. A hot key with a fixed time-to-live can therefore alternate between cheap cache hits and a burst of origin demand.

Coalescing collapses the concurrent miss burst, but it does not remove the sharp expiration boundary. Other cache strategies change that boundary itself. Refresh-ahead starts replacement work before expiry. Stale-while-revalidate permits some callers to receive an older acceptable value while one refresh proceeds. Randomized expiration can spread refresh times across related entries.

These mechanisms solve adjacent problems and can be combined. Coalescing controls duplicate in-flight work. Stale serving controls whether callers must wait during refresh. Expiration policy controls when refresh pressure appears.

The distinction is useful because each mechanism carries different correctness conditions. Serving stale data requires an explicit tolerance for age. Coalescing does not require stale reads; it can make all waiters block for a fresh value. Refresh-ahead requires a trigger before expiry and may perform work for entries that receive no later request.

Process boundaries limit the effect

An in-memory coalescer coordinates only callers that reach the same process. With eight application instances, the same cold key can still produce as many as eight concurrent origin operations if each instance elects its own producer.

That may be entirely acceptable. Reducing a burst from thousands of calls to one call per instance can be sufficient, and local coordination avoids adding a distributed lock or coordination service to the read path.

Cross-process coalescing is possible, but it changes the failure model. A distributed lock or lease needs ownership, expiry, and stale-holder semantics. Waiters also need a way to discover completion, commonly by rechecking the cache rather than receiving an in-process future. The coordination mechanism can become more expensive than the duplicated origin work it was meant to suppress.

The appropriate scope follows the cost being controlled. If the origin tolerates one request per application instance, process-local coalescing keeps the protocol small. If even that multiplicity is unacceptable, the design has moved into distributed coordination and should be evaluated with those semantics in view.

Shared work is its own state

The central design shift is to represent an in-progress computation as something callers can join. Without that representation, a cache miss is only absence, so every caller is free to react independently. With it, absence and active retrieval become different states.

That distinction reaches beyond caches. Metadata refreshes, token renewal, configuration fetches, compilation, and other keyed computations can all have periods where concurrent callers request the same result before it exists. Coalescing applies when those calls are genuinely equivalent and sharing one completion preserves their contract.

Its value is therefore not that it makes a cache smarter. It makes concurrency around missing state explicit. Once in-flight work has identity, the system can reason separately about completed values, active producers, waiting callers, cancellation, and capacity instead of hiding all of them behind the single word miss.

Software Engineering Caching Concurrency Backpressure