Backpressure

Request Coalescing Turns Cache Miss Bursts Into Shared Work

A cache entry expires at a single instant, but requests for that entry do not necessarily arrive one at a time. If twenty callers observe the same miss before any caller has repopulated the cache, a conventional cache-aside path can send twenty equivalent reads to the origin. The cache is functioning according to its contract; the concurrency around the miss is creating duplicated work. Request coalescing changes that boundary. Instead of treating each miss as permission to start an origin operation, callers for the same key can share one in-flight operation. One caller becomes the producer of the pending result. Other callers wait for that result rather than starting equivalent work.