Apps Artificial Intelligence Cloud Computing CSS Cybersecurity Data Science Database Go JavaScript Linux Python Rust Software Engineering Web Development

Coalesce Duplicate Work with Single-Flight Patterns in Go

3 min read .
Published on
Coalesce Duplicate Work with Single-Flight Patterns in Go

Concurrent services often receive bursts of requests for the same expensive value: configuration, a database row, a rendered artifact, or a remote API response. A cache helps after the first request completes, but it does not stop ten simultaneous cache misses from doing the same work ten times.

A single-flight pattern lets one caller perform the work while other callers wait for that result.

The cache-miss stampede problem

Without coordination, several callers can observe the same miss and all call the dependency. Single-flight changes that behavior so one request becomes the leader and later requests for the same key become followers.

This is sometimes called request collapsing or request coalescing.

Coalescing is not caching

A cache reuses a completed result across time. Single-flight shares work only while that work is currently running.

A common flow is:

  1. check the cache;
  2. on a miss, join or create an in-flight operation for the key;
  3. perform the expensive call once;
  4. populate the cache;
  5. return the shared result.

Without step two, concurrent misses can still multiply load.

Choose the coalescing key carefully

Only callers that can safely share the exact result should use the same key.

A good key may include:

tenant_id + resource_id + representation_version

A key based only on resource_id may be unsafe if authorization, locale, pricing tier, or request options change the response.

At the other extreme, including highly variable request values can make every key unique and eliminate the benefit.

Cancellation needs explicit semantics

The hardest design question is what happens when one waiting caller cancels.

Usually, one caller disconnecting should not cancel shared work that other callers still need. If the leader’s request context directly controls the upstream operation, its cancellation can incorrectly abort everyone.

A robust design separates each caller’s waiting context from the lifetime of the shared operation. A follower may stop waiting when its own context is done while the shared operation continues for remaining consumers.

Bound shared work with its own timeout

Coalescing does not make a slow dependency safe. The shared operation still needs a deadline.

Do not automatically inherit the shortest caller deadline for all participants. Give the underlying operation a service-level timeout appropriate to the dependency, then let individual callers stop waiting independently.

This prevents one impatient client from controlling work that is useful to others.

Decide how errors are shared

If the upstream call fails, all callers currently joined to that flight will usually observe the same error. That is often correct, but it affects caching.

Do not store transient failures in a long-lived success cache unless negative caching is intentional. One temporary timeout should not poison future requests.

Stable “not found” results may justify short negative caching, but distinguish those from connection failures and overloaded dependencies.

Prevent unbounded key growth

A single-flight registry normally keeps one entry per active key. Entries must be removed after completion, including failure paths.

If arbitrary keys can be created and operations can hang indefinitely, the registry becomes a memory pressure point. Apply operation timeouts and normalize keys before using them for coordination.

Observe the pattern

Useful metrics include:

  • number of newly started flights;
  • number of followers that joined existing flights;
  • shared operation duration;
  • errors by operation type;
  • number of active keys.

A high follower ratio may indicate healthy request collapsing, or it may reveal a cache entry that expires too aggressively.

When not to coalesce

Do not use single-flight for operations with per-request side effects such as creating orders, consuming one-time tokens, or incrementing counters. Sharing those operations changes semantics.

The pattern is best for idempotent reads or deterministic computation where every caller for a key is entitled to the same result.

Also avoid adding coordination when the work is already cheap. Locks, bookkeeping, and cancellation behavior add complexity.

Combine it with expiration strategy

Single-flight controls the burst after expiration, but other techniques can reduce the burst further:

  • randomize TTLs so many keys do not expire together;
  • serve stale data while one request refreshes it where business rules permit;
  • proactively refresh extremely hot keys.

These mechanisms solve different parts of the same reliability problem.

Conclusion

Single-flight is a concurrency-control tool, not merely a performance trick. It limits duplicate work during simultaneous misses and protects expensive dependencies. The important design choices are the key, cancellation policy, error handling, and operation deadline. Define those semantics before adding request coalescing to production code.

Golang Concurrency Backend Performance

Related Posts

Bounded Concurrency in Go with a Worker Pool
Published on
Bounded Concurrency in Go with a Worker Pool
Goroutines are cheap, but the resources they call are often not. Starting one goroutine for every item in a large batch can overwhelm a database connection pool, trigger API rate limits, exhaust file descriptors, or create avoidable memory pressure. Bounded concurrency solves this by allowing only a fixed number of operations to run at the same time. A worker pool is one of the simplest standard-library patterns for implementing that limit in Go.
Golang Concurrency Backend Performance
Request Coalescing in Go Without Extra Dependencies
Published on
Request Coalescing in Go Without Extra Dependencies
When many requests ask for the same expensive resource at the same time, running identical work for every caller can overload a database, API, or filesystem. A cache can help after a result exists, but it does not necessarily prevent several concurrent cache misses from triggering the same backend operation. Request coalescing solves a different problem: while one operation for a key is already running, later callers wait for that operation and share its result. After the operation finishes, the result is forgotten. The next request starts fresh work.
Golang Concurrency Backend Performance
Atomic File Writes in Go: Prevent Partial and Corrupted Files
Published on
Atomic File Writes in Go: Prevent Partial and Corrupted Files
Writing a file with os.WriteFile is simple, but it is not always the safest choice for configuration files, generated metadata, caches, state files, or other data that must never be left half-written. If a process crashes or the machine loses power while a file is being replaced, readers may observe incomplete content. A common way to reduce this risk is an atomic file write: write the new content to a temporary file first, then replace the destination with a rename.
Golang Files Backend Reliability
Exponential Backoff with Jitter in Go
Published on
Exponential Backoff with Jitter in Go
Retries can make distributed systems more resilient, but immediate retries can also make an outage worse. If thousands of clients retry at the same moment, a recovering dependency receives another synchronized burst of traffic before it has time to stabilize. A common solution is exponential backoff with jitter: increase the maximum delay after each failure, then randomize the actual wait. This article builds that pattern with Go’s standard library and shows where retry logic belongs—and where it does not.
Golang Retries Reliability Backend
Go Context Timeouts and Request Cancellation
Published on
Go Context Timeouts and Request Cancellation
A Go HTTP handler can outlive the request that started it unless the work inside the handler pays attention to cancellation. That matters when a client disconnects, an upstream request takes too long, or a database query is no longer useful. Go solves this with context.Context. Every incoming *http.Request already has a context, and that context is canceled when the client connection closes, the request is canceled by HTTP/2, or the handler returns. You can also derive a shorter deadline for work that should not consume the entire request lifetime.
Golang Backend HTTP
Go Error Wrapping with errors.Is and errors.As
Published on
Go Error Wrapping with errors.Is and errors.As
Errors often cross several layers of a Go application. A low-level function may know that a file is missing, while a higher-level function needs to add context about which operation failed. Go error wrapping lets you add that context without losing information callers need for reliable handling. Why error strings are fragile Do not make program logic depend on error wording. Adding a filename or changing punctuation can break string comparisons even when the underlying condition is unchanged. Prefer semantic checks:
Golang Errors Backend Best Practices
chevron-up