Request Coalescing in Go Without Extra Dependencies

When many requests ask for the same expensive resource at the same time, running identical work for every caller can overload a database, API, or filesystem. A cache can help after a result exists, but it does not necessarily prevent several concurrent cache misses from triggering the same backend operation.

Request coalescing solves a different problem: while one operation for a key is already running, later callers wait for that operation and share its result. After the operation finishes, the result is forgotten. The next request starts fresh work.

This pattern is useful for cache fills, metadata lookups, configuration refreshes, expensive calculations, and calls to rate-limited services.

Request coalescing is not caching

Suppose 100 goroutines simultaneously request user 42, and the value is not cached.

Without coalescing, all 100 goroutines may query the database. With coalescing, one goroutine performs the query while the other 99 wait. All callers receive the same result when the query completes.

The important distinction is lifetime:

• a cache stores a result for later requests;
• request coalescing shares only an operation that is currently in flight.

The two techniques are often used together. Coalescing protects the backend during a cache miss, while caching prevents later requests from reaching the backend at all.

A small standard-library implementation

The following implementation uses a mutex and a per-key completion channel. It requires no third-party packages.

package coalesce

import "sync"

type call struct {
    done  chan struct{}
    value string
    err   error
}

type Group struct {
    mu sync.Mutex
    m  map[string]*call
}

func (g *Group) Do(key string, fn func() (string, error)) (string, error) {
    g.mu.Lock()

    if g.m == nil {
        g.m = make(map[string]*call)
    }

    if c, ok := g.m[key]; ok {
        g.mu.Unlock()
        <-c.done
        return c.value, c.err
    }

    c := &call{done: make(chan struct{})}
    g.m[key] = c
    g.mu.Unlock()

    c.value, c.err = fn()

    g.mu.Lock()
    delete(g.m, key)
    close(c.done)
    g.mu.Unlock()

    return c.value, c.err
}

The map contains only operations that are currently running. A key disappears as soon as its operation finishes.

How the synchronization works

The first caller for a key creates a call, stores it in the map, releases the mutex, and executes fn.

A concurrent caller for the same key finds that call. It releases the mutex and waits on c.done instead of running fn again.

When the first caller finishes, it stores the value and error, removes the map entry, and closes c.done. Closing a channel wakes every goroutine waiting to receive from it.

The waiting goroutines then read the completed value and error.

Why the expensive function runs outside the mutex

Holding g.mu while calling fn would serialize unrelated keys. A slow operation for user:42 would prevent another goroutine from starting work for user:99.

The mutex should protect only the shared map. Expensive work belongs outside the critical section.

Verifying that duplicate work runs once

A test can launch several callers for the same key and count how many times the underlying function executes.

package coalesce

import (
    "sync"
    "sync/atomic"
    "testing"
    "time"
)

func TestGroupCoalescesSameKey(t *testing.T) {
    var g Group
    var calls atomic.Int32

    const workers = 20
    start := make(chan struct{})

    var wg sync.WaitGroup
    wg.Add(workers)

    for i := 0; i < workers; i++ {
        go func() {
            defer wg.Done()
            <-start

            value, err := g.Do("settings", func() (string, error) {
                calls.Add(1)
                time.Sleep(50 * time.Millisecond)
                return "ready", nil
            })
            if err != nil {
                t.Errorf("Do returned error: %v", err)
                return
            }
            if value != "ready" {
                t.Errorf("value = %q, want %q", value, "ready")
            }
        }()
    }

    close(start)
    wg.Wait()

    if got := calls.Load(); got != 1 {
        t.Fatalf("function ran %d times, want 1", got)
    }
}

The start channel releases all workers together. The artificial delay keeps the first operation in flight long enough for the other workers to join it.

Run the test with the race detector when developing concurrency primitives:

go test -race ./...

A successful run ends with output similar to:

ok      example/coalesce

The exact module path and timing reported by go test depend on the project.

Different keys can run concurrently

Coalescing should group equivalent work, not globally serialize the application. Calls with different keys receive different call objects and can execute at the same time.

Choose keys that describe the actual unit of work. For example:

key := "user:" + userID

For a request whose result also depends on locale, the locale must be part of the key:

key := "user:" + userID + ":locale:" + locale

If two requests can legitimately produce different results, they should not share a key.

Use it around cache misses

A common production pattern is to check a cache first, then coalesce the backend lookup.

value, ok := cache.Get(key)
if ok {
    return value, nil
}

return group.Do(key, func() (string, error) {
    // Check again because another caller may have filled the cache
    // before this function started.
    if value, ok := cache.Get(key); ok {
        return value, nil
    }

    value, err := loadFromDatabase(key)
    if err != nil {
        return "", err
    }

    cache.Set(key, value)
    return value, nil
})

The second cache lookup is important. A result may have appeared between the first cache miss and entering the coalesced operation.

Errors are shared too

Every caller waiting for an in-flight operation receives its error as well as its value. This is usually desirable: starting 100 identical retries immediately after one backend failure can make an outage worse.

However, coalescing is not a retry policy. Decide separately whether an error is retryable and how much backoff to apply.

Because this implementation forgets the operation after completion, a later request can try again immediately.

Add cancellation carefully

The minimal implementation deliberately does not accept a context. Cancellation introduces an important policy question: should one caller cancel the shared backend operation for every other caller?

Usually, the answer is no. A caller that stops waiting should be able to return without necessarily cancelling work that other callers still need.

One approach is to let waiting callers select between their context and the completion channel:

select {
case <-ctx.Done():
    return "", ctx.Err()
case <-c.done:
    return c.value, c.err
}

Cancelling the underlying shared operation requires more bookkeeping, such as tracking how many callers are still interested. Avoid connecting the backend operation directly to the first caller’s context unless that ownership model is intentional.

Handle panics in production implementations

The compact example assumes fn returns normally. If fn panics, waiting goroutines would otherwise remain blocked because c.done would never close.

A reusable library implementation should use deferred cleanup so the map entry is removed and waiters are released even when unexpected control flow occurs. You must also decide whether a panic should be propagated, converted to an error, or recovered at a higher application boundary.

Keeping these semantics explicit is one reason mature concurrency helpers are often preferable once requirements grow beyond a small internal utility.

Common pitfalls

Using keys that are too broad

If unrelated requests share a key, one caller can receive a result computed for different inputs. Include every input that affects the result.

Using keys that are too specific

A key containing irrelevant request-specific data, such as a trace ID, prevents equivalent operations from being coalesced.

Treating coalescing as a cache

Once the operation completes, the map entry is deleted. A request arriving afterward runs the function again. Add a real cache when results should survive beyond the in-flight operation.

Holding the mutex during slow work

Only map access needs the mutex. Network calls, database queries, parsing, and other expensive work should run after unlocking it.

Forgetting failure behavior

A burst of requests can still create repeated backend attempts if each failed operation finishes before the next request arrives. Combine coalescing with backoff, rate limiting, circuit breaking, or short-lived negative caching when appropriate.

When to use this pattern

Request coalescing is especially effective when traffic contains bursts of identical work and the backend operation is expensive relative to synchronization overhead.

It is less useful when requests rarely share keys, operations are extremely cheap, or callers require different cancellation and result-sharing semantics.

Start by measuring the backend behavior. If concurrent cache misses or duplicate lookups are a meaningful source of load, coalescing can remove that amplification with a small amount of synchronization.

Summary

Request coalescing ensures that duplicate concurrent work for the same key runs once while all interested callers share the result. Unlike caching, it stores no completed result.

A practical implementation should:

1. keep one in-flight operation per key;
2. run expensive work outside the map mutex;
3. wake all waiters when the operation completes;
4. remove completed operations so future calls can run again;
5. define cancellation, error, and panic behavior deliberately.

Used around cache misses or expensive backend calls, this pattern can significantly reduce load during traffic bursts without changing how long application data is cached.