Exponential Backoff with Jitter in Go

Retries can make distributed systems more resilient, but immediate retries can also make an outage worse. If thousands of clients retry at the same moment, a recovering dependency receives another synchronized burst of traffic before it has time to stabilize.

A common solution is exponential backoff with jitter: increase the maximum delay after each failure, then randomize the actual wait. This article builds that pattern with Go’s standard library and shows where retry logic belongs—and where it does not.

The implementation below uses math/rand/v2, which was added in Go 1.22. Use Go 1.22 or newer for the example as written.

Retry only transient failures

A retry is useful when another attempt may succeed without changing the request. Typical examples include a temporary network interruption, an overloaded upstream service, or a database connection that briefly becomes unavailable.

Do not blindly retry permanent failures. Invalid input, authentication failures, permission errors, and most other client mistakes should normally fail immediately.

For HTTP clients, status codes such as 429 Too Many Requests, 502 Bad Gateway, 503 Service Unavailable, and 504 Gateway Timeout may be retryable depending on the API contract. A server-provided Retry-After value should generally take precedence over a locally calculated delay when the API documents that behavior.

Why exponential backoff helps

A simple exponential schedule starts with a small delay and doubles its upper bound after each failed attempt:

attempt 1: up to 200 ms
attempt 2: up to 400 ms
attempt 3: up to 800 ms
attempt 4: up to 1.6 s

The increasing delay reduces pressure on a struggling dependency. However, using exactly the same delay on every client can still synchronize retries.

Add jitter to spread retries

Jitter randomizes the delay. One straightforward strategy, often called full jitter, chooses a random duration between zero and the current exponential cap.

capDelay := base << attempt
delay := time.Duration(rand.Int64N(int64(capDelay) + 1))

If the cap is 800 milliseconds, different callers may wait 73 ms, 421 ms, or 756 ms instead of all retrying at 800 ms.

In production code, also cap the exponential value at a configured maximum so long retry sequences cannot produce impractically large delays or overflow a duration.

A cancellation-aware retry function

The following helper retries an operation a fixed number of times. It uses a timer instead of time.Sleep so cancellation can interrupt the wait.

package retry

import (
    "context"
    "fmt"
    "math/rand/v2"
    "time"
)

func Do(ctx context.Context, maxAttempts int, base time.Duration, fn func() error) error {
    if maxAttempts < 1 {
        return fmt.Errorf("maxAttempts must be at least 1")
    }
    if base <= 0 {
        return fmt.Errorf("base delay must be positive")
    }

    var err error

    for attempt := 0; attempt < maxAttempts; attempt++ {
        if err = fn(); err == nil {
            return nil
        }

        if attempt == maxAttempts-1 {
            break
        }

        capDelay := base << attempt
        delay := time.Duration(rand.Int64N(int64(capDelay) + 1))

        timer := time.NewTimer(delay)
        select {
        case <-ctx.Done():
            if !timer.Stop() {
                <-timer.C
            }
            return ctx.Err()
        case <-timer.C:
        }
    }

    return fmt.Errorf("operation failed after %d attempts: %w", maxAttempts, err)
}

The function returns the last operation error with %w, so callers can still inspect it with errors.Is or errors.As.

Why use a timer instead of Sleep?

time.Sleep cannot be interrupted. If a request is canceled while a retry loop is sleeping for several seconds, the goroutine remains blocked until the sleep finishes.

A timer combined with select lets the function return as soon as ctx.Done() becomes ready.

Put a ceiling on the delay

The compact example above is intentionally small, but a reusable retry package should accept a maximum delay. Calculate the exponential cap, then clamp it before choosing the jittered value:

capDelay := base << attempt
if capDelay > maxDelay {
    capDelay = maxDelay
}

delay := time.Duration(rand.Int64N(int64(capDelay) + 1))

Be careful when attempt can become large: the shift itself can overflow before the comparison. A robust library should either limit the attempt count to a safe range or calculate the next delay with explicit overflow checks.

Keep the total retry budget bounded

Limiting attempts is useful, but a context deadline gives the caller control over the complete operation budget.

ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()

err := retry.Do(ctx, 5, 200*time.Millisecond, func() error {
    return callDependency(ctx)
})

The operation now stops when it succeeds, exhausts five attempts, or reaches the five-second deadline—whichever happens first.

A deadline is especially important in request handlers. An upstream retry loop should not continue working after the original client has already timed out.

Respect idempotency

Retries are safest for operations that are idempotent: performing the same logical request more than once has the same intended effect as performing it once.

HTTP GET and HEAD requests are commonly safe to retry when the API follows HTTP semantics. Mutation requests require more care. Retrying a payment, order creation, or message publication after an ambiguous network failure can duplicate the side effect.

For non-idempotent operations, use an API-supported idempotency key or another deduplication mechanism before enabling automatic retries.

Avoid retry multiplication

Retries at several layers can multiply unexpectedly. If an application retries three times, its service client retries three times, and a proxy also retries three times, one user request can generate many upstream attempts.

Choose where retry responsibility lives and keep the overall request budget visible. Observability should record attempt counts and final outcomes so retry storms are detectable.

Common mistakes

Retrying every error

Classify failures first. Permanent errors should fail fast rather than consume time and capacity.

Using fixed delays everywhere

A fixed delay can synchronize a large fleet. Jitter reduces the chance that all clients wake at once.

Ignoring Retry-After

When an upstream explicitly tells clients when to retry, ignoring that signal can violate its rate-limit or recovery policy.

Retrying forever

Every retry policy needs a bound: attempts, elapsed time, a context deadline, or preferably a combination of them.

Forgetting side effects

A timeout does not prove that the server failed to process a request. Retrying a mutation without idempotency protection can execute it twice.

Testing retry code

Randomized waits should not make unit tests slow or flaky. For a reusable retry component, inject the delay generator and waiting mechanism so tests can provide deterministic values and avoid real sleeping.

Tests should cover at least successful first attempts, eventual success, exhausted attempts, context cancellation, permanent-error short-circuiting when classification is supported, and maximum-delay behavior.

Practical checklist

Before adding automatic retries, answer these questions:

1. Which failures are actually transient?
2. Is the operation safe to repeat?
3. Does the upstream provide Retry-After or another retry signal?
4. What is the maximum number of attempts?
5. What is the maximum delay and total time budget?
6. Does cancellation stop both the operation and the backoff wait?
7. Can metrics and logs reveal excessive retry activity?

Retries are a load-management mechanism as much as an error-handling mechanism. Exponential backoff slows repeated attempts, jitter prevents clients from moving in lockstep, and bounded cancellation-aware policies keep recovery behavior predictable.