Apps Artificial Intelligence Cloud Computing CSS Cybersecurity Data Science Database Go JavaScript Linux Python Rust Software Engineering Web Development

Idempotency Keys for Safe API Retries

7 min read .
Published on
Idempotency Keys for Safe API Retries

Retries are essential in distributed systems. Networks fail, clients time out, load balancers reset connections, and responses sometimes disappear after a server has already committed a write.

The dangerous case is a retry of a non-idempotent operation. If a client sends POST /orders, times out, and sends the same request again, the server may create two orders even though the user intended one.

An idempotency key gives the client a stable identifier for one logical operation. The server remembers the result associated with that key and can return the same result when the request is retried.

Why ordinary retries can duplicate writes

Consider this sequence:

  1. A client sends a request to create an order.
  2. The server writes the order to the database.
  3. The response is lost before it reaches the client.
  4. The client sees a timeout and retries.
  5. The server creates a second order.

From the client’s perspective, retrying was reasonable. From the server’s perspective, both HTTP requests were valid. The missing piece is a way to tell that both requests represent the same logical operation.

What an idempotency key does

The client generates a unique key before the first attempt and reuses it for every retry of that operation:

POST /orders HTTP/1.1
Content-Type: application/json
Idempotency-Key: 01JEXAMPLEKEY

{
  "product_id": "sku-123",
  "quantity": 2
}

The exact key format is an API design choice. A sufficiently random UUID or another high-entropy identifier is common. The important rule is that a new logical operation gets a new key, while retries reuse the original key.

On the server, the key typically maps to information such as:

  • a fingerprint of the request,
  • whether processing is still in progress,
  • the final HTTP status,
  • the response body or a durable reference to the created resource,
  • an expiration time.

Store a request fingerprint

A key alone is not enough. A buggy client could accidentally reuse the same key with different input. The server should detect that conflict instead of returning an unrelated earlier result.

A request fingerprint can be derived from the fields that define the operation. In Go, a small JSON request can be hashed with the standard library:

package idempotency

import (
    "crypto/sha256"
    "encoding/hex"
    "encoding/json"
)

type CreateOrderRequest struct {
    ProductID string `json:"product_id"`
    Quantity  int    `json:"quantity"`
}

func Fingerprint(req CreateOrderRequest) (string, error) {
    data, err := json.Marshal(req)
    if err != nil {
        return "", err
    }

    sum := sha256.Sum256(data)
    return hex.EncodeToString(sum[:]), nil
}

For this fixed struct, encoding/json produces a deterministic field order, so identical values produce the same SHA-256 fingerprint. If the input contains maps or semantically equivalent values that can have different representations, define a canonical representation before hashing.

Do not use a fingerprint as the idempotency key itself. The client-generated key identifies the logical operation; the fingerprint checks that subsequent uses of that key describe the same operation.

Make the database claim atomic

The most important implementation detail is not the hash. It is the database operation that claims an idempotency key.

A simplified table might contain:

CREATE TABLE idempotency_keys (
    key            TEXT PRIMARY KEY,
    request_hash   TEXT NOT NULL,
    status         TEXT NOT NULL,
    response_code  INTEGER,
    response_body  TEXT,
    expires_at     TIMESTAMP NOT NULL
);

The primary key or a unique constraint is essential. Two application instances can receive retries at nearly the same time, so an in-memory if key exists check is not sufficient.

A robust flow is:

  1. Start a database transaction.
  2. Attempt to insert the key with an in_progress state.
  3. If the insert wins, perform or coordinate the protected operation.
  4. Store the completed result before making it available for replay.
  5. If the key already exists, compare the stored request fingerprint.
  6. Replay a completed result, wait or reject if it is still in progress, or reject the request if the fingerprint differs.

The exact transaction boundaries depend on the database and the side effect. When the protected write and the idempotency record live in the same transactional database, keeping them in one transaction can eliminate important failure windows.

Decide what to do with concurrent duplicates

Retries are not always sequential. Mobile clients, browser code, proxies, or job workers can send the same operation concurrently.

When a second request finds the key in an in_progress state, an API can choose among several policies:

  • wait briefly for the first request and then replay its result,
  • return a conflict or retryable response,
  • return an accepted response with a resource that can be polled.

There is no universal status code for this state. Document the policy so clients know whether and when to retry.

Replay the result, not the side effect

After the first request completes, later requests with the same key should not execute the operation again. They should receive a representation of the original outcome.

For small responses, storing the HTTP status and serialized response body can be convenient. For larger responses, store the created resource ID and reconstruct the response when needed.

Be careful with responses that contain timestamps, expiring URLs, or other transient data. Decide whether replay means returning the exact original bytes or returning the current representation of the original resource.

Choose an expiration policy

Idempotency records do not usually need to live forever. Their retention period should cover the maximum realistic retry window for clients and infrastructure.

Deleting a key too early is dangerous: a very late retry can be treated as a new operation. Keeping every key forever creates unnecessary storage growth.

Document the retention period as part of the API contract. Clients should not assume a key remains replayable indefinitely unless the API explicitly guarantees it.

Scope keys carefully

A globally unique key space is simple but not always necessary. Many systems scope keys by account, tenant, endpoint, or operation type.

For example, a database uniqueness constraint might apply to (account_id, idempotency_key) rather than the key alone. Whatever scope you choose must match the lookup logic consistently on every application instance.

Never allow one tenant to retrieve another tenant’s stored response merely by guessing or reusing an idempotency key.

Idempotency is not the same as deduplication by payload

Two requests with identical JSON are not necessarily duplicates. A customer may intentionally order the same product twice. Automatically deduplicating requests only because their bodies match can suppress legitimate operations.

An idempotency key expresses client intent: these attempts belong to one logical operation. That is stronger and safer than guessing from payload similarity.

External side effects need extra care

A local database transaction cannot automatically make an external API call transactional. Suppose an application charges a payment provider and then crashes before recording the successful result locally. A retry may attempt the charge again.

Prefer downstream services that support their own idempotency keys and pass a stable operation identifier through the call chain. For asynchronous work, patterns such as a transactional outbox can help coordinate database state with message publication.

Idempotency reduces duplicate effects only when every important boundary has a recovery strategy.

Common pitfalls

Generating a new key for every retry

If retry code generates a fresh key after a timeout, the server correctly treats every attempt as a new operation. Generate the key once, before the retry loop.

Reusing one key for different operations

A key should represent one logical operation. Reusing it for different payloads should produce a conflict rather than silently replaying the first result.

Keeping idempotency state only in memory

An in-memory map does not coordinate multiple application instances and disappears during a restart. Use shared durable storage when duplicate side effects matter.

Checking and inserting in separate steps

A SELECT followed by an unconditional INSERT has a race window. Enforce uniqueness in the database and use an atomic insert or transaction appropriate for the database engine.

Caching only successful responses

If the operation committed but the application failed before recording its response, the retry path must still be able to determine what happened. Design the protected operation and idempotency record together instead of treating idempotency as a thin response cache.

A practical checklist

Before adding automatic retries to a write endpoint, answer these questions:

  • Which operations require an idempotency key?
  • Who generates the key and when?
  • What scope makes a key unique?
  • Which request fields are included in the fingerprint?
  • How is the key claimed atomically across server instances?
  • What happens when two matching requests arrive concurrently?
  • How are conflicting payloads rejected?
  • What result is stored for replay?
  • How long are records retained?
  • How are downstream side effects made idempotent or recoverable?

Conclusion

Retries are unavoidable in reliable distributed systems, but duplicate writes do not have to be. An idempotency key lets clients retry an uncertain request while giving the server enough information to recognize that the operation has already started or completed.

The reliable pattern is more than an HTTP header: generate one key per logical operation, store a request fingerprint, claim the key atomically in durable storage, replay completed results, and define how concurrent and expired requests behave. With those pieces in place, write APIs can tolerate common network failures without turning a harmless retry into a duplicate side effect.

Backend API HTTP Reliability

Related Posts

Liveness and Readiness Health Checks for Backend Services
Published on
Liveness and Readiness Health Checks for Backend Services
Health endpoints look simple, but their semantics directly affect how a production platform routes traffic and restarts applications. A poorly designed check can turn a temporary database slowdown into a restart loop or send requests to an instance that has not finished initializing. The most useful model separates two questions: Liveness: Is this process still capable of running? Readiness: Should this instance receive new traffic right now? Those questions sound similar, but they should usually have different answers and different failure behavior.
Backend Reliability HTTP DevOps
Atomic File Writes in Go: Prevent Partial and Corrupted Files
Published on
Atomic File Writes in Go: Prevent Partial and Corrupted Files
Writing a file with os.WriteFile is simple, but it is not always the safest choice for configuration files, generated metadata, caches, state files, or other data that must never be left half-written. If a process crashes or the machine loses power while a file is being replaced, readers may observe incomplete content. A common way to reduce this risk is an atomic file write: write the new content to a temporary file first, then replace the destination with a rename.
Golang Files Backend Reliability
Exponential Backoff with Jitter in Go
Published on
Exponential Backoff with Jitter in Go
Retries can make distributed systems more resilient, but immediate retries can also make an outage worse. If thousands of clients retry at the same moment, a recovering dependency receives another synchronized burst of traffic before it has time to stabilize. A common solution is exponential backoff with jitter: increase the maximum delay after each failure, then randomize the actual wait. This article builds that pattern with Go’s standard library and shows where retry logic belongs—and where it does not.
Golang Retries Reliability Backend
Go Context Timeouts and Request Cancellation
Published on
Go Context Timeouts and Request Cancellation
A Go HTTP handler can outlive the request that started it unless the work inside the handler pays attention to cancellation. That matters when a client disconnects, an upstream request takes too long, or a database query is no longer useful. Go solves this with context.Context. Every incoming *http.Request already has a context, and that context is canceled when the client connection closes, the request is canceled by HTTP/2, or the handler returns. You can also derive a shorter deadline for work that should not consume the entire request lifetime.
Golang Backend HTTP
Graceful HTTP Server Shutdown in Go
Published on
Graceful HTTP Server Shutdown in Go
Stopping a web server with Ctrl+C looks harmless during development, but production deployments need a more careful shutdown process. If a process exits immediately, active HTTP requests can be interrupted, clients may receive connection errors, and in-flight work can be left unfinished. Go’s standard library already provides the pieces needed for a clean shutdown. The main tools are os/signal, context, and http.Server.Shutdown. This guide shows a practical pattern for shutting down an HTTP server when the process receives SIGINT or SIGTERM.
Golang HTTP Context Backend
Keyset Pagination in SQL for Fast, Stable APIs
Published on
Keyset Pagination in SQL for Fast, Stable APIs
Pagination looks simple until a table becomes large or new rows are inserted while a client is paging through results. LIMIT and OFFSET are easy to understand, but deep offsets can become expensive and changing data can make rows appear twice or disappear between requests. Keyset pagination, also called seek pagination, avoids those problems by asking for rows after a known position instead of asking the database to skip a number of rows.
SQL Databases Backend API
chevron-up