Fencing Tokens Stop Stale Lease Holders from Writing

A distributed lease can decide which client currently owns a resource, but lease expiry does not instantly stop the previous holder. A process can pause, lose network access, or stall long enough for its lease to expire. Another client then acquires the lease. If the old process resumes and still has access to the protected storage or service, both clients can issue writes.

The lease manager has already moved ownership forward, yet the protected resource has no basis for distinguishing a current holder from a stale one. A fencing token closes that gap by attaching an ordered generation number to each successful acquisition. The resource accepts operations only when their token is at least as recent as the greatest token it has already accepted.

client A acquires lease -> token 41
A pauses
lease expires
client B acquires lease -> token 42
B writes with 42       -> accepted
A resumes and writes 41 -> rejected

The critical check occurs at the resource receiving the write, not only inside the lease service.

Lease expiry is not process revocation

A lease is authority bounded by time. Once its term ends, the coordination service may grant authority to another client. That transition does not imply that the former holder has stopped executing.

Long garbage-collection pauses, VM suspension, scheduler starvation, network partitions, and delayed messages can all create a stale actor. The actor may retain open connections and local state from the period when its lease was valid.

A local check such as lease_expiry > now is insufficient when the process clock or cached state is stale. Renewing before each operation narrows some windows but still cannot retract a request that was sent before expiry and delayed in transit.

Fencing moves the decision to the component that can enforce ordering on the side effect.

Tokens must advance with ownership

A fencing token is commonly a monotonically increasing integer issued when lease ownership is granted:

acquisition 1 -> token 101
acquisition 2 -> token 102
acquisition 3 -> token 103

The token does not need to encode time. Its useful property is order: a later successful acquisition receives a value greater than earlier acquisitions for the same protected scope.

The protected resource keeps the greatest accepted token, either directly or as part of its version state:

if request.token < highest_token:
    reject_stale_request()
else:
    highest_token = request.token
    apply_write()

The comparison and write need an atomic relationship appropriate to the resource. Checking a token in one transaction and applying the mutation later without the same ordering guarantee can reopen the race.

The resource must participate

Issuing tokens at the lock or lease service is not enough. If downstream storage ignores them, an old holder can still mutate state after a new holder has taken over.

This requirement shapes where fencing can be used. A database can store the token beside a row and reject lower generations with a conditional update. A custom service can retain the greatest generation for a resource and validate every mutating request. A storage API that offers no conditional write, version check, or comparable enforcement point may not support effective fencing directly.

The enforcement scope also has to match the lease scope. A token for one shard should not accidentally fence unrelated shards unless the design intentionally uses a global generation.

Fencing differs from a random lock identifier

A random lease ID can establish identity but not recency. Suppose client A has ID a7f2 and client B later has ID 91cd. A resource receiving both values cannot infer which acquisition came later.

An ordered token carries that information:

41 < 42 < 43

Random IDs can still be useful for safe release operations. A client should not delete a lock merely because the key exists; it should confirm that the stored lock identity is still its own. That protects the coordination record from an old client’s cleanup. Fencing addresses a separate problem: stale operations reaching the protected resource.

The two mechanisms can coexist. Identity protects lease management, while generation ordering protects side effects.

Renewal normally keeps the same generation

A successful renewal extends the current ownership term; it does not represent a transfer to a new owner. Keeping the same fencing token across ordinary renewals makes the generation correspond to ownership epochs rather than heartbeat count.

A new token is needed when ownership is granted after the previous term is no longer authoritative. If acquisition semantics permit a new owner, that owner must receive a generation ordered after the prior one.

The token allocator therefore needs durability and ordering compatible with the lease service’s failure model. Reusing an old generation after coordinator recovery can make a stale request indistinguishable from a current one.

Delayed requests are the core case

Fencing is valuable even when processes terminate promptly after losing a lease. Networks and queues can preserve old requests independently of the process that created them.

Consider this sequence:

t0  A holds token 7
t1  A sends write(7), packet is delayed
t2  A's lease expires
t3  B acquires token 8
t4  B sends write(8), resource accepts it
t5  delayed write(7) arrives

Without fencing, the request from t1 can overwrite state produced at t4. With generation validation, arrival order no longer grants stale authority. The resource rejects token 7 after observing token 8.

This is also the reason a client-side lease check immediately before sending a request is not a complete substitute. The request can become stale after that check.

Tokens do not impose full transaction ordering

A fencing token orders ownership epochs. It does not automatically serialize every operation inside one epoch, provide exactly-once delivery, or resolve application-level conflicts among writes carrying the same token.

If a holder sends two concurrent mutations with token 12, the token alone does not say which mutation should win. Sequence numbers, database transactions, compare-and-swap versions, idempotency keys, or application-specific ordering may still be required.

Fencing also does not repair a lease algorithm that can grant overlapping authority contrary to its stated model. It provides a downstream guard against older generations; the coordinator still needs a coherent rule for issuing generations and ownership.

Failure handling should preserve the fence

A client that receives a stale-token rejection should treat its authority as lost. Retrying the same mutation with the same token cannot make that generation current again.

The safe recovery path usually returns to coordination: stop work tied to the old lease, release local resources, and reacquire only when the application intends to start a new ownership epoch. Blindly requesting a newer token solely to force an old operation through defeats the ownership protocol.

Observability should expose both lease transitions and rejected stale operations. Useful fields include the resource key, presented token, highest accepted token, lease owner identity, acquisition generation, and rejection count. These records make rare pause-and-resume races visible without treating every lease timeout as a data corruption event.

The fence belongs next to the side effect

Distributed coordination cannot forcibly erase execution already in progress on another machine. A lease can declare that authority ended, but stale code and delayed packets can survive that declaration.

Fencing tokens turn each ownership transfer into an ordered generation that travels with mutating operations. Once the protected resource has accepted a newer generation, earlier holders can no longer write through that boundary. The lease decides current ownership; the resource enforces that decision where state actually changes.