A remote call can fail in a few milliseconds or consume its entire timeout budget before returning an error. If callers keep issuing equivalent requests while the dependency remains unable to serve them, each attempt spends resources on an outcome that recent evidence already suggests is unavailable. Retries can increase that pressure because one logical operation may create several physical calls.
A circuit breaker changes call admission rather than the remote protocol. It records recent outcomes, moves between explicit states, and can reject new calls locally for a bounded period. After that period, it permits limited probes to test whether normal traffic can resume.
That description sounds close to a boolean switch, but the useful engineering properties sit in the transitions. The breaker needs a failure signal, a sampling rule, a recovery policy, and a concurrency model. Each choice defines what the breaker actually protects and which failures it can distinguish.
Closed state still admits failure
In the closed state, calls pass through to the dependency. The breaker observes their outcomes and updates its failure evidence. It does not prevent individual failures; it needs observations before it can decide that continued admission should stop.
A threshold therefore describes a detection policy, not a guarantee. A breaker configured to open after five qualifying failures can still admit at least those failures. Under concurrency, more calls may already be in flight when the threshold is crossed. Opening the circuit normally affects subsequent admission rather than cancelling work that has already passed the boundary.
The measured event also matters. A connection refusal, transport timeout, HTTP 503 response, application validation error, and caller cancellation are all failures in some sense, but they do not necessarily indicate the same dependency condition. Counting every non-success outcome can make the breaker react to errors that the remote service cannot correct.
A useful failure predicate is tied to the protected operation. If an HTTP client receives a 400 response because its request is invalid, opening a breaker against the server does not address that request defect. A sequence of transport failures or selected server errors can provide different evidence. The exact classification depends on the protocol and application contract.
Thresholds need a sampling model
A statement such as “open after five failures” leaves an important question unresolved: five failures across what population?
A consecutive-failure counter is one model. A single success can reset the sequence, depending on the implementation. A rolling window is another model, retaining outcomes over a fixed count or time interval and opening when a minimum sample size and failure ratio are reached. These policies can produce different state transitions from the same stream of results.
Consider ten calls with alternating outcomes:
F S F S F S F S F SA consecutive threshold of three never opens. A window that evaluates ten calls and opens at a fifty-percent failure ratio can open after the sample becomes eligible. Neither policy is intrinsically correct; they encode different interpretations of degraded service.
Small samples deserve special care. A ratio without a minimum observation count can turn one failure into a one-hundred-percent failure rate. A minimum sample requirement prevents the ratio from carrying more confidence than the available observations support. It also means a low-traffic dependency can remain closed for longer because it takes longer to collect enough evidence.
Time-based windows introduce another boundary. Old failures must expire according to the window definition. Count-based windows instead retain the most recent number of observations regardless of elapsed time. The first model represents recent wall-clock activity; the second represents recent request history. Traffic rate changes their behavior in different ways.
Open state is local rejection
Once open, the breaker stops admitting ordinary calls through its protected boundary. A rejected call has not reached the dependency, so its error should remain distinguishable from a remote failure when the API permits that distinction.
That separation matters for composition. A caller may treat a local open-circuit result as a signal to use a fallback, defer work, or return an unavailable response. Treating it as another remote timeout hides the fact that no network attempt occurred.
The open interval is not evidence that the dependency has recovered. It is a quiet period during which ordinary traffic is withheld. When the interval expires, immediately restoring full traffic would discard the breaker’s accumulated caution at the exact point where recovery is still unverified.
This creates the need for a probe state, often called half-open. The name is less important than its admission rule: permit a limited amount of trial traffic, observe the result, then decide whether to restore ordinary admission or return to open state.
Recovery probes need their own capacity rule
If one thousand callers arrive just after the open interval expires, allowing all one thousand to become probes makes the recovery state nearly equivalent to closing the breaker immediately. A bounded probe count preserves the distinction between testing recovery and resuming normal load.
The probe limit is a concurrency decision. With one permitted probe, other callers can be rejected or held according to the implementation contract while the probe is unresolved. With several probes, the breaker gathers more observations but also sends more work to a dependency whose recovery has not yet been established.
Probe success criteria also need precision. Some breakers close after one successful probe. Others require several successful calls or evaluate a fresh sample. A mixed set of probe outcomes can therefore produce implementation-specific transitions.
This is one reason state names alone are insufficient documentation. Two libraries can both expose closed, open, and half-open states while differing materially in threshold calculation, probe concurrency, state duration, and transition conditions.
The breaker boundary defines the failure domain
Placement determines which calls share failure evidence. A breaker around an entire service endpoint can combine operations with different behavior. A breaker per host can isolate hosts but combine all routes on each host. A breaker per operation can distinguish routes but create many independent state machines.
The correct granularity follows the failure correlation the application intends to model. If two operations use independent downstream capacity, combining their outcomes can cause failures in one path to reject healthy calls in the other. If they depend on the same constrained resource, separate breakers can continue admitting aggregate traffic even after one breaker has opened.
Dynamic destinations make this more visible. A client that talks to many tenant-specific hosts and allocates one breaker per host also creates breaker state proportional to observed destinations unless old entries are retired. A single global breaker avoids that state growth but merges unrelated destination health. The breaker key is therefore part of resource management as well as failure classification.
The same issue appears when a load balancer sits between the caller and several backend instances. A client-side breaker keyed only by the load-balancer address observes the aggregate behavior visible through that address. It does not independently know the health of each backend unless the routing layer exposes that distinction.
Retries and breakers act at different boundaries
A retry decides whether one logical operation should make another attempt. A breaker decides whether an attempt is admitted at all. Their composition changes both traffic volume and recorded evidence.
If the breaker wraps the entire retry operation, it may observe one final outcome after several physical attempts. If each physical attempt passes through the breaker, the breaker can observe every failed attempt and may reach its threshold sooner. These arrangements answer different questions.
The distinction is concrete. Suppose one logical request is allowed three attempts and all three fail. An outer breaker can record one failed operation. An inner breaker can record three failed calls. With a threshold based on event count, those structures cannot be assumed to open at the same point.
Timeout placement has a similar effect. A timeout outside a retry loop can bound the whole logical operation, while per-attempt timeouts bound individual calls. A breaker observing only inner calls sees a different stream from a breaker observing the final outer result.
No ordering makes every composition correct. The desired boundary depends on whether breaker evidence is intended to represent physical dependency attempts or completed logical operations. Making that choice explicit is more useful than treating retry, timeout, and breaker decorators as freely interchangeable layers.
A breaker does not prove dependency health
Closing a circuit means its recovery policy has accepted enough evidence to resume admission. It does not establish that the dependency will remain available, that every operation is healthy, or that latency has returned to a previous distribution.
Likewise, an open circuit records a local decision based on observed events. The dependency might recover before the open interval expires. It might also remain unavailable after the interval. The breaker trades immediate observation for controlled admission during the open period.
That trade is central to the pattern. A breaker is not a health oracle and does not replace server-side overload control. It is a client-side state machine that uses recent call outcomes to decide whether more calls should cross a boundary.
State transitions are the contract
The most informative breaker description is not “three states around a remote call.” It is the transition contract: which outcomes count, which sample is evaluated, when ordinary admission stops, how long it remains stopped, how many probes can pass, and what evidence restores normal admission.
Those details also determine observability. Metrics for current state, rejected calls, admitted probes, and transition counts expose different facts from raw dependency error counts. A local rejection should not be mistaken for a remote error, and a probe should remain identifiable as deliberately limited recovery traffic.
A circuit breaker earns its place when that admission contract matches a real failure boundary. Its value comes from making repeated call admission conditional on recent evidence, then treating recovery as a controlled state transition rather than an immediate return to full traffic.