https://nalar.dev/tags/distributed-systems/ Recent content in Distributed Systems on Nalar Hugo en-us Tue, 01 Sep 2026 00:00:00 +0700 https://nalar.dev/contract-tests-for-service-boundaries/ Tue, 01 Sep 2026 00:00:00 +0700 https://nalar.dev/contract-tests-for-service-boundaries/ <p>Distributed systems fail in an awkward place: each service can pass its own tests while the interaction between two services is incompatible.</p> <p>A provider may rename a JSON field, tighten validation, change an enum, or stop returning a value that a consumer quietly depends on. Contract tests make those cross-service assumptions executable.</p> <h2 id="what-a-contract-is">What a contract is</h2> <p>A contract describes an observable interaction between a consumer and a provider.</p> <p>For an HTTP API, it might specify:</p> https://nalar.dev/circuit-breakers-cloud-service-dependencies/ Tue, 01 Sep 2026 00:00:00 +0700 https://nalar.dev/circuit-breakers-cloud-service-dependencies/ <p>When a downstream service is failing, continuing to send every request can waste threads, sockets, and latency budget while increasing load on the unhealthy dependency. A circuit breaker temporarily fails calls fast after failures cross a threshold.</p> <h2 id="understand-the-three-states">Understand the three states</h2> <p>A typical breaker is <strong>closed</strong> while calls flow normally. It moves <strong>open</strong> after a configured failure policy is exceeded. After a recovery interval, it becomes <strong>half-open</strong> and allows a limited number of probes. Successful probes close the circuit; failures open it again.</p> https://nalar.dev/idempotent-event-consumers-at-least-once-delivery/ Tue, 01 Sep 2026 00:00:00 +0700 https://nalar.dev/idempotent-event-consumers-at-least-once-delivery/ <p>Many queues and event brokers provide at-least-once delivery: a message that has been accepted can be delivered again when acknowledgements are lost, consumers crash, visibility timeouts expire, or the broker retries after uncertain outcomes.</p> <p>Duplicates are therefore not exceptional. A robust consumer should assume that the same logical event can arrive more than once.</p> <h2 id="why-duplicates-happen">Why duplicates happen</h2> <p>Consider this sequence:</p> <ol> <li>a consumer receives an event;</li> <li>it updates the database successfully;</li> <li>the process crashes before acknowledging the message;</li> <li>the broker makes the message visible again;</li> <li>another consumer receives it.</li> </ol> <p>The broker cannot know that the database update happened. Redelivery is the safer choice.</p> https://nalar.dev/propagate-timeout-budgets-across-cloud-services/ Tue, 01 Sep 2026 00:00:00 +0700 https://nalar.dev/propagate-timeout-budgets-across-cloud-services/ <p>A request that crosses several cloud services does not have one timeout. It has a chain of deadlines: client, edge proxy, application, database, and downstream APIs.</p> <p>When those limits are configured independently, an upstream service can give up while downstream work continues consuming connections and CPU for a response nobody will use.</p> <p>An end-to-end timeout budget gives the request one bounded lifetime and lets each hop consume part of it.</p>