SRE

Load Shedding and Overload Protection for Cloud Services

Autoscaling is useful, but it is not instantaneous. Traffic can rise faster than new instances start, a dependency can slow down, or a retry storm can multiply work. When demand exceeds safe capacity, accepting every request can make the entire service slower until almost nothing completes. Load shedding is the deliberate rejection or degradation of work to keep the system inside a recoverable operating range. Overload is often a queueing problem Imagine a service that safely handles 200 concurrent requests. A downstream dependency slows from 50 ms to 2 seconds.