System Design

Token Bucket Rate Limiting for Controlled Bursts

Token Bucket Rate Limiting for Controlled Bursts A service may handle 100 requests per second comfortably on average while still needing to accept a short burst of 300 requests after a client reconnects. A rigid per-second limit treats those situations as the same problem: once the current window is full, otherwise acceptable work is rejected. Token bucket rate limiting gives you a more useful control. It separates two decisions: how quickly permission to do work is replenished and how much permission may accumulate for a burst. Once you understand those two numbers, you can reason about the limiter without depending on a particular library or platform.