Rate Limiting
System Design: lesson 12 of 15
Cap the flow before it caps your service.
Lesson 12 of 15 · 5 min
Rate Limiting
Step 1 of 11
A limiter counts requests per caller and rejects the excess. First, identify who is asking.
The Idea
A rate limiter counts requests per client and rejects the excess, normally with 429 and a Retry-After header saying when to come back. A token bucket refills at a steady rate up to a maximum, so a client may burst up to the bucket size, then settles to the refill rate.
Real-World Example
A council recycling centre allows each household twelve visits a year and at most three in any one week. You can clear a whole garage over a weekend, but you cannot do it every weekend — exactly the shape of a bucket that refills slowly and only holds so much.
The Tradeoff
Limits protect the service and punish legitimate bursts, and a counter shared across many servers costs a round trip on every request. A fixed window is the cheapest to build and the easiest to abuse: a client can spend a full budget either side of the boundary.
limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;
location /api/ {
limit_req zone=api burst=20 nodelay; # 10/s sustained, 20 in hand
limit_req_status 429;
}
Your turn
Put the steps in the right order.
- Reject with 429 and a Retry-After header once the bucket is empty
- Identify the caller: API key, account, or address
- Spend one token if the bucket holds at least one
- Refill that caller's bucket for the time since their last request
Mini quiz
1 / 3