DevNet - Leaky Bucket
API is Cool And Leaky
Who needs a leaky bucket? Not me, but API would not mind one.
Why?
To protect web services, some measures must be implemented to control API requests and avoid DDoS and overloading, to maintain good quality of service for all users.
So there are a few algorithms which limit rate of requests
- Leaky bucket
- Token bucket
- Fixed window counter
- Sliding window counter
Look at a good implementation of different API rate limit algorithms.
Leaky bucket
Leaky bucket means that all requests are served with a constant rate, like water drops from a leaky bucket but all incoming requests accumulated in a queue and served on a first come first serve basis. Basically it means that some requests are going to be delayed, and some even rejected. An explanation found on YouTube
Comparison Leaky bucket to token bucket
Token bucket
A certain number of tokens generated at certain rate. Before serving a user’s request,
the web service checks token storage. If a token present, it removed from the
storage (bucket) and the request is served. No tokens - no service, requests rejected,
until new token generated.
Comparison Leaky bucket to token bucket
Fixed window
Only a certain number of requests are allowed at a fixed time period / window. Every request from a user increments a counter and recorded with a timestamp. The service records the number of requests and compares timestamps to the current window timestamps. The requests with exceeding counter get rejected. New time period starts, the counter is zeroed.
Sliding window
All request counted, and only certain number of requests are served at certain period of time. However, the counter is not set to 0 when time window ends. It is incremented with new request beyond the window limit. To make decision, the service has to check and compare the number of requests during this window and during previous window.
What Is The Limit
A client can obviously check the limits and know what are the limits usually by checking headers:
- X-RateLimit-Limit
- X-RateLimit-Remaining
- X-RateLimit-Reset
Deal with rejection
- 429 - Too many requests
- 403 - Forbidden
When you get these 2 errors it might mean that you’ve been too mean to the service and your rate limit is exceeded. Wait. Check SLA. Ask forgivness. Check your stupid script that sends too many request. Give it a break, will ya?