How to implement a token bucket rate limiter in Python
A thread-safe in-memory token bucket rate limiter that tracks per-key tokens with refill logic, including a usage example after a timed refill.
Python code
34 linesimport time
import threading
class TokenBucketRateLimiter:
def __init__(self, capacity, refill_rate):
self.capacity = capacity
self.refill_rate = refill_rate
self.tokens = capacity
self.last_refill_time = time.time()
self.lock = threading.Lock()
def allow_request(self, key):
with self.lock:
now = time.time()
elapsed = now - self.last_refill_time
self.tokens = min(self.capacity, self.tokens + elapsed * self.refill_rate)
self.last_refill_time = now
if self.tokens >= 1:
self.tokens -= 1
return True
return False
if __name__ == "__main__":
limiter = TokenBucketRateLimiter(capacity=3, refill_rate=1)
keys = ["user:123", "user:123", "user:123", "user:123"]
for key in keys:
allowed = limiter.allow_request(key)
print(f"{key}: {'Allowed' if allowed else 'Rate limited'} (tokens left: {limiter.tokens:.2f})")
time.sleep(1)
allowed = limiter.allow_request("user:123")
print(f"After refill - user:123: {'Allowed' if allowed else 'Rate limited'} (tokens left: {limiter.tokens:.2f})")
Output
user:123: Allowed (tokens left: 2.00)
user:123: Allowed (tokens left: 1.00)
user:123: Allowed (tokens left: 0.00)
user:123: Rate limited (tokens left: 0.00)
After refill - user:123: Allowed (tokens left: 0.00)
How it works
The token bucket algorithm caps tokens at a fixed capacity and refills them continuously at a given rate. Entering the method with a threading.Lock guarantees that concurrent calls can't double-spend tokens. time.time() measures elapsed wall-clock time, so refills accumulate naturally across requests. Each successful request subtracts one token; when the bucket is empty the call is rejected. The example sleeps one second so the refill rate of 1 token/sec restores a token before the final check.
Common mistakes
- Using time.sleep in production instead of calculating elapsed time — it blocks the thread and skews refills.
- Ignoring thread safety on shared state if the limiter is used from multiple workers.
- Resetting `last_refill_time` on every request instead of only when refilling tokens.
Variations
- Use a Redis INCR + EXPIRE script for distributed rate limiting across multiple app instances.
- Implement a sliding window counter instead, tracking request timestamps per key in a deque.
Real-world use cases
- Protect an API endpoint with per-user rate limits to prevent abuse and control cost.
- Throttle outgoing webhook calls to a third-party service that enforces its own quotas.
- Limit batch job retries per worker to avoid hammering a database or external system.
Sponsored
More from Caching & Redis
- Cache Asides in Python with a Read-Through Loader easy
- Cache Data in Redis with Python easy
- Cache Penetration Null Object Mock in Python medium
- Cache Stampede Prevention with SingleFlight in Python medium
- Cache Warming with Python: Preload Hot Keys easy
- Coalescing duplicate in-flight requests: one shared result for concurrent callers hard
Keep learning
Related tutorials and quizzes for this topic.