Token bucket rate limiter in Python (in-memory)
Implement a thread-safe in-memory token bucket rate limiter that throttles requests based on a steady refill rate.
Python code
36 linesimport time
import threading
class TokenBucket:
def __init__(self, capacity, refill_rate, refill_interval=1.0):
self.capacity = capacity
self.tokens = capacity
self.refill_rate = refill_rate
self.refill_interval = refill_interval
self.last_refill = time.monotonic()
self.lock = threading.Lock()
def _refill(self):
now = time.monotonic()
elapsed = now - self.last_refill
if elapsed > 0:
new_tokens = elapsed * self.refill_rate
self.tokens = min(self.capacity, self.tokens + new_tokens)
self.last_refill = now
def consume(self, tokens=1):
with self.lock:
self._refill()
if self.tokens >= tokens:
self.tokens -= tokens
return True
return False
if __name__ == "__main__":
bucket = TokenBucket(capacity=5, refill_rate=2)
for _ in range(8):
print(f"consume 1 -> {bucket.consume()}")
time.sleep(1)
print(f"after 1s refill -> {bucket.consume()}")
Output
consume 1 -> True
consume 1 -> True
consume 1 -> True
consume 1 -> True
consume 1 -> True
consume 1 -> False
consume 1 -> False
consume 1 -> False
after 1s refill -> True
How it works
The token bucket algorithm works by maintaining a token count that increases over time at a fixed refill_rate. The _refill method calculates elapsed time using time.monotonic, adds the appropriate number of tokens, and caps the total at capacity to avoid overflow. consume acquires a lock to protect the token state for thread safety, ensuring multiple threads cannot accidentally over-spend tokens. In the example, the bucket starts full with 5 tokens so the first 5 consumes return True, then subsequent calls return False until enough time passes for a refill.
Common mistakes
- Using `time.time()` instead of `time.monotonic()`, which can jump backward due to system clock adjustments.
- Forgetting to cap tokens at capacity, causing the bucket to accumulate a large burst.
- Not using a lock when accessing `tokens` from multiple threads, leading to race conditions.
Variations
- Use `threading.Condition` to block when tokens are unavailable instead of returning False.
- Create a decorator that applies the bucket to a function call.
Real-world use cases
- Throttling outgoing API requests to stay within an external provider's quota.
- Limiting login attempts per user to prevent brute-force attacks.
- Capping click-through or event processing rates in a real-time pipeline.
Sponsored
More from Reliability & rate limiting
- At Least Once with Idempotent Consumer in Python medium
- Build a Rate Limiter Decorator in Python easy
- Build a queue-based admission control system in Python easy
- Chaos Inject Random Failures in Python easy
- Circuit breaker failure threshold count in Python medium
- Exactly Once Processing Dedupe Mock in Python easy
Keep learning
Related tutorials and quizzes for this topic.