How to implement a token bucket rate limiter in Python

A thread-safe in-memory token bucket rate limiter that tracks per-key tokens with refill logic, including a usage example after a timed refill.

Easy Python 3.9+ Aug 9, 2026 Caching & Redis 12 views 0 copies

Python code

34 lines
Python 3.9+
import time
import threading

class TokenBucketRateLimiter:
    def __init__(self, capacity, refill_rate):
        self.capacity = capacity
        self.refill_rate = refill_rate
        self.tokens = capacity
        self.last_refill_time = time.time()
        self.lock = threading.Lock()

    def allow_request(self, key):
        with self.lock:
            now = time.time()
            elapsed = now - self.last_refill_time
            self.tokens = min(self.capacity, self.tokens + elapsed * self.refill_rate)
            self.last_refill_time = now

            if self.tokens >= 1:
                self.tokens -= 1
                return True
            return False

if __name__ == "__main__":
    limiter = TokenBucketRateLimiter(capacity=3, refill_rate=1)
    keys = ["user:123", "user:123", "user:123", "user:123"]
    
    for key in keys:
        allowed = limiter.allow_request(key)
        print(f"{key}: {'Allowed' if allowed else 'Rate limited'} (tokens left: {limiter.tokens:.2f})")
    
    time.sleep(1)
    allowed = limiter.allow_request("user:123")
    print(f"After refill - user:123: {'Allowed' if allowed else 'Rate limited'} (tokens left: {limiter.tokens:.2f})")

Output

stdout
user:123: Allowed (tokens left: 2.00)
user:123: Allowed (tokens left: 1.00)
user:123: Allowed (tokens left: 0.00)
user:123: Rate limited (tokens left: 0.00)
After refill - user:123: Allowed (tokens left: 0.00)

How it works

The token bucket algorithm caps tokens at a fixed capacity and refills them continuously at a given rate. Entering the method with a threading.Lock guarantees that concurrent calls can't double-spend tokens. time.time() measures elapsed wall-clock time, so refills accumulate naturally across requests. Each successful request subtracts one token; when the bucket is empty the call is rejected. The example sleeps one second so the refill rate of 1 token/sec restores a token before the final check.

Common mistakes

  • Using time.sleep in production instead of calculating elapsed time — it blocks the thread and skews refills.
  • Ignoring thread safety on shared state if the limiter is used from multiple workers.
  • Resetting `last_refill_time` on every request instead of only when refilling tokens.

Variations

  1. Use a Redis INCR + EXPIRE script for distributed rate limiting across multiple app instances.
  2. Implement a sliding window counter instead, tracking request timestamps per key in a deque.

Real-world use cases

  • Protect an API endpoint with per-user rate limits to prevent abuse and control cost.
  • Throttle outgoing webhook calls to a third-party service that enforces its own quotas.
  • Limit batch job retries per worker to avoid hammering a database or external system.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Caching & Redis

Related tutorials and quizzes for this topic.