Reference library

Reliability & rate limiting

Retries, exponential backoff, circuit breakers, token buckets, and idempotent handlers.

4 matches

How to Implement Hedged Requests in Python

This code demonstrates a hedged request pattern using threading, which sends duplicate calls and returns the first result that arrives within a timeout.

hedged-requests threading timeout

Python

import time
from unittest.mock import Mock

def hedged_request(call, timeout=0.05):
    """Execute two duplicate calls, return first result within timeout."""
    result_container = {}

    def run_and_store():
        result_container['result'] = call()
        result_container['done'] = True

    # Simulate slow cal…

16 0 Open

Reliability & rate limiting medium

How to Implement a Token Bucket Rate Limiter per Client IP in Python

Implements a simple sliding-window rate limiter using a dictionary of timestamp lists per client IP to limit requests per window.

rate-limiting sliding-window ip

Python

from time import time
from collections import defaultdict

class RateLimiter:
    def __init__(self, max_requests: int, window_seconds: int):
        self.max_requests = max_requests
        self.window_seconds = window_seconds
        self.clients = defaultdict(list)

    def allow(self, ip: str) -> bool:
        now…

13 0 Open

Reliability & rate limiting medium

How to implement rate limiting per API key in Python

A simple sliding-window rate limiter that tracks request timestamps per API key and rejects requests exceeding the configured limit.

rate-limiting api time-window

Python

import time

API_RATE_LIMITS = {"api_key_1": 5, "api_key_2": 3}  # max requests per window
WINDOW_SECONDS = 10

class RateLimiter:
    def __init__(self, limits, window):
        self.limits = limits
        self.window = window
        self.requests = {key: [] for key in limits}

    def allow(self, api_key):
       …

13 0 Open

Reliability & rate limiting medium

Token bucket rate limiter in Python (in-memory)

Implement a thread-safe in-memory token bucket rate limiter that throttles requests based on a steady refill rate.

rate-limiting token-bucket threading

Python

import time
import threading


class TokenBucket:
    def __init__(self, capacity, refill_rate, refill_interval=1.0):
        self.capacity = capacity
        self.tokens = capacity
        self.refill_rate = refill_rate
        self.refill_interval = refill_interval
        self.last_refill = time.monotonic()
       …

14 0 Open

Browse by section

Each section groups closely related Python snippets.