Reference library

Reliability & rate limiting

Retries, exponential backoff, circuit breakers, token buckets, and idempotent handlers.

5 matches

How to Implement Hedged Requests in Python

This code demonstrates a hedged request pattern using threading, which sends duplicate calls and returns the first result that arrives within a timeout.

hedged-requests threading timeout

Python

import time
from unittest.mock import Mock

def hedged_request(call, timeout=0.05):
    """Execute two duplicate calls, return first result within timeout."""
    result_container = {}

    def run_and_store():
        result_container['result'] = call()
        result_container['done'] = True

    # Simulate slow cal…

16 0 Open

Reliability & rate limiting medium

How to Implement a Token Bucket Rate Limiter per Client IP in Python

Implements a simple sliding-window rate limiter using a dictionary of timestamp lists per client IP to limit requests per window.

rate-limiting sliding-window ip

Python

from time import time
from collections import defaultdict

class RateLimiter:
    def __init__(self, max_requests: int, window_seconds: int):
        self.max_requests = max_requests
        self.window_seconds = window_seconds
        self.clients = defaultdict(list)

    def allow(self, ip: str) -> bool:
        now…

13 0 Open

Reliability & rate limiting medium

How to Implement an Adaptive Rate Limiter in Python

Build an adaptive rate limiter that adjusts request intervals dynamically based on recent error rates, slowing down when failures spike.

rate-limiting backoff adaptive

Python

import time
import random

class AdaptiveRateLimiter:
    """Simple adaptive rate limiter that reduces requests when error rate is high."""
    
    def __init__(self, min_interval=0.1, max_interval=2.0, error_threshold=0.3):
        self.min_interval = min_interval
        self.max_interval = max_interval
        sel…

12 0 Open

Reliability & rate limiting medium

How to implement rate limiting per API key in Python

A simple sliding-window rate limiter that tracks request timestamps per API key and rejects requests exceeding the configured limit.

rate-limiting api time-window

Python

import time

API_RATE_LIMITS = {"api_key_1": 5, "api_key_2": 3}  # max requests per window
WINDOW_SECONDS = 10

class RateLimiter:
    def __init__(self, limits, window):
        self.limits = limits
        self.window = window
        self.requests = {key: [] for key in limits}

    def allow(self, api_key):
       …

13 0 Open

Reliability & rate limiting medium

Token bucket rate limiter in Python (in-memory)

Implement a thread-safe in-memory token bucket rate limiter that throttles requests based on a steady refill rate.

rate-limiting token-bucket threading

Python

import time
import threading


class TokenBucket:
    def __init__(self, capacity, refill_rate, refill_interval=1.0):
        self.capacity = capacity
        self.tokens = capacity
        self.refill_rate = refill_rate
        self.refill_interval = refill_interval
        self.last_refill = time.monotonic()
       …

14 0 Open

Browse by section

Each section groups closely related Python snippets.