How to Add TTL Jitter to Cache Expiration in Python

A Python decorator that adds random jitter to cache TTLs, staggering expiration times to prevent cache avalanche.

Medium Python 3.9+ Aug 9, 2026 Caching & Redis 15 views 0 copies

Python code

43 lines
Python 3.9+
import random
import time
from functools import wraps

def add_jitter(ttl: float, jitter_range: float = 0.1) -> float:
    """Add random jitter (as % of TTL) to stagger cache expiration and prevent avalanche."""
    jitter = random.uniform(-jitter_range, jitter_range)
    return ttl * (1 + jitter)

def cache_with_jitter(ttl: float, jitter_range: float = 0.1):
    """Simple mock cache decorator with staggered TTL."""
    cache = {}

    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            key = (args, tuple(kwargs.items()))
            now = time.time()

            if key in cache:
                value, expiry = cache[key]
                if now < expiry:
                    return value
                else:
                    del cache[key]

            value = func(*args, **kwargs)
            effective_ttl = add_jitter(ttl, jitter_range)
            cache[key] = (value, now + effective_ttl)
            return value

        return wrapper
    return decorator

@cache_with_jitter(ttl=10, jitter_range=0.2)
def get_user_data(user_id: int) -> dict:
    return {"user_id": user_id, "name": f"User{user_id}"}

if __name__ == "__main__":
    # Simulate cache behavior for multiple items
    for _ in range(5):
        print(get_user_data(1))
    print("Cache hits with staggered TTL values (10s ± 20%)")

Output

stdout
{'user_id': 1, 'name': 'User1'}
{'user_id': 1, 'name': 'User1'}
{'user_id': 1, 'name': 'User1'}
{'user_id': 1, 'name': 'User1'}
{'user_id': 1, 'name': 'User1'}
Cache hits with staggered TTL values (10s ± 20%)

How it works

The cache_with_jitter decorator stores computed values in an in-memory dictionary keyed by function arguments. On each cache miss, it computes a new TTL that is randomly adjusted by a percentage (jitter_range), then stores the value with an absolute expiry timestamp. On subsequent calls, it checks if the current time is before the expiry; if so, it returns the cached value, otherwise it evicts and recomputes. The jitter helps distribute expiration times, preventing simultaneous cache expirations that cause thundering herd issues.

Common mistakes

  • Forgetting to use `time.time()` as the base for expiry, causing float comparison bugs
  • Not handling mutable arguments like lists or dicts as cache keys
  • Setting `jitter_range` too large, which can over-stretch TTLs and reduce cache effectiveness

Variations

  1. Use `random.gauss(mu=ttl, sigma=ttl*jitter_range*0.5)` for a normal distribution instead of uniform
  2. Use a decorator parameter to allow per-function jitter settings

Real-world use cases

  • Prevent DB load spikes in microservices by staggering cache expirations when many keys are set with similar TTLs.
  • Add jitter to session cache expirations in distributed auth services to avoid synchronized re-authentication storms.
  • Stagger TTL of configuration caches in deployment pipelines to avoid mass evictions during rolling releases.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Caching & Redis

Related tutorials and quizzes for this topic.