How to Add TTL Jitter to Cache Expiration in Python
A Python decorator that adds random jitter to cache TTLs, staggering expiration times to prevent cache avalanche.
Python code
43 linesimport random
import time
from functools import wraps
def add_jitter(ttl: float, jitter_range: float = 0.1) -> float:
"""Add random jitter (as % of TTL) to stagger cache expiration and prevent avalanche."""
jitter = random.uniform(-jitter_range, jitter_range)
return ttl * (1 + jitter)
def cache_with_jitter(ttl: float, jitter_range: float = 0.1):
"""Simple mock cache decorator with staggered TTL."""
cache = {}
def decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
key = (args, tuple(kwargs.items()))
now = time.time()
if key in cache:
value, expiry = cache[key]
if now < expiry:
return value
else:
del cache[key]
value = func(*args, **kwargs)
effective_ttl = add_jitter(ttl, jitter_range)
cache[key] = (value, now + effective_ttl)
return value
return wrapper
return decorator
@cache_with_jitter(ttl=10, jitter_range=0.2)
def get_user_data(user_id: int) -> dict:
return {"user_id": user_id, "name": f"User{user_id}"}
if __name__ == "__main__":
# Simulate cache behavior for multiple items
for _ in range(5):
print(get_user_data(1))
print("Cache hits with staggered TTL values (10s ± 20%)")
Output
{'user_id': 1, 'name': 'User1'}
{'user_id': 1, 'name': 'User1'}
{'user_id': 1, 'name': 'User1'}
{'user_id': 1, 'name': 'User1'}
{'user_id': 1, 'name': 'User1'}
Cache hits with staggered TTL values (10s ± 20%)
How it works
The cache_with_jitter decorator stores computed values in an in-memory dictionary keyed by function arguments. On each cache miss, it computes a new TTL that is randomly adjusted by a percentage (jitter_range), then stores the value with an absolute expiry timestamp. On subsequent calls, it checks if the current time is before the expiry; if so, it returns the cached value, otherwise it evicts and recomputes. The jitter helps distribute expiration times, preventing simultaneous cache expirations that cause thundering herd issues.
Common mistakes
- Forgetting to use `time.time()` as the base for expiry, causing float comparison bugs
- Not handling mutable arguments like lists or dicts as cache keys
- Setting `jitter_range` too large, which can over-stretch TTLs and reduce cache effectiveness
Variations
- Use `random.gauss(mu=ttl, sigma=ttl*jitter_range*0.5)` for a normal distribution instead of uniform
- Use a decorator parameter to allow per-function jitter settings
Real-world use cases
- Prevent DB load spikes in microservices by staggering cache expirations when many keys are set with similar TTLs.
- Add jitter to session cache expirations in distributed auth services to avoid synchronized re-authentication storms.
- Stagger TTL of configuration caches in deployment pipelines to avoid mass evictions during rolling releases.
Sponsored
More from Caching & Redis
- Cache Asides in Python with a Read-Through Loader easy
- Cache Data in Redis with Python easy
- Cache Penetration Null Object Mock in Python medium
- Cache Stampede Prevention with SingleFlight in Python medium
- Cache Warming with Python: Preload Hot Keys easy
- Coalescing duplicate in-flight requests: one shared result for concurrent callers hard
Keep learning
Related tutorials and quizzes for this topic.