Retry with Exponential Backoff and Jitter in Python
A decorator-style retry wrapper that retries a flaky function with exponential backoff plus random jitter, then raises after the last attempt fails.
Python code
23 linesimport random
import time
def retry_with_backoff(func, max_retries=3, base_delay=0.5, max_jitter=0.1):
for attempt in range(max_retries + 1):
try:
return func()
except Exception as e:
if attempt == max_retries:
raise
delay = base_delay * (2 ** attempt) + random.uniform(0, max_jitter)
print(f"Attempt {attempt + 1} failed: {e}. Retrying in {delay:.2f}s...")
time.sleep(delay)
def flaky_operation():
if random.random() < 0.6:
raise ValueError("Temporary failure")
return "Success"
if __name__ == "__main__":
random.seed(42)
result = retry_with_backoff(flaky_operation)
print(f"Final result: {result}")
Output
Attempt 1 failed: Temporary failure. Retrying in 0.53s...
Attempt 2 failed: Temporary failure. Retrying in 1.02s...
Final result: Success
How it works
This pattern retries transient failures while avoiding thundering-herd retries from many clients hitting a service at once. random.uniform adds jitter to each backoff delay, spreading retries across time. The loop runs from attempt 0 to max_retries, raising only when the final attempt also fails. Using time.sleep blocks the thread, so this is ideal for scripts or single-threaded workers.
Common mistakes
- Forgetting to set max_retries to the number of retries, not attempts, causing one extra try.
- Calling time.sleep with no jitter, causing synchronized retries across many clients.
- Catching broad Exception instead of specific transient exceptions, masking permanent failures.
- Not importing random or time, leading to NameError at runtime.
Variations
- Use `retrying` or `tenacity` library for more configurable retry policies.
- Add exponential backoff with a cap to prevent extremely long waits.
- Implement retries in an async function with `asyncio.sleep` instead of `time.sleep`.
Real-world use cases
- Retrying HTTP calls to a flaky third-party API that returns 5xx errors.
- Retrying database connections during a brief network blip or connection pool exhaustion.
- Retrying message-publishing attempts to a queue like Kafka when the broker is briefly unavailable.
Sponsored
More from Reliability & rate limiting
- At Least Once with Idempotent Consumer in Python medium
- Build a Rate Limiter Decorator in Python easy
- Build a queue-based admission control system in Python easy
- Chaos Inject Random Failures in Python easy
- Circuit breaker failure threshold count in Python medium
- Exactly Once Processing Dedupe Mock in Python easy
Keep learning
Related tutorials and quizzes for this topic.