Python Exponential Backoff Retry Example

Retry a flaky function with exponential backoff and jitter-free delays, printing each attempt and finally returning the successful result.

Medium Python 3.6+ Aug 9, 2026 Data pipelines & processing 16 views 0 copies

retry backoff exception-handling flaky-functions

Python code

26 lines

Python 3.6+

import random
import time


def flaky_function():
    if random.random() < 0.6:
        raise ConnectionError("Temporary network error")
    return "success"


def retry_with_exponential_backoff(func, max_retries=5, base_delay=1.0):
    for attempt in range(max_retries + 1):
        try:
            return func()
        except Exception as e:
            if attempt == max_retries:
                raise
            delay = base_delay * (2 ** attempt)
            print(f"Attempt {attempt + 1} failed: {e}. Retrying in {delay}s...")
            time.sleep(delay)


if __name__ == "__main__":
    random.seed(42)
    stage_result = retry_with_exponential_backoff(flaky_function)
    print(f"Stage completed: {stage_result}")

Output

stdout

Attempt 1 failed: Temporary network error. Retrying in 1.0s...
Attempt 2 failed: Temporary network error. Retrying in 2.0s...
Attempt 3 failed: Temporary network error. Retrying in 4.0s...
Stage completed: success

How it works

The retry function runs a loop that attempts func() up to max_retries + 1 times. When an exception occurs, it checks whether it's the last allowed attempt; if so, it re-raises the exception. Otherwise, it calculates a delay using base_delay * (2 ** attempt), which produces 1s, 2s, 4s, etc., and sleeps. The random seed gives a deterministic sequence, so with seed 42, the flaky function fails three times then succeeds on the fourth call. This pattern ensures transient failures are retried with increasing intervals, giving the system time to recover.

Common mistakes

Not including jitter, which can cause thundering herd problems when many clients retry simultaneously.
Swallowing exceptions permanently instead of re-raising after the final attempt.
Omitting a `time.sleep` call, making the backoff ineffective.
Forgetting that `time.sleep` blocks the thread, which can be problematic in asynchronous code.

Variations

Add random jitter like `delay = base_delay * (2 ** attempt) + random.uniform(0, 1)` to avoid synchronized retries.
Use `tenacity` or `backoff` library for more robust retry logic with decorators and configurable conditions.

Real-world use cases

Retrying fetching data from external APIs when temporary network failures occur in batch pipelines.
Reconnecting to a database or message broker after transient connection losses.
Handling rate-limited service calls by retrying with exponential delays.

Python Exponential Backoff Retry Example

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Data pipelines & processing

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Data pipelines & processing

Keep learning

Tutorials

Quizzes