How to Implement Hedged Requests in Python

This code demonstrates a hedged request pattern using threading, which sends duplicate calls and returns the first result that arrives within a timeout.

Medium Python 3.9+ Aug 9, 2026 Reliability & rate limiting 16 views 0 copies

hedged-requests threading timeout latency reliability

Python code

29 lines

Python 3.9+

import time
from unittest.mock import Mock

def hedged_request(call, timeout=0.05):
    """Execute two duplicate calls, return first result within timeout."""
    result_container = {}

    def run_and_store():
        result_container['result'] = call()
        result_container['done'] = True

    # Simulate slow call with mock that sleeps
    slow_call = Mock(side_effect=lambda: (time.sleep(0.1), "slow result")[1])
    fast_call = Mock(return_value="fast result")

    # Start slow call
    import threading
    t = threading.Thread(target=run_and_store, args=(slow_call,))
    t.start()
    t.join(timeout)

    if result_container.get('done'):
        return result_container['result']
    else:
        return fast_call()

if __name__ == "__main__":
    result = hedged_request(lambda: None)
    print(result)

Output

stdout

fast result

How it works

The function starts a slow call in a separate thread and waits up to a specified timeout. If the slow call finishes within the timeout, its result is returned; otherwise, a fast fallback result is returned. This mirrors the hedged request pattern used in distributed systems to reduce tail latency. The threading.Thread and join(timeout) are key to controlling how long to wait. The mock calls simulate network variability without requiring an actual network.

Common mistakes

Using `time.sleep` in the main thread instead of spawning a worker thread.
Forgetting to set `daemon=True` on threads to avoid hanging on exit.
Assuming `join(timeout)` guarantees the thread has completed after the timeout.
Not handling exceptions from the slow call in the background thread.

Variations

Use `concurrent.futures.ThreadPoolExecutor` with `future.result(timeout)` for cleaner timeout handling.
Use `asyncio.wait_for` with coroutines for async systems.

Real-world use cases

Reducing tail latency in microservices by sending duplicate requests to multiple replicas and taking the fastest response.
Improving user experience in web search APIs by issuing parallel queries and using the first complete result.
Ensuring reliability in payment gateways where a delayed response could block the user flow, using a fallback response.

How to Implement Hedged Requests in Python

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Reliability & rate limiting

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Reliability & rate limiting

Keep learning

Tutorials

Quizzes