How to Run Blocking Code in an Executor with asyncio in Python
This code runs blocking functions concurrently without stalling the event loop by offloading them to thread pool executors via asyncio.
Python code
22 linesimport asyncio
import time
def blocking_task(name: str, duration: float) -> str:
"""Simulate a blocking operation."""
time.sleep(duration)
return f"Finished {name} after {duration}s"
async def main() -> None:
loop = asyncio.get_running_loop()
results = await asyncio.gather(
loop.run_in_executor(None, blocking_task, "A", 1.0),
loop.run_in_executor(None, blocking_task, "B", 0.5),
loop.run_in_executor(None, blocking_task, "C", 0.2),
)
print(results)
if __name__ == "__main__":
asyncio.run(main())Output
['Finished A after 1.0s', 'Finished B after 0.5s', 'Finished C after 0.2s']How it works
asyncio.get_running_loop() retrieves the current event loop. loop.run_in_executor(None, func, *args) schedules func to run in a default thread pool executor, returning a coroutine that you can await. asyncio.gather runs multiple such coroutines concurrently, so the blocking sleeps happen in parallel threads. The event loop stays responsive because time.sleep blocks only the executor threads, not the loop itself.
Common mistakes
- Using `time.sleep` directly inside an async function, which blocks the event loop.
- Forgetting to await the result of `run_in_executor`, causing the task never to execute.
Variations
- Use `asyncio.to_thread(func, *args)` for a simpler one-liner in Python 3.9+.
- Pass a custom executor (e.g., `ThreadPoolExecutor(max_workers=5)`) to control thread count.
Real-world use cases
- Offloading CPU-bound or blocking database queries from a FastAPI endpoint to keep the server responsive.
- Performing legacy synchronous file I/O inside an async web crawler without pausing other requests.
- Running blocking third-party SDK calls (e.g., requests) inside an asyncio-based microservice.
Sponsored
More from Concurrency & performance
- Benchmark list.append vs deque.append in Python medium
- Build a Python Performance Profiler That Generates Readable Reports medium
- Graceful Shutdown Executor Context Manager in Python medium
- How to Build a Producer-Consumer Pattern with asyncio.Queue in Python medium
- How to Cancel an asyncio Task with Graceful Cleanup in Python medium
- How to Convert Data in Parallel with ThreadPoolExecutor in Python easy
Keep learning
Related tutorials and quizzes for this topic.