Run Synchronous Code in Threadpools

Your FastAPI endpoint returns JSON in milliseconds — until the day a synchronous database call, an image resize, or a legacy SDK blocks the entire event loop. Every request in the queue stalls behind that one blocking operation, and your async def endpoints suddenly feel like a queue at a busy post office. The fix isn't to rewrite your code with asyncio magic — it's to run that synchronous code in a threadpool, letting FastAPI (and Starlette underneath) do the heavy lifting for you.

The Problem This Lesson Solves

FastAPI's async model makes concurrency look effortless, but it has a hidden trap: synchronous blocking code inside an async endpoint. In an async def endpoint, the event loop is the single thread that processes all events — incoming requests, file I/O, timer callbacks. When your code calls time.sleep(2) or a blocking requests.get() directly, it blocks that one thread. The event loop freezes, and every other request on the server waits.

Imagine a busy coffee shop with one barista. If the barista stops to hand-grind coffee beans for two minutes, every customer in line stands still. Now, if your barista delegates that grinding task to a separate helper worker in the back, the barista keeps taking orders and pouring espresso. That's the threadpool: a group of helper threads that let the event loop stay responsive while your synchronous tasks finish in the background.

The pain point is real: in production, one slow synchronous call — a legacy REST API, a psycopg2 query, a CPU-bound hash operation — can tank your API's throughput from thousands of requests per second to a handful. This lesson gives you the tool to isolate that damage.

Core Concept / Mental Model

Thread pool basics

A threadpool is a fixed-size collection of pre-spawned threads that wait for tasks. When you submit a task, the pool assigns it to an available thread. Threads are lightweight compared to processes, but they still share memory and can hit the dreaded Global Interpreter Lock (GIL) in CPU-bound work — though for I/O-bound tasks, the GIL is released during the wait, making threads a great fit.

Python's standard library ships concurrent.futures.ThreadPoolExecutor for this exact purpose. You can submit a function to the pool and get back a Future object. You can wait for the result with .result() or hand the whole thing to asyncio via loop.run_in_executor().

Where FastAPI fits in

Here's the elegant part: FastAPI automatically runs your def (non-async) path operations in a threadpool. Starlette, the underlying ASGI framework, detects that a route function is synchronous and calls it using run_in_executor under the hood. So simply declaring your route as def instead of async def moves the blocking code off the event loop — no extra syntax needed.

But you can also go one step further: inside an async def endpoint, you can explicitly offload a synchronous function to a threadpool using fastapi.concurrency.run_in_threadpool or the lower-level loop.run_in_executor. This gives you precise control when you must mix async and sync code in the same endpoint.

Key mental model: The event loop is the dispatcher; the threadpool is the team of workers. The dispatcher never gets stuck waiting for a worker — it schedules other work while workers handle the blocking stuff.

How It Works Step by Step

Step 1: Identify the blocking call

Look for I/O operations that don't natively support async — typical culprits:

• Database drivers that are not async (e.g., psycopg2, asyncpg is the exception)
• Third-party SDKs that only offer synchronous APIs (e.g., boto3, requests)
• CPU-heavy calculations like image processing or JSON serialization

Step 2: Choose your offload strategy

You have three main paths:

1. Declare your endpoint as def (simple, automatic, and best when the entire endpoint is blocking).
2. Use run_in_threadpool explicitly inside an async def endpoint for targeted offloading of one block.
3. Use loop.run_in_executor for complete control, e.g., with a custom executor.

Step 3: Respect the pool size

The default threadpool in FastAPI (via Starlette) uses anyio's default capacity — typically 40 threads. That's fine for most APIs, but if your blocking tasks are long-running, you may want to customize the executor. You can do this by creating your own ThreadPoolExecutor and passing it to run_in_executor. Just be aware: spawning too many threads can cause memory pressure and context-switching overhead.

Step 4: Handle the result

Whether you use automatic offloading or explicit run_in_threadpool, the result of your synchronous function is returned as if it were asynchronous. Exceptions propagate normally — but see troubleshooting for timeout nuances.

Hands-On Walkthrough

Let's build a small FastAPI app that demonstrates all three patterns. First, install FastAPI and an HTTP client:

pip install fastapi uvicorn httpx

Create main.py with the following code:

import time
from fastapi import FastAPI
from fastapi.concurrency import run_in_threadpool

app = FastAPI()

def blocking_work(seconds: int) -> str:
    """A blocking synchronous function (simulating I/O or CPU work)."""
    time.sleep(seconds)
    return f"Done after {seconds}s"

@app.get("/auto")
def auto_sync():
    """FastAPI runs this in a threadpool automatically."""
    return {"result": blocking_work(2)}

@app.get("/explicit")
async def explicit_async():
    """Mix async and sync: offload only the blocking part."""
    # Do some async work first
    await asyncio.sleep(0.1)
    result = await run_in_threadpool(blocking_work, 2)
    return {"result": result}

import asyncio  # added for completeness

Run the server:

uvicorn main:app --reload

Now fire two requests at the same time:

curl http://localhost:8000/auto &
curl http://localhost:8000/explicit &

Observe that both finish in about 2 seconds, not 4 — the threadpool runs them concurrently.

Expected output (when run with time or a concurrency tester): both /auto and /explicit return after ~2s, proving the event loop didn't block.

Now, the perils of doing it wrong: change /auto to async def auto_sync(): and call blocking_work() directly — no threadpool — and you'll see requests queue up, doubling response times.

Compare Options / When to Choose What

When to choose what: If your endpoint is 100% synchronous, just make it def — you get threadpooling for free. If you need to mix async operations (like calling an async HTTP client) with a legacy sync function, use run_in_threadpool for the sync part. Reach for run_in_executor only when you need a custom thread pool or want to pass a custom executor.

Troubleshooting & Edge Cases

1. Blocking the event loop anyway

If you forget to offload, your async def endpoint blocks. Symptom: response times grow linearly with number of concurrent requests. Fix: either change to def or wrap the blocking call with run_in_threadpool.

2. Threadpool saturation

If you have more concurrent blocking tasks than threads, they queue up. The default pool (~40) is enough for most, but if your tasks are CPU-heavy or long, you may starve other sync endpoints. Monitor thread usage; in extreme cases, consider a separate executor for different task classes.

3. Exceptions and cancellation

Exceptions in the threadpool propagate to the caller via the future, so try/except around await run_in_threadpool(...) works. However, cancellation is tricky: if a client disconnects while a thread is running, the thread continues to completion — you can't force-kill it. Design your sync functions to be interruptible (e.g., use time.sleep with cancel() handling) or accept that they'll finish in background.

4. The GIL and CPU-bound tasks

Threads won't speed up pure CPU-bound Python code due to the GIL. For def endpoints doing CPU-heavy work, threads will actually serialize — no speedup, but at least the event loop stays responsive. For true parallelism, use a process pool (ProcessPoolExecutor) or offload to an external service.

5. Creating threads too often

Avoid creating a new ThreadPoolExecutor on every request. Create it once at module level and reuse, or rely on FastAPI's global pool.

What You Learned & What's Next

You've mastered running synchronous code in threadpools. Let's recap the core skills you've built:

• You can identify blocking calls in async endpoints that stall the event loop.
• You know three ways to offload sync work: def endpoints (auto), run_in_threadpool (explicit), and run_in_executor (low-level).
• You can compare options and choose the right one based on your endpoint's structure.
• You can troubleshoot common issues like threadpool saturation and GIL limitations.

Your next step in the FastAPI Backend Development track is ready. You've solved the concurrency problem — now you'll learn how to take it further with asynchronous background tasks to handle long-running jobs after sending the response.

To practice what you've learned, open your FastAPI project and refactor one of your async def endpoints that calls a blocking library. Convert it to use run_in_threadpool, and test with concurrent requests. Notice how response times improve — and you've just made your API more production-ready.