How to Limit Concurrent Requests with a Semaphore in Python

Use threading.Semaphore with a ThreadPoolExecutor to cap how many worker threads run simultaneously, preventing resource overload.

Medium Python 3.9+ Aug 9, 2026 System design patterns 14 views 0 copies

concurrency semaphore threading threadpoolexecutor rate-limit

Python code

21 lines

Python 3.9+

import threading
import time
from concurrent.futures import ThreadPoolExecutor

def worker(name, semaphore, results):
    with semaphore:
        results.append(f"start {name}")
        time.sleep(0.5)  # simulate async work
        results.append(f"done {name}")

def main():
    sem = threading.Semaphore(2)  # max 2 concurrent workers
    results = []
    with ThreadPoolExecutor(max_workers=5) as executor:
        for i in range(5):
            executor.submit(worker, f"task-{i}", sem, results)
    for line in results:
        print(line)

if __name__ == "__main__":
    main()

Output

stdout

start task-0
start task-1
done task-0
done task-1
start task-2
start task-3
done task-2
done task-3
start task-4
done task-4

How it works

A threading.Semaphore maintains an internal counter initialized to the max concurrency. Each with semaphore: block acquires the semaphore before running and releases it on exit, so only the permitted number of workers proceed at once. Combined with a ThreadPoolExecutor(max_workers=5), five tasks are submitted, but the semaphore throttles execution to two at a time. The time.sleep(0.5) simulates external work like an API call, making the scheduling visible in the output. This pattern is a classic concurrency limiter for rate-limited endpoints.

Common mistakes

Forgetting to call `release()` on error – use the context manager to avoid leaks
Using `Semaphore(0)` accidentally, which blocks all workers forever
Assuming semaphore limits the number of submitted tasks, not the active ones

Variations

Use `asyncio.Semaphore` with `asyncio.gather` or `create_task` for async code
Replace `ThreadPoolExecutor` with `ProcessPoolExecutor` for CPU-bound work, still using the same semaphore approach

Real-world use cases

Throttling outgoing HTTP requests to a third-party API that allows only a few concurrent calls.
Capping database connection usage from a pool of threads to avoid overwhelming the server.
Limiting concurrent file writes or cloud storage uploads in a batch job to prevent I/O saturation.

How to Limit Concurrent Requests with a Semaphore in Python

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from System design patterns

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from System design patterns

Keep learning

Tutorials

Quizzes