How to Implement a Bulkhead Pattern with Threading in Python
Implement a bulkhead pattern in Python that isolates concurrent tasks with a bounded semaphore, limiting active workers to prevent resource exhaustion.
Python code
42 linesimport threading
import time
import random
class Bulkhead:
def __init__(self, workers: int):
self._semaphore = threading.BoundedSemaphore(workers)
self._lock = threading.Lock()
self._active = 0
def run(self, task):
with self._semaphore:
with self._lock:
self._active += 1
try:
task()
finally:
with self._lock:
self._active -= 1
@property
def active(self):
with self._lock:
return self._active
def task(name):
print(f"Start {name}, active={bulkhead.active}")
time.sleep(random.uniform(0.05, 0.2))
print(f"End {name}, active={bulkhead.active}")
if __name__ == "__main__":
bulkhead = Bulkhead(workers=2)
names = [f"task-{i}" for i in range(6)]
threads = [threading.Thread(target=bulkhead.run, args=(lambda n=n: task(n),)) for n in names]
for t in threads:
t.start()
for t in threads:
t.join()
print(f"Final active: {bulkhead.active}")
Output
Start task-0, active=1
Start task-1, active=2
End task-0, active=1
Start task-2, active=2
End task-1, active=1
Start task-3, active=2
End task-2, active=1
Start task-4, active=2
End task-3, active=1
Start task-5, active=2
End task-4, active=1
End task-5, active=1
Final active: 0
How it works
The Bulkhead class uses a BoundedSemaphore to cap the number of concurrent tasks at the configured worker count. Each run call acquires the semaphore, increments an active counter under a lock, executes the task, and decrements the counter in a finally block to guarantee cleanup. The active property provides thread-safe visibility of current in-flight tasks. This isolates the workload so failures or slow tasks in one bulkhead don't block the whole system, improving fault tolerance.
Common mistakes
- Forgetting to use `finally` to release the semaphore and decrement the counter on exceptions
- Using a plain `Semaphore` instead of `BoundedSemaphore`, which allows over-release bugs
- Not using a lock when updating shared counters, causing race conditions with concurrent threads
Variations
- Use `asyncio.Semaphore` to implement the same pattern for async coroutines
- Wrap the task with `timeout` handling using `threading.Timer` to prevent stuck workers
Real-world use cases
- Isolating database connection pools per service tier so one slow query doesn't exhaust global resources.
- Limiting concurrent calls to a third-party API to stay within rate limits and avoid throttling.
- Separating CPU-heavy and I/O-heavy workloads so they don't compete for the same thread budget.
Sponsored
More from Reliability & rate limiting
- At Least Once with Idempotent Consumer in Python medium
- Build a Rate Limiter Decorator in Python easy
- Build a queue-based admission control system in Python easy
- Chaos Inject Random Failures in Python easy
- Circuit breaker failure threshold count in Python medium
- Exactly Once Processing Dedupe Mock in Python easy
Keep learning
Related tutorials and quizzes for this topic.