Implement Bulkhead Thread Pool Isolation in Python
Create isolated thread pools with a bulkhead pattern to protect different services from cascading failures.
Python code
61 linesimport threading
import time
import random
from concurrent.futures import ThreadPoolExecutor
class Bulkhead:
"""Simple bulkhead isolation: separate thread pools for different tasks."""
def __init__(self, max_workers):
self.executor = ThreadPoolExecutor(max_workers=max_workers)
self.active = 0
self.lock = threading.Lock()
def submit(self, fn, *args, **kwargs):
with self.lock:
if self.active >= self.executor._max_workers:
raise RuntimeError("Bulkhead full — task rejected")
self.active += 1
def wrapped():
try:
return fn(*args, **kwargs)
finally:
with self.lock:
self.active -= 1
return self.executor.submit(wrapped)
def slow_task(name, delay):
time.sleep(delay)
return f"{name} done in {delay:.1f}s"
def fast_task(name):
return f"{name} immediate"
if __name__ == "__main__":
# Two isolated bulkheads: one for slow DB work, one for quick caching
db_bulkhead = Bulkhead(max_workers=2)
cache_bulkhead = Bulkhead(max_workers=3)
# Saturate the DB bulkhead
db_futures = [db_bulkhead.submit(slow_task, f"db-{i}", random.uniform(0.2, 0.5))
for i in range(2)]
# Cache bulkhead is independent — keeps working while DB is full
cache_results = [cache_bulkhead.submit(fast_task, f"cache-{i}") for i in range(3)]
print("Cache results:", [f.result() for f in cache_results])
# Third DB task should fail — pool is exhausted
try:
db_bulkhead.submit(slow_task, "db-overload", 0.1)
print("Unexpected success — bulkhead should have rejected")
except RuntimeError as e:
print(f"Rejected as expected: {e}")
for f in db_futures:
print(f.result())
Output
Cache results: ['cache-0 immediate', 'cache-1 immediate', 'cache-2 immediate']
Rejected as expected: Bulkhead full — task rejected
db-0 done in 0.4s
db-1 done in 0.2s
How it works
The Bulkhead class wraps a ThreadPoolExecutor and tracks active tasks with a lock. When submitting, it checks if the pool is saturated and raises RuntimeError if so. The wrapped function decrements the counter in a finally block, ensuring thread safety. This isolates slow operations (like DB calls) from fast ones (like cache reads), preventing a slow service from exhausting all threads and blocking other services. The pattern prevents cascading failures in distributed systems by limiting concurrent load per dependency.
Common mistakes
- Forgetting the `finally` block — if the task fails, the active count stays high and the pool becomes permanently blocked.
- Using `executor._max_workers` directly — it's private API; better to pass `max_workers` to the class or expose it explicitly.
- Not raising an exception on saturation — silently queuing can still cause unbounded resource consumption.
Variations
- Use a `Semaphore` to gate submissions instead of tracking with a lock, or mix both for timeouts.
- Add a queue with a max size to reject excess tasks instead of blocking.
Real-world use cases
- Isolating synchronous database calls from cache lookups in a web backend to avoid one slow dependency blocking the other.
- Protecting third-party API calls from S3 or external HTTP endpoints by limiting per-dependency concurrency.
- Separating compute-heavy batch jobs from latency-sensitive real-time request handling in a service.
Sponsored
More from System design patterns
- Build a BFF (Backend for Frontend) Mock Aggregator in Python medium
- Builder pattern for mocking complex objects in Python easy
- Circuit Breaker Pattern in Python: Closed, Open, and Half-Open States medium
- Create a Data Helper Class in Python easy
- Domain Driven Design Aggregate Root Example in Python medium
- Facade Pattern in Python with Mock Simplification medium
Keep learning
Related tutorials and quizzes for this topic.