Implement Bulkhead Thread Pool Isolation in Python

Create isolated thread pools with a bulkhead pattern to protect different services from cascading failures.

Medium Python 3.9+ Aug 9, 2026 System design patterns 13 views 0 copies

Python code

61 lines
Python 3.9+
import threading
import time
import random
from concurrent.futures import ThreadPoolExecutor


class Bulkhead:
    """Simple bulkhead isolation: separate thread pools for different tasks."""

    def __init__(self, max_workers):
        self.executor = ThreadPoolExecutor(max_workers=max_workers)
        self.active = 0
        self.lock = threading.Lock()

    def submit(self, fn, *args, **kwargs):
        with self.lock:
            if self.active >= self.executor._max_workers:
                raise RuntimeError("Bulkhead full — task rejected")
            self.active += 1

        def wrapped():
            try:
                return fn(*args, **kwargs)
            finally:
                with self.lock:
                    self.active -= 1

        return self.executor.submit(wrapped)


def slow_task(name, delay):
    time.sleep(delay)
    return f"{name} done in {delay:.1f}s"


def fast_task(name):
    return f"{name} immediate"


if __name__ == "__main__":
    # Two isolated bulkheads: one for slow DB work, one for quick caching
    db_bulkhead = Bulkhead(max_workers=2)
    cache_bulkhead = Bulkhead(max_workers=3)

    # Saturate the DB bulkhead
    db_futures = [db_bulkhead.submit(slow_task, f"db-{i}", random.uniform(0.2, 0.5))
                  for i in range(2)]

    # Cache bulkhead is independent — keeps working while DB is full
    cache_results = [cache_bulkhead.submit(fast_task, f"cache-{i}") for i in range(3)]
    print("Cache results:", [f.result() for f in cache_results])

    # Third DB task should fail — pool is exhausted
    try:
        db_bulkhead.submit(slow_task, "db-overload", 0.1)
        print("Unexpected success — bulkhead should have rejected")
    except RuntimeError as e:
        print(f"Rejected as expected: {e}")

    for f in db_futures:
        print(f.result())

Output

stdout
Cache results: ['cache-0 immediate', 'cache-1 immediate', 'cache-2 immediate']
Rejected as expected: Bulkhead full — task rejected
db-0 done in 0.4s
db-1 done in 0.2s

How it works

The Bulkhead class wraps a ThreadPoolExecutor and tracks active tasks with a lock. When submitting, it checks if the pool is saturated and raises RuntimeError if so. The wrapped function decrements the counter in a finally block, ensuring thread safety. This isolates slow operations (like DB calls) from fast ones (like cache reads), preventing a slow service from exhausting all threads and blocking other services. The pattern prevents cascading failures in distributed systems by limiting concurrent load per dependency.

Common mistakes

  • Forgetting the `finally` block — if the task fails, the active count stays high and the pool becomes permanently blocked.
  • Using `executor._max_workers` directly — it's private API; better to pass `max_workers` to the class or expose it explicitly.
  • Not raising an exception on saturation — silently queuing can still cause unbounded resource consumption.

Variations

  1. Use a `Semaphore` to gate submissions instead of tracking with a lock, or mix both for timeouts.
  2. Add a queue with a max size to reject excess tasks instead of blocking.

Real-world use cases

  • Isolating synchronous database calls from cache lookups in a web backend to avoid one slow dependency blocking the other.
  • Protecting third-party API calls from S3 or external HTTP endpoints by limiting per-dependency concurrency.
  • Separating compute-heavy batch jobs from latency-sensitive real-time request handling in a service.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from System design patterns

Related tutorials and quizzes for this topic.