Bulkhead Thread Pool per Service Mock in Python

Simulates a bulkhead pattern with per-service thread pools and semaphore-based rejection to isolate failures between dependent services.

Medium Python 3.9+ Aug 9, 2026 Microservices patterns 11 views 0 copies

Python code

53 lines
Python 3.9+
import threading
import time
import random
from concurrent.futures import ThreadPoolExecutor

class ServiceBulkhead:
    def __init__(self, name, max_threads, max_queue):
        self.name = name
        self.executor = ThreadPoolExecutor(max_workers=max_threads)
        self.semaphore = threading.Semaphore(max_threads + max_queue)
        self.active = 0
        self.lock = threading.Lock()

    def call(self, fn, *args, **kwargs):
        if not self.semaphore.acquire(blocking=False):
            print(f"{self.name}: REJECTED - bulkhead full")
            return None
        try:
            with self.lock:
                self.active += 1
            result = self.executor.submit(fn, *args, **kwargs).result()
            return result
        finally:
            with self.lock:
                self.active -= 1
            self.semaphore.release()


def mock_service_call(service_name, delay):
    time.sleep(delay)
    return f"{service_name} response after {delay:.1f}s"


if __name__ == "__main__":
    payment = ServiceBulkhead("payment", max_threads=2, max_queue=2)
    shipping = ServiceBulkhead("shipping", max_threads=1, max_queue=1)

    def send_to_service(bulkhead, service):
        print(f"Sending to {service}...")
        result = bulkhead.call(mock_service_call, service, random.uniform(0.2, 0.8))
        if result:
            print(f"Result: {result}")

    threads = []
    for i in range(5):
        threads.append(threading.Thread(target=send_to_service, args=(payment, "payment")))
    for i in range(3):
        threads.append(threading.Thread(target=send_to_service, args=(shipping, "shipping")))

    for t in threads:
        t.start()
    for t in threads:
        t.join()

Output

stdout
Sending to payment...
Sending to shipping...
Sending to payment...
Sending to payment...
Sending to payment...
Sending to payment...
Sending to shipping...
Sending to shipping...
payment: REJECTED - bulkhead full
payment: REJECTED - bulkhead full
Result: payment response after 0.4s
Result: payment response after 0.6s
Result: payment response after 0.8s
Result: shipping response after 0.3s
Result: shipping response after 0.5s
Sending to shipping...
shipping: REJECTED - bulkhead full
payment: REJECTED - bulkhead full

How it works

Each service gets its own ThreadPoolExecutor and a Semaphore to limit concurrency. The semaphore counts both active tasks and queued items; if it can't acquire, the call is rejected immediately. The lock protects the active counter for accurate stats. Because threads run concurrently, rejection and completion order may vary, but the pattern isolates failures: a spike in one service doesn't exhaust resources of another.

Common mistakes

  • Forgetting to use blocking=False on acquire, which would block indefinitely instead of rejecting
  • Setting semaphore only to max_threads, ignoring the queue capacity
  • Not releasing the semaphore in a finally block, causing resource leaks
  • Sharing a single executor across services, defeating the bulkhead isolation

Variations

  1. Use `asyncio.Semaphore` and `asyncio.create_task` for async services
  2. Implement with `queue.Queue` and worker threads for more explicit control

Real-world use cases

  • Isolating calls to third-party payment and shipping APIs so one slow service doesn't block others.
  • Applying per-dependency thread limits in a microservices gateway to prevent cascading timeouts.
  • Managing connection pools for multiple backend databases with distinct capacity quotas.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Microservices patterns

Related tutorials and quizzes for this topic.