Bulkhead Thread Pool per Service Mock in Python
Simulates a bulkhead pattern with per-service thread pools and semaphore-based rejection to isolate failures between dependent services.
Python code
53 linesimport threading
import time
import random
from concurrent.futures import ThreadPoolExecutor
class ServiceBulkhead:
def __init__(self, name, max_threads, max_queue):
self.name = name
self.executor = ThreadPoolExecutor(max_workers=max_threads)
self.semaphore = threading.Semaphore(max_threads + max_queue)
self.active = 0
self.lock = threading.Lock()
def call(self, fn, *args, **kwargs):
if not self.semaphore.acquire(blocking=False):
print(f"{self.name}: REJECTED - bulkhead full")
return None
try:
with self.lock:
self.active += 1
result = self.executor.submit(fn, *args, **kwargs).result()
return result
finally:
with self.lock:
self.active -= 1
self.semaphore.release()
def mock_service_call(service_name, delay):
time.sleep(delay)
return f"{service_name} response after {delay:.1f}s"
if __name__ == "__main__":
payment = ServiceBulkhead("payment", max_threads=2, max_queue=2)
shipping = ServiceBulkhead("shipping", max_threads=1, max_queue=1)
def send_to_service(bulkhead, service):
print(f"Sending to {service}...")
result = bulkhead.call(mock_service_call, service, random.uniform(0.2, 0.8))
if result:
print(f"Result: {result}")
threads = []
for i in range(5):
threads.append(threading.Thread(target=send_to_service, args=(payment, "payment")))
for i in range(3):
threads.append(threading.Thread(target=send_to_service, args=(shipping, "shipping")))
for t in threads:
t.start()
for t in threads:
t.join()
Output
Sending to payment...
Sending to shipping...
Sending to payment...
Sending to payment...
Sending to payment...
Sending to payment...
Sending to shipping...
Sending to shipping...
payment: REJECTED - bulkhead full
payment: REJECTED - bulkhead full
Result: payment response after 0.4s
Result: payment response after 0.6s
Result: payment response after 0.8s
Result: shipping response after 0.3s
Result: shipping response after 0.5s
Sending to shipping...
shipping: REJECTED - bulkhead full
payment: REJECTED - bulkhead full
How it works
Each service gets its own ThreadPoolExecutor and a Semaphore to limit concurrency. The semaphore counts both active tasks and queued items; if it can't acquire, the call is rejected immediately. The lock protects the active counter for accurate stats. Because threads run concurrently, rejection and completion order may vary, but the pattern isolates failures: a spike in one service doesn't exhaust resources of another.
Common mistakes
- Forgetting to use blocking=False on acquire, which would block indefinitely instead of rejecting
- Setting semaphore only to max_threads, ignoring the queue capacity
- Not releasing the semaphore in a finally block, causing resource leaks
- Sharing a single executor across services, defeating the bulkhead isolation
Variations
- Use `asyncio.Semaphore` and `asyncio.create_task` for async services
- Implement with `queue.Queue` and worker threads for more explicit control
Real-world use cases
- Isolating calls to third-party payment and shipping APIs so one slow service doesn't block others.
- Applying per-dependency thread limits in a microservices gateway to prevent cascading timeouts.
- Managing connection pools for multiple backend databases with distinct capacity quotas.
Sponsored
More from Microservices patterns
- BFF aggregation pattern: combine multiple service responses in Python easy
- Backward Compatible Schema Evolution in Python medium
- CQRS with Separate Read and Write Repositories in Python medium
- Cache-Aside Pattern in Python: Per-Service Mock easy
- Consumer Driven Contract Pact Mock in Python medium
- Correlation ID HTTP header mock in Python easy
Keep learning
Related tutorials and quizzes for this topic.