How to Drain a Connection Pool Before Exit in Python

Gracefully close all pooled sockets using a thread-safe ConnectionPool that drains connections before program exit.

Medium Python 3.9+ Aug 9, 2026 Production deployment patterns 13 views 0 copies

Python code

61 lines
Python 3.9+
import socket
import threading
import time
import random

class ConnectionPool:
    def __init__(self, size=5):
        self.pool = []
        self.lock = threading.Lock()
        self.closed = False
        for _ in range(size):
            self.pool.append(self.create_connection())
    
    def create_connection(self):
        mock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        mock.settimeout(1)
        return mock
    
    def acquire(self):
        with self.lock:
            if self.closed:
                raise RuntimeError("Pool is closed")
            if self.pool:
                return self.pool.pop()
            return self.create_connection()
    
    def release(self, conn):
        with self.lock:
            if not self.closed and conn.fileno() != -1:
                self.pool.append(conn)
            else:
                conn.close()
    
    def drain(self):
        with self.lock:
            self.closed = True
            connections = self.pool
            self.pool = []
        for conn in connections:
            try:
                conn.close()
                print(f"Closed connection fd={conn.fileno()} (now -1)")
            except OSError as e:
                print(f"Error closing: {e}")
        print(f"Drained {len(connections)} connections")

def simulate_work(pool):
    conn = pool.acquire()
    time.sleep(random.uniform(0.01, 0.05))
    pool.release(conn)

if __name__ == "__main__":
    pool = ConnectionPool(3)
    threads = [threading.Thread(target=simulate_work, args=(pool,)) for _ in range(5)]
    for t in threads:
        t.start()
    for t in threads:
        t.join()
    
    pool.drain()
    print("All connections drained before exit")

Output

stdout
Closed connection fd=3 (now -1)
Closed connection fd=4 (now -1)
Closed connection fd=5 (now -1)
Drained 3 connections
All connections drained before exit

How it works

This code builds a minimal connection pool where every socket is wrapped with a lock for thread safety. The drain() method marks the pool closed and removes all cached sockets from the internal list before closing each one individually. Because close() is called outside the lock, other threads won't block while file descriptors are being released. Using fileno() == -1 detects sockets that are already closed and avoids double-close errors. This pattern is a production-ready way to ensure no connections leak when an application shuts down.

Common mistakes

  • Calling close() while still holding the lock, which can block or deadlock other threads.
  • Forgetting to set a closed flag before draining, allowing new acquire() calls to grab stale sockets.
  • Closing sockets that are still in use by other threads without proper synchronization.

Variations

  1. Use a dedicated pool library like `urllib3.PoolManager` that handles draining internally.
  2. Wrap the drain logic in a `contextlib.closing` block to ensure cleanup even on exceptions.

Real-world use cases

  • Shutting down a web scraper gracefully so all HTTP connections are closed before the process exits.
  • Flushing a database connection pool when a microservice receives a SIGTERM during container shutdown.
  • Releasing pooled sockets in a chat server when the application needs to restart cleanly.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Production deployment patterns

Related tutorials and quizzes for this topic.