How to Drain a Connection Pool Before Exit in Python
Gracefully close all pooled sockets using a thread-safe ConnectionPool that drains connections before program exit.
Python code
61 linesimport socket
import threading
import time
import random
class ConnectionPool:
def __init__(self, size=5):
self.pool = []
self.lock = threading.Lock()
self.closed = False
for _ in range(size):
self.pool.append(self.create_connection())
def create_connection(self):
mock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
mock.settimeout(1)
return mock
def acquire(self):
with self.lock:
if self.closed:
raise RuntimeError("Pool is closed")
if self.pool:
return self.pool.pop()
return self.create_connection()
def release(self, conn):
with self.lock:
if not self.closed and conn.fileno() != -1:
self.pool.append(conn)
else:
conn.close()
def drain(self):
with self.lock:
self.closed = True
connections = self.pool
self.pool = []
for conn in connections:
try:
conn.close()
print(f"Closed connection fd={conn.fileno()} (now -1)")
except OSError as e:
print(f"Error closing: {e}")
print(f"Drained {len(connections)} connections")
def simulate_work(pool):
conn = pool.acquire()
time.sleep(random.uniform(0.01, 0.05))
pool.release(conn)
if __name__ == "__main__":
pool = ConnectionPool(3)
threads = [threading.Thread(target=simulate_work, args=(pool,)) for _ in range(5)]
for t in threads:
t.start()
for t in threads:
t.join()
pool.drain()
print("All connections drained before exit")
Output
Closed connection fd=3 (now -1)
Closed connection fd=4 (now -1)
Closed connection fd=5 (now -1)
Drained 3 connections
All connections drained before exit
How it works
This code builds a minimal connection pool where every socket is wrapped with a lock for thread safety. The drain() method marks the pool closed and removes all cached sockets from the internal list before closing each one individually. Because close() is called outside the lock, other threads won't block while file descriptors are being released. Using fileno() == -1 detects sockets that are already closed and avoids double-close errors. This pattern is a production-ready way to ensure no connections leak when an application shuts down.
Common mistakes
- Calling close() while still holding the lock, which can block or deadlock other threads.
- Forgetting to set a closed flag before draining, allowing new acquire() calls to grab stale sockets.
- Closing sockets that are still in use by other threads without proper synchronization.
Variations
- Use a dedicated pool library like `urllib3.PoolManager` that handles draining internally.
- Wrap the drain logic in a `contextlib.closing` block to ensure cleanup even on exceptions.
Real-world use cases
- Shutting down a web scraper gracefully so all HTTP connections are closed before the process exits.
- Flushing a database connection pool when a microservice receives a SIGTERM during container shutdown.
- Releasing pooled sockets in a chat server when the application needs to restart cleanly.
Sponsored
More from Production deployment patterns
- Auto Rollback on Error Rate Exceeded in Python medium
- Automate Semantic Versioning with Conventional Commits in Python medium
- Design a Data Helper for Beginners in Python easy
- Docker healthcheck CMD mock in Python easy
- Generate a Mock Artifact Version Tag in Python easy
- Generate a docker-compose.yml with mock services in Python easy
Keep learning
Related tutorials and quizzes for this topic.