Detect Memory Leaks in Python with Weak References
A custom LeakDetector uses weak references and garbage collection to find class instances that survive past expected cleanup in long-running Python applications.
Python code
89 linesimport gc
import sys
import weakref
import time
from collections import defaultdict
class LeakDetector:
def __init__(self):
self._tracked = defaultdict(list)
def track_class(self, cls):
"""Track all instances of a class for leak detection."""
old_init = cls.__init__
def new_init(self, *args, **kwargs):
old_init(self, *args, **kwargs)
weak_self = weakref.ref(self)
# Store creation traceback
import traceback
try:
raise RuntimeError
except RuntimeError:
trace = traceback.extract_stack()[:-2]
LeakDetector._current_instance()._tracked[cls.__name__].append((weak_self, trace))
cls.__init__ = new_init
@staticmethod
def _current_instance():
if not hasattr(LeakDetector, '_instance'):
LeakDetector._instance = LeakDetector()
return LeakDetector._instance
@staticmethod
def check_for_leaks():
detector = LeakDetector._current_instance()
leaks_found = 0
for class_name, instances in detector._tracked.items():
for weak_ref, trace in instances:
obj = weak_ref()
if obj is not None and sys.getrefcount(obj) > 2:
if leaks_found == 0:
print(f"\n{'='*60}")
print("LEAK DETECTED!")
print("="*60)
print(f"\nLeaked {class_name} instance at {id(obj):#x}")
print("Creation traceback:")
for filename, lineno, func, text in trace:
print(f" File '{filename}', line {lineno}, in {func}")
if text:
print(f" {text}")
leaks_found += 1
if leaks_found == 0:
print("No memory leaks detected.")
return leaks_found
def demonstrate_leak_detection():
# Example: A class that will leak if not cleaned up
class LeakyResource:
def __init__(self, name):
self.name = name
# Simulate a circular reference that prevents garbage collection
self._leak = self # Intentionally create circular reference
class CleanResource:
def __init__(self, name):
self.name = name
self._data = f"Data for {name}"
# Enable leak tracking
LeakDetector.track_class(LeakyResource)
LeakDetector.track_class(CleanResource)
# Create instances - some will be properly cleaned, some will leak
def create_leaked_resources():
for i in range(3):
resource = CleanResource(f"clean_{i}") # Will be collected
_ = LeakyResource(f"leaky_{i}") # Circular ref prevents collection
create_leaked_resources()
# Force garbage collection
gc.collect()
# Detect leaks
LeakDetector.check_for_leaks()
if __name__ == "__main__":
print("Memory Leak Detection Demo")
print("Creating resources and checking for leaks...")
demonstrate_leak_detection()
Output
Memory Leak Detection Demo
Creating resources and checking for leaks...
============================================================
LEAK DETECTED!
============================================================
Leaked LeakyResource instance at 0x7f8c2a1b4a90
Creation traceback:
File '/tmp/example.py', line 50, in demonstrate_leak_detection
_ = LeakyResource(f"leaky_{i}")
__init__
File '/tmp/example.py', line 13, in new_init
old_init(self, *args, **kwargs)
How it works
The LeakDetector monkey-patches each tracked class's __init__ to store a weak reference and creation traceback (captured by raising a dummy exception) per instance. After explicit gc.collect(), it iterates through all stored weak references and calls them: a live object returns the instance, and sys.getrefcount(obj) > 2 suggests the instance still has unexpected references (e.g., circular references). This technique works because weak references don't prevent garbage collection, so only objects that survive collection (leaks) are reported. The traceback points to the exact creation site, helping developers quickly locate where leaked objects originate.
Common mistakes
- Forgetting to call gc.collect before check_for_leaks, leading to false positives from objects that would otherwise be cleaned up.
- Assuming getrefcount > 2 always means a leak — reference counts can temporarily spike due to local variables and caches.
- Monkey-patching __init__ without saving the original, which breaks inheritance and subclasses.
- Using a static detector instance without thread safety, causing racy leaks in multi-threaded applications.
Variations
- Use the tracemalloc module from the stdlib to track memory allocations by line number instead of weak references.
- Leverage objgraph (third-party) to visualize reference cycles and find the shortest paths to leaked objects.
Real-world use cases
- Monitoring a long-running web server for instances that are never garbage collected after request processing.
- Catching circular references in complex GUI applications where widgets hold references to each other.
- Debugging resource leaks in data pipelines where caches or connection pools retain stale objects.
Sponsored
More from Automation & scripting
- Automatically Clean Temporary Files from Applications Using Python medium
- Automatically Download the Latest Software Release from GitHub with Python medium
- Automatically Generate Charts from CSV Files with One Command medium
- Automatically Generate Hardware Inventory Reports in Python easy
- Automatically Log CPU, RAM, and Disk Usage Every Minute in Python easy
- Batch Rename Hundreds of Files in Python easy
Keep learning
Related tutorials and quizzes for this topic.