Debugging Python Memory Leaks: A Practical Guide

You’re running a Python script, maybe a web server or a data pipeline, and after a few hours, it starts slowing down. Then your system starts swapping. Eventually, the process is killed by the OOM (Out Of Memory) killer. Sound familiar?

Memory leaks in Python aren’t as obvious as in compiled languages, but they happen more often than most developers admit. The good news is that with the right tools and a systematic approach, you can find and fix them without tearing your hair out.

What actually causes memory leaks in Python?

Python’s garbage collector is generally reliable, but it can’t clean up everything. The main culprits are:

• Circular references that the GC fails to collect (rare with modern CPython)
• Unclosed resources like file handles, database connections, or network sockets
• Cached objects that grow unboundedly (e.g., dictionaries or lists used as caches without eviction)
• C extension modules that allocate memory outside Python’s control
• Forgotten listeners or callbacks that hold references to objects

I’ve seen a Django app that leaked 500MB/hour because a logging handler wasn’t properly closed. The code looked fine on the surface, but the handler kept all log messages in memory for “debugging purposes.”

Step 1: Detect the leak before it kills your app

Don’t wait for the process to crash. Use Python’s built-in tools to spot memory growth early.

import tracemalloc
import objgraph

# Start tracing
tracemalloc.start()

# Take a snapshot after your app has been running for a while
snapshot1 = tracemalloc.take_snapshot()

# ... let your app run some operations ...

snapshot2 = tracemalloc.take_snapshot()

# Compare snapshots to see what's growing
top_stats = snapshot2.compare_to(snapshot1, 'lineno')
for stat in top_stats[:10]:
    print(f"{stat.size_diff / 1024:.1f} KB - {stat.count_diff} objects in {stat.traceback}")

This will show you exactly which lines of code are allocating memory that never gets freed. I’ve used this technique at PythonSkillset to debug a Celery worker that was keeping task results in memory because of a missing .forget() call.

Step 2: Find what’s holding references

Sometimes you know something is leaking, but you don’t know what is holding it alive. That’s where objgraph shines.

import objgraph

# Show the 5 most common types in memory
objgraph.show_most_common_types(limit=5)

# Find what's keeping a list object alive
objgraph.find_backref_chain(
    some_leaking_list,
    objgraph.is_proper_module
)

The find_backref_chain function is gold. It traces back from your suspect object to everything that holds a reference to it. I once found a memory leak in a Flask app where a request object was being captured inside a closure that was never released. The chain showed it was being held by a thread-local variable.

Step 3: Use gc module to inspect the garbage collector

For stubborn circular references, Python’s gc module can help you see what the collector knows:

import gc

# Enable debug output
gc.set_debug(gc.DEBUG_SAVEALL)

# Force a collection
unreachable = gc.collect()

if unreachable:
    print(f"Found {len(unreachable)} unreachable objects")
    for obj in unreachable[:5]:
        print(repr(obj)[:200])

This saved my team at PythonSkillset when we had a leak in a real-time dashboard. A custom class was holding references to itself in a __del__ method, creating a circular reference that the GC couldn’t clean because __del__ prevented it.

Step 4: Profile memory usage with memory_profiler

For fine-grained analysis, memory_profiler gives you line-by-line memory usage:

@profile
def process_data():
    # Your code here
    data = load_large_file()  # This line might be the culprit
    # ... more code ...

Run with: python -m memory_profiler your_script.py

The output shows memory usage per line. I’ve seen cases where a simple list comprehension was holding twice as much memory as expected because of how Python evaluated the expression.

Step 5: Practical fix strategies

Once you find the leak, here’s how to fix it:

• For caches: Use functools.lru_cache with a maxsize parameter, or implement weakref.WeakValueDictionary for caches that don’t need strong references.
• For forgotten callbacks: Unregister listeners in __del__ or use weak references (weakref.ref) for callbacks.
• For unclosed resources: Use context managers (with statements) religiously, especially for files, sockets, and database connections.
• For C extensions: Check if they provide cleanup functions, or consider using ctypes instead.

At PythonSkillset, we added a weekly cron job that runs memory profiling on our production-like environment and alerts if any process grows more than 10% per hour. It catches leaks before they hit production.

When to use external tools

If your leak is in production and you can’t reproduce it locally, use these:

• Pympler – For production memory analysis without heavy overhead
• Fil profiler – Shows memory per line with minimal CPU overhead
• Memray – Modern C-level memory profiler (works with Python 3.6+)

I recommend starting with tracemalloc and objgraph — they’re built into Python and cover 90% of cases. Only reach for external tools if you’re dealing with C extensions or extremely high-throughput systems.

A quick checklist for your next leak

1. Run tracemalloc to see what’s growing
2. Use objgraph to find what holds references
3. Check for unclosed file handles and connections
4. Review your cache implementations for unbounded growth
5. Look for circular references with custom __del__ methods
6. Profile memory in production with pympler if needed

Memory leaks are frustrating, but they’re also predictable once you know where to look. The next time your Python process gets killed by OOM, you’ll have the tools and the steps to get to the bottom of it.