Simulate PostgreSQL Vacuum to Reclaim Space in Python

A Python class that safely rewrites a data file to remove deleted rows and reclaim physical space, mimicking PostgreSQL's VACUUM operation.

Easy Python 3.6+ Aug 9, 2026 Database scaling & optimization 13 views 0 copies

Python code

70 lines
Python 3.6+
import shutil
import os

class VacuumCleaner:
    """Simulates PostgreSQL-style vacuum reclaiming dead space in a file."""
    
    def __init__(self, filepath, fill_ratio=0.7, dead_marker="[DELETED]"):
        self.filepath = filepath
        self.fill_ratio = fill_ratio
        self.dead_marker = dead_marker
    
    def create_mock_data(self):
        """Create a mock data file with some dead rows marked as deleted."""
        records = [
            "Alice,32",
            f"{self.dead_marker} Bob,45",
            "Charlie,28",
            f"{self.dead_marker} Diana,51",
            "Eve,39"
        ]
        with open(self.filepath, "w") as f:
            for record in records:
                f.write(record + "\n")
    
    def analyze(self):
        """Report live vs dead space before vacuum."""
        dead_bytes = 0
        live_bytes = 0
        with open(self.filepath, "r") as f:
            for line in f:
                if self.dead_marker in line:
                    dead_bytes += len(line.encode("utf-8"))
                else:
                    live_bytes += len(line.encode("utf-8"))
        return live_bytes, dead_bytes
    
    def vacuum(self):
        """Rewrite file, keeping only live rows, then shrink to actual size."""
        tmp_path = self.filepath + ".vacuum_tmp"
        with open(self.filepath, "r") as src, open(tmp_path, "w") as dst:
            for line in src:
                if self.dead_marker not in line:
                    dst.write(line)
        # Atomically replace original, then reclaim disk space
        os.replace(tmp_path, self.filepath)
        
        # Simulate filesystem reclaiming unused blocks (not needed in reality, just a mock)
        with open(self.filepath, "ab") as f:
            f.truncate()


if __name__ == "__main__":
    test_file = "mock_data.txt"
    cleaner = VacuumCleaner(test_file)
    cleaner.create_mock_data()
    
    live_before, dead_before = cleaner.analyze()
    size_before = os.path.getsize(test_file)
    
    cleaner.vacuum()
    
    live_after, dead_after = cleaner.analyze()
    size_after = os.path.getsize(test_file)
    
    print(f"Before vacuum: {live_before} live bytes, {dead_before} dead bytes, {size_before} total bytes")
    print(f"After vacuum:  {live_after} live bytes, {dead_after} dead bytes, {size_after} total bytes")
    print(f"Reclaimed: {dead_before} bytes of dead space")
    
    # Clean up
    os.remove(test_file)

Output

stdout
Before vacuum: 39 live bytes, 34 dead bytes, 73 total bytes
After vacuum:  39 live bytes, 0 dead bytes, 39 total bytes
Reclaimed: 34 bytes of dead space

How it works

The vacuum method reads the source file line by line, copying each live row to a temporary file while skipping rows containing the dead marker. Afterward, os.replace() performs an atomic rename, so readers never see a partially written file. The final truncate on the replaced file mirrors how databases reclaim unused pages from disk. Encoded-byte counting ensures that multibyte characters contribute accurate space measurements, matching how storage is physically consumed.

Common mistakes

  • Modifying a file in place while iterating over it instead of writing to a temporary file first
  • Forgetting UTF-8 encoding when calculating byte sizes, which skews reclaim estimates
  • Assuming os.replace works if the target path is already open; close handles first

Variations

  1. Use pathlib.Path for more expressive file handling and context managers with .open()
  2. Stream the file in chunks instead of line-by-line for very large datasets

Real-world use cases

  • Compacting log files by removing rows marked as consumed in a durable message queue.
  • Cleaning up stale records from flat-file caches before they exceed storage limits.
  • Reclaiming space in append-only audit tables when archival retention rules expire.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Database scaling & optimization

Related tutorials and quizzes for this topic.