Simulate PostgreSQL Vacuum to Reclaim Space in Python
A Python class that safely rewrites a data file to remove deleted rows and reclaim physical space, mimicking PostgreSQL's VACUUM operation.
Python code
70 linesimport shutil
import os
class VacuumCleaner:
"""Simulates PostgreSQL-style vacuum reclaiming dead space in a file."""
def __init__(self, filepath, fill_ratio=0.7, dead_marker="[DELETED]"):
self.filepath = filepath
self.fill_ratio = fill_ratio
self.dead_marker = dead_marker
def create_mock_data(self):
"""Create a mock data file with some dead rows marked as deleted."""
records = [
"Alice,32",
f"{self.dead_marker} Bob,45",
"Charlie,28",
f"{self.dead_marker} Diana,51",
"Eve,39"
]
with open(self.filepath, "w") as f:
for record in records:
f.write(record + "\n")
def analyze(self):
"""Report live vs dead space before vacuum."""
dead_bytes = 0
live_bytes = 0
with open(self.filepath, "r") as f:
for line in f:
if self.dead_marker in line:
dead_bytes += len(line.encode("utf-8"))
else:
live_bytes += len(line.encode("utf-8"))
return live_bytes, dead_bytes
def vacuum(self):
"""Rewrite file, keeping only live rows, then shrink to actual size."""
tmp_path = self.filepath + ".vacuum_tmp"
with open(self.filepath, "r") as src, open(tmp_path, "w") as dst:
for line in src:
if self.dead_marker not in line:
dst.write(line)
# Atomically replace original, then reclaim disk space
os.replace(tmp_path, self.filepath)
# Simulate filesystem reclaiming unused blocks (not needed in reality, just a mock)
with open(self.filepath, "ab") as f:
f.truncate()
if __name__ == "__main__":
test_file = "mock_data.txt"
cleaner = VacuumCleaner(test_file)
cleaner.create_mock_data()
live_before, dead_before = cleaner.analyze()
size_before = os.path.getsize(test_file)
cleaner.vacuum()
live_after, dead_after = cleaner.analyze()
size_after = os.path.getsize(test_file)
print(f"Before vacuum: {live_before} live bytes, {dead_before} dead bytes, {size_before} total bytes")
print(f"After vacuum: {live_after} live bytes, {dead_after} dead bytes, {size_after} total bytes")
print(f"Reclaimed: {dead_before} bytes of dead space")
# Clean up
os.remove(test_file)
Output
Before vacuum: 39 live bytes, 34 dead bytes, 73 total bytes
After vacuum: 39 live bytes, 0 dead bytes, 39 total bytes
Reclaimed: 34 bytes of dead space
How it works
The vacuum method reads the source file line by line, copying each live row to a temporary file while skipping rows containing the dead marker. Afterward, os.replace() performs an atomic rename, so readers never see a partially written file. The final truncate on the replaced file mirrors how databases reclaim unused pages from disk. Encoded-byte counting ensures that multibyte characters contribute accurate space measurements, matching how storage is physically consumed.
Common mistakes
- Modifying a file in place while iterating over it instead of writing to a temporary file first
- Forgetting UTF-8 encoding when calculating byte sizes, which skews reclaim estimates
- Assuming os.replace works if the target path is already open; close handles first
Variations
- Use pathlib.Path for more expressive file handling and context managers with .open()
- Stream the file in chunks instead of line-by-line for very large datasets
Real-world use cases
- Compacting log files by removing rows marked as consumed in a durable message queue.
- Cleaning up stale records from flat-file caches before they exceed storage limits.
- Reclaiming space in append-only audit tables when archival retention rules expire.
Sponsored
More from Database scaling & optimization
- Approximate Count with HyperLogLog in Python medium
- B-Tree Insert and In-Order Traversal in Python hard
- Broadcast a Small Reference Table in Python easy
- Build a Full Text Search Index in Python medium
- Build a Partial Index Mock in Python for Database Filtering easy
- Composite index leftmost prefix in Python medium
Keep learning
Related tutorials and quizzes for this topic.