Python: Archive Old Logs by Compressing Gzip by Age
A Python script that finds .log files older than a specified age and compresses them into .gz archives while removing the originals.
Python code
42 linesimport gzip
import os
import shutil
from pathlib import Path
def archive_logs(log_dir: str, max_age_days: int) -> list[str]:
"""Compress log files older than max_age_days into .gz archives.
Returns a list of compressed file paths.
"""
cutoff = time.time() - max_age_days * 86400
compressed = []
for log_file in Path(log_dir).glob("*.log"):
if log_file.stat().st_mtime < cutoff:
gz_path = log_file.with_suffix(".log.gz")
with log_file.open("rb") as f_in, gzip.open(gz_path, "wb") as f_out:
shutil.copyfileobj(f_in, f_out)
log_file.unlink()
compressed.append(str(gz_path))
return compressed
if __name__ == "__main__":
import time
# Create sample logs
test_dir = Path("/tmp/sample_logs")
test_dir.mkdir(exist_ok=True)
for name in ["recent.log", "old.log"]:
path = test_dir / name
path.write_text("sample log content\n")
# Make old.log 10 days old
old_ts = time.time() - 10 * 86400
os.utime(test_dir / "old.log", (old_ts, old_ts))
result = archive_logs(str(test_dir), max_age_days=7)
print(f"Compressed: {result}")
print(f"Remaining files: {sorted(p.name for p in test_dir.iterdir())}")
Output
Compressed: ['/tmp/sample_logs/old.log.gz']
Remaining files: ['recent.log']
How it works
The script uses pathlib.Path.glob to find all .log files, then compares each file's modification time (via stat().st_mtime) against a cutoff computed by subtracting max_age_days from the current time (in seconds). Files that are older than the cutoff are opened in binary mode and copied into a gzip archive using shutil.copyfileobj, which efficiently streams the data. After the archive is created, the original file is deleted with unlink() to save space. The function returns a list of compressed file paths for logging or further processing.
Common mistakes
- Forgetting to pass a timestamp tuple to os.utime — it needs both atime and mtime
- Comparing mtime against current time instead of using a cutoff
- Deleting the original file before verifying the archive was written successfully
Variations
- Use `log_file.with_suffix('.log.gz')` for clarity, or build the path manually with `log_file.stem`
- Add error handling with try/except to skip files that can't be compressed
Real-world use cases
- A cron job that nightly compresses token-safe logs older than 30 days to save disk space.
- A log rotation system that archives debug logs to .gz before deletion, keeping audit trails.
- A backup automation that runs periodically on app servers to gzip stale log files.
Sponsored
More from Automation & scripting
- Aggregate Log Errors Count by Hour in Python easy
- Automate Tweeting New Blog Posts in Python easy
- Automatically Clean Temporary Files from Applications Using Python medium
- Automatically Download the Latest Software Release from GitHub with Python medium
- Automatically Generate Charts from CSV Files with One Command medium
- Automatically Generate Hardware Inventory Reports in Python easy
Keep learning
Related tutorials and quizzes for this topic.