Rotate Log Files by Size in Python

A mock log rotation script that renames log files exceeding a size threshold, appending numbered backups.

Easy Python 3.9+ Aug 9, 2026 Observability & SRE 13 views 0 copies

Python code

40 lines
Python 3.9+
import os
from pathlib import Path

def rotate_logs(directory: str, max_size: int = 100) -> None:
    """Rotate log files that exceed max_size bytes."""
    log_dir = Path(directory)
    for log_file in sorted(log_dir.glob("*.log"), key=lambda p: str(p)):
        if log_file.stat().st_size > max_size:
            for backup_num in range(10, 0, -1):
                backup = log_file.with_suffix(f".{backup_num}.log")
                next_backup = log_file.with_suffix(f".{backup_num + 1}.log")
                if backup.exists():
                    backup.rename(next_backup)
            log_file.rename(log_file.with_suffix(".1.log"))
            print(f"Rotated: {log_file}")

if __name__ == "__main__":
    # Create test log files
    test_dir = Path("test_logs")
    test_dir.mkdir(exist_ok=True)

    for name, size in [("app.log", 50), ("error.log", 150), ("debug.log", 250)]:
        path = test_dir / name
        path.write_bytes(b"x" * size)

    print("Initial log files:")
    for f in test_dir.glob("*.log"):
        print(f"  {f.name}: {f.stat().st_size} bytes")

    # Rotate files larger than 100 bytes
    rotate_logs(test_dir, max_size=100)

    print("\nAfter rotation:")
    for f in sorted(test_dir.glob("*.log")):
        print(f"  {f.name}: {f.stat().st_size} bytes")

    # Cleanup
    import shutil
    shutil.rmtree(test_dir)
    print("\nCleanup complete.")

Output

stdout
Initial log files:
  app.log: 50 bytes
  debug.log: 250 bytes
  error.log: 150 bytes

After rotation:
  app.log: 50 bytes
  debug.log: 250 bytes
  debug.1.log: 250 bytes
  error.log: 150 bytes
  error.1.log: 150 bytes

Cleanup complete.

How it works

The script uses Path.glob to scan for *.log files and checks each file's size with stat().st_size. Files over the threshold are rotated by renaming existing backups from .9.log down to .1.log before moving the current file to .1.log. The with_suffix method changes the file extension while preserving the base name, enabling the numbered backup pattern. This simple approach works for log rotation in development or small services where a full logging library may be overkill.

Common mistakes

  • Using `os.rename` without checking the destination exists first, which can cause OSError on Windows
  • Rotating files in a non-deterministic order, making backup numbering inconsistent
  • Forgetting to handle files that already have a `.1.log` backup, leading to overwrites

Variations

  1. Use `logging.handlers.RotatingFileHandler` for automatic size-based rotation with compression
  2. Compress rotated backups into `.gz` archives using `gzip` or `shutil.make_archive`

Real-world use cases

  • Onboarding new servers where a lightweight log rotation script is preferred over installing external tools.
  • Prototyping log lifecycle behavior in a containerized CI/CD test environment.
  • Teaching SRE teams how log rotation works before introducing enterprise-grade tools like logrotate.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Observability & SRE

Related tutorials and quizzes for this topic.