Rotate Log Files in Python by Size
This code rotates a log file when its size exceeds a threshold, keeping a specified number of backups.
Python code
33 linesimport os
import glob
from pathlib import Path
def rotate_log(log_path, max_size_bytes=1024, max_backups=3):
log_file = Path(log_path)
if log_file.stat().st_size <= max_size_bytes:
print(f"Log size {log_file.stat().st_size} bytes <= threshold, no rotation")
return
for i in range(max_backups, 0, -1):
src = log_file.with_suffix(f".log.{i-1}" if i > 1 else ".log")
dst = log_file.with_suffix(f".log.{i}")
if src.exists():
os.replace(src, dst)
if log_file.exists():
os.replace(log_file, log_file.with_suffix(".log.1"))
log_file.touch()
print(f"Rotated: created {log_file.name}, moved old to {log_file.name}.1")
if __name__ == "__main__":
test_file = Path("test.log")
test_file.write_text("x" * 100) # 100 bytes, below threshold
rotate_log(test_file, max_size_bytes=1024)
test_file.write_text("y" * 2000) # 2000 bytes, above threshold
rotate_log(test_file, max_size_bytes=1024)
# Cleanup
for f in glob.glob("test.log*"):
Path(f).unlink()
Output
Log size 100 bytes <= threshold, no rotation
Rotated: created test.log, moved old to test.log.1
How it works
The rotate_log function checks the current size of the log file using stat().st_size. If the file is smaller than the threshold, it returns early. Otherwise, it shifts existing backups: the newest backup becomes .log.1, the previous becomes .log.2, and so on, up to max_backups. The os.replace function is used for atomic file moves. After rotation, a new empty log file is created with touch(), and a message confirms the action.
Common mistakes
- Forgetting to check if the log file exists before calling `stat()`
- Not using `os.replace` leading to issues on Windows when destination exists
- Mishandling the loop range causing incorrect backup numbering
Variations
- Use `logging.handlers.RotatingFileHandler` for built-in size-based rotation
- Use `shutil.move` instead of `os.replace` if compatibility with older Python is needed
Real-world use cases
- Preventing log files from growing unbounded in long-running services.
- Rotating application logs in production to preserve disk space and ease debugging.
- Implementing a custom log rotation policy when external tools like logrotate are unavailable.
Sponsored
More from Files & data
- Append a Line to a Log File in Python easy
- Audit File Permissions Across a Project in Python easy
- Automatically Detect Corrupted Files Using SHA-256 Checksums in Python easy
- Automatically Highlight Data Validation Errors Inside Excel Files in Python easy
- Build a Command-Line To-Do List Application with Data Persistence in Python easy
- Build a File Index by Relative Path Hash Map in Python easy
Keep learning
Related tutorials and quizzes for this topic.