How to Sync Two Directories in Python (rsync-like)
Mirror a source directory into a destination by copying new or changed files and deleting extras, similar to rsync.
Python code
41 linesimport os
import shutil
import sys
from pathlib import Path
def sync_dirs(src: Path, dst: Path):
"""Mirror src into dst: copy new files, overwrite changed, delete extras."""
dst.mkdir(parents=True, exist_ok=True)
for dst_entry in dst.rglob('*'):
rel = dst_entry.relative_to(dst)
src_entry = src / rel
if not src_entry.exists():
if dst_entry.is_dir():
shutil.rmtree(dst_entry)
else:
dst_entry.unlink()
print(f"Deleted: {rel}")
for src_entry in src.rglob('*'):
rel = src_entry.relative_to(src)
dst_entry = dst / rel
if src_entry.is_dir():
dst_entry.mkdir(exist_ok=True)
else:
if not dst_entry.exists() or src_entry.stat().st_mtime != dst_entry.stat().st_mtime:
dst_entry.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(src_entry, dst_entry)
print(f"Copied/Updated: {rel}")
if __name__ == "__main__":
src_dir = Path("source_dir")
dst_dir = Path("mirror_dir")
src_dir.mkdir(exist_ok=True)
(src_dir / "file1.txt").write_text("hello")
(src_dir / "file2.txt").write_text("world")
existing = dst_dir / "stale.txt"
existing.parent.mkdir(exist_ok=True)
existing.write_text("old")
sync_dirs(src_dir, dst_dir)
print("Destination contents:", sorted(p.name for p in dst_dir.iterdir()))
shutil.rmtree(src_dir)
shutil.rmtree(dst_dir)
Output
Copied/Updated: file1.txt
Copied/Updated: file2.txt
Deleted: stale.txt
Destination contents: ['file1.txt', 'file2.txt']
How it works
The function sync_dirs first scans the destination with rglob('*') and removes any entries that don't exist in the source using unlink() or rmtree(). It then iterates source files, creating missing directories with mkdir(exist_ok=True) and copying newer files using shutil.copy2 which preserves timestamps. The mtime comparison detects changed files without hashing contents. The cleanup at the end removes the temporary directories used for the demo.
Common mistakes
- Comparing only file size instead of mtime or checksum, missing content changes
- Forgetting to create parent directories before copying files
- Using `rglob('*')` without excluding directories when processing deletions
- Not handling symlinks, which can cause infinite recursion or broken copies
Variations
- Use `filecmp.cmp()` for byte-level comparison instead of mtime for accuracy
- Add a `--dry-run` mode using `print` statements to preview actions before executing
Real-world use cases
- Synchronizing configuration files from a master server to multiple worker nodes in a fleet.
- Mirroring build artifacts from a CI pipeline to an S3 bucket or FTP staging area.
- Syncing a local development folder to a mounted network drive or container volume.
Sponsored
More from Automation & scripting
- Aggregate Log Errors Count by Hour in Python easy
- Automate Tweeting New Blog Posts in Python easy
- Automatically Clean Temporary Files from Applications Using Python medium
- Automatically Download the Latest Software Release from GitHub with Python medium
- Automatically Generate Charts from CSV Files with One Command medium
- Automatically Generate Hardware Inventory Reports in Python easy
Keep learning
Related tutorials and quizzes for this topic.