How to Compare Directory Trees in Python
This code recursively scans two directory trees and reports files that exist in only one directory, as well as files present in both but with different content.
Python code
48 linesfrom pathlib import Path
def compare_directories(path1, path2):
dir1 = Path(path1)
dir2 = Path(path2)
if not dir1.is_dir() or not dir2.is_dir():
raise ValueError("Both paths must be directories.")
files1 = {p.relative_to(dir1) for p in dir1.rglob("*") if p.is_file()}
files2 = {p.relative_to(dir2) for p in dir2.rglob("*") if p.is_file()}
only_in_1 = files1 - files2
only_in_2 = files2 - files1
common = files1 & files2
different = []
for rel_path in common:
file1 = dir1 / rel_path
file2 = dir2 / rel_path
if file1.read_bytes() != file2.read_bytes():
different.append(str(rel_path))
return {
"only_in_first": sorted(str(p) for p in only_in_1),
"only_in_second": sorted(str(p) for p in only_in_2),
"different_content": sorted(different),
}
if __name__ == "__main__":
import tempfile
import os
with tempfile.TemporaryDirectory() as d1, tempfile.TemporaryDirectory() as d2:
os.makedirs(os.path.join(d1, "sub"))
os.makedirs(os.path.join(d2, "sub"))
Path(d1, "same.txt").write_text("hello")
Path(d2, "same.txt").write_text("hello")
Path(d1, "diff.txt").write_text("version 1")
Path(d2, "diff.txt").write_text("version 2")
Path(d1, "only1.txt").write_text("only here")
Path(d2, "only2.txt").write_text("only there")
result = compare_directories(d1, d2)
print(result)
Output
{'only_in_first': ['only1.txt'], 'only_in_second': ['only2.txt'], 'different_content': ['diff.txt']}
How it works
The rglob("*") method recursively walks all paths under each directory, and is_file() filters out directories. Using set operations on relative paths makes it easy to identify files that are missing from one side or the other. For common files, reading bytes with read_bytes() allows a fast exact comparison. The result is a clean dictionary with three categories of differences.
Common mistakes
- Forgetting that rglob returns directories too; always filter with is_file()
- Comparing full paths instead of relative paths, which causes false mismatches
- Not handling permission errors when reading files inside large trees
- Assuming files are identical based on name alone without comparing content
Variations
- Use filecmp.cmp() for a more memory-efficient byte comparison
- Compare file metadata (size, mtime) first for a fast pre-filter before byte comparison
Real-world use cases
- Deploying to production: verifying a release artifact tree matches the staging environment exactly.
- Synchronizing assets between cloud storage buckets by identifying what needs uploading or deleting.
- Validating that a backup copy on removable media contains all current project files without corruption.
Sponsored
More from Files & data
- Append a Line to a Log File in Python easy
- Audit File Permissions Across a Project in Python easy
- Automatically Detect Corrupted Files Using SHA-256 Checksums in Python easy
- Automatically Highlight Data Validation Errors Inside Excel Files in Python easy
- Build a Command-Line To-Do List Application with Data Persistence in Python easy
- Build a File Index by Relative Path Hash Map in Python easy
Keep learning
Related tutorials and quizzes for this topic.