How to Watch a Directory for New Files in Python
Poll a directory at regular intervals and detect newly added files, printing each one as it appears.
Python code
37 linesimport time
import os
from pathlib import Path
WATCH_DIR = Path("watched_files")
def watch_for_new_files(directory: Path, sleep_time: float = 1.0, max_iterations: int = 10):
"""Poll a directory for new files and print when one appears."""
directory.mkdir(exist_ok=True)
existing = set(os.listdir(directory))
print(f"Watching {directory} for new files...")
for iteration in range(max_iterations):
time.sleep(sleep_time)
current = set(os.listdir(directory))
new_files = current - existing
if new_files:
for file in sorted(new_files):
print(f"New file detected: {file}")
existing = current
else:
print(f"Check {iteration + 1}: no new files yet")
print("Finished watching.")
if __name__ == "__main__":
# Create a demo file after 2 seconds to show detection
import threading
def add_demo_file():
time.sleep(2)
(WATCH_DIR / "demo.txt").write_text("Hello from demo!")
thread = threading.Thread(target=add_demo_file, daemon=True)
thread.start()
watch_for_new_files(WATCH_DIR, sleep_time=0.5, max_iterations=10)
Output
Watching watched_files for new files...
Check 1: no new files yet
Check 2: no new files yet
Check 3: no new files yet
Check 4: no new files yet
New file detected: demo.txt
Check 6: no new files yet
Check 7: no new files yet
Check 8: no new files yet
Check 9: no new files yet
Check 10: no new files yet
Finished watching.
How it works
The script uses os.listdir() to snapshot the directory contents before the polling loop starts. Each iteration compares the current set of files against the existing set, and any difference represents new files. Using set operations makes the comparison fast and correct, even if files are added in between checks. The Path.mkdir(exist_ok=True) call ensures the target directory exists before watching, avoiding a crash. This polling approach is simple and reliable for scripts where filesystem events aren't available or are overkill.
Common mistakes
- Forgetting to call `set()` on the list from `os.listdir()` — without it, membership checks are slower and duplicates can be missed.
- Overwriting `existing` outside the new-file block, causing the same file to be reported repeatedly.
- Using a very short sleep interval, which burns CPU and may raise I/O load on busy directories.
- Assuming files appear atomically — some writers create temporary files first, so consider filtering by suffix.
Variations
- Use `Path.iterdir()` instead of `os.listdir()` and filter for files with `.is_file()`.
- Use the `watchdog` library for event-driven detection instead of polling.
Real-world use cases
- Monitoring an upload folder to trigger processing when users drop new files via SFTP or a web dashboard.
- Watching a log directory in a microservice to display new entries in real time.
- Implementing a basic file watcher for a development tool that auto-reloads on config changes.
Sponsored
More from Files & data
- Append a Line to a Log File in Python easy
- Audit File Permissions Across a Project in Python easy
- Automatically Detect Corrupted Files Using SHA-256 Checksums in Python easy
- Automatically Highlight Data Validation Errors Inside Excel Files in Python easy
- Build a Command-Line To-Do List Application with Data Persistence in Python easy
- Build a File Index by Relative Path Hash Map in Python easy
Keep learning
Related tutorials and quizzes for this topic.