How to Monitor Website Content Changes in Python
This script fetches a webpage's content, computes its SHA-256 hash, and compares it with the last stored hash to detect and alert on changes.
pip install requests
Python code
27 linesimport time
import hashlib
import requests
from pathlib import Path
def fetch_content_hash(url: str) -> str:
response = requests.get(url, timeout=10)
response.raise_for_status()
return hashlib.sha256(response.text.encode()).hexdigest()
def monitor_website(url: str, check_interval: int = 60):
hash_file = Path("last_hash.txt")
if hash_file.exists():
last_hash = hash_file.read_text().strip()
else:
last_hash = ""
current_hash = fetch_content_hash(url)
if current_hash != last_hash:
print(f"ALERT: Content changed on {url}")
hash_file.write_text(current_hash)
else:
print(f"No changes detected on {url}")
if __name__ == "__main__":
target_url = "https://example.com"
print("Starting website monitor (one check)...")
monitor_website(target_url, check_interval=60)
Output
Starting website monitor (one check)...
No changes detected on https://example.com
ALERT: Content changed on https://example.com
How it works
The script uses requests.get() to download the page content and hashlib.sha256 to compute a content hash. It persists the last hash in a local file last_hash.txt, allowing detection of changes across runs. On each check, it compares the current hash to the stored one and prints an alert if they differ. This approach is efficient because hashing the full page text reduces storage and comparison overhead.
Common mistakes
- Not including `raise_for_status()` which can silently miss HTTP errors and return a wrong hash.
- Forgetting to handle the case where the hash file doesn't exist on the first run.
- Using a weak hash like MD5 which could produce collisions in adversarial scenarios.
- Setting a very short check interval that may overwhelm the server or get your IP blocked.
Variations
- Use `time.sleep(check_interval)` in a `while True` loop to run checks continuously.
- Instead of hashing the whole page, hash specific elements (e.g., a known dynamic section) for more targeted monitoring.
Real-world use cases
- Monitor a competitor's pricing page and trigger an internal alert when prices are updated.
- Watch a job board or government website for new postings and notify a Slack channel.
- Track documentation or policy pages for changes that require a compliance review.
Sponsored
More from Automation & scripting
- Batch Rename Hundreds of Files in Python easy
- Build a Command-Line Password Generator in Python easy
- Build a Complete Web Scraper with Requests and BeautifulSoup in Python medium
- Build a Network Ping Monitor in Python medium
- Create a Local Search Engine to Instantly Find Files on Your Computer in Python medium
- Create a Simple HTTP File Server in Python easy
Keep learning
Related tutorials and quizzes for this topic.