Files & data
Read and write files safely; parse JSON, CSV, and common text formats.
Find Duplicate Web Pages by Content Similarity in Python
Compute SHA-256 hashes of file contents to detect and report duplicate HTML pages or any files in a directory.
import hashlib
import os
from collections import defaultdict
def get_file_hash(filepath):
"""Compute SHA-256 hash of file contents."""
sha256 = hashlib.sha256()
with open(filepath, 'rb') as f:
for chunk in iter(lambda: f.read(4096), b''):
sha256.update(chunk)
return sha256.hexdiges…How to Find Duplicate Files by Size and Hash in Python
Recursively scan a directory, group files by size, then hash candidates to identify exact duplicate files.
import hashlib
from pathlib import Path
def hash_file(path, chunk_size=8192):
hasher = hashlib.md5()
with open(path, 'rb') as f:
while chunk := f.read(chunk_size):
hasher.update(chunk)
return hasher.hexdigest()
def find_duplicates(directory):
size_map = {}
for path in Path(dir…Browse by section
Each section groups closely related Python snippets.
Files & data — Python code examples
What you will find here
This page collects files & data snippets — short, copy-ready Python you can paste into our free online IDE and run without installing anything. Each sample includes a plain-English explanation and the full source code.
Samples vs tutorials and challenges
Samples are quick reference — one concept per page. For step-by-step teaching, use our Python tutorials. To test yourself, try quizzes or coding challenges. Clean up style with the Python formatter.