Python Code
Samples
Copy-ready Python snippets by topic and difficulty — short, focused, and runnable in the browser editor.
Deduplicate events by ID within a window in Python
Deduplicate event streams by ID within sliding time windows, keeping the newest occurrence per window using heaps and sets.
import heapq
from collections import defaultdict
def deduplicate_events(events, window_size):
"""Return events deduplicated by id, keeping newest within each sliding window."""
# Index events by (timestamp, id) for deterministic ordering
events_by_id = defaultdict(list)
for ts, eid, *payload in events…Generate a Deterministic Hash for Deduplication in Python
Create a stable SHA-256 fingerprint from nested data and file contents to deduplicate records in a data pipeline.
import hashlib
import json
from pathlib import Path
def natural_key_hash(data, salt=""):
"""
Generate a deterministic fingerprint from raw data (dict/list/str).
Uses JSON canonical-ish serialization with sorted keys and SHA-256.
"""
canonical = json.dumps(data, sort_keys=True, separators=(",", ":"…How to Deduplicate Events with At-Least-Once Delivery in Python
Implements an exactly-once processing pattern for at-least-once event delivery by tracking seen event IDs in a set, skipping duplicates.
seen_ids = set()
def process_event(event_id: str, payload: dict) -> dict:
"""Process an event exactly once, ignoring duplicates."""
if event_id in seen_ids:
return {"status": "duplicate", "event_id": event_id}
seen_ids.add(event_id)
return {"status": "processed", "event_id": event_id, **payloa…How to Validate Fact Table Grain Row Counts in Python
Validate fact table grain by checking dimension key references, unique grain combinations, duplicate rows, and dimension cardinality from a CSV file.
import csv
import hashlib
from pathlib import Path
def validate_fact_grain(fact_file: Path, expected_dim_keys: dict[str, set[str]]) -> dict:
"""
Validate fact table grain by checking each row's dimension keys exist
in expected dimension tables and row count consistency.
"""
dim_references = {}
…Implement Exactly-Once Transaction Log in Python
A mock transaction log that deduplicates transaction IDs so each is recorded only once, with a dataclass for records and simple in-memory storage.
from dataclasses import dataclass
from typing import Dict, Optional
@dataclass
class TxnRecord:
txn_id: str
status: str
class ExactlyOnceTxnLog:
def __init__(self) -> None:
self._log: Dict[str, TxnRecord] = {}
self._processed_ids: set = set()
def record(self, txn_id: str, status: s…Count Unique Contributors from Git Shortlog in Python
Parses git shortlog -sn output to count the number of unique contributors, handling duplicate entries and variable whitespace.
import subprocess
from collections import Counter
# Mock shortlog output as a list of lines (simulating git shortlog -sn output)
MOCK_SHORTLOG = """ 120 Alice Johnson
88 Bob Smith
45 Alice Johnson
30 Carol Williams
25 Bob Smith
10 Dave Brown
"""
def count_contributors_from_shortlog(text):
"…How to Mock CloudFront Invalidation Paths in Python
Build a sorted, deduplicated list of CloudFront invalidation paths from a set of file paths, adding implicit index.html entries.
import argparse
def build_invalidation_paths(files, include_index=True):
"""
Create CloudFront invalidation paths from a list of files.
Converts file names to root-relative paths and optionally adds /index.html.
"""
paths = []
for f in files:
f = f.strip()
if not f:
…Idempotent Consumer: Store Processed IDs in Python
Implement an idempotent consumer that persists processed message IDs to a JSON file, skipping duplicates on restart.
import json
from pathlib import Path
class IdempotentStore:
def __init__(self, storage_path: str = "processed_ids.json"):
self.storage_path = Path(storage_path)
self.processed_ids = self._load()
def _load(self) -> set:
if self.storage_path.exists():
with self.storage_path…Inbox pattern consumer dedupe mock in Python
Implements a mock inbox consumer that deduplicates incoming messages by ID, with automatic eviction of old seen IDs to prevent unbounded memory growth.
import json
from collections import deque
from dataclasses import dataclass, field
from hashlib import sha256
from typing import Any
@dataclass
class InboxConsumer:
max_seen: int = 1000
seen_ids: set = field(default_factory=set)
seen_history: deque = field(default_factory=deque)
def _mark_seen(self,…How to Build an Idempotency-Key POST Handler in Python
Python HTTP server mock that accepts POST requests and deduplicates them using an Idempotency-Key header, returning the same response for repeated calls.
import hashlib
import json
from http.server import BaseHTTPRequestHandler, HTTPServer
from urllib.parse import urlparse
class MockAPI(BaseHTTPRequestHandler):
responses = {}
def do_POST(self):
length = int(self.headers.get("Content-Length", 0))
body = self.rfile.read(length).decode("utf-8")
…Exactly Once Idempotent Consumer Store in Python
A mock key-value store that guarantees exactly-once processing by rejecting duplicate message keys in a message or event stream.
from collections import defaultdict
class ExactlyOnceStore:
def __init__(self):
self.processed = defaultdict(set)
self.data = {}
def consume(self, key, value):
if key in self.data:
return False
self.data[key] = value
return True
def get_processed_count…How to deduplicate messages by ID in Python
Track seen message IDs in a set to skip duplicate messages and store unique content in a dict, with exact output showing which messages were added or skipped.
import time
class MessageStore:
def __init__(self):
self.seen_ids = set()
self.messages = {}
def add(self, message_id, content, timestamp=None):
timestamp = timestamp or time.time()
if message_id in self.seen_ids:
return False
self.seen_ids.add(message_…Cache Stampede Prevention with SingleFlight in Python
Implements a SingleFlight pattern in Python to deduplicate concurrent cache-miss computations and prevent cache stampede.
import threading
import time
from functools import wraps
class SingleFlight:
def __init__(self):
self._lock = threading.Lock()
self._inflight = None
def do(self, key, fn):
with self._lock:
if self._inflight is not None:
return self._inflight[1]
…Coalescing duplicate in-flight requests: one shared result for concurrent callers
Runs identical concurrent requests through a single shared call, caching the result while it's in flight and returning the same value to all callers.
import time
import threading
from collections import defaultdict
class CoalescingExecutor:
def __init__(self):
self._locks = defaultdict(threading.Lock)
self._in_flight = {}
def execute(self, key, func):
with self._locks[key]:
if key in self._in_flight:
re…At Least Once with Idempotent Consumer in Python
Implements a thread-safe idempotent consumer that processes each unique message exactly once, even when a producer sends duplicates under an at-least-once delivery model.
import threading
import time
import uuid
from collections import Counter
class IdempotentConsumer:
def __init__(self):
self.processed = set()
self._lock = threading.Lock()
def consume(self, message_id, payload):
with self._lock:
if message_id in self.processed:
…How to Deduplicate Messages in Python by ID
This code consumes a mock inbox of JSON messages and deduplicates them by message ID, keeping either the first or last occurrence.
import json
from collections import OrderedDict
mock_inbox = [
{"id": 1, "message": "hello", "timestamp": "2024-01-01T10:00:00Z"},
{"id": 2, "message": "world", "timestamp": "2024-01-01T10:01:00Z"},
{"id": 1, "message": "hello", "timestamp": "2024-01-01T10:00:00Z"},
{"id": 3, "message": "test", "times…How to Implement Hedged Requests in Python
This code demonstrates a hedged request pattern using threading, which sends duplicate calls and returns the first result that arrives within a timeout.
import time
from unittest.mock import Mock
def hedged_request(call, timeout=0.05):
"""Execute two duplicate calls, return first result within timeout."""
result_container = {}
def run_and_store():
result_container['result'] = call()
result_container['done'] = True
# Simulate slow cal…How to implement an idempotency key store in Python
Build an in-memory idempotency key store with TTL that processes a request once and reuses the cached result for duplicate calls.
import hashlib
import time
from typing import Dict, Optional
class IdempotencyStore:
"""Simple in-memory idempotency key store with mock processing."""
def __init__(self, ttl_seconds: int = 3600) -> None:
self.ttl = ttl_seconds
self._store: Dict[str, tuple[str, float]] = {}
def _is_expi…How to Deduplicate Events in Python with SHA256 Hashing
Build an event deduplicator that identifies duplicate inbox messages using SHA256 hashes and tracks duplicate counts per event type.
```python
import hashlib
import json
from collections import defaultdict
class EventDeduplicator:
def __init__(self):
self.seen_hashes = set()
self.duplicate_counts = defaultdict(int)
def process_event(self, event):
event_key = f"{event['event_id']}:{event['timestamp']}"
even…How to Implement an Exactly-Once Deduplication Store in Python
Implement a Python class that deduplicates keys exactly once, tracking first-seen timestamps and duplicate counts.
from datetime import datetime
from typing import Any, Hashable
class ExactlyOnceStore:
def __init__(self) -> None:
self._seen: set[Hashable] = set()
self._first_seen: dict[Hashable, datetime] = {}
self._counts: dict[Hashable, int] = {}
def add(self, key: Hashable, value: Any = None) …Idempotent Consumer Event Processing in Python
Track processed event IDs to skip duplicates and count event types for a reliable, idempotent consumer.
import json
from collections import defaultdict
class EventProcessor:
def __init__(self):
self.processed_ids = set()
self.counts = defaultdict(int)
def process_event(self, event):
event_id = event["id"]
if event_id in self.processed_ids:
return {"status": "skipped"…How to enforce a unique index constraint in Python
Mock a database unique index in Python that rejects duplicate rows based on one or more columns.
class MockIndex:
def __init__(self, columns):
self.columns = columns
self._values = set()
def insert(self, row):
key = tuple(row[col] for col in self.columns)
if key in self._values:
raise ValueError(f"Duplicate key {key} for columns {self.columns}")
self._v…Idempotent Writes for Sharded Databases in Python
Implement a mock shard with idempotent write support using request IDs to prevent duplicate writes and track the latest value per key.
import json
class ShardMock:
"""Mock distributed shard with idempotent write support."""
def __init__(self, shard_id):
self.shard_id = shard_id
self._store = {}
def write(self, key, value, request_id):
"""Write value only if request_id not yet processed; idempotent."""
i…Browse by section
Each section groups closely related Python snippets.
Guide: free Python code samples library
Copy-ready Python snippets for learners and developers
PythonSkillset code samples are short, focused examples organised by topic and difficulty. Every snippet is server-rendered HTML — readable by search engines and easy to copy. Open any sample, read the notes, copy the code, then press Try in editor to run it in the browser with Pyodide.
How to use this library
- Pick a topic section — strings, lists, files, functions, and more
- Open a sample, read How it works, and copy the code block
- Run it in the IDE, tweak values, then take a related quiz or tutorial lesson
Samples vs tutorials and challenges
Samples are quick reference — one concept per page. For step-by-step teaching, use our Python tutorials. To test yourself, try quizzes or coding challenges. Clean up style with the Python formatter.