Reference library

Python Code Samples

Medium snippets you can copy, study, and run in the browser editor.

33 matches
Big data & Spark medium

Approximate Distinct Count in Python with HyperLogLog

Mock a large data stream and estimate the number of distinct items with a HyperLogLog-style probabilistic counter to save memory.

hyperloglog distinct-count probabilistic
Python
import random
import string
from collections import Counter
import math

class ApproxCountDistinct:
    def __init__(self, num_buckets=16):
        self.num_buckets = num_buckets
        self.max_zeros = [0] * num_buckets
        
    def _hash(self, item):
        # Simple string hash to a 32-bit integer
        h = …
15 0 Open
Big data & Spark medium

How to Implement a Streaming Watermark in Python

Mock structured streaming watermarks in Python to track late event times and compute a watermark for windowed processing.

streaming watermark spark
Python
from datetime import datetime, timedelta
import time

class StreamingWatermark:
    """Mock watermark tracker for structured streaming."""

    def __init__(self, watermark_delay_seconds):
        self.watermark_delay = timedelta(seconds=watermark_delay_seconds)
        self.max_event_time = None

    def observe_even…
14 0 Open
Big data & Spark medium

How to Mock Spark Streaming Micro-Batches in Python

Simulate Spark's micro-batch streaming with a simple deque-based class that collects events over time and processes them in timed batches.

spark streaming micro-batch
Python
import time
from collections import deque
from datetime import datetime


class MicroBatchStream:
    def __init__(self, batch_interval_sec=2):
        self.batch_interval = batch_interval_sec
        self.source = deque()
        self.processed = []

    def add_events(self, events):
        self.source.extend(events…
13 0 Open
Big data & Spark medium

How to Mock and Test a Rate-Limited Source Stream in Python

Build a class that rate-limits emitted items using a sliding window and test it with a simulated stream in Python.

rate-limiting mock-testing streaming
Python
import time
from collections import deque


class RateLimitedSource:
    def __init__(self, max_rate, window=1.0):
        self.max_rate = max_rate
        self.window = window
        self._timestamps = deque()

    def emit(self, item):
        now = time.monotonic()
        while self._timestamps and self._timestam…
16 0 Open
Big data & Spark medium

How to implement a tumbling window aggregation in Python

Build a mock tumbling window aggregator in Python that groups streaming events into fixed time intervals and computes count, sum, and average per window.

tumbling-window streaming aggregation
Python
import time
from collections import deque

class TumblingWindow:
    def __init__(self, duration_seconds):
        self.duration = duration_seconds
        self.buffer = deque()
        self.window_start = None

    def add(self, item):
        current_time = time.time()
        if self.window_start is None:
         …
13 0 Open
Big data & Spark medium

How to use foreachBatch with a mock sink in PySpark

Demonstrates using Spark Structured Streaming's foreachBatch sink to capture and verify streaming batches by writing them into a custom mock sink object.

pyspark structured-streaming foreachbatch
Python
from pyspark.sql import SparkSession
from pyspark.sql.functions import col, lit

class MockSink:
    def __init__(self):
        self.batches = []
    
    def write_batch(self, batch_df, batch_id):
        # Collect batch data as list of dicts for verification
        records = batch_df.collect()
        self.batches…
14 0 Open
A/B testing & experimentation medium

Benjamini Hochberg FDR Correction in Python

Implement the Benjamini-HHochberg false discovery rate (FDR) procedure in Python to control the expected proportion of false positives among rejected hypotheses.

fdr multiple testing hypothesis testing
Python
import numpy as np

def benjamini_hochberg(p_values, alpha=0.05):
    p_values = np.array(p_values)
    n = len(p_values)
    sorted_idx = np.argsort(p_values)
    sorted_p = p_values[sorted_idx]
    
    thresholds = (np.arange(1, n + 1) / n) * alpha
    significant = sorted_p <= thresholds
    
    if not significan…
15 0 Open
Auth & security at scale medium

How to Implement a Vault Dynamic Database Credentials Mock in Python

A Python dataclass-based mock of HashiCorp Vault that issues short-lived database credentials, tracks leases, and revokes them, demonstrating dynamic secrets rotation.

vault secrets database
Python
import time
import json
from dataclasses import dataclass, field
from typing import Dict


@dataclass
class DynamicCredential:
    username: str
    password: str
    lease_duration: int
    created_at: float = field(default_factory=time.time)

    def is_valid(self) -> bool:
        return time.time() - self.created_…
16 0 Open
Production deployment patterns medium

How to Expand a Contract and Migrate Data in Python

Expand an old data contract by renaming fields and adding defaults, then migrate to a final version with deepcopy isolation.

contract migration deepcopy
Python
import json
from copy import deepcopy

# Mock data representing a user record (old contract)
old_contract = {
    "id": 1,
    "name": "Alice",
    "email": "alice@example.com",
    "age": 30,
    "status": "active"
}

# Expanded contract: adds fields with defaults and renames some fields
expand_rules = {
    "id": "u…
14 0 Open

Browse by section

Each section groups closely related Python snippets.

Guide: free Python code samples library

Copy-ready Python snippets for learners and developers

PythonSkillset code samples are short, focused examples organised by topic and difficulty. Every snippet is server-rendered HTML — readable by search engines and easy to copy. Open any sample, read the notes, copy the code, then press Try in editor to run it in the browser with Pyodide.

How to use this library

  1. Pick a topic section — strings, lists, files, functions, and more
  2. Open a sample, read How it works, and copy the code block
  3. Run it in the IDE, tweak values, then take a related quiz or tutorial lesson

Samples vs tutorials and challenges

Samples are quick reference — one concept per page. For step-by-step teaching, use our Python tutorials. To test yourself, try quizzes or coding challenges. Clean up style with the Python formatter.