A/B testing & experimentation
User bucketing, experiment metrics, statistical comparison, and rollout guardrails.
Difference in Differences Mock in Python
Generate mock panel data with a known treatment effect and compute a difference-in-differences estimate using group and period means.
import numpy as np
import pandas as pd
# Generate mock panel data: 2 groups (control=0, treatment=1) × 2 periods (pre=0, post=1)
rng = np.random.default_rng(42)
n_per_cell = 50
data = []
for group in [0, 1]:
for period in [0, 1]:
# True effect: treatment increases outcome by 5 in the post period
…Generate a Mock Multi-Armed Bandit Report in Python
Simulate a multi-armed bandit experiment with random pulls and rewards, then output a JSON report with per-arm statistics.
import random
import json
def generate_mock_bandit_report(num_arms=5, num_rounds=100, seed=42):
random.seed(seed)
arms = ["A", "B", "C", "D", "E"][:num_arms]
true_means = {arm: random.uniform(0.3, 0.7) for arm in arms}
pulls = {arm: 0 for arm in arms}
rewards = {arm: 0 for arm in arms}
for _ …How to Build a Guardrail Metrics Monitor in Python
This code implements a mock monitor that records metric values, checks them against thresholds, and summarizes pass/alert statistics.
import random
import time
from collections import defaultdict
class GuardrailMetricsMonitor:
def __init__(self):
self.metrics = defaultdict(list)
self.thresholds = {
"prompt_toxicity": 0.8,
"response_length": 500,
"latency_ms": 1000,
}
def record(s…How to Build a Simple Binary Protocol Parser Mock in Python
Defines a mock binary protocol with field definitions, encoding, and decoding to simulate network packet parsing for A/B testing and experiment setup.
class SimpleProtocol:
def __init__(self, name, version):
self.name = name
self.version = version
self.fields = []
def add_field(self, field_name, field_size):
self.fields.append((field_name, field_size))
def parse(self, data):
if len(data) != sum(size for _, size i…How to Calculate Weighted Grades and Generate Mock Notes in Python
Compute a weighted physics grade from exam and homework scores, then generate a performance-based mock note with percentage and feedback.
def get_physics_grade(exam_score, homework_score):
"""Calculate final grade from exam and homework scores."""
exam_weight = 0.7
homework_weight = 0.3
return (exam_score * exam_weight) + (homework_score * homework_weight)
def mock_note(correct_score, max_score, student_name):
"""Generate a mock no…How to Create a Mock That Returns Inverse Counter Values in Python
Builds a Mock whose side_effect returns the inverse (1/count) of each Counter value, defaulting to 0.0 for unseen keys.
from collections import Counter
from unittest.mock import Mock
def inverse_mock(counter: Counter) -> Mock:
"""
Return a Mock that mimics the inverse of a Counter:
each key returns a value representing the inverse of its count.
The Mock's side_effect maps keys to their inverse counts.
"""
mock …How to Create a Sticky Consistent Mock with unittest.mock in Python
Shows how to use unittest.mock.patch.object to mock a method consistently across multiple calls, returning a sticky value every time.
from unittest.mock import patch
class Database:
def fetch(self, key):
return f"real value for {key}"
def get_value(db, key):
return db.fetch(key)
if __name__ == "__main__":
db = Database()
with patch.object(db, "fetch", return_value="sticky value") as mock_fetch:
result1 = get_value(…How to Define a Mock Primary Metric in Python
Define a mock primary metric object with a name, value, and unit, and serialize it to a dictionary for experimentation and testing.
class Metric:
def __init__(self, name, value, unit=None):
self.name = name
self.value = value
self.unit = unit
def to_dict(self):
result = {"name": self.name, "value": self.value}
if self.unit:
result["unit"] = self.unit
return result
def __repr…How to Do Random Assignment in Python for A/B Tests
Assign each item to a binary group (0 or 1) with uniform probability using a small reusable function, optionally weighted, for A/B testing mocks.
import random
def random_assignment_uniform_mock(items, weights=None):
"""Assign each item to a group (0 or 1) with uniform probability."""
if weights is None:
# Default: each item independently gets 0 or 1 with 50% probability
return [random.randint(0, 1) for _ in items]
# Optional weight…How to Generate Multivariate JSON Mock Data in Python
This script generates mock multivariate JSON-compatible data with measurements and boolean flags for testing and experimentation pipelines.
import json
def multivariate_mock(row_count: int = 3) -> list:
"""Generate mock multivariate data as list of JSON-compatible dicts."""
records = []
for i in range(row_count):
record = {
"id": i + 1,
"measurements": {
"temperature": 20.5 + i * 1.5,
…How to Mock Stratified Assignment by Segment in Python
Simulate stratified assignment for A/B experiments by sampling a fixed proportion of units from each segment, with deterministic seeds for reproducibility.
import random
def stratified_assignment(segments, seed=None):
"""
Mock stratified assignment: given a dict of segment -> population size,
return a dict of segment -> sampled unit ids (deterministic with seed).
"""
if seed is not None:
random.seed(seed)
rng = random.Random(seed)
res…How to Mock a Confidence Interval for a Proportion in Python
Simulate a Bernoulli sample and compute a 95% confidence interval for a proportion using the normal approximation in Python.
import random
import math
def mock_ci(n=100, p_true=0.5, z=1.96, seed=42):
"""Simulate a sample proportion and compute its 95% confidence interval."""
random.seed(seed)
successes = sum(1 for _ in range(n) if random.random() < p_true)
p_hat = successes / n
se = math.sqrt(p_hat * (1 - p_hat) / n)
…How to Mock a Remote Config Fetch in Python
Simulate a remote config API response with metadata, timestamps, and mock data for testing or local development.
import json
from datetime import datetime
from typing import Any, Dict
def fetch_remote_config(mock_data: Dict[str, Any]) -> Dict[str, Any]:
"""Simulate fetching a remote config with metadata and timestamps."""
return {
"status": "success",
"source": "mock",
"fetched_at": datetime.utcn…How to Mock an Exposure Event Log Record in Python
Generate a realistic exposure event record with UUID, UTC timestamp, and risk level for testing or experimentation.
import uuid
from datetime import datetime, timezone
def mock_exposure_event(person_id: str, location: str, duration_minutes: int) -> dict:
return {
"event_id": str(uuid.uuid4()),
"person_id": person_id,
"location": location,
"duration_minutes": duration_minutes,
"timestamp…How to Simulate Fixed-Horizon Testing in Python
Simulate a fixed-horizon experiment by labeling data before the horizon as warmup and after as active/inactive, then summarize via CSV.
import csv
import io
def fixed_horizon_mock(data: list[tuple[float, float, float]], horizon: int) -> str:
"""Simulate fixed-horizon testing, then summarize with CSV output."""
output = io.StringIO()
writer = csv.writer(output)
writer.writerow(["day", "value", "signal", "status"])
for day, value,…Simulate a Ramp Rollout Percentage in Python
Simulates a percentage-based ramp rollout with deterministic seeding, returning success/failure/in-progress counts for a mock user population.
import random
from enum import Enum
class RolloutStatus(Enum):
SUCCESS = "success"
FAILED = "failed"
IN_PROGRESS = "in_progress"
def simulate_ramp_rollout(total_users: int, percentage: int, seed: int = 42) -> dict:
"""
Simulates a mock ramp rollout for a given percentage of users.
Returns sta…Browse by section
Each section groups closely related Python snippets.
A/B testing & experimentation — Python code examples
What you will find here
This page collects a/b testing & experimentation snippets — short, copy-ready Python you can paste into our free online IDE and run without installing anything. Each sample includes a plain-English explanation and the full source code.
Samples vs tutorials and challenges
Samples are quick reference — one concept per page. For step-by-step teaching, use our Python tutorials. To test yourself, try quizzes or coding challenges. Clean up style with the Python formatter.