A/B testing & experimentation
User bucketing, experiment metrics, statistical comparison, and rollout guardrails.
Difference in Differences Mock in Python
Generate mock panel data with a known treatment effect and compute a difference-in-differences estimate using group and period means.
import numpy as np
import pandas as pd
# Generate mock panel data: 2 groups (control=0, treatment=1) × 2 periods (pre=0, post=1)
rng = np.random.default_rng(42)
n_per_cell = 50
data = []
for group in [0, 1]:
for period in [0, 1]:
# True effect: treatment increases outcome by 5 in the post period
…Generate a Mock Multi-Armed Bandit Report in Python
Simulate a multi-armed bandit experiment with random pulls and rewards, then output a JSON report with per-arm statistics.
import random
import json
def generate_mock_bandit_report(num_arms=5, num_rounds=100, seed=42):
random.seed(seed)
arms = ["A", "B", "C", "D", "E"][:num_arms]
true_means = {arm: random.uniform(0.3, 0.7) for arm in arms}
pulls = {arm: 0 for arm in arms}
rewards = {arm: 0 for arm in arms}
for _ …How to Mock a Confidence Interval for a Proportion in Python
Simulate a Bernoulli sample and compute a 95% confidence interval for a proportion using the normal approximation in Python.
import random
import math
def mock_ci(n=100, p_true=0.5, z=1.96, seed=42):
"""Simulate a sample proportion and compute its 95% confidence interval."""
random.seed(seed)
successes = sum(1 for _ in range(n) if random.random() < p_true)
p_hat = successes / n
se = math.sqrt(p_hat * (1 - p_hat) / n)
…How to Simulate Fixed-Horizon Testing in Python
Simulate a fixed-horizon experiment by labeling data before the horizon as warmup and after as active/inactive, then summarize via CSV.
import csv
import io
def fixed_horizon_mock(data: list[tuple[float, float, float]], horizon: int) -> str:
"""Simulate fixed-horizon testing, then summarize with CSV output."""
output = io.StringIO()
writer = csv.writer(output)
writer.writerow(["day", "value", "signal", "status"])
for day, value,…Simulate a Ramp Rollout Percentage in Python
Simulates a percentage-based ramp rollout with deterministic seeding, returning success/failure/in-progress counts for a mock user population.
import random
from enum import Enum
class RolloutStatus(Enum):
SUCCESS = "success"
FAILED = "failed"
IN_PROGRESS = "in_progress"
def simulate_ramp_rollout(total_users: int, percentage: int, seed: int = 42) -> dict:
"""
Simulates a mock ramp rollout for a given percentage of users.
Returns sta…Browse by section
Each section groups closely related Python snippets.
A/B testing & experimentation — Python code examples
What you will find here
This page collects a/b testing & experimentation snippets — short, copy-ready Python you can paste into our free online IDE and run without installing anything. Each sample includes a plain-English explanation and the full source code.
Samples vs tutorials and challenges
Samples are quick reference — one concept per page. For step-by-step teaching, use our Python tutorials. To test yourself, try quizzes or coding challenges. Clean up style with the Python formatter.