A/B testing & experimentation
User bucketing, experiment metrics, statistical comparison, and rollout guardrails.
Bootstrap Confidence Interval in Python
Estimates a confidence interval for a statistic (like the mean) using bootstrap resampling in pure Python.
import random
def bootstrap_ci(data, statistic, n_bootstraps=1000, ci_level=0.95, seed=42):
random.seed(seed)
n = len(data)
boot_stats = []
for _ in range(n_bootstraps):
sample = [random.choice(data) for _ in range(n)]
boot_stats.append(statistic(sample))
boot_stats.sort()
l…Check Sample Ratio Mismatch in Python
Estimates the probability that a simple random sample's proportion differs from the population proportion by more than 10% using simulation.
import random
def sample_ratio_mismatch(population_size: int, sample_size: int, p: float) -> float:
"""
Estimate the probability that a simple random sample's proportion
differs from the population proportion by more than 10%.
"""
total_counts = [0, 0]
for _ in range(10000):
sample = …How to Mock Stratified Assignment by Segment in Python
Simulate stratified assignment for A/B experiments by sampling a fixed proportion of units from each segment, with deterministic seeds for reproducibility.
import random
def stratified_assignment(segments, seed=None):
"""
Mock stratified assignment: given a dict of segment -> population size,
return a dict of segment -> sampled unit ids (deterministic with seed).
"""
if seed is not None:
random.seed(seed)
rng = random.Random(seed)
res…Thompson Sampling Mock Bandit in Python
Implement a Thompson sampling multi-armed bandit to explore and exploit reward probabilities across multiple options, updating Beta distributions over time.
import random
class ThompsonSamplingBandit:
def __init__(self, num_arms, alpha=1.0, beta=1.0):
self.num_arms = num_arms
self.alpha = [alpha] * num_arms
self.beta = [beta] * num_arms
def select_arm(self):
samples = [random.betavariate(a, b) for a, b in zip(self.alpha, self.beta…Browse by section
Each section groups closely related Python snippets.
A/B testing & experimentation — Python code examples
What you will find here
This page collects a/b testing & experimentation snippets — short, copy-ready Python you can paste into our free online IDE and run without installing anything. Each sample includes a plain-English explanation and the full source code.
Samples vs tutorials and challenges
Samples are quick reference — one concept per page. For step-by-step teaching, use our Python tutorials. To test yourself, try quizzes or coding challenges. Clean up style with the Python formatter.