A/B testing & experimentation
User bucketing, experiment metrics, statistical comparison, and rollout guardrails.
Benjamini Hochberg FDR Correction in Python
Implement the Benjamini-HHochberg false discovery rate (FDR) procedure in Python to control the expected proportion of false positives among rejected hypotheses.
import numpy as np
def benjamini_hochberg(p_values, alpha=0.05):
p_values = np.array(p_values)
n = len(p_values)
sorted_idx = np.argsort(p_values)
sorted_p = p_values[sorted_idx]
thresholds = (np.arange(1, n + 1) / n) * alpha
significant = sorted_p <= thresholds
if not significan…Check Sample Ratio Mismatch in Python
Estimates the probability that a simple random sample's proportion differs from the population proportion by more than 10% using simulation.
import random
def sample_ratio_mismatch(population_size: int, sample_size: int, p: float) -> float:
"""
Estimate the probability that a simple random sample's proportion
differs from the population proportion by more than 10%.
"""
total_counts = [0, 0]
for _ in range(10000):
sample = …Browse by section
Each section groups closely related Python snippets.
A/B testing & experimentation — Python code examples
What you will find here
This page collects a/b testing & experimentation snippets — short, copy-ready Python you can paste into our free online IDE and run without installing anything. Each sample includes a plain-English explanation and the full source code.
Samples vs tutorials and challenges
Samples are quick reference — one concept per page. For step-by-step teaching, use our Python tutorials. To test yourself, try quizzes or coding challenges. Clean up style with the Python formatter.