A/B testing & experimentation
User bucketing, experiment metrics, statistical comparison, and rollout guardrails.
How to Compute CUPED Variance Reduction in Python
Implement CUPED in Python to reduce variance of A/B test treatment effect estimates using pre-experiment covariates.
import numpy as np
def compute_cuped_reduction(control, variant, covariate):
"""
Compute variance reduction using CUPED (Controlled Experiment with
Pre-Experiment Data). Uses pre-experiment covariate values to
reduce variance of the treatment effect estimate.
"""
control = np.asarray(control, …How to Generate an Orthogonal Array for A/B Testing in Python
Generate a mock orthogonal array for multi-layer experiments with NumPy, ensuring balanced level combinations across experiment groups.
import numpy as np
def orthogonal_mock_layers(n_experiments: int, n_layers: int, n_levels: int) -> np.ndarray:
"""Generate an orthogonal array for multi-layer experiment design using base-level logic."""
ortho = np.indices((n_levels,) * n_layers).reshape(n_layers, -1).T
ortho = ortho % n_levels # Classic…How to Simulate Geo Experiments in Python
Build a mock geo experiment simulator with ramp-up/down periods, measuring weekly lift between treatment and control markets.
import random
import math
from dataclasses import dataclass
@dataclass
class GeoMarket:
name: str
base_demand: float
geo_coefficient: float
def simulate_geo_experiment(markets, weeks=12, control_weeks=6):
"""
Simulates a geo experiment with ramp-up and ramp-down periods.
Returns weekly lift p…Browse by section
Each section groups closely related Python snippets.
A/B testing & experimentation — Python code examples
What you will find here
This page collects a/b testing & experimentation snippets — short, copy-ready Python you can paste into our free online IDE and run without installing anything. Each sample includes a plain-English explanation and the full source code.
Samples vs tutorials and challenges
Samples are quick reference — one concept per page. For step-by-step teaching, use our Python tutorials. To test yourself, try quizzes or coding challenges. Clean up style with the Python formatter.