A/B testing & experimentation
User bucketing, experiment metrics, statistical comparison, and rollout guardrails.
Difference in Differences Mock in Python
Generate mock panel data with a known treatment effect and compute a difference-in-differences estimate using group and period means.
import numpy as np
import pandas as pd
# Generate mock panel data: 2 groups (control=0, treatment=1) × 2 periods (pre=0, post=1)
rng = np.random.default_rng(42)
n_per_cell = 50
data = []
for group in [0, 1]:
for period in [0, 1]:
# True effect: treatment increases outcome by 5 in the post period
…How to Calculate Secondary Metrics in Python
Computes distribution, variability, and spread of a numeric dataset using Python's statistics and collections modules.
import random
import statistics
from collections import Counter
def explore_secondary_metrics(data):
"""Calculate secondary metrics: distribution, variability, and spread."""
if not data:
return "No data provided"
total = sum(data)
mean = statistics.mean(data)
median = statistics.medi…How to Define a Mock Primary Metric in Python
Define a mock primary metric object with a name, value, and unit, and serialize it to a dictionary for experimentation and testing.
class Metric:
def __init__(self, name, value, unit=None):
self.name = name
self.value = value
self.unit = unit
def to_dict(self):
result = {"name": self.name, "value": self.value}
if self.unit:
result["unit"] = self.unit
return result
def __repr…How to Generate Multivariate JSON Mock Data in Python
This script generates mock multivariate JSON-compatible data with measurements and boolean flags for testing and experimentation pipelines.
import json
def multivariate_mock(row_count: int = 3) -> list:
"""Generate mock multivariate data as list of JSON-compatible dicts."""
records = []
for i in range(row_count):
record = {
"id": i + 1,
"measurements": {
"temperature": 20.5 + i * 1.5,
…How to Mock a Remote Config Fetch in Python
Simulate a remote config API response with metadata, timestamps, and mock data for testing or local development.
import json
from datetime import datetime
from typing import Any, Dict
def fetch_remote_config(mock_data: Dict[str, Any]) -> Dict[str, Any]:
"""Simulate fetching a remote config with metadata and timestamps."""
return {
"status": "success",
"source": "mock",
"fetched_at": datetime.utcn…How to Simulate Fixed-Horizon Testing in Python
Simulate a fixed-horizon experiment by labeling data before the horizon as warmup and after as active/inactive, then summarize via CSV.
import csv
import io
def fixed_horizon_mock(data: list[tuple[float, float, float]], horizon: int) -> str:
"""Simulate fixed-horizon testing, then summarize with CSV output."""
output = io.StringIO()
writer = csv.writer(output)
writer.writerow(["day", "value", "signal", "status"])
for day, value,…Browse by section
Each section groups closely related Python snippets.
A/B testing & experimentation — Python code examples
What you will find here
This page collects a/b testing & experimentation snippets — short, copy-ready Python you can paste into our free online IDE and run without installing anything. Each sample includes a plain-English explanation and the full source code.
Samples vs tutorials and challenges
Samples are quick reference — one concept per page. For step-by-step teaching, use our Python tutorials. To test yourself, try quizzes or coding challenges. Clean up style with the Python formatter.