A/B testing & experimentation
User bucketing, experiment metrics, statistical comparison, and rollout guardrails.
How to Mock a Remote Config Fetch in Python
Simulate a remote config API response with metadata, timestamps, and mock data for testing or local development.
import json
from datetime import datetime
from typing import Any, Dict
def fetch_remote_config(mock_data: Dict[str, Any]) -> Dict[str, Any]:
"""Simulate fetching a remote config with metadata and timestamps."""
return {
"status": "success",
"source": "mock",
"fetched_at": datetime.utcn…How to Mock an Exposure Event Log Record in Python
Generate a realistic exposure event record with UUID, UTC timestamp, and risk level for testing or experimentation.
import uuid
from datetime import datetime, timezone
def mock_exposure_event(person_id: str, location: str, duration_minutes: int) -> dict:
return {
"event_id": str(uuid.uuid4()),
"person_id": person_id,
"location": location,
"duration_minutes": duration_minutes,
"timestamp…How to Perform Intent-to-Treat Analysis in Python
Runs an intent-to-treat analysis on mock A/B test data, comparing outcomes by initial group assignment with a t-test for significance.
import pandas as pd
import numpy as np
def intent_to_treat_analysis(data):
"""Perform intent-to-treat (ITT) analysis.
ITT compares outcomes based on initial treatment assignment,
regardless of whether participants actually received the treatment.
"""
# Create a copy to avoid mutating the origina…How to Run a Permutation Test in Python
Run a Monte Carlo permutation test to compute a p-value for comparing two group means without parametric assumptions.
import random
import statistics
def permutation_test(group_a, group_b, n_permutations=10000, seed=42):
random.seed(seed)
combined = group_a + group_b
observed_diff = abs(statistics.mean(group_a) - statistics.mean(group_b))
count = 0
n = len(group_a)
for _ in range(n_permutations):
…How to Simulate Fixed-Horizon Testing in Python
Simulate a fixed-horizon experiment by labeling data before the horizon as warmup and after as active/inactive, then summarize via CSV.
import csv
import io
def fixed_horizon_mock(data: list[tuple[float, float, float]], horizon: int) -> str:
"""Simulate fixed-horizon testing, then summarize with CSV output."""
output = io.StringIO()
writer = csv.writer(output)
writer.writerow(["day", "value", "signal", "status"])
for day, value,…How to Simulate Geo Experiments in Python
Build a mock geo experiment simulator with ramp-up/down periods, measuring weekly lift between treatment and control markets.
import random
import math
from dataclasses import dataclass
@dataclass
class GeoMarket:
name: str
base_demand: float
geo_coefficient: float
def simulate_geo_experiment(markets, weeks=12, control_weeks=6):
"""
Simulates a geo experiment with ramp-up and ramp-down periods.
Returns weekly lift p…How to create a global control holdout group in Python
This code implements a deterministic global control holdout group, randomly selecting a fraction of users to be excluded from feature rollouts for experiment validation.
import random
class GlobalControl:
def __init__(self, population_size, holdout_fraction=0.2, seed=42):
random.seed(seed)
self.population_size = population_size
self.holdout_fraction = holdout_fraction
self.holdout_size = int(population_size * holdout_fraction)
self.holdout_…How to join assignment logs with outcomes in Python
Merge submission log entries with grading outcomes using left join and full outer join patterns in pure Python.
from datetime import datetime, timedelta
class AssignmentLog:
def __init__(self):
self.logs = [
{"assignment_id": 101, "student_id": "S001", "submitted_at": "2024-03-01 10:30:00"},
{"assignment_id": 101, "student_id": "S002", "submitted_at": "2024-03-02 14:15:00"},
{"as…How to simulate a contextual bandit in Python
Simulate a contextual multi-armed bandit with random features and epsilon-greedy action selection in Python.
import random
class ContextualBandit:
def __init__(self, n_actions=3, n_features=4):
self.n_actions = n_actions
self.n_features = n_features
self.theta = [random.random() for _ in range(n_actions * n_features)]
def mock_context(self):
return [random.uniform(-1, 1) for _ in ra…Simulate a Ramp Rollout Percentage in Python
Simulates a percentage-based ramp rollout with deterministic seeding, returning success/failure/in-progress counts for a mock user population.
import random
from enum import Enum
class RolloutStatus(Enum):
SUCCESS = "success"
FAILED = "failed"
IN_PROGRESS = "in_progress"
def simulate_ramp_rollout(total_users: int, percentage: int, seed: int = 42) -> dict:
"""
Simulates a mock ramp rollout for a given percentage of users.
Returns sta…Synthetic Control in Python: Mock Example
Implements synthetic control from scratch: learns donor weights via ridge regression on pre-period data, then predicts a counterfactual for the treated unit.
import numpy as np
class SyntheticControl:
def __init__(self, data, treated_index, pre_periods, post_periods):
self.data = np.array(data, dtype=float)
self.treated_index = treated_index
self.pre_periods = pre_periods
self.post_periods = post_periods
def fit_weights(sel…Thompson Sampling Mock Bandit in Python
Implement a Thompson sampling multi-armed bandit to explore and exploit reward probabilities across multiple options, updating Beta distributions over time.
import random
class ThompsonSamplingBandit:
def __init__(self, num_arms, alpha=1.0, beta=1.0):
self.num_arms = num_arms
self.alpha = [alpha] * num_arms
self.beta = [beta] * num_arms
def select_arm(self):
samples = [random.betavariate(a, b) for a, b in zip(self.alpha, self.beta…UCB1 Bandit Algorithm in Python
This code implements the UCB1 multi-armed bandit algorithm, balancing exploration and exploitation to identify the best arm while maximizing cumulative reward.
import math
import random
def ucb1(means, n_iterations=1000, exploration_weight=2.0):
"""Run UCB1 bandit algorithm on arms with given true means."""
n_arms = len(means)
counts = [0] * n_arms
rewards = [0.0] * n_arms
for t in range(1, n_iterations + 1):
# UCB1 selection
if t <…Browse by section
Each section groups closely related Python snippets.
A/B testing & experimentation — Python code examples
What you will find here
This page collects a/b testing & experimentation snippets — short, copy-ready Python you can paste into our free online IDE and run without installing anything. Each sample includes a plain-English explanation and the full source code.
Samples vs tutorials and challenges
Samples are quick reference — one concept per page. For step-by-step teaching, use our Python tutorials. To test yourself, try quizzes or coding challenges. Clean up style with the Python formatter.