A/B testing & experimentation
User bucketing, experiment metrics, statistical comparison, and rollout guardrails.
How to Do Random Assignment in Python for A/B Tests
Assign each item to a binary group (0 or 1) with uniform probability using a small reusable function, optionally weighted, for A/B testing mocks.
import random
def random_assignment_uniform_mock(items, weights=None):
"""Assign each item to a group (0 or 1) with uniform probability."""
if weights is None:
# Default: each item independently gets 0 or 1 with 50% probability
return [random.randint(0, 1) for _ in items]
# Optional weight…How to Mock Stratified Assignment by Segment in Python
Simulate stratified assignment for A/B experiments by sampling a fixed proportion of units from each segment, with deterministic seeds for reproducibility.
import random
def stratified_assignment(segments, seed=None):
"""
Mock stratified assignment: given a dict of segment -> population size,
return a dict of segment -> sampled unit ids (deterministic with seed).
"""
if seed is not None:
random.seed(seed)
rng = random.Random(seed)
res…How to Perform Intent-to-Treat Analysis in Python
Runs an intent-to-treat analysis on mock A/B test data, comparing outcomes by initial group assignment with a t-test for significance.
import pandas as pd
import numpy as np
def intent_to_treat_analysis(data):
"""Perform intent-to-treat (ITT) analysis.
ITT compares outcomes based on initial treatment assignment,
regardless of whether participants actually received the treatment.
"""
# Create a copy to avoid mutating the origina…How to hash user IDs to experiment buckets in Python
Deterministically map a user ID to an experiment bucket using MD5 hashing, ensuring stable and consistent assignment for A/B testing.
import hashlib
def hash_user_to_bucket(user_id: str, num_buckets: int = 10) -> int:
"""Deterministically map a user ID to an experiment bucket (0..num_buckets-1)."""
digest = hashlib.md5(user_id.encode("utf-8")).hexdigest()
return int(digest, 16) % num_buckets
if __name__ == "__main__":
mock_users …How to join assignment logs with outcomes in Python
Merge submission log entries with grading outcomes using left join and full outer join patterns in pure Python.
from datetime import datetime, timedelta
class AssignmentLog:
def __init__(self):
self.logs = [
{"assignment_id": 101, "student_id": "S001", "submitted_at": "2024-03-01 10:30:00"},
{"assignment_id": 101, "student_id": "S002", "submitted_at": "2024-03-02 14:15:00"},
{"as…Browse by section
Each section groups closely related Python snippets.
A/B testing & experimentation — Python code examples
What you will find here
This page collects a/b testing & experimentation snippets — short, copy-ready Python you can paste into our free online IDE and run without installing anything. Each sample includes a plain-English explanation and the full source code.
Samples vs tutorials and challenges
Samples are quick reference — one concept per page. For step-by-step teaching, use our Python tutorials. To test yourself, try quizzes or coding challenges. Clean up style with the Python formatter.