Reference library

A/B testing & experimentation

User bucketing, experiment metrics, statistical comparison, and rollout guardrails.

5 matches

Generate a Mock Multi-Armed Bandit Report in Python

Simulate a multi-armed bandit experiment with random pulls and rewards, then output a JSON report with per-arm statistics.

bandit simulation random

Python

import random
import json

def generate_mock_bandit_report(num_arms=5, num_rounds=100, seed=42):
    random.seed(seed)
    arms = ["A", "B", "C", "D", "E"][:num_arms]
    true_means = {arm: random.uniform(0.3, 0.7) for arm in arms}
    pulls = {arm: 0 for arm in arms}
    rewards = {arm: 0 for arm in arms}

    for _ …

16 0 Open

A/B testing & experimentation easy

How to Do Random Assignment in Python for A/B Tests

Assign each item to a binary group (0 or 1) with uniform probability using a small reusable function, optionally weighted, for A/B testing mocks.

random ab-testing assignment

Python

import random

def random_assignment_uniform_mock(items, weights=None):
    """Assign each item to a group (0 or 1) with uniform probability."""
    if weights is None:
        # Default: each item independently gets 0 or 1 with 50% probability
        return [random.randint(0, 1) for _ in items]
    # Optional weight…

13 0 Open

A/B testing & experimentation easy

How to Mock Stratified Assignment by Segment in Python

Simulate stratified assignment for A/B experiments by sampling a fixed proportion of units from each segment, with deterministic seeds for reproducibility.

ab-testing sampling random

Python

import random

def stratified_assignment(segments, seed=None):
    """
    Mock stratified assignment: given a dict of segment -> population size,
    return a dict of segment -> sampled unit ids (deterministic with seed).
    """
    if seed is not None:
        random.seed(seed)
    rng = random.Random(seed)
    res…

12 0 Open

A/B testing & experimentation easy

How to create a global control holdout group in Python

This code implements a deterministic global control holdout group, randomly selecting a fraction of users to be excluded from feature rollouts for experiment validation.

ab-testing holdout global-control

Python

import random

class GlobalControl:
    def __init__(self, population_size, holdout_fraction=0.2, seed=42):
        random.seed(seed)
        self.population_size = population_size
        self.holdout_fraction = holdout_fraction
        self.holdout_size = int(population_size * holdout_fraction)
        self.holdout_…

11 0 Open

A/B testing & experimentation easy

Simulate a Ramp Rollout Percentage in Python

Simulates a percentage-based ramp rollout with deterministic seeding, returning success/failure/in-progress counts for a mock user population.

rollout simulation random

Python

import random
from enum import Enum

class RolloutStatus(Enum):
    SUCCESS = "success"
    FAILED = "failed"
    IN_PROGRESS = "in_progress"

def simulate_ramp_rollout(total_users: int, percentage: int, seed: int = 42) -> dict:
    """
    Simulates a mock ramp rollout for a given percentage of users.
    Returns sta…

14 0 Open

Browse by section

Each section groups closely related Python snippets.