Reference library

A/B testing & experimentation

User bucketing, experiment metrics, statistical comparison, and rollout guardrails.

6 matches

How to Compute Mann-Whitney U Test in Python

Compute the Mann-Whitney U statistic and p-value manually in Python with tie correction and a normal approximation for independent samples.

statistics hypothesis-testing ab-testing

Python

import numpy as np
from scipy import stats

def mann_whitney_u_mock(sample_a, sample_b):
    """Compute Mann-Whitney U and p-value manually."""
    # Combine and rank
    combined = sample_a + sample_b
    n_a, n_b = len(sample_a), len(sample_b)
    n_total = n_a + n_b
    
    # Rank with ties handling (average ranks…

12 0 Open

A/B testing & experimentation easy

How to Create a Sticky Consistent Mock with unittest.mock in Python

Shows how to use unittest.mock.patch.object to mock a method consistently across multiple calls, returning a sticky value every time.

unittest mock testing

Python

from unittest.mock import patch

class Database:
    def fetch(self, key):
        return f"real value for {key}"

def get_value(db, key):
    return db.fetch(key)

if __name__ == "__main__":
    db = Database()
    with patch.object(db, "fetch", return_value="sticky value") as mock_fetch:
        result1 = get_value(…

15 0 Open

A/B testing & experimentation easy

How to Do Random Assignment in Python for A/B Tests

Assign each item to a binary group (0 or 1) with uniform probability using a small reusable function, optionally weighted, for A/B testing mocks.

random ab-testing assignment

Python

import random

def random_assignment_uniform_mock(items, weights=None):
    """Assign each item to a group (0 or 1) with uniform probability."""
    if weights is None:
        # Default: each item independently gets 0 or 1 with 50% probability
        return [random.randint(0, 1) for _ in items]
    # Optional weight…

13 0 Open

A/B testing & experimentation easy

How to Hash a User ID to an Experiment Bucket in Python

Deterministically map a user ID to one of N experiment buckets using MD5 hashing and modulo arithmetic.

hashing ab-testing bucketing

Python

import hashlib

def hash_to_bucket(user_id: str, num_buckets: int = 10) -> int:
    """Deterministically map a user_id to a bucket (0 to num_buckets-1)."""
    digest = hashlib.md5(user_id.encode("utf-8")).hexdigest()
    return int(digest[:8], 16) % num_buckets

if __name__ == "__main__":
    # Mock experiment: split…

14 0 Open

A/B testing & experimentation medium

How to Mock Sequential Calls in Python with unittest.mock

Use Mock.side_effect to return a different result for each sequential call and verify the call order with assert_has_calls.

mock unittest testing

Python

import unittest
from unittest.mock import Mock

class Service:
    def fetch(self, item_id):
        raise NotImplementedError

def process_items(service, ids):
    results = []
    for item_id in ids:
        result = service.fetch(item_id)
        results.append(result)
    return results

if __name__ == "__main__":…

14 0 Open

A/B testing & experimentation easy

How to hash user IDs to experiment buckets in Python

Deterministically map a user ID to an experiment bucket using MD5 hashing, ensuring stable and consistent assignment for A/B testing.

hashing ab-testing experiments

Python

import hashlib


def hash_user_to_bucket(user_id: str, num_buckets: int = 10) -> int:
    """Deterministically map a user ID to an experiment bucket (0..num_buckets-1)."""
    digest = hashlib.md5(user_id.encode("utf-8")).hexdigest()
    return int(digest, 16) % num_buckets


if __name__ == "__main__":
    mock_users …

12 0 Open

Browse by section

Each section groups closely related Python snippets.