Epsilon Greedy Bandit Mock in Python

A simple epsilon-greedy multi-armed bandit simulation that balances exploration and exploitation to estimate true means of several Bernoulli-like reward distributions.

Medium Python 3.9+ Aug 9, 2026 A/B testing & experimentation 12 views 0 copies

bandit epsilon-greedy exploration exploitation reinforcement-learning ab-testing

Python code

34 lines

Python 3.9+

import random


class Bandit:
    def __init__(self, true_mean):
        self.true_mean = true_mean
        self.estimated_mean = 0.0
        self.n_pulls = 0

    def pull(self):
        return random.gauss(self.true_mean, 1.0)

    def update(self, reward):
        self.n_pulls += 1
        self.estimated_mean += (reward - self.estimated_mean) / self.n_pulls


def epsilon_greedy_bandit(bandits, epsilon, n_pulls=1000, seed=42):
    random.seed(seed)
    for _ in range(n_pulls):
        if random.random() < epsilon:
            chosen = random.choice(bandits)
        else:
            chosen = max(bandits, key=lambda b: b.estimated_mean)
        reward = chosen.pull()
        chosen.update(reward)
    return [(b.true_mean, b.estimated_mean, b.n_pulls) for b in bandits]


if __name__ == "__main__":
    bandits = [Bandit(1.0), Bandit(2.0), Bandit(3.0)]
    results = epsilon_greedy_bandit(bandits, epsilon=0.1)
    for true, estimated, count in results:
        print(f"True: {true:.1f}, Estimated: {estimated:.2f}, Pulls: {count}")

Output

stdout

True: 1.0, Estimated: 1.05, Pulls: 34
True: 2.0, Estimated: 2.10, Pulls: 45
True: 3.0, Estimated: 2.98, Pulls: 921

How it works

This code implements an epsilon-greedy strategy: with probability epsilon it explores by choosing a random bandit, otherwise it exploits by picking the bandit with the highest estimated mean. Each bandit tracks its running average reward through incremental updates, which keeps memory O(1). The random seed ensures reproducible results, making it ideal for experiments. Estimated means converge toward the true means as pulls accumulate, demonstrating the exploration-exploitation tradeoff.

Common mistakes

Forgetting to seed random for reproducible runs
Not resetting bandit state between experiments
Using a fixed epsilon that is too high or too low for long runs

Variations

Use a decreasing epsilon schedule (epsilon=1/t) for faster convergence
Replace gaussian noise with Bernoulli outcomes for click-through vs. no-click

Real-world use cases

A/B testing platforms that allocate traffic between variants to maximize conversions.
Recommendation systems that balance showing popular vs. new content to learn user preferences.
Online advertising systems that choose between creative variations to optimize click-through rates.

Epsilon Greedy Bandit Mock in Python

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from A/B testing & experimentation

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from A/B testing & experimentation

Keep learning

Tutorials

Quizzes