Champion Challenger Deployment Mock in Python

Simulates an A/B champion-challenger ML deployment workflow — comparing two mock model accuracies and deciding which to promote to production.

Easy Python 3.9+ Aug 9, 2026 ML engineering pipelines 13 views 0 copies

ml deployment champion-challenger simulation mocking

Python code

47 lines

Python 3.9+

import random
import time

class ModelMocker:
    def __init__(self, name="Model", accuracy=0.85):
        self.name = name
        self.accuracy = accuracy

    def predict(self, data):
        """Simulate prediction with some randomness."""
        time.sleep(0.005)  # simulate compute time
        return 1 if random.random() < self.accuracy else 0

def deploy_mock():
    """Simulate champion-challenger deployment workflow."""
    champion = ModelMocker("Champion", accuracy=0.90)
    challenger = ModelMocker("Challenger", accuracy=0.93)

    sample_data = [42, 17, 8, 99]

    champion_pred = champion.predict(sample_data)
    challenger_pred = challenger.predict(sample_data)

    # Compare and decide which to promote
    if challenger.accuracy > champion.accuracy:
        promoted = challenger
        decision = "PROMOTE_CHALLENGER"
    else:
        promoted = champion
        decision = "KEEP_CHAMPION"

    results = {
        "champion_pred": champion_pred,
        "challenger_pred": challenger_pred,
        "decision": decision,
        "promoted_model": promoted.name,
        "challenger_accuracy": challenger.accuracy,
        "champion_accuracy": champion.accuracy
    }

    print(f"Deployment decision: {results['decision']}")
    print(f"Promoted model: {results['promoted_model']}")
    print(f"Predictions — Champion: {champion_pred}, Challenger: {challenger_pred}")
    return results

if __name__ == "__main__":
    deploy_mock()

Output

stdout

Deployment decision: PROMOTE_CHALLENGER
Promoted model: Challenger
Predictions — Champion: 1, Challenger: 1

How it works

The ModelMocker class uses Python's random module to simulate predictions with a deterministic accuracy threshold. Each call to predict() adds a small time.sleep to mimic real inference latency, making the mock realistic for pipeline testing. The deploy function compares champion and challenger accuracy metrics, then outputs a promotion decision — a pattern that mirrors shadow-deployment evaluation. if __name__ == __main__ guards the execution so the script can be imported as a module without triggering a deployment run.

Common mistakes

Assuming the challenged model always wins — accuracy must be compared, not just predictions
Hardcoding sleep delays that make tests slow in CI pipelines
Forgetting that random results make output non-deterministic for assertions
Not separating the deployment decision from the mock prediction logic

Variations

Replace accuracy with actual offline metrics (AUC, F1) computed on a holdout set before comparison
Add a traffic-split percentage to route a small slice of real requests to the challenger

Real-world use cases

Automated CI/CD pipelines that run shadow deployments to evaluate new model versions against current production.
Internal ML experiments where you need a lightweight stub to test downstream API contracts before the real model is ready.
Training a rolling-canary rollout script that promotes a challenger only when its live metrics beat the champion over a window.

Champion Challenger Deployment Mock in Python

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from ML engineering pipelines

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from ML engineering pipelines

Keep learning

Tutorials

Quizzes