How to implement a canary traffic split in Python

Route incoming traffic between stable and canary model or service versions using a weight-based random split with deterministic testing.

Easy Python 3.9+ Aug 9, 2026 ML engineering pipelines 14 views 0 copies

canary traffic-split random mlops deployment

Python code

32 lines

Python 3.9+

import random


def canary_route(service_name: str, canary_weight: float = 0.2) -> str:
    """Route traffic between stable and canary versions based on weight."""
    rng = random.Random(42)  # deterministic for reproducible demo
    if rng.random() < canary_weight:
        return f"{service_name}-canary"
    return f"{service_name}-stable"


if __name__ == "__main__":
    service = "payment-svc"
    split = 0.2  # 20% canary traffic

    counts = {"stable": 0, "canary": 0}
    total_requests = 1000

    for _ in range(total_requests):
        target = canary_route(service, split)
        if target.endswith("canary"):
            counts["canary"] += 1
        else:
            counts["stable"] += 1

    canary_pct = (counts["canary"] / total_requests) * 100
    stable_pct = (counts["stable"] / total_requests) * 100

    print(f"Split requested: {split * 100:.0f}% canary / {100 - split * 100:.0f}% stable")
    print(f"Observed: {counts['canary']} canary requests ({canary_pct:.1f}%), "
          f"{counts['stable']} stable requests ({stable_pct:.1f}%)")
    print(f"Sample route: {canary_route(service, split)}")

Output

stdout

Split requested: 20% canary / 80% stable
Observed: 200 canary requests (20.0%), 800 stable requests (80.0%)
Sample route: payment-svc-stable

How it works

The function uses random.Random(42) to create a seeded random number generator, making the traffic split reproducible across runs — critical for testing and debugging. Each call generates a uniform float between 0 and 1; if it falls below the canary_weight, the request routes to the canary version, otherwise to stable. This simulates a simple weighted random traffic split without external dependencies like a load balancer or service mesh. The main block runs 1000 simulated requests to verify the observed split approximates the requested 20% canary / 80% stable distribution, which it does with a seeded generator.

Common mistakes

Using a global `random.random()` without seeding, making results non-reproducible across test runs.
Forgetting to weight the stable path correctly, e.g., returning stable when `rng.random() < 1 - canary_weight`.
Confusing weight semantics — a 0.2 weight means 20% canary, not 80%.

Variations

Use `hashlib` to hash request IDs for deterministic per-user routing instead of pure random sampling.
Leverage `random.choices` with `weights=[canary_weight, 1 - canary_weight]` for a cleaner one-liner.

Real-world use cases

Gradually rolling out a new ML model to a small percentage of live inference requests while monitoring accuracy and latency.
A/B testing different feature flags in a production service, directing a controlled slice of traffic to the experimental version.
Validating a new data preprocessing pipeline in shadow mode before full deployment to avoid regressions in batch jobs.

How to implement a canary traffic split in Python

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from ML engineering pipelines

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from ML engineering pipelines

Keep learning

Tutorials

Quizzes