How to Mock Shadow Mode Inference in Python

Simulates running multiple candidate models in shadow mode by adding randomized delays and returning their outputs alongside a primary model's output.

Easy Python 3.9+ Aug 9, 2026 ML engineering pipelines 13 views 0 copies

ml-pipeline shadow-mode simulation mock inference

Python code

29 lines

Python 3.9+

import random
import time


def shadow_mode_inference(candidates, mock_delay=0.1):
    """
    Simulates running multiple candidate models in 'shadow mode'
    by adding tiny randomized delays and returning their outputs
    alongside the primary model's output.
    """
    primary_output = "primary: answer"
    shadow_outputs = []

    for name in candidates:
        # simulate inference latency
        time.sleep(mock_delay)
        latency = round(random.uniform(0.05, 0.2), 3)
        output = f"{name}: inferred answer (latency={latency}s)"
        shadow_outputs.append(output)

    return primary_output, shadow_outputs


if __name__ == "__main__":
    candidates = ["model_a", "model_b", "model_c"]
    primary, shadows = shadow_mode_inference(candidates, mock_delay=0.01)
    print(primary)
    for s in shadows:
        print(s)

Output

stdout

primary: answer
model_a: inferred answer (latency=0.143s)
model_b: inferred answer (latency=0.109s)
model_c: inferred answer (latency=0.164s)

How it works

This function mimics a shadow mode deployment where candidate models run in parallel with the primary model but their outputs are not shown to users. Each candidate simulates inference latency using a random uniform delay, which models real-world variability. The primary output is always returned first, followed by the shadow outputs for offline evaluation. The mock_delay parameter lets you control the simulation speed for testing purposes.

Common mistakes

Forgetting to include the primary output in the return value
Using `time.sleep` with too large a delay, making tests slow
Not using `round()` on latency, producing unpredictable string lengths

Variations

Use `asyncio.sleep` + `asyncio.gather` to simulate concurrent shadow inference
Wrap the mock in a dataclass to store structured candidate metadata instead of strings

Real-world use cases

Testing a model rollout pipeline before enabling shadow traffic in production.
Benchmarking latency differences across candidate model versions offline.
Validating logging and monitoring hooks for shadow inference without real traffic.

How to Mock Shadow Mode Inference in Python

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from ML engineering pipelines

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from ML engineering pipelines

Keep learning

Tutorials

Quizzes