Compare Model A vs Model B Metrics in Python

A script that simulates and compares metrics between two ML models, showing a formatted diff table for quick insight.

Easy Python 3.9+ Aug 9, 2026 ML engineering pipelines 13 views 0 copies

model comparison mock metrics ml reporting

Python code

19 lines

Python 3.9+

import random


def compare_a_b(samples=5):
    """Mock comparison of model A vs model B predictions."""
    metrics = ["accuracy", "precision", "recall", "f1"]
    print(f"{'Metric':<12}{'Model A':>10}{'Model B':>10}{'Diff':>10}")
    print("-" * 42)

    random.seed(42)
    for metric in metrics:
        a = round(random.uniform(0.75, 0.95), 3)
        b = round(random.uniform(0.75, 0.95), 3)
        diff = round(a - b, 3)
        print(f"{metric:<12}{a:>10.3f}{b:>10.3f}{diff:>+10.3f}")


if __name__ == "__main__":
    compare_a_b()

Output

stdout

Metric      Model A    Model B       Diff
------------------------------------------
accuracy       0.882      0.762     +0.120
precision      0.778      0.859     -0.081
recall         0.940      0.751     +0.189
f1             0.763      0.837     -0.074

How it works

The script seed the random generator so runs are reproducible — the same numbers appear every execution. Each metric value is drawn from a uniform distribution between 0.75 and 0.95, then rounded to three decimals. The formatted print statement aligns columns using f-string width specifiers, making the output easy to read in a terminal or CI log. This mock removes the need for a real evaluation harness while still exercising the same reporting code path.

Common mistakes

Forgetting to seed random, making each run produce different comparison results
Not accounting for negative diffs in the formatting, which can misalign columns
Rounding too early before computing the diff, losing precision in the comparison

Variations

Use numpy arrays and pandas DataFrame for easier statistical aggregation
Read real model metrics from a JSON or CSV file instead of mocking values

Real-world use cases

A/B testing two model versions in staging with quick synthetic metrics before a full offline eval.
Generating sample comparison output in a Jupyter notebook to validate a reporting function.
In CI, smoke-testing a model comparison pipeline with mocked data before wiring real predictions.

Compare Model A vs Model B Metrics in Python

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from ML engineering pipelines

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from ML engineering pipelines

Keep learning

Tutorials

Quizzes