ML engineering pipelines
Feature prep, batch inference, model-serving hooks, and production ML workflow glue.
How to Compute a Confusion Matrix in Python
Compute a multi-class confusion matrix from true and predicted labels using pure Python dictionaries and nested lists, then format it for readable output.
from collections import defaultdict
def compute_confusion_matrix(y_true, y_pred, labels):
"""Compute confusion matrix using Python dicts and nested lists."""
label_index = {label: i for i, label in enumerate(labels)}
matrix = [[0] * len(labels) for _ in range(len(labels))]
for true, pred in zip(y…How to Define Dagster ML Assets in Python
Define a chain of Dagster software-defined assets that compute raw features, normalized features, and predictions for an ML pipeline.
from dagster import asset
@asset
def raw_features():
return {"sepal_length": [5.1, 4.9, 6.2], "sepal_width": [3.5, 3.0, 3.4]}
@asset
def normalized_features(raw_features):
values = raw_features["sepal_length"]
mean = sum(values) / len(values)
std = (sum((x - mean) ** 2 for x in values) / len(values…How to Evaluate Accuracy, Precision, and Recall in Python
Compute accuracy, precision, and recall for a binary classification model using scikit-learn's metrics functions.
from sklearn.metrics import accuracy_score, precision_score, recall_score
if __name__ == "__main__":
y_true = [0, 1, 1, 0, 1, 0, 1, 1]
y_pred = [0, 1, 0, 0, 1, 0, 1, 1]
accuracy = accuracy_score(y_true, y_pred)
precision = precision_score(y_true, y_pred)
recall = recall_score(y_true, y_pred)
…How to Impute Missing Values with Mean in Python
Replace None values in a list with the mean of the existing values using Python's statistics module.
import statistics
from statistics import mean
def impute_mean(values):
"""Replace None with the mean of the non-None values."""
# Filter out None to compute the mean of existing values
valid = [v for v in values if v is not None]
if not valid:
return values # nothing to impute if all are Non…How to Mock Cron Schedule in Python
Compute the next scheduled run time for a cron expression using a pure-Python mock parser.
import re
from datetime import datetime, timedelta
class CronMock:
def __init__(self, expression):
self.expression = expression
self.minutes = self._parse_field(expression.split()[0], 0, 59)
self.hours = self._parse_field(expression.split()[1], 0, 23)
self.days = self._parse_field(…How to Mock Kedro Pipeline Nodes in Python
Create a modular Kedro pipeline with node functions, namespacing, and input/output mapping to mock pipeline execution locally.
from kedro.pipeline import Pipeline, node
from kedro.pipeline.modular_pipeline import pipeline as modular_pipeline
def preprocess(data: list) -> list:
"""Clean data by removing None values."""
return [item for item in data if item is not None]
def transform(data: list) -> list:
"""Add 1 to each numeric…How to Mock ROC AUC in Python
Compute ROC AUC from scratch in Python using pairwise comparisons between positive and negative score distributions, ideal for testing ML models without sklearn.
import random
from math import comb
def mock_roc_auc(scores, labels):
"""Compute mock ROC AUC by simulating a classifier's score distribution."""
random.seed(42)
n = len(labels)
pos_scores = [scores[i] for i in range(n) if labels[i] == 1]
neg_scores = [scores[i] for i in range(n) if labels[i] == …How to Mock Shadow Mode Inference in Python
Simulates running multiple candidate models in shadow mode by adding randomized delays and returning their outputs alongside a primary model's output.
import random
import time
def shadow_mode_inference(candidates, mock_delay=0.1):
"""
Simulates running multiple candidate models in 'shadow mode'
by adding tiny randomized delays and returning their outputs
alongside the primary model's output.
"""
primary_output = "primary: answer"
shado…K-Fold Cross Validation in Python: A Simple Implementation
Implements k-fold cross validation from scratch, splitting data into folds and computing MSE scores for a baseline mean-predictor model.
import random
from statistics import mean
def cross_validation_scores(data, labels, k=5, seed=42):
random.seed(seed)
indices = list(range(len(data)))
random.shuffle(indices)
fold_size = len(indices) // k
folds = []
for i in range(k):
if i == k - 1:
folds.append(indices[i *…Mock a Flyte ML workflow in Python
Build a lightweight mock of a Flyte ML pipeline with dataclasses and a simple execution loop that passes outputs between tasks.
from dataclasses import dataclass, field
from typing import List, Dict, Optional
import time
@dataclass
class FlyteTask:
name: str
inputs: Dict = field(default_factory=dict)
outputs: Dict = field(default_factory=dict)
def run(self) -> Dict:
time.sleep(0.1) # simulate work
return sel…Browse by section
Each section groups closely related Python snippets.
ML engineering pipelines — Python code examples
What you will find here
This page collects ml engineering pipelines snippets — short, copy-ready Python you can paste into our free online IDE and run without installing anything. Each sample includes a plain-English explanation and the full source code.
Samples vs tutorials and challenges
Samples are quick reference — one concept per page. For step-by-step teaching, use our Python tutorials. To test yourself, try quizzes or coding challenges. Clean up style with the Python formatter.