Reference library

ML engineering pipelines

Feature prep, batch inference, model-serving hooks, and production ML workflow glue.

4 matches

How to Create a Mock ONNX Model in Python

Build and export a minimal mock ONNX model with a Reshape and Gemm layer using the onnx helper API.

import onnx
import numpy as np
from onnx import helper, TensorProto

def create_mock_model():
    # Define input and output tensors
    input_tensor = helper.make_tensor_value_info('input', TensorProto.FLOAT, [1, 3, 224, 224])
    output_tensor = helper.make_tensor_value_info('output', TensorProto.FLOAT, [1, 10])

   …

16 0 Open

ML engineering pipelines medium

How to Detect Data Drift with PSI in Python

Calculate the Population Stability Index (PSI) in Python to compare expected vs actual distributions and detect data drift in machine learning pipelines.

data drift psi monitoring

Python

import numpy as np

def calculate_psi(expected, actual, buckets=10):
    """Calculate Population Stability Index (PSI) between two distributions."""
    # Create bucket edges based on expected distribution percentiles
    edges = np.percentile(expected, np.linspace(0, 100, buckets + 1))
    edges[-1] = np.inf  # Ensur…

13 0 Open

ML engineering pipelines medium

How to Mock Kedro Pipeline Nodes in Python

Create a modular Kedro pipeline with node functions, namespacing, and input/output mapping to mock pipeline execution locally.

kedro pipeline modular

Python

from kedro.pipeline import Pipeline, node
from kedro.pipeline.modular_pipeline import pipeline as modular_pipeline


def preprocess(data: list) -> list:
    """Clean data by removing None values."""
    return [item for item in data if item is not None]


def transform(data: list) -> list:
    """Add 1 to each numeric…

15 0 Open

ML engineering pipelines medium

How to Mock a Kubeflow Pipeline in Python

Build a minimal in-memory mock of a Kubeflow pipeline DAG using dataclasses and OrderedDict to chain component functions.

kubeflow pipelines mlops

Python

from typing import Dict, Any
from dataclasses import dataclass, field
from collections import OrderedDict


@dataclass
class KubeflowPipelineMock:
    """A minimal mock of a Kubeflow pipeline DAG."""
    name: str
    components: OrderedDict[str, callable] = field(default_factory=OrderedDict)

    def add_component(se…

14 0 Open

Browse by section

Each section groups closely related Python snippets.