Reference library

ML engineering pipelines

Feature prep, batch inference, model-serving hooks, and production ML workflow glue.

3 matches

How to Mock a Kubeflow Pipeline in Python

Build a minimal in-memory mock of a Kubeflow pipeline DAG using dataclasses and OrderedDict to chain component functions.

kubeflow pipelines mlops

Python

from typing import Dict, Any
from dataclasses import dataclass, field
from collections import OrderedDict


@dataclass
class KubeflowPipelineMock:
    """A minimal mock of a Kubeflow pipeline DAG."""
    name: str
    components: OrderedDict[str, callable] = field(default_factory=OrderedDict)

    def add_component(se…

14 0 Open

ML engineering pipelines easy

How to ordinal encode categorical data in Python with sklearn

Convert job title categories into ordinal numeric labels using sklearn's OrdinalEncoder with explicit ordering.

ordinal-encoding sklearn categorical-data

Python

from sklearn.preprocessing import OrdinalEncoder
import numpy as np

# Mock data: small job title categories with known ordering
data = np.array([
    ["intern"],
    ["junior"],
    ["mid"],
    ["senior"],
    ["lead"]
])

# Define the ordinal order (lowest to highest)
categories = [["intern", "junior", "mid", "seni…

15 0 Open

ML engineering pipelines medium

Training Pipeline Orchestration Mock DAG in Python

Build a mock DAG orchestrator that runs ML pipeline stages in dependency order using topological sorting (Kahn's algorithm).

dag pipeline topological-sort

Python

from collections import deque
from dataclasses import dataclass, field


@dataclass
class DAGNode:
    name: str
    task: callable
    dependencies: list[str] = field(default_factory=list)


class MockDAG:
    def __init__(self, nodes: list[DAGNode]):
        self.nodes = {n.name: n for n in nodes}
        self.execu…

13 0 Open

Browse by section

Each section groups closely related Python snippets.