Reference library

ML engineering pipelines

Feature prep, batch inference, model-serving hooks, and production ML workflow glue.

4 matches

How to Build a Data Validation Schema in Python

Create a lightweight validation schema using dataclasses and lambda validators to check fields in a dictionary.

import re
from dataclasses import dataclass, field
from typing import Any, Callable


@dataclass
class Field:
    name: str
    validator: Callable[[Any], bool]
    required: bool = True

    def validate(self, value: Any) -> bool:
        if not self.required and value is None:
            return True
        return …

12 0 Open

ML engineering pipelines medium

How to Mock a Kubeflow Pipeline in Python

Build a minimal in-memory mock of a Kubeflow pipeline DAG using dataclasses and OrderedDict to chain component functions.

kubeflow pipelines mlops

Python

from typing import Dict, Any
from dataclasses import dataclass, field
from collections import OrderedDict


@dataclass
class KubeflowPipelineMock:
    """A minimal mock of a Kubeflow pipeline DAG."""
    name: str
    components: OrderedDict[str, callable] = field(default_factory=OrderedDict)

    def add_component(se…

14 0 Open

ML engineering pipelines medium

How to Stage ML Model Workflows with Python Classes

Defines a Stage class to model ML pipeline stages with variants and mocks, printing grammar for Model, Staging, and Production stages.

ml-pipelines stages model-deployment

Python

class Stage:
    def __init__(self, name):
        self.name = name
        self.mocks = []
        self.variants = []

    def add_mock(self, mock_name):
        self.mocks.append(mock_name)

    def add_variant(self, variant_name, productions=()):
        self.variants.append((variant_name, list(productions)))

    …

12 0 Open

ML engineering pipelines medium

Mock a Flyte ML workflow in Python

Build a lightweight mock of a Flyte ML pipeline with dataclasses and a simple execution loop that passes outputs between tasks.

flyte ml-pipeline dataclass

Python

from dataclasses import dataclass, field
from typing import List, Dict, Optional
import time


@dataclass
class FlyteTask:
    name: str
    inputs: Dict = field(default_factory=dict)
    outputs: Dict = field(default_factory=dict)

    def run(self) -> Dict:
        time.sleep(0.1)  # simulate work
        return sel…

16 0 Open

Browse by section

Each section groups closely related Python snippets.