Reference library

ML engineering pipelines

Feature prep, batch inference, model-serving hooks, and production ML workflow glue.

4 matches

How to Build a Data Validation Schema in Python

Create a lightweight validation schema using dataclasses and lambda validators to check fields in a dictionary.

import re
from dataclasses import dataclass, field
from typing import Any, Callable


@dataclass
class Field:
    name: str
    validator: Callable[[Any], bool]
    required: bool = True

    def validate(self, value: Any) -> bool:
        if not self.required and value is None:
            return True
        return …

12 0 Open

ML engineering pipelines easy

How to Create a Mock Metaflow Flow in Python

Build a minimal Metaflow flow with two sequential steps that pass data between them using instance attributes.

metaflow ml-pipelines workflow

Python

from metaflow import FlowSpec, step, current


class MockFlow(FlowSpec):
    """A minimal Metaflow flow to demonstrate basic steps and branching."""

    @step
    def start(self):
        self.category = "mock"
        print(f"Start step for {self.category} flow")
        self.next(self.process)

    @step
    def pr…

15 0 Open

ML engineering pipelines easy

How to Generate Experiment Tracking Run IDs in Python

Generate unique experiment run IDs with timestamps and random suffixes for tracking ML pipeline executions.

run-ids experiment-tracking ml-pipelines

Python

import random
import string
import time

def generate_run_id(prefix="exp"):
    timestamp = time.strftime("%Y%m%d_%H%M%S")
    suffix = "".join(random.choices(string.ascii_lowercase + string.digits, k=6))
    return f"{prefix}_{timestamp}_{suffix}"

if __name__ == "__main__":
    # Simulate tracking three experiment r…

13 0 Open

ML engineering pipelines easy

How to Load CSV Training Data in Python Without Pandas

Load CSV training data using Python's standard library and mock it with io.StringIO for testing, returning headers and rows as dictionaries.

csv ml-pipelines io-stringio

Python

import csv
from pathlib import Path


def load_csv_training_data(file_path: str | Path) -> tuple[list[str], list[dict[str, str]]]:
    """Load CSV training data and return headers plus rows as dictionaries."""
    with open(file_path, mode="r", newline="", encoding="utf-8") as csv_file:
        reader = csv.DictReader…

14 0 Open

Browse by section

Each section groups closely related Python snippets.