Reference library

ML engineering pipelines

Feature prep, batch inference, model-serving hooks, and production ML workflow glue.

6 matches

Build a Data Helper Class in Python for ML Pipelines

A beginner-friendly Python class that summarizes, filters, and exports ML dataset rows as JSON.

from typing import List, Dict, Any
import json

class DataHelper:
    """Beginner-friendly helpers for ML data pipelines."""
    
    def __init__(self, data: List[Dict[str, Any]]):
        self.data = data
        self.keys = list(data[0].keys()) if data else []
    
    def summary(self) -> Dict[str, Any]:
        "…

17 0 Open

ML engineering pipelines easy

How to Build a Data Validation Schema in Python

Create a lightweight validation schema using dataclasses and lambda validators to check fields in a dictionary.

validation dataclasses ml-pipelines

Python

import re
from dataclasses import dataclass, field
from typing import Any, Callable


@dataclass
class Field:
    name: str
    validator: Callable[[Any], bool]
    required: bool = True

    def validate(self, value: Any) -> bool:
        if not self.required and value is None:
            return True
        return …

12 0 Open

ML engineering pipelines easy

How to Create a Mock Metaflow Flow in Python

Build a minimal Metaflow flow with two sequential steps that pass data between them using instance attributes.

metaflow ml-pipelines workflow

Python

from metaflow import FlowSpec, step, current


class MockFlow(FlowSpec):
    """A minimal Metaflow flow to demonstrate basic steps and branching."""

    @step
    def start(self):
        self.category = "mock"
        print(f"Start step for {self.category} flow")
        self.next(self.process)

    @step
    def pr…

15 0 Open

ML engineering pipelines easy

How to Generate Experiment Tracking Run IDs in Python

Generate unique experiment run IDs with timestamps and random suffixes for tracking ML pipeline executions.

run-ids experiment-tracking ml-pipelines

Python

import random
import string
import time

def generate_run_id(prefix="exp"):
    timestamp = time.strftime("%Y%m%d_%H%M%S")
    suffix = "".join(random.choices(string.ascii_lowercase + string.digits, k=6))
    return f"{prefix}_{timestamp}_{suffix}"

if __name__ == "__main__":
    # Simulate tracking three experiment r…

13 0 Open

ML engineering pipelines easy

How to Load CSV Training Data in Python Without Pandas

Load CSV training data using Python's standard library and mock it with io.StringIO for testing, returning headers and rows as dictionaries.

csv ml-pipelines io-stringio

Python

import csv
from pathlib import Path


def load_csv_training_data(file_path: str | Path) -> tuple[list[str], list[dict[str, str]]]:
    """Load CSV training data and return headers plus rows as dictionaries."""
    with open(file_path, mode="r", newline="", encoding="utf-8") as csv_file:
        reader = csv.DictReader…

14 0 Open

ML engineering pipelines easy

One Hot Encode Categories in Python

Convert a list of categorical strings into one-hot encoded numeric vectors using pure Python and NumPy.

one-hot encoding categorical numpy

Python

import numpy as np

categories = ["red", "green", "blue", "red", "blue", "green", "red"]

unique = sorted(set(categories))
lookup = {cat: i for i, cat in enumerate(unique)}

one_hot = []
for cat in categories:
    row = [0] * len(unique)
    row[lookup[cat]] = 1
    one_hot.append(row)

print("Categories:", categories…

13 0 Open

Browse by section

Each section groups closely related Python snippets.