Reference library

ML engineering pipelines

Feature prep, batch inference, model-serving hooks, and production ML workflow glue.

4 matches

How to Build a Mock TFX Pipeline in Python

Simulate a TFX-style ML pipeline with simple Python functions to understand component orchestration, data flow, and artifact passing.

tfx ml-pipeline orchestration

Python

# Mock TFX pipeline to illustrate component orchestration

def CsvExampleGen(data_path):
    """Mock component: Simulates reading CSV data."""
    print(f"ExampleGen: Reading from {data_path}")
    return {"records": 100, "name": "examples"}

def StatisticsGen(example_artifact):
    """Mock component: Simulates genera…

15 0 Open

ML engineering pipelines easy

How to Create a Mock Metaflow Flow in Python

Build a minimal Metaflow flow with two sequential steps that pass data between them using instance attributes.

metaflow ml-pipelines workflow

Python

from metaflow import FlowSpec, step, current


class MockFlow(FlowSpec):
    """A minimal Metaflow flow to demonstrate basic steps and branching."""

    @step
    def start(self):
        self.category = "mock"
        print(f"Start step for {self.category} flow")
        self.next(self.process)

    @step
    def pr…

15 0 Open

ML engineering pipelines easy

How to Evaluate Accuracy, Precision, and Recall in Python

Compute accuracy, precision, and recall for a binary classification model using scikit-learn's metrics functions.

metrics classification scikit-learn

Python

from sklearn.metrics import accuracy_score, precision_score, recall_score

if __name__ == "__main__":
    y_true = [0, 1, 1, 0, 1, 0, 1, 1]
    y_pred = [0, 1, 0, 0, 1, 0, 1, 1]

    accuracy = accuracy_score(y_true, y_pred)
    precision = precision_score(y_true, y_pred)
    recall = recall_score(y_true, y_pred)

   …

13 0 Open

ML engineering pipelines easy

How to Load, Save, and Split JSON Data in Python

Provides helper functions to load, save, and split JSON dictionary data for simple ML pipeline preprocessing.

json data-splitting ml-pipeline

Python

import json
from pathlib import Path


def load_json_data(file_path):
    """Load JSON data from a file, returning an empty dict if missing."""
    path = Path(file_path)
    if path.exists():
        with path.open("r", encoding="utf-8") as f:
            return json.load(f)
    return {}


def save_json_data(data, f…

13 0 Open

Browse by section

Each section groups closely related Python snippets.