Reference library

ML engineering pipelines

Feature prep, batch inference, model-serving hooks, and production ML workflow glue.

3 matches

How to Stage ML Model Workflows with Python Classes

Defines a Stage class to model ML pipeline stages with variants and mocks, printing grammar for Model, Staging, and Production stages.

ml-pipelines stages model-deployment

Python

class Stage:
    def __init__(self, name):
        self.name = name
        self.mocks = []
        self.variants = []

    def add_mock(self, mock_name):
        self.mocks.append(mock_name)

    def add_variant(self, variant_name, productions=()):
        self.variants.append((variant_name, list(productions)))

    …

12 0 Open

ML engineering pipelines medium

How to mock an artifact store with local paths in Python for ML pipelines

Create a temporary local artifact store with dummy files and metadata to test ML pipeline code without real storage.

ml-pipelines mock tempfile

Python

import tempfile
from pathlib import Path
import json


def create_artifact_store_mock(base_path: Path = None):
    """Create a local artifact store mock directory structure."""
    if base_path is None:
        base_path = Path(tempfile.mkdtemp())

    store_layout = {
        "artifacts": [
            {"name": "mode…

13 0 Open

ML engineering pipelines medium

Mock a Flyte ML workflow in Python

Build a lightweight mock of a Flyte ML pipeline with dataclasses and a simple execution loop that passes outputs between tasks.

flyte ml-pipeline dataclass

Python

from dataclasses import dataclass, field
from typing import List, Dict, Optional
import time


@dataclass
class FlyteTask:
    name: str
    inputs: Dict = field(default_factory=dict)
    outputs: Dict = field(default_factory=dict)

    def run(self) -> Dict:
        time.sleep(0.1)  # simulate work
        return sel…

16 0 Open

Browse by section

Each section groups closely related Python snippets.