Reference library

ML engineering pipelines

Feature prep, batch inference, model-serving hooks, and production ML workflow glue.

3 matches

How to Build a Mock TFX Pipeline in Python

Simulate a TFX-style ML pipeline with simple Python functions to understand component orchestration, data flow, and artifact passing.

tfx ml-pipeline orchestration

Python

# Mock TFX pipeline to illustrate component orchestration

def CsvExampleGen(data_path):
    """Mock component: Simulates reading CSV data."""
    print(f"ExampleGen: Reading from {data_path}")
    return {"records": 100, "name": "examples"}

def StatisticsGen(example_artifact):
    """Mock component: Simulates genera…

15 0 Open

ML engineering pipelines medium

How to Mock Kedro Pipeline Nodes in Python

Create a modular Kedro pipeline with node functions, namespacing, and input/output mapping to mock pipeline execution locally.

kedro pipeline modular

Python

from kedro.pipeline import Pipeline, node
from kedro.pipeline.modular_pipeline import pipeline as modular_pipeline


def preprocess(data: list) -> list:
    """Clean data by removing None values."""
    return [item for item in data if item is not None]


def transform(data: list) -> list:
    """Add 1 to each numeric…

15 0 Open

ML engineering pipelines easy

How to do feature selection with VarianceThreshold in Python

This code demonstrates how to use scikit-learn's VarianceThreshold to remove low-variance features from a NumPy array, keeping only those that vary enough to be useful for modeling.

feature selection sklearn machine learning

Python

import numpy as np
from sklearn.feature_selection import VarianceThreshold

def main():
    # Mock dataset: 4 samples, 5 features
    X = np.array([
        [0.1, 0.2, 1.0, 1.0, 0.5],
        [0.2, 0.2, 0.0, 1.0, 0.4],
        [0.1, 0.2, 1.0, 1.0, 0.6],
        [0.3, 0.2, 1.0, 0.0, 0.5]
    ])

    # Select features w…

14 0 Open

Browse by section

Each section groups closely related Python snippets.