How to Define Dagster ML Assets in Python

Define a chain of Dagster software-defined assets that compute raw features, normalized features, and predictions for an ML pipeline.

Easy Python 3.9+ Aug 9, 2026 ML engineering pipelines 13 views 0 copies

dagster ml-pipeline asset etl orchestration
Requires third-party packages — install first
pip install dagster

Python code

29 lines
Python 3.9+ 
from dagster import asset


@asset
def raw_features():
    return {"sepal_length": [5.1, 4.9, 6.2], "sepal_width": [3.5, 3.0, 3.4]}


@asset
def normalized_features(raw_features):
    values = raw_features["sepal_length"]
    mean = sum(values) / len(values)
    std = (sum((x - mean) ** 2 for x in values) / len(values)) ** 0.5
    normalized = [(x - mean) / std for x in values]
    return {**raw_features, "sepal_length_normalized": normalized}


@asset
def predictions(normalized_features):
    return [round(0.5 + 0.3 * v, 3) for v in normalized_features["sepal_length_normalized"]]


if __name__ == "__main__":
    raw = raw_features()
    norm = normalized_features(raw)
    preds = predictions(norm)
    print(f"Raw features: {raw}")
    print(f"Normalized features: {norm}")
    print(f"Predictions: {preds}")

Output

stdout 
Raw features: {'sepal_length': [5.1, 4.9, 6.2], 'sepal_width': [3.5, 3.0, 3.4]}
Normalized features: {'sepal_length': [5.1, 4.9, 6.2], 'sepal_width': [3.5, 3.0, 3.4], 'sepal_length_normalized': [0.9284766908852594, -1.1352928444153171, 0.20681615353005768]}
Predictions: [0.779, 0.159, 0.562]

How it works

Dagster assets are plain Python functions decorated with @asset, and their return values become the asset's data. Asset dependencies are declared by naming a function parameter after another asset, which lets Dagster build the execution graph automatically. This example mimics a small feature-engineering and inference pipeline where each asset consumes the previous one's output. The normalization uses the standard z-score formula on the sepal_length column, and predictions are a simple linear transform of the normalized values. Running the file directly calls each function manually to demonstrate the flow outside of a full Dagster instance.

Common mistakes

  • Forgetting that asset function parameters must match the names of other upstream assets exactly.
  • Mixing up validation mode: scheduling the asset pipeline vs. calling functions directly in `__main__`.
  • Hardcoding input data instead of reading from source systems, which limits production reuse.
  • Ignoring type hints, which makes asset dependencies harder to track in larger pipelines.

Variations

  1. Use `@asset(io_manager_key='...')` to connect a custom I/O manager for storing outputs.
  2. Replace mock data with a data loader asset that reads from a database or file.

Real-world use cases

  • Building a modular feature-engineering pipeline where each transformation is a separate, testable asset.
  • Scheduling machine learning training and inference jobs in production with Dagster's orchestrator.
  • Keeping data transforms reproducible and lineage-tracked for audit and debugging in ML platforms.

Sponsored

Run locally

This sample needs third-party packages, so it cannot run in the browser IDE. Copy the code above, install the packages shown at the top, then run it in your own Python environment.

More from ML engineering pipelines

Keep learning

Related tutorials and quizzes for this topic.

Tutorials

Quizzes