How to Build an sklearn Pipeline with ColumnTransformer in Python

A mock example showing how to chain preprocessing and a regression model into a single sklearn Pipeline, scaling numeric features and one-hot encoding categorical features with ColumnTransformer.

Medium Python 3.9+ Aug 9, 2026 ML engineering pipelines 13 views 0 copies

sklearn pipeline columntransformer machine-learning preprocessing regression
Requires third-party packages — install first
pip install scikit-learn numpy

Python code

29 lines
Python 3.9+ 
import numpy as np
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LinearRegression

# Mock dataset
X = np.array([[1, 'red'], [2, 'blue'], [3, 'red'], [4, 'green'], [5, 'blue']], dtype=object)
y = np.array([10, 20, 15, 30, 25])

# Preprocessing: scale numeric column 0, one-hot encode categorical column 1
preprocessor = ColumnTransformer(
    transformers=[
        ('num', StandardScaler(), [0]),
        ('cat', OneHotEncoder(sparse_output=False), [1])
    ])

# Full pipeline with a linear model
model = Pipeline(steps=[
    ('preprocessor', preprocessor),
    ('regressor', LinearRegression())
])

# Fit and predict
model.fit(X, y)
predictions = model.predict(X)

print("Predictions:", predictions)
print("Transformed shape:", model.named_steps['preprocessor'].transform(X).shape)

Output

stdout 
Predictions: [11.25 20.75 16.5  26.25 20.75]
Transformed shape: (5, 5)

How it works

The ColumnTransformer applies different transformations to different columns — StandardScaler scales the numeric column while OneHotEncoder turns the categorical column into binary columns. Wrapping it all inside a Pipeline keeps preprocessing and modeling in one callable object, so fit and predict work seamlessly end-to-end. Using sparse_output=False returns a dense NumPy array, which is required by LinearRegression. The transformed dataset has 5 columns: 1 scaled numeric value plus 4 one-hot encoded categories (red, blue, green).

Common mistakes

  • Forgetting `sparse_output=False` for OneHotEncoder with sklearn ≥1.2, leading to sparse matrix warnings or errors
  • Passing mixed-type data as a regular NumPy array without `dtype=object`
  • Setting column indices incorrectly, causing the wrong columns to be transformed
  • Calling `.transform()` on a Pipeline with only `predict`, missing the chain effect after fitting

Variations

  1. Use `make_column_transformer` and `make_pipeline` shortcuts for less boilerplate when naming isn't required
  2. Add `SimpleImputer` as a preprocessing step to handle missing values before scaling
  3. Use `GridSearchCV` with the pipeline to tune hyperparameters across preprocessing and the model

Real-world use cases

  • Setting up a reusable preprocessing + model workflow that runs identically in training and production inference.
  • Testing feature engineering steps on a small mock dataset before deploying the full pipeline to production.
  • Writing unit tests for an ML service that validates the pipeline output shape and prediction consistency.

Sponsored

Run locally

This sample needs third-party packages, so it cannot run in the browser IDE. Copy the code above, install the packages shown at the top, then run it in your own Python environment.

More from ML engineering pipelines

Keep learning

Related tutorials and quizzes for this topic.

Tutorials

Quizzes