Prepare Data for ML

You found a dataset, loaded it into pandas, and even made a few plots. But the moment you try to feed it to scikit-learn, you get a wall of cryptic errors: ValueError: Input contains NaN, could not convert string to float, or maybe it runs but the accuracy is embarrassingly low. Sound familiar? The truth is, machine learning models are picky eaters — they need clean, numeric, and correctly scaled data. In this lesson, you'll master the essential steps to prepare data for machine learning: understanding why raw data almost never works, building a robust pipeline to clean, split, and scale your data, and applying it in a hands-on exercise. By the end, you'll never fear the fit() call again.

The problem this lesson solves

Raw data is messy. Real-world datasets come with missing values, inconsistent text, and wildly different scales. If you throw that straight into a model, you're asking for trouble. Here's what happens when you skip preparation:

• Missing values — Many algorithms can't handle NaN and will crash or silently produce garbage.
• Non-numeric data — Models operate on numbers; strings like "red" or "high" are meaningless until encoded.
• Feature scale mismatch — A feature measured in kilograms (0-200) will dominate one in meters (1-2), skewing distance-based models.
• Data leakage — If you scale or fill using the entire dataset before splitting, your test set no longer represents unseen data, and your evaluation is toast.

This lesson solves all of that. You'll learn a repeatable process to transform raw, messy data into a clean, model-ready format — one you can trust in production.

Core concept / mental model

Think of data preparation like prepping ingredients before cooking. You wouldn't throw unwashed vegetables and a raw chicken straight into a pan. Similarly, you need to wash (drop or fill missing), chop (encode categories), and season (scale features) before the model (the chef) can work its magic.

Here's the mental model in three acts:

1. Cleaning — Tidy the table: handle missing values and remove duplicates.
2. Transforming — Convert everything to numeric: encode categorical texts, bin numeric values if needed.
3. Scaling — Normalize or standardize so no single feature dominates.

And crucially, you do all of this after splitting your data to avoid leakage. The pipeline is: split → clean → transform → scale → train. This order is non-negotiable.

How it works step by step

Here's the step-by-step flow you'll use every time you prepare data for machine learning:

1. Load and inspect — Read your data, check shape, head, and info().
2. Split first — train_test_split to separate features (X) and target (y). Why? So any statistics (mean, median) computed from the training set are applied to the test set — no peeking.
3. Handle missing values — Choose a strategy: drop rows (if missing is few), fill with median (robust to outliers), or use a model to predict missing values. For beginners, SimpleImputer is your friend.
4. Encode categorical variables — Convert strings to numbers. Use OneHotEncoder for nominal categories (colors, countries) and OrdinalEncoder for ordered (low, medium, high). For binary, a simple LabelEncoder works.
5. Scale numeric features — StandardScaler (zero mean, unit variance) works for most models; MinMaxScaler (0-1 range) is better for neural networks and distance-based algorithms.
6. Verify and proceed — Confirm no NaN, all numeric, and then train your model.

To automate this in a repeatable way, scikit-learn's Pipeline is a lifesaver — it bundles imputer, encoder, and scaler into one object that can be fit once and reused.

Hands-on walkthrough

Let's put it into practice. We'll use a tiny toy dataset that mimics real messy data: missing values, a categorical column, and features on different scales.

Step 1: Setup and data inspection

import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer

# Simulated raw data
data = {
    'age': [25, 30, np.nan, 45, 22, np.nan, 50],
    'income': [50000, 60000, 55000, np.nan, 48000, 65000, 72000],
    'city': ['NYC', 'LA', 'NYC', 'LA', 'SF', 'SF', 'NYC'],
    'purchased': [0, 1, 0, 1, 0, 1, 1]
}
df = pd.DataFrame(data)
print(df)
print("\nInfo:")
print(df.info())

Expected output: You'll see NaN in age and income, and a non-numeric city column.

Step 2: Split the data (before any cleaning!)

X = df.drop('purchased', axis=1)
y = df['purchased']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
print("Train shape:", X_train.shape, "Test shape:", X_test.shape)

Why split first? If you compute the median of income from the whole data and then fill, your test set has influenced the training imputation — that's leakage.

Step 3: Build a preprocessing pipeline

# Identify column types
numeric_features = ['age', 'income']
categorical_features = ['city']

# Numeric pipeline: impute median, then scale
numeric_transformer = Pipeline(steps=[
    ('imputer', SimpleImputer(strategy='median')),
    ('scaler', StandardScaler())
])

# Categorical pipeline: impute 'missing' then one-hot encode
categorical_transformer = Pipeline(steps=[
    ('imputer', SimpleImputer(strategy='constant', fill_value='missing')),
    ('onehot', OneHotEncoder(handle_unknown='ignore'))
])

# Combine with ColumnTransformer
preprocessor = ColumnTransformer(
    transformers=[
        ('num', numeric_transformer, numeric_features),
        ('cat', categorical_transformer, categorical_features)
    ])

# Fit and transform training data, transform test data
X_train_prepared = preprocessor.fit_transform(X_train)
X_test_prepared = preprocessor.transform(X_test)

print("Prepared train shape:", X_train_prepared.shape)
print("Prepared test shape:", X_test_prepared.shape)

Expected output: Both arrays are fully numeric and dense (or sparse — check with .toarray() if needed).

Step 4: Train a model and evaluate

from sklearn.linear_model import LogisticRegression

model = LogisticRegression(max_iter=1000)
model.fit(X_train_prepared, y_train)
accuracy = model.score(X_test_prepared, y_test)
print(f"Test accuracy: {accuracy:.2f}")

On such a tiny random dataset, accuracy may vary, but the pipeline works without errors — that's the win.

Compare options / when to choose what

When to choose what? Default to StandardScaler for most algorithms. If your data has extreme outliers, RobustScaler is safer. For deep learning, MinMaxScaler is common. Always use a pipeline to avoid leakage.

Troubleshooting & edge cases

• ValueError: Input contains NaN, infinity or a value too large for dtype('float64') — You forgot to impute. Ensure your pipeline imputes all columns that could have missing values. Check with X_train.isna().sum().
• could not convert string to float: 'NYC' — You didn't encode categoricals. Make sure OneHotEncoder (or OrdinalEncoder) is applied inside the transformer.
• Losing column structure — ColumnTransformer may reorder columns. To see what you get, call preprocessor.get_feature_names_out() to map.
• Sparse matrix issues — One-hot encoding produces sparse arrays. Some models handle it fine; if you need a dense array, call .toarray() but beware memory.
• Shape mismatch after transform — If you added a category in test that wasn't in train, set handle_unknown='ignore' in OneHotEncoder.
• Data leakage in scaling — Never call fit_transform on the whole dataset. Always split first, then fit on train only, transform test.
• Inconsistent accuracies on tiny datasets — With 7 rows, random split can skew results. That's expected; use cross-validation for stable estimates.

What you learned & what's next

You've learned the essential steps to prepare data for machine learning: split first, handle missing values, encode categories, and scale numeric features — all wrapped in a reusable pipeline. You can now explain why raw data fails, and you've completed a hands-on exercise that cleans and transforms data, ready for modeling. This is the foundation for every machine learning project.

Next in this track, you'll dive into building and evaluating your first models — logistic regression and decision trees — using the prepared data you just learned to create. You'll also learn cross-validation to get robust accuracy scores. Master data prep now, and every modeling lesson becomes smoother.