Handle Imbalanced Classes with SMOTE

You’ve built a classifier, trained it, and watched it hit 95% accuracy — but then you look at the confusion matrix and realize it simply predicts the majority class every time. That’s the silent killer of applied AI projects: imbalanced classes. When fraud is 1% of transactions or churn is 5% of customers, raw accuracy becomes meaningless, and your model learns to ignore the rare cases you actually care about. This lesson shows you how to stop that pattern using SMOTE (Synthetic Minority Over-sampling Technique), a powerful, battle-tested method that generates realistic synthetic samples of your minority class — so your model finally sees the signal in the noise.

The problem this lesson solves

Imbalanced datasets are everywhere in production AI: fraud detection, medical diagnostics, fault prediction, churn modeling. In each case, the event you want to predict is rare, often comprising less than 10% of your data. If you train a model on that raw data, it quickly discovers the easiest path to high accuracy: always predict the majority class.

Consider a dataset with 99% non-fraud and 1% fraud. A model that labels everything "non-fraud" gets 99% accuracy — impressive on paper, disastrous in practice. Every fraud case slips through undetected. This is why accuracy is the wrong metric for imbalanced problems, and why you need active techniques to rebalance the data.

The pain is real: you spend days engineering features, and your model still performs no better than a dumb baseline. You try class weights, but the model still struggles to draw boundaries around sparse minority clusters. You need a way to give the minority class more presence in the training set — without simply duplicating the same samples and risking overfitting. That’s precisely where SMOTE comes in.

Core concept / mental model

Think of your dataset as a map of feature space. The majority class forms dense, crowded regions; the minority class is a sparse, thinly populated area with a few scattered points. SMOTE works by interpolating between existing minority samples to create new, synthetic points that lie along the lines connecting nearby minority neighbors. It’s like drawing a smoother, denser map of the minority territory instead of just stamping extra copies of the same coordinates.

The algorithm focuses on the feature vector of each minority sample and its nearest neighbors. For a chosen neighbor, it picks a random point along the line segment between the two vectors. This creates a synthetic sample that’s different from any real data point, yet stays within the plausible region of the minority class. The result is a balanced training set without redundant duplicates, which helps the model learn more robust decision boundaries.

Key terms to remember:

• Minority class — the rare class you care about (e.g., fraud, defect).
• Majority class — the common class.
• Synthetic sample — a new data point created by interpolation, not a duplicate.
• Over-sampling — increasing the number of minority samples.

💡 Pro tip: SMOTE operates in feature space — it doesn’t know anything about the meaning of features. That’s why standardizing or scaling features before applying SMOTE is critical; otherwise, features with larger scales dominate the distance calculations.

How it works step by step

SMOTE follows a clear, repeatable process:

1. Select a minority sample — pick a random data point from the minority class.
2. Find its k-nearest neighbors (usually k=5) among minority samples, computed in feature space.
3. Choose one neighbor randomly from those k neighbors.
4. Create a synthetic sample by interpolating: take the difference between the feature vector of the chosen neighbor and the original sample, multiply it by a random number between 0 and 1, and add it back to the original vector. Mathematically, this creates a point along the line segment between the two.
5. Repeat until the minority class reaches the desired count (e.g., 50% of the majority).

This synthetic sample sits close to real minority examples, so the model sees a denser, more varied representation of the rare class. Because the points are generated from local neighborhoods, SMOTE effectively forces the model to carve out more nuanced decision regions — regions it previously ignored.

The beauty of this approach is that it doesn’t erase information from the majority class (like under-sampling would), and it adds genuinely new data rather than copying. The result is a training set that better reflects the underlying distribution of the minority class, allowing algorithms like logistic regression, trees, and SVMs to learn patterns they otherwise would miss.

Hands-on walkthrough

Let’s put SMOTE into practice with a Python example using the popular imbalanced-learn library (install with pip install imbalanced-learn). We’ll build a synthetic dataset with a 1:99 imbalance, apply SMOTE, and compare the model performance before and after.

# install: pip install imbalanced-learn scikit-learn
import numpy as np
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report
from imblearn.over_sampling import SMOTE

# Create imbalanced dataset: 1% positive class
X, y = make_classification(
    n_samples=1000, n_features=10, weights=[0.99], random_state=42
)
print("Class distribution before:", {0: sum(y==0), 1: sum(y==1)})

# Split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# Apply SMOTE to training data only
sm = SMOTE(random_state=42)
X_train_res, y_train_res = sm.fit_resample(X_train, y_train)

print("Class distribution after SMOTE:", {0: sum(y_train_res==0), 1: sum(y_train_res==1)})
print("Original training size:", X_train.shape[0])
print("Resampled training size:", X_train_res.shape[0])

Expected output:

Class distribution before: {0: 990, 1: 10}
Class distribution after SMOTE: {0: 694, 1: 694}
Original training size: 700
Resampled training size: 1388

Now train a logistic regression on both the original and resampled data, then evaluate on the untouched test set:

# Model on original data
model_orig = LogisticRegression(max_iter=1000)
model_orig.fit(X_train, y_train)

# Model on SMOTE-resampled data
model_smote = LogisticRegression(max_iter=1000)
model_smote.fit(X_train_res, y_train_res)

# Evaluate on test set
y_pred_orig = model_orig.predict(X_test)
y_pred_smote = model_smote.predict(X_test)

print("--- Original data ---")
print(classification_report(y_test, y_pred_orig))
print("--- After SMOTE ---")
print(classification_report(y_test, y_pred_smote))

Expected outcome (values vary slightly):

--- Original data ---
              precision    recall  f1-score   support

           0       0.99      1.00      0.99       297
           1       0.00      0.00      0.00         3

--- After SMOTE ---
              precision    recall  f1-score   support

           0       1.00      0.92      0.96       297
           1       0.11      1.00      0.19         3

The recall for the minority class jumps from 0% to 100% after SMOTE. That’s the power of the technique — the model finally detects the rare cases you care about.

⚠️ Important: Always apply SMOTE after splitting the data, and only to the training portion. If you resample the full dataset, information from the test set leaks into training, giving you a falsely optimistic performance estimate.

Compare options / when to choose what

SMOTE is not the only way to handle imbalanced classes. Here’s a comparison of common strategies:

Variations to explore

• Borderline SMOTE — focuses synthesis near class boundaries, where misclassifications happen.
• SVMSMOTE — uses SVM algorithms to identify support vectors as seeds for synthesis.
• SMOTE-NC — handles mixed categorical and numeric features.

In practice, SMOTE is the sweet spot for most tabular problems. It’s built into the imbalanced-learn library, works with any scikit-learn compatible model, and is well-documented in production pipelines.

Troubleshooting & edge cases

Even with SMOTE, things can go wrong. Here are common issues and fixes:

• SMOTE fails with mixed data types (categorical + numeric): The default SMOTE assumes numeric features only. If you have categorical variables, use SMOTE-NC from imblearn.over_sampling, which handles categorical features by using the mode of neighbors for categorical values.

• Model still performs poorly: SMOTE adds synthetic samples, but it doesn’t magically solve all problems. If your minority class is extremely small (e.g., 10 samples), the synthesized points might be too artificial. Consider using SMOTEENN to clean up noisy overlapping samples after synthesis.

• Leaking test data: Never apply SMOTE to the entire dataset before splitting. This is the most common mistake. Always split first, then resample the training set only.

• Features on different scales: SMOTE uses Euclidean distance to find neighbors. If one feature spans 0-1000 and another 0-1, the distance is dominated by the large-scale feature. Standardize features with StandardScaler before SMOTE.

• Extreme class imbalance (0.1%): With very few minority samples, SMOTE may create unrealistic overlapping points. In that case, try combining SMOTE with under-sampling the majority class, or use anomaly detection algorithms.

• Performance metrics: Don't rely on accuracy after SMOTE. Use precision, recall, F1-score, and AUC-ROC. SMOTE may lower accuracy but increase recall, which is often what you want.

💡 Pro tip: Always visualize the data (e.g., t-SNE or PCA) before and after SMOTE to confirm the synthetic points are realistic and not noise.

What you learned & what's next

In this lesson, you learned how to handle imbalanced classes with SMOTE. You now understand:

• The core idea: synthetic minority samples created by interpolation.
• The step-by-step workflow, from splitting data to evaluating with appropriate metrics.
• How to implement SMOTE in Python using imbalanced-learn.
• How it compares to other techniques like random sampling and class weighting.
• The critical edge cases and mistakes to avoid.

This skill directly applies to real-world AI systems where rare events matter most. Next in the Applied AI engineering track, you’ll dive into evaluation frameworks for imbalanced problems, learning advanced metrics like precision-recall curves and the impact of thresholds. You’ll also explore how to integrate SMOTE into full machine learning pipelines with cross-validation.

Keep your newly rebalanced model in mind — it’s about to get a lot more useful.