Evaluate Regression with R² and MAE

Wondering if your regression model actually learned something, or if it's just memorizing noise? R-squared and Mean Absolute Error (MAE) are the two numbers that tell you instantly whether your model is a trusted crystal ball or a broken thermometer. In this lesson, you'll learn to evaluate regression with R-squared and MAE — two complementary metrics that reveal both the proportion of variance explained and the typical error in your model's predictions. By the end, you'll know exactly when to trust, tune, or trash your model.

The problem this lesson solves

You just trained a regression model — maybe a linear regression or a gradient boosting machine — and you're staring at the test predictions. Did your model really learn the underlying pattern, or is it just repeating the average? A single accuracy number like 80% correct doesn't make sense for regression; you need to measure how close the predictions are. Without proper evaluation, you'll ship a model that looks impressive in training but fails in production, or you'll discard a perfectly decent model because you misread a metric.

The core pain: R² alone is misleading, and MAE alone is uninformative. R² can look high even when predictions are systematically off by a fixed amount, and MAE can be tiny while the model misses the overall trend. Only together do they give you a full picture.

Core concept / mental model

Think of your regression model as an archer shooting arrows at a target. Each arrow is a prediction.

• Mean Absolute Error (MAE) measures precision — how tight your arrows cluster around the bullseye on average. A low MAE means your arrows land close to the target, regardless of where the target is.
• R-squared (R²) measures accuracy — how consistently your arrows land relative to the overall trend. An R² of 0.85 means your model explains 85% of the variability in the target; the remaining 15% is just scatter.

Pro tip: MAE tells you how wrong your predictions are in the same units as your target (e.g., dollars, sales, degrees). R² tells you how much better your model is than just predicting the average.

Definitions you'll use forever:

• Residual: actual − predicted (the vertical distance between your point and the regression line).
• MAE: the average of the absolute residuals. It ignores direction — a +10 and −10 error cancel out in your head but not in MAE.
• R²: 1 − (SS_res / SS_tot) where SS_res is the sum of squared residuals and SS_tot is the sum of squared differences from the mean. It's a unitless value between −∞ and 1.

How it works step by step

Before you write a single line of code, understand the mechanics:

1. Compute the residuals. For each test point, subtract the predicted value from the actual value.
2. Compute MAE. Sum the absolute residuals, divide by the number of points. This gives you the average error in your target's units.
3. Compute R². Divide the sum of squared residuals by the sum of squared total variance around the mean. Subtract that ratio from 1.
4. Interpret both together. - High R² (>0.7) + low MAE → strong, tight model. - High R² + high MAE → model explains variance but has large absolute errors (possible outliers). - Low R² + low MAE → model is close on average but fails to capture variability (like always predicting the mean).
5. Check the units. R² is unitless; MAE is in target units. Always compare MAE to the scale of your target variable.

Hands-on walkthrough

Let's put this into practice with a simple linear regression on scikit-learn's Boston house prices dataset (still available via load_boston in older versions; we'll use a synthetic example to keep it clean).

First, generate a synthetic dataset:

import numpy as np
import pandas as pd
from sklearn.datasets import make_regression

# Generate 200 samples, 5 features, some noise
X, y = make_regression(n_samples=200, n_features=5, noise=20, random_state=42)

# Split into train/test
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

Now train a linear regression and compute the metrics:

from sklearn.linear_model import LinearRegression
from sklearn.metrics import r2_score, mean_absolute_error

model = LinearRegression()
model.fit(X_train, y_train)
y_pred = model.predict(X_test)

r2 = r2_score(y_test, y_pred)
mae = mean_absolute_error(y_test, y_pred)

print(f"R²: {r2:.3f}")
print(f"MAE: {mae:.2f}")

Expected output (your exact numbers may vary slightly):

R²: 0.994
MAE: 15.87

Here, R² is extremely high (the synthetic data has a clear linear signal) and MAE is about 16 on a target that ranges from about −200 to +200 — so the average error is tiny relative to the spread. Excellent model.

Now let's compare with a baseline model that always predicts the mean of the training target. This is the worst-case model any regression can beat:

baseline_pred = np.full_like(y_test, np.mean(y_train))
baseline_r2 = r2_score(y_test, baseline_pred)
baseline_mae = mean_absolute_error(y_test, baseline_pred)

print(f"Baseline R²: {baseline_r2:.3f}")
print(f"Baseline MAE: {baseline_mae:.2f}")

Expected output:

Baseline R²: -0.001
Baseline MAE: 88.43

Your model beats the baseline by a huge margin — R² went from 0 to 0.994, MAE dropped from ~88 to ~16. That's the kind of comparison that tells you your model isn't just guessing.

Pro tip: Always compare your model's R² against a dummy regressor that predicts the mean. If your model's R² isn't significantly above 0, you're not learning anything.

Now let's simulate a bad model — one that only captures part of the pattern — and see how the metrics degrade:

# Corrupt the predictions: add noise and a bias
noisy_pred = y_pred + np.random.normal(0, 50, y_pred.shape)
noisy_pred += 20  # systematic overestimate

r2_noisy = r2_score(y_test, noisy_pred)
mae_noisy = mean_absolute_error(y_test, noisy_pred)

print(f"Noisy model R²: {r2_noisy:.3f}")
print(f"Noisy model MAE: {mae_noisy:.2f}")

Expected output:

Noisy model R²: 0.881
MAE: 89.23

Notice the R² dropped only to 0.88 — still looks decent — but MAE exploded to 89 (close to the baseline). The systematic bias inflated MAE while R² stayed high because R² measures variance explained, not absolute error. This is exactly why you need both metrics.

Compare options / when to choose what

While R² and MAE are the focus, other regression metrics exist. Here's a quick comparison:

When to use each:

• Use R² when you want a universal, unitless measure of model quality that you can compare across different datasets or features.
• Use MAE when you need to communicate error in the business metric (e.g., "our model is off by $15 on average").
• Use both whenever you present results — R² for the big picture, MAE for the practical impact.
• Use RMSE instead of MAE if you want to penalize large errors more heavily (e.g., when a big miss is catastrophic).

Pro tip: If your residuals have a heavy tail (few huge errors), RMSE will be much larger than MAE. That gap itself is diagnostic — it flags outliers.

Troubleshooting & edge cases

Here are the most common issues you'll hit when evaluating regression:

1. R² is negative. That's not a bug. A negative R² means your model is worse than predicting the mean. Check your code, your data split, or your feature engineering — you likely overfit or have a data leak.
2. MAE is huge but R² is high. This happens when there's a systematic bias (like the noisy model above). Plot residuals vs. predicted values — if you see a line, you're missing a feature.
3. Length mismatch in y_pred and y_test. Make sure you're evaluating on the same test set. Use train_test_split with a fixed random_state to ensure reproducibility.
4. R² close to 1 on tiny datasets. With few samples, you can get lucky. Always cross-validate (e.g., cross_val_score) to get a stable estimate.
5. Units misinterpreted. MAE in dollars vs. euros is meaningless if you forget the scale. Always normalize or use percentage errors when comparing across datasets.

What you learned & what's next

Congratulations! You can now explain the core idea behind R² and MAE, compute them with scikit-learn, and interpret them together. You know that R² measures variance explained and MAE measures average error, and you've seen why you need both — R² can look good while MAE is terrible, and vice versa.

In this lesson, you've covered: - The problem of evaluating regression without accuracy scores - The archer analogy and formal definitions of MAE and R² - Step-by-step computation and hands-on Python examples - Comparison with other metrics and when to use each - Real-world troubleshooting tips

Next step: Now that you know how to evaluate a regression model, you're ready to learn about cross-validation — the technique that gives you reliable performance estimates without wasting your data. In the next lesson, you'll see how to combine R² and MAE with cross-validation to build robust, production-ready models.

Keep practicing — your models are only as trustworthy as your evaluation metrics!