R-squared for Regression

Your regression model predicts a house price of $350,000, but the actual sale is $410,000. Is that error acceptable? You check the mean absolute error and it looks small, but you can't tell if your model is truly capturing the pattern or just guessing the average. This is the exact moment you need R-squared — the most widely used (and misused) metric for evaluating regression models. By the end of this lesson, you'll not only calculate R-squared in Python, but you'll know exactly what it does, what it hides, and when to trust it.

The problem this lesson solves

After fitting a linear regression with sklearn.linear_model.LinearRegression, you print the coefficients and see numbers. But those numbers only tell you direction and strength of each feature — not whether the overall model is any good. Without a single, interpretable number, you can't answer simple questions like: "Did my model learn anything?" or "Is my model better than predicting the mean?"

Other metrics like Mean Squared Error (MSE) or Mean Absolute Error (MAE) give you an error magnitude, but they are unit-dependent. An MSE of 12,000 means something very different for house prices in USD versus temperatures in Celsius. You need a relative, unitless measure that tells you how much of the variance in your target is explained by your features. That's precisely the gap R-squared (also written as R², or the coefficient of determination) fills.

R-squared is the default "goodness-of-fit" metric in scikit-learn's score() method for regression models. It is the first number most recruiters, blog posts, and Kaggle notebooks will ask about — but it is also the most frequently misunderstood. Mastering it now will save you from making false confidence mistakes later in your data science journey.

Core concept / mental model

Think of R-squared as a score of "how much better than the average" you are. Your baseline guess for any target variable is its mean — if someone asked you to predict a house price with no other information, the safest guess is the average price of all houses you've seen. This baseline model will have some error. R-squared measures how much of that error your model eliminates.

Formally, R-squared is defined as:

R² = 1 - (SS_res / SS_tot)

Where:

• SS_res (Residual Sum of Squares) = sum of squared differences between actual and predicted values
• SS_tot (Total Sum of Squares) = sum of squared differences between actual values and the mean of actual values

In plain terms, SS_tot represents the total variance in your target — a measure of how spread out your data is. SS_res is the variance that still remains after your model makes its predictions. The ratio tells you what fraction of the original variance your model failed to explain. Subtracting that from 1 gives you the fraction it did explain.

Mental model: Imagine you're in a dark room trying to guess the height of people. Without any model, your best guess is the average height — that's SS_tot. A linear regression is like a flashlight that illuminates a pattern (e.g., height vs. shoe size). The brighter the flashlight, the less growing error remains, and the higher your R².

Interpreting the values

• R² = 1.0: Perfect prediction — every predicted point exactly equals the actual value. This is almost always a sign of overfitting in real-world data.
• R² = 0.7: Your model explains 70% of the variance in the target. The remaining 30% is due to factors not in your model or irreducible noise.
• R² = 0.0: Your model is no better than simply predicting the mean. It has learned nothing useful about the relationship.
• R² < 0: Your model is worse than predicting the mean. This happens when you fit the wrong functional form (e.g., a straight line to data that curves sharply) or when you evaluate your model on data very different from training data.

How it works step by step

Calculating R-squared by hand involves four clear steps. Even though libraries do this for you, understanding the mechanics helps you debug and interpret results.

1. Compute the mean of the actual target values — this is your baseline. Call it y_mean.
2. Calculate the total sum of squares (SS_tot) — for each actual value y_i, compute (y_i - y_mean)² and sum all these squares. This represents the total variance you're trying to explain.
3. Calculate the residual sum of squares (SS_res) — for each actual value y_i and its prediction ŷ_i, compute (y_i - ŷ_i)² and sum all these squares. This is the error left over after your model makes predictions.
4. Apply the formula R² = 1 - (SS_res / SS_tot).

The ratio SS_res / SS_tot is the fraction of variance not explained. Subtracting it from 1 flips that into the fraction explained.

Let's illustrate with a tiny example by hand. Suppose you have three points: actual values [2, 4, 6] and your model predicts [3, 5, 5].

• Mean of actuals: (2+4+6)/3 = 4
• SS_tot = (2-4)² + (4-4)² + (6-4)² = 4 + 0 + 4 = 8
• SS_res = (2-3)² + (4-5)² + (6-5)² = 1 + 1 + 1 = 3
• R² = 1 - (3/8) = 0.625

Your model explains 62.5% of the variance in this tiny dataset. Not bad for three points!

Hands-on walkthrough

Now let's bring this to life with Python. We'll use scikit-learn, numpy, and pandas to build a simple linear regression and evaluate it with R-squared in three different ways.

Setup and dataset

First, create a synthetic dataset with a clear linear relationship and a bit of noise:

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import r2_score

# Set seed for reproducibility
np.random.seed(42)

# Generate features: house size in square feet
X = np.random.uniform(1000, 3000, 200).reshape(-1, 1)

# True relationship: price = 150 * size + 30000, plus noise
y = 150 * X[:, 0] + 30000 + np.random.normal(0, 30000, 200)

# Split into train/test
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Fit model
model = LinearRegression()
model.fit(X_train, y_train)

# Predictions on test set
y_pred = model.predict(X_test)

Three ways to compute R-squared

Method 1: Using model.score() — the scikit-learn built-in for regression models:

# Method 1: model.score()
r2_sklearn = model.score(X_test, y_test)
print(f"R² from model.score(): {r2_sklearn:.4f}")

Method 2: Using r2_score from metrics — this is more flexible because it works with any predictions, not just fitted models:

# Method 2: r2_score function
from sklearn.metrics import r2_score
r2_metrics = r2_score(y_test, y_pred)
print(f"R² from r2_score(): {r2_metrics:.4f}")

Method 3: Manual computation — to verify the library matches the formula:

# Method 3: Manual calculation
ss_res = np.sum((y_test - y_pred) ** 2)
ss_tot = np.sum((y_test - np.mean(y_test)) ** 2)
r2_manual = 1 - (ss_res / ss_tot)
print(f"R² from manual formula: {r2_manual:.4f}")

Expected output (yours may vary slightly due to randomness):

R² from model.score(): 0.9502
R² from r2_score(): 0.9502
R² from manual formula: 0.9502

All three methods give the same value, confirming that scikit-learn calculates R-squared exactly as the formula we studied.

Adding multiple features

R-squared works just as well with multiple predictors. Let's add the number of bedrooms:

# Create a second feature
bedrooms = np.random.randint(1, 6, 200)
X_multi = np.column_stack([X[:, 0], bedrooms])

# Fit a multiple linear regression
model_multi = LinearRegression()
model_multi.fit(X_train_multi, y_train_multi)  # Note: need to split X_multi similarly

# Evaluate
r2_multi = model_multi.score(X_test_multi, y_test_multi)
print(f"R² with two features: {r2_multi:.4f}")

Pro tip: Always evaluate R-squared on a held-out test set, not on the training data. Training R² will almost always be higher (sometimes misleadingly perfect) because the model has already seen those points.

Compare options / when to choose what

R-squared is not the only metric for evaluating regression. Here's how it stacks up against the common alternatives:

Key considerations:

• If you only care about explaining variance, R-squared is a natural fit.
• If you need to compare error magnitudes between models, RMSE or MAE are more practical because they use the target's units.
• Adjusted R² is a variation that penalizes adding features that don't contribute much, helping you avoid overfitting. Use it when you're doing feature selection.
• R-squared is not appropriate for non-linear models like decision trees or neural networks (though sklearn still computes it); those models have their own evaluation metrics (e.g., log-loss, accuracy).

Variations you'll encounter:

• Out-of-sample R²: computed on a test set, which is what you should report in practice.
• Pseudo-R²: for logistic regression or other non-linear models, a different formula is used (e.g., McFadden's R²).
• Weighted R²: when some data points have higher importance, you can compute a weighted version.

Troubleshooting & edge cases

R-squared is negative

You might get a negative R² even though your model was trained fine. This is normal when the model is a poor fit for the test data — often because the test set is very different from training, or you're using a linear model on non-linear data. For example, fitting a straight line to exponential growth data will give negative R² on the test set.

Fix: Check your data's distribution, consider polynomial features or a non-linear model, and visualize residuals.

R-squared = 1.0

A perfect 1.0 R² on training data is a huge red flag. It almost always means overfitting — the model has memorized the data instead of learning the underlying pattern. On test data, perfect R² is practically impossible unless your data is synthetic and noise-free.

Fix: Use cross-validation, regularize your model (e.g., Ridge or Lasso), or collect more data.

R-squared is too high due to outliers

Outliers can inflate or deflate R-squared dramatically. If you have a few extreme points, they dominate the sum of squares and can make a mediocre model look great (or awful).

Fix: Plot residuals to spot outliers, and consider robust regression techniques or transform your target (e.g., log transformation).

Floating-point precision

In edge cases where SS_tot is extremely small, manual division can lead to numerical errors. The sklearn implementation is robust, but when you compute manually, use np.float64 and avoid dividing by zero by checking SS_tot > 0.

Using score() on wrong data

You might forget to pass features vs. target to score(). The signature is score(X, y), not score(y_pred, y). Passing predictions will throw an error.

What you learned & what's next

You've now mastered evaluate regression with r-squared: you can explain its core concept (proportion of variance explained), calculate it using both scikit-learn and manual formulas, interpret its values, and know when to pair it with other metrics. You can also troubleshoot common issues like negative values and overfitting.

In the next lesson of the Python for data science track, you'll build on this foundation by learning cross-validation techniques — how to split data into multiple folds to get a more reliable estimate of model performance, which will directly complement your R-squared skills.

Keep practicing: calculate R-squared on a new dataset, compare it with RMSE, and always ask yourself "Is my model really explaining the variance, or just fitting the mean?"