Build simple linear regression models

You have CSV files full of numbers, but when a stakeholder asks, "If I raise the price by 10%, what happens to sales?" — you freeze. You know correlation exists, but you can't quantify the relationship or make a prediction. That's the exact pain this lesson solves: building simple linear regression models in Python that turn scattered data points into a line of best fit — and let you make confident, data-backed predictions.

The problem this lesson solves

Guessing is not analysis. Eye-balling a scatter plot and saying "looks like an upward trend" doesn't survive a follow-up question like "How much will sales increase, exactly?" Without a formal model, you can't:

• Quantify the strength of the relationship between two variables
• Predict the target value for a new input you've never seen
• Separate signal from noise — is that trend real or just random?

Simple linear regression answers these by fitting a straight line to your data. It's the stepping stone to every more advanced model you'll meet later: multiple regression, polynomial regression, and even neural networks. Master this, and you'll have a reusable mental framework for all of them.

Why now? You've already learned NumPy for numeric manipulation and pandas for data wrangling. Regression is the first predictive tool in your data science toolbox — and it's the foundation for the classification and clustering lessons coming next.

Core concept / mental model

Think of simple linear regression as drawing the best possible straight line through a cloud of points. "Best" means the line that minimizes the total vertical distance between each point and the line itself. That distance is called the residual — the error between the actual value and the line's prediction.

The model is written as:

y = b0 + b1 * x

• y — the dependent variable (what you're predicting, e.g., sales)
• x — the independent variable (what you're using to predict, e.g., price)
• b0 — the intercept (predicted y when x = 0)
• b1 — the slope (change in y for a one-unit change in x)

Visually, imagine a scatter plot of house sizes vs. prices. A line snakes through the middle of the points. Some houses sit above the line (model underestimates), some below (model overestimates). The line is positioned so those errors cancel out as much as possible — that's the least squares method.

The key idea: correlation tells you if two variables move together; regression tells you how much.

How it works step by step

Building a simple linear regression model in Python is a pipeline. Each step matters:

1. Load and inspect your data — read the CSV, check for missing values, and sanity-check the column names.
2. Explore the relationship — make a scatter plot or compute the correlation coefficient to confirm a linear relationship exists.
3. Split the data — separate features (X) from target (y). In simple linear regression, X is a single column.
4. Create the model — instantiate LinearRegression from sklearn.linear_model.
5. Fit the model — .fit(X, y) learns the slope (b1) and intercept (b0).
6. Evaluate — score the model with R² (how much variance in y is explained by x) and look at the coefficients.
7. Predict — call .predict(X_new) to make forecasts for new inputs.

Steps 1–2 are often rushed, but they're critical. A model fitted to non-linear data will be misleading, no matter how good the code is.

Hands-on walkthrough

Let's build a complete example. We'll use the classic Boston housing-style dataset (available via sklearn.datasets) — but to keep it clean, we'll simulate our own data: house size (square feet) vs. price ($).

Setup and data preparation

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
from sklearn.metrics import r2_score, mean_squared_error

# Simulated data: 100 houses
np.random.seed(42)
sizes = np.random.randint(800, 4000, 100)
prices = 50 + 0.3 * sizes + np.random.normal(0, 40, 100)

df = pd.DataFrame({'size': sizes, 'price': prices})
print(df.head())

Output (first few rows):

   size       price
0  3575  1057.231040
1  1400   541.513596
2  1059   385.365870
3  2619   840.745475
4  2144   699.103856

Visualize the relationship

plt.scatter(df['size'], df['price'], alpha=0.6)
plt.title('House Size vs. Price')
plt.xlabel('Size (sq ft)')
plt.ylabel('Price ($k)')
plt.show()

You'll see a clear upward trend — as size increases, so does price. That's the linear pattern we're going to model.

Fit the model and make predictions

X = df[['size']]  # must be 2D for sklearn
 y = df['price']

model = LinearRegression()
model.fit(X, y)

# Model parameters
print(f"Intercept (b0): {model.intercept_:.2f}")
print(f"Slope (b1):     {model.coef_[0]:.2f}")

# R-squared
print(f"R² score: {model.score(X, y):.3f}")

# Predict for a new house
new_size = [[2500]]  # 2500 sq ft
predicted = model.predict(new_size)
print(f"Predicted price for 2500 sq ft: ${predicted[0]:.2f}k")

Expected output (values will vary slightly):

Intercept (b0): 51.65
Slope (b1):     0.28
R² score: 0.997
Predicted price for 2500 sq ft: $756.00k

The slope of 0.28 means each additional square foot adds about $280 to the price. The intercept is the base price when size hits zero (theoretical, not meaningful here). The R² of 0.997 means the model explains 99.7% of the variance in price — excellent fit.

Evaluate with residuals

residuals = y - model.predict(X)
plt.scatter(X, residuals, alpha=0.6)
plt.axhline(0, color='red', linestyle='--')
plt.title('Residual Plot')
plt.xlabel('Size')
plt.ylabel('Residuals')
plt.show()

If residuals are randomly scattered around zero with no pattern, your linear model is a good fit. A funnel shape (residuals widening as x increases) would suggest heteroscedasticity — a non-constant variance you'll learn to handle later.

Pro tip: Always scale your features if you plan to compare coefficients across multiple variables (future lesson). For simple regression with one variable, scaling isn't necessary.

Compare options / when to choose what

Simple linear regression is not the only tool. Here's how it stacks up against alternatives:

Choose simple linear regression when:

• You have exactly one relevant predictor variable
• The scatter plot shows a roughly straight-line relationship
• You need a fast, explainable model to communicate to stakeholders

If you have more features or the relationship is clearly curved, move to multiple or polynomial regression (upcoming lessons in this track).

Variations

• statsmodels is a library that gives you detailed statistical summaries (p-values, confidence intervals). Great for inferential analysis, not prediction alone.
• scipy.stats.linregress is a lightweight alternative for quick slope/intercept without full sklearn.
• numpy.polyfit with degree=1 also fits a line, but sklearn is more consistent with the rest of the scikit-learn ecosystem.

Troubleshooting & edge cases

Problem 1: Model scores terrible R² (e.g., 0.01)

Cause: No linear relationship between variables, or data is too noisy. Fix: Go back to the scatter plot — you may need non-linear features or a different model.

Problem 2: You get a ValueError: Expected 2D array, got 1D array

Your X must be a 2D array (a DataFrame or a reshaped array). Use df[['size']] instead of df['size'].

Problem 3: Intercept is absurdly high or slope is negative when you expected positive

Check for outliers or data entry errors. A single extreme value can pull the line in the wrong direction. Also verify units — mixing meters and feet will break everything.

Problem 4: Residuals show a clear curve

Your assumption of linearity is wrong. Try adding a squared term (polynomial regression later) or log-transforming the target.

Problem 5: Missing values cause fit() to fail

Scikit-learn won't handle NaNs automatically. Drop or impute them first:

df = df.dropna()  # or use SimpleImputer

What you learned & what's next

You now know how to build simple linear regression models in Python: from loading data, visualizing relationships, fitting a model with sklearn, interpreting coefficients, and evaluating with R² and residuals. You understand the mental model of a best-fit line and the trade-offs versus other predictive techniques.

Next up: In the next lesson, we’ll expand to multiple linear regression — handling several predictor variables at once, checking for multicollinearity, and using feature scaling to compare coefficients. You'll apply what you learned here, but with richer data.

Take a moment to reflect: you've gone from raw data to a predictive model that answers real business questions. That's a major milestone in your data science journey.