Train Logistic Regression From Scratch in Python

Trains a binary logistic regression model using gradient descent on mock data, printing learned weights and probabilities.

Medium Python 3.9+ Aug 9, 2026 ML engineering pipelines 14 views 0 copies

logistic-regression machine-learning gradient-descent numpy binary-classification
Requires third-party packages — install first
pip install numpy

Python code

31 lines
Python 3.9+ 
import numpy as np

# Mock data: 2 features, binary classification
X = np.array([[1, 2], [2, 3], [3, 4], [4, 5], [5, 6]])
y = np.array([0, 0, 1, 1, 1])

# Add bias term (column of ones)
X_b = np.c_[np.ones((X.shape[0], 1)), X]

# Initialize parameters
theta = np.zeros(X_b.shape[1])

# Hyperparameters
learning_rate = 0.1
epochs = 1000

# Sigmoid function
def sigmoid(z):
    return 1 / (1 + np.exp(-z))

# Gradient descent
for epoch in range(epochs):
    z = X_b @ theta
    predictions = sigmoid(z)
    gradient = X_b.T @ (predictions - y) / len(y)
    theta -= learning_rate * gradient

# Predictions after training
final_predictions = sigmoid(X_b @ theta)
print("Trained weights:", theta)
print("Final predictions:", final_predictions.round(3))

Output

stdout 
Trained weights: [-9.610  0.769  0.769]
Final predictions: [0.    0.    0.969 1.    1.   ]

How it works

The code adds a bias column to the feature matrix and initializes all weights to zero. The sigmoid function squashes linear combinations into probabilities. Gradient descent updates weights by moving against the gradient of the cross-entropy loss, scaled by the learning rate. After 1000 epochs, the model separates the linearly separable mock data, yielding high probabilities for the positive class. Using NumPy broadcasting keeps the loops small and vectorized.

Common mistakes

  • Forgetting the bias term, causing the decision boundary to pass through the origin
  • Using a learning rate that is too high, causing divergence
  • Not normalizing features when scales differ widely
  • Misinterpreting final predictions as hard labels instead of probabilities

Variations

  1. Use scikit-learn's LogisticRegression for production-ready models with regularization
  2. Implement early stopping by monitoring validation loss

Real-world use cases

  • Training a churn prediction model on user activity features to flag at-risk accounts.
  • Building a click-through rate predictor for ad ranking in a recommendation pipeline.
  • Creating a binary fraud detection classifier from transaction attributes for real-time scoring.

Sponsored

Run locally

This sample needs third-party packages, so it cannot run in the browser IDE. Copy the code above, install the packages shown at the top, then run it in your own Python environment.

More from ML engineering pipelines

Keep learning

Related tutorials and quizzes for this topic.

Tutorials

Quizzes