Implement Online Learning with SGD

Your model was trained once, on a fixed batch of data, and now it’s already stale. In production, data doesn’t stop arriving — clickstreams, sensor readings, transaction feeds — and retraining from scratch every hour is slow, expensive, and often impossible. The pain is real: models that can’t adapt to new patterns silently degrade, and by the time you notice, your predictions are hurting your business. That’s why you need online learning — the ability to update a model incrementally with each new sample, or small mini-batches, using the same workhorse optimizer you already know: stochastic gradient descent (SGD). In this lesson, you’ll implement online learning with SGD in Python, so you can keep your models fresh without rebuilding them from zero.

The problem this lesson solves

Imagine you’ve deployed a churn-prediction model. The customer base changes — new pricing, new behavior, a seasonal spike — and your model’s accuracy drifts. A traditional batch-training pipeline would collect data for a month, retrain a new model, validate it, and deploy it. That’s days of delay, and your model is already behind.

The core problem: static models can’t adapt to non-stationary data. In many real-world applications, data streams continuously, concepts shift, and you need a system that updates fast.

• Latency of retraining: Batch retraining takes hours or days.
• Storage burden: You’d have to store all historical data forever to retrain.
• Concept drift: The underlying relationship between features and target changes over time.

Online learning solves this by updating the model parameters incrementally — one sample at a time or in small mini-batches — so the model is always learning from the most recent data. But naive implementation can be unstable or forget old patterns. That’s where SGD shines: it’s the perfect vehicle for online updates because it already operates on one sample at a time.

By the end of this lesson, you’ll be able to explain the core idea, implement online learning with SGD in a practical exercise, and connect it to your broader AI engineering toolkit.

Core concept / mental model

Think of online learning as teaching a student one flashcard at a time, instead of giving them the whole textbook. With batch learning, you read the entire book, then take the test. With online learning, you get a flashcard, answer it, immediately see the correct answer, adjust your thinking, and move to the next card. Each card refines your knowledge, but you never have to re-read the whole book.

In mathematical terms, you’re minimizing a loss function incrementally. For each new sample ((x_t, y_t)), you compute the gradient of the loss with respect to the current parameters (\theta), then update:

[ \theta_{t+1} = \theta_t - \eta \cdot \nabla_\theta L(\hat{y}_t, y_t) ]

Where (\eta) is the learning rate. The key difference from batch gradient descent is that you use only the current sample’s gradient, not the average over the entire dataset. This is what makes it stochastic — noisy but fast, and perfect for streaming.

Definitions you’ll need

• Online learning: A learning paradigm where the model updates after each sample (or small mini-batch), without revisiting past data.
• SGD: Stochastic Gradient Descent — an optimization algorithm that updates parameters using the gradient of the loss on a single sample or mini-batch.
• Learning rate (η): Controls how big a step you take in the direction of the gradient. Too high — you overshoot; too low — you crawl.
• Epoch: In batch learning, one full pass over the dataset. In online learning, there’s no fixed epoch — you just keep updating forever.

Why SGD fits online learning

Your model sees one flashcard at a time — that’s exactly what SGD is designed for. Batch gradient descent would need the whole dataset, which defeats the purpose. Mini-batch SGD with size 32 or 64 is a nice middle ground: less noisy than pure SGD, but still online-friendly.

How it works step by step

Let’s walk through the mechanics of implementing online learning with SGD. You’ll need a model (e.g., linear regression or a small neural net), a loss function, and the SGD update rule. The steps are:

1. Initialize parameters — weights and bias, often randomly or to zero.
2. Receive a new sample — in streaming, each new observation arrives one by one.
3. Compute prediction — pass features through the current model.
4. Compute loss — compare prediction to the true label.
5. Compute gradient — of the loss w.r.t. each parameter.
6. Update parameters — using the SGD update rule.
7. Repeat — continue for each incoming sample, optionally with a decaying learning rate.

Cause → effect

• If learning rate is too high → parameters oscillate, loss increases.
• If learning rate is too low → model adapts too slowly to new patterns.
• If you don’t handle feature scaling → gradients explode or vanish, causing instability.
• If you use pure SGD → high variance in updates, but fast adaptation.

For a linear model, the update for weight (w_j) is:

[ w_j \leftarrow w_j - \eta \cdot (\hat{y} - y) \cdot x_j ]

That’s it. No matrix inversions, no full dataset scans — just a few multiplications per sample.

Hands-on walkthrough

Let’s implement online learning with SGD in Python from scratch, then compare it to a batch-trained model. We’ll simulate a streaming data source.

1. Implementing a simple online linear regression with SGD

import numpy as np

def sgd_update(weight, bias, x, y, learning_rate):
    """One SGD step for linear regression with MSE loss."""
    pred = np.dot(weight, x) + bias
    error = pred - y
    # Gradient of MSE = 2 * error * x (we'll drop the 2 for simplicity)
    weight -= learning_rate * error * x
    bias -= learning_rate * error
    return weight, bias

# Simulate a streaming data source
def stream_data(n_samples=500):
    """Yields (x, y) pairs with a gradual slope shift (concept drift)."""
    true_slope = 2.0
    for i in range(n_samples):
        # Drift: slope increases slightly over time
        true_slope += 0.001
        x = np.random.randn(5)
        noise = np.random.randn() * 0.1
        y = np.dot(true_slope * np.ones(5), x) + 1.5 + noise
        yield x, y

# Initialize
weight = np.zeros(5)
bias = 0.0
learning_rate = 0.01
losses = []

# Online learning: update after every sample
for i, (x, y) in enumerate(stream_data()):
    weight, bias = sgd_update(weight, bias, x, y, learning_rate)
    # Track loss on a fixed validation set (not shown)
    if i % 50 == 0:
        print(f"Step {i}: weight[0]={weight[0]:.3f}, bias={bias:.3f}")

# Expected output (approx):
# Step 0: weight[0]=0.000, bias=0.000
# Step 50: weight[0]=0.452, bias=0.456
# Step 100: weight[0]=0.873, bias=0.898
# ... converging toward the true slope with drift
print("Final weight:", weight)

2. Using scikit-learn’s SGDRegressor with partial_fit

You don’t always need to write from scratch. Scikit-learn provides partial_fit for online learning.

from sklearn.linear_model import SGDRegressor
import numpy as np

# Create an online linear regression model
model = SGDRegressor(max_iter=1000, tol=1e-3, learning_rate='adaptive', eta0=0.01)

# Simulate a streaming source
def stream_data(n_samples=1000):
    for i in range(n_samples):
        x = np.random.randn(5)
        # Non-stationary: target shifts with time
        y = 2 * x[0] + 0.5 * x[1] + i * 0.001 + np.random.randn() * 0.1
        yield x.reshape(1, -1), np.array([y])

# Online learning loop
losses = []
for i, (x_batch, y_batch) in enumerate(stream_data()):
    # partial_fit updates the model with this batch (size 1)
    model.partial_fit(x_batch, y_batch)
    if i % 100 == 0:
        # Evaluate on a small held-out set (simulated)
        pred = model.predict(x_batch)
        loss = (pred[0] - y_batch[0])**2
        losses.append(loss)
        print(f"Step {i}: loss={loss:.4f}, weight[0]={model.coef_[0]:.3f}")

# Expected output: loss decreasing, weights converging (with drift)

Pro tip: partial_fit in scikit-learn is the direct API for online learning. It expects a single sample or a mini-batch, and it updates the model in place. Always call it with the same feature names/order.

3. Handling mini-batches

For efficiency, you might batch small chunks from a stream.

from sklearn.linear_model import SGDClassifier
from sklearn.preprocessing import StandardScaler
import numpy as np

def stream_batches(batch_size=32):
    while True:
        X = np.random.randn(batch_size, 10)
        y = (X[:, 0] + 0.5 * X[:, 1] > 0).astype(int)  # simple binary pattern
        yield X, y

# Online logistic regression
model = SGDClassifier(loss='log_loss', learning_rate='adaptive', eta0=0.01)
scaler = StandardScaler()  # first batch for scaling
stream = stream_batches()

# First batch to initialize scaler
X_batch, y_batch = next(stream)
scaler.fit(X_batch)
X_scaled = scaler.transform(X_batch)
model.partial_fit(X_scaled, y_batch, classes=[0, 1])

# Continue streaming
for step in range(100):
    X_batch, y_batch = next(stream)
    X_scaled = scaler.transform(X_batch)  # use running stats
    model.partial_fit(X_scaled, y_batch)

print("Model trained on 100 mini-batches.")

Compare options / when to choose what

When to choose what: - If you need a quick, reliable update in a Python pipeline, use SGDRegressor/SGDClassifier with partial_fit. - If you’re building a custom deep-learning model in PyTorch, you already have SGD built in — just call optimizer.zero_grad(); loss.backward(); optimizer.step() on each batch from your data stream. - If your data arrives in large chunks, use mini-batch SGD to amortize compute. - Avoid pure SGD when you have highly varying feature scales; always standardize first.

Troubleshooting & edge cases

• Loss explodes or oscillates — learning rate too high. Reduce it, or use an adaptive schedule like learning_rate='adaptive' in scikit-learn.
• Model never converges — feature scaling is off. Use StandardScaler and update its statistics as new data comes (e.g., exponential moving average).
• partial_fit throws ValueError: The classes ... are not in the data — you must provide all possible classes on the first call with the classes parameter.
• Model forgets old patterns (catastrophic forgetting) — for abrupt shifts, you might want to reset or decay the learning rate over time. For gradual drift, SGD handles it naturally.
• Online updates are too noisy — increase mini-batch size (e.g., 32 or 64).
• Data arrives with missing values — impute before calling partial_fit, or use models that handle NaN.

Pro tip: Always log metrics (loss, accuracy) on a rolling window to detect drift and ensure your online model is improving, not oscillating.

What you learned & what's next

You now understand the core idea behind online learning with SGD: update your model incrementally as new data arrives, without retraining from scratch. You implemented it both from scratch and with scikit-learn’s partial_fit, and you saw how to handle mini-batches and common pitfalls.

Key points covered: - Explain the core idea of online learning with SGD. - Complete a practical exercise for implementing online learning with SGD. - Connect it to your broader AI engineering toolkit.

What’s next: In the next lesson, you’ll explore concept drift detection — how to automatically detect when your online model’s performance degrades and trigger a model update. That’s a natural extension of the streaming mindset you just built.