Containerize ML Models with Docker

You've trained a model that achieves great metrics in your notebook, but when your colleague tries to run the same code on their machine, it fails with cryptic dependency errors. Or your production server runs a different Python version, so your model outputs gibberish. This is the classic "works on my machine" problem, and it's a major source of friction in data science. But there's a proven solution: containerizing your ML models with Docker. By packaging your code, dependencies, and runtime into a self-contained unit, you can ship models that run identically anywhere—on your laptop, your colleague's desktop, or a cloud server. In this lesson, you'll learn the core concepts and hands-on steps to containerize your ML models, making your work reproducible, portable, and ready for production.

The problem this lesson solves

You've just spent hours tuning a Random Forest classifier and the results are brilliant—in your Jupyter notebook. But when you try to deploy it as a REST API on a server, everything falls apart. Why? Here are the usual culprits:

• Dependency hell: Your model needs scikit-learn==1.2.0, but the server has 1.1.0, and the API behaves differently.
• Python version mismatch: Your code uses match statements (Python 3.10+), but the server runs Python 3.8.
• System-level packages: Your model relies on a compiled library like libgomp that isn't installed on the target machine.

Even in a team environment, sharing your model means sharing a whole environment—which is fragile and error-prone. The pain is real: hours of debugging that should take minutes.

Containerization fixes this by bundling your model, code, dependencies, and runtime into a single artifact that runs anywhere Docker is installed. That's the promise of Docker for ML.

Core concept / mental model

Think of Docker as a shipping container for software. In logistics, a shipping container holds goods securely and can be moved by ship, train, or truck without opening it. Similarly, a Docker container holds your application—code, libraries, system tools, settings—and can run on any system with Docker installed, without conflicts.

Key definitions

• Image: A read-only template. Think of a snapshot of your environment, including your code and all dependencies.
• Container: A running instance of an image. It's like starting a process from the snapshot.
• Dockerfile: A text file with instructions on how to build an image. It's like a recipe.

Why this matters for ML

Machine learning models are particularly sensitive to environment changes. The version of NumPy, the exact implementation of a solver, or a random seed can alter predictions. By containerizing your model, you ensure that the same code always produces the same result, regardless of where it runs.

Here's a simple analogy: your model is a pizza. The Dockerfile is the recipe. The image is the frozen pizza kit. The container is the cooked pizza. You can hand out the kit, and everyone gets the same tasty result.

Pro tip: Docker also solves the "but it works on my machine" problem for your future self. Even you will forget the exact setup six months from now—a container preserves it forever.

How it works step by step

The process of containerizing an ML model has a clear workflow:

1. Write a Dockerfile: Define the base image, copy your code, install dependencies, and set the command to run.
2. Build the image: Use docker build to create the image from the Dockerfile.
3. Run the container: Use docker run to execute the model inside the container.

Let's explore each step with a focus on ML-specific considerations.

Step 1: Write a Dockerfile

The Dockerfile is the heart of containerization. Here's a minimal example for an ML model that loads a saved model and makes predictions:

# Use an official Python runtime as the base image
FROM python:3.10-slim

# Set the working directory inside the container
WORKDIR /app

# Copy the requirements file first (leverage Docker layer caching)
COPY requirements.txt .

# Install dependencies
RUN pip install --no-cache-dir -r requirements.txt

# Copy the rest of the application code
COPY . .

# Command to run the model server
CMD ["python", "predict.py"]

Step 2: Build the image

Navigate to the folder containing your Dockerfile and run:

docker build -t my-ml-model .

This creates an image tagged my-ml-model. Docker executes each instruction in the Dockerfile, creating a new layer for each step. This layering is powerful: if you change only your code, Docker can reuse cached layers for the unaffected steps, speeding up subsequent builds.

Step 3: Run the container

docker run -p 5000:5000 my-ml-model

This starts a container from the image and maps port 5000 of your host to port 5000 in the container, so your API is accessible at http://localhost:5000.

Hands-on walkthrough

Let's put this into practice with a complete example. We'll create a simple ML model that predicts house prices, then containerize it.

1. Create your model files

First, create a directory and add the following files.

model.py – trains a simple linear regression model and saves it:

import numpy as np
from sklearn.linear_model import LinearRegression
import joblib

# Dummy data: house size in square feet vs price
X = np.array([[750], [800], [850], [900], [950]])
y = np.array([150000, 160000, 170000, 180000, 190000])

# Train
model = LinearRegression()
model.fit(X, y)

# Save
joblib.dump(model, 'house_price_model.pkl')
print("Model trained and saved.")

predict.py – loads the model and provides a prediction function:

import joblib
import numpy as np

# Load the model
model = joblib.load('house_price_model.pkl')

# Simple prediction
size = float(input("Enter house size (sq ft): "))
prediction = model.predict(np.array([[size]]))[0]
print(f"Predicted price: ${prediction:,.2f}")

requirements.txt – specify dependencies:

scikit-learn==1.2.0
numpy==1.24.0
joblib==1.2.0

2. Train and save the model

Run model.py to generate the model file:

python model.py

You should see Model trained and saved.

3. Write the Dockerfile

As shown earlier, but with our specific files:

FROM python:3.10-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

CMD ["python", "predict.py"]

4. Build the image

docker build -t housing-price-predictor .

Expected output ends with something like:

Successfully built a1b2c3d4e5f6
Successfully tagged housing-price-predictor:latest

5. Run the container

Since predict.py expects input, we'll run it interactively:

docker run -it housing-price-predictor

You'll be prompted:

Enter house size (sq ft):

Enter 1200 and you'll see:

Predicted price: $???,???

You've just executed your ML model inside a container! This same container runs identically on any machine with Docker.

Compare options / when to choose what

Docker is not the only way to package and deploy ML models. Here's a comparison to help you choose:

For most scenarios, Docker is the go-to choice because it gives you total control and works universally. If you need experiment tracking alongside deployment, MLflow is a great complement—you can even run MLflow models inside Docker containers.

Pro tip: If your model is a simple API, consider using a lightweight base image like python:3.10-alpine to reduce size and startup time.

Troubleshooting & edge cases

Even with Docker, things can go wrong. Here are common issues and fixes:

docker: command not found

Docker isn't installed. Download and install Docker Desktop from the official website, then restart your terminal.

Build errors: ERROR: No matching distribution found for scikit-learn==1.2.0

Your base image's Python version may not support that package version. Check your python version in the image (docker run python:3.10-slim python --version) and ensure compatibility.

Model file not found

If the model isn't in the container, you likely forgot to COPY it. Make sure house_price_model.pkl exists before building, and that your Dockerfile includes COPY . ..

Input 0 of layer ... is incompatible with the layer errors

This happens when the model expects a different input shape than what you're passing. Debug by printing the model's expected shape on the host, then match it in your prediction code.

Large image size

ML models and libraries can make images huge. Use multi-stage builds or a slim base to minimize size.

What you learned & what's next

You now understand how to containerize ML models with Docker. You can:

• Explain why containerization solves reproducibility and portability problems.
• Write a basic Dockerfile for an ML model.
• Build and run a Docker image with your model.

This is a game-changer for your data science workflow: you can share your model confidently, deploy it to any server, and ensure consistent results.

What's next? In the next lesson in the Python for data science track, you'll explore deploying models as REST APIs with FastAPI—a natural companion to Docker. You'll learn how to turn your containerized model into a web service that others can query, completing the journey from notebook to production.

To solidify this lesson, try this mini-exercise: modify the example to include a simple API (e.g., using Flask or FastAPI) inside the container. Then rebuild and run it, accessing the API from your browser. This will prepare you for the next step.