Containerize AI Apps with Docker

You've spent hours fine-tuning a model, wiring up an API, and testing your AI app locally — it works perfectly on your machine. Then you share it with a teammate, and suddenly it fails with a missing dependency or a CUDA error. This is the classic "it works on my machine" problem, and it gets worse with AI apps that depend on exact library versions, GPU drivers, and system packages. Docker solves this by packaging your entire application — code, dependencies, and runtime — into a single, portable container that runs identically anywhere. In this lesson, you'll learn how to containerize AI apps with Docker, from writing a Dockerfile to handling GPU support and troubleshooting common pitfalls.

The Problem This Lesson Solves

AI applications are notoriously fragile. A single version mismatch in a Python library like numpy or torch can break your entire pipeline. Environment drift — when the development environment differs from production — leads to hours of debugging. Docker isolates your app in a lightweight, self-contained environment that includes everything it needs to run, making your AI app reproducible and deployable on any machine that supports Docker.

Pro tip: Think of Docker as a shipping container for your software. Just as a shipping container standardizes cargo transport, Docker standardizes how apps run across different environments.

Without containerization, you face these pain points:

• "It works on my machine" — dependencies are installed globally, so subtle differences in OS or Python version cause failures.
• GPU driver conflicts — AI models often need specific CUDA versions that clash with the host.
• Scaling is hard — if your app runs only on your laptop, you can't easily deploy it to a cloud server or a cluster.

Core Concept / Mental Model

Think of a Docker container as a miniature virtual machine — but much lighter. Instead of virtualizing an entire operating system, Docker shares the host's kernel while providing an isolated userspace. Your AI app, its Python environment, and all its dependencies live inside this container.

A Docker image is a read-only template — like a blueprint or a snapshot. When you run an image, you get a container — a running instance with its own filesystem, processes, and network. Images are built from a Dockerfile, a text file that lists the instructions to assemble the image.

Here's the mental model in words:

Dockerfile (instructions) → Build → Docker Image (read-only template) → Run → Container (isolated process)

For AI apps, the image typically includes:

• A base image with Python (e.g., python:3.10-slim)
• System libraries (e.g., libgomp1 for OpenMP, or CUDA libraries for GPU)
• All Python dependencies (pip install -r requirements.txt)
• Your application code and model files

How It Works Step by Step

Containerizing an AI app follows a logical sequence. Let's break it down:

1. Define the base image — Choose a base that matches your needs. For AI, you often want a Python image that includes common scientific libraries.
2. Copy your code and dependencies — Use COPY to place your app files into the image, and pip install to install dependencies.
3. Set the working directory and expose ports — Define where your app runs and which ports it listens on (for web APIs).
4. Set the startup command — Tell Docker what to run when the container starts (e.g., CMD ["python", "app.py"]).
5. Build the image — Use docker build to create the image from the Dockerfile.
6. Run the container — Use docker run to start your app in an isolated environment.
7. Optional: Add GPU support — For CUDA-based models, use a GPU-enabled base image and run with --gpus all.

Each step has a cause-and-effect relationship. For example, if you don't pin your dependency versions, a future pip install might pull in a breaking version — causing your container to fail later. That's why you should always use a requirements.txt with exact versions.

Hands-On Walkthrough

Let's containerize a simple AI app — a FastAPI service that uses a pre-trained Hugging Face model to perform sentiment analysis. We'll build a Docker image and run it locally.

Step 1: Create a Simple AI App

Create a file named app.py:

from fastapi import FastAPI
from pydantic import BaseModel
from transformers import pipeline

app = FastAPI()
classifier = pipeline("sentiment-analysis", model="distilbert-base-uncased-finetuned-sst-2-english")

class TextRequest(BaseModel):
    text: str

@app.post("/analyze")
def analyze(request: TextRequest):
    result = classifier(request.text)[0]
    return {"label": result["label"], "score": result["score"]}

Step 2: List Dependencies

Create requirements.txt:

fastapi==0.104.1
uvicorn==0.24.0
transformers==4.35.0
torch==2.1.0
pydantic==2.5.0

Why pin exact versions? AI libraries change rapidly; pinning ensures your container always behaves the same way.

Step 3: Write the Dockerfile

Create a Dockerfile:

# Use a Python slim image to keep size small
FROM python:3.10-slim

# Set environment variables to reduce Python output buffering and avoid pyc files
ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1

# Set working directory
WORKDIR /app

# Copy requirements and install dependencies first (caching benefit)
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy the rest of the app
COPY . .

# Expose the port for the API
EXPOSE 8000

# Start the app
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

Step 4: Build and Run

# Build the image (tag it as my-ai-app)
docker build -t my-ai-app .

# Run the container, mapping port 8000 of the container to 8000 on the host
docker run -p 8000:8000 my-ai-app

Expected output:

INFO:     Uvicorn running on http://0.0.0.0:8000
INFO:     Application startup complete.

Now you can send a POST request to http://localhost:8000/analyze with JSON {"text": "I love this!"} and get a sentiment result.

Step 5: Add GPU Support (Optional)

If you're using a GPU, replace the base image with a CUDA-enabled one:

FROM nvidia/cuda:12.2.0-runtime-ubuntu22.04

# Install Python and pip
RUN apt-get update && apt-get install -y python3 python3-pip

# ... rest is similar

Then run with --gpus all:

docker run --gpus all -p 8000:8000 my-ai-app

Step 6: Test the Container

To verify your container works, you can use docker exec to run commands inside the running container:

# List files inside the container
docker exec -it <container-id> ls -la

# Check that Python and dependencies are present
docker exec -it <container-id> python -c "import torch; print(torch.__version__)"

Compare Options / When to Choose What

When containerizing AI apps, you have several choices. Here's a comparison to help you decide:

Also consider Docker Compose for multi-container apps (e.g., API + database + model server). It lets you define services in a docker-compose.yml file and start them with one command.

Rule of thumb: Start with a slim base for CPU-only apps. Add GPU support only when you truly need it — the image size will balloon.

Troubleshooting & Edge Cases

Common issues when containerizing AI apps, and their fixes:

• ModuleNotFoundError: No module named 'torch' — The container doesn't have PyTorch. Make sure requirements.txt is correct and that the base image has pip. Use pip list inside the container to verify.
• CUDA error: no kernel image is available for execution on the device — The CUDA version in the image doesn't match your host driver. Either use the same CUDA version as your host driver or revert to CPU-only.
• Image too large — AI models can be gigabytes. Use .dockerignore to exclude model files if they are downloaded at runtime, or use a smaller base image.
• User permissions — If your app needs to write files (e.g., model cache), run the container with a non-root user or mount a volume.
• Port already in use — The -p flag maps a host port; if it's taken, change the host port (e.g., -p 8001:8000).
• Slow build due to pip downloads — Use --no-cache-dir and copy requirements.txt before the rest of the code to leverage layer caching.

What You Learned & What's Next

You now understand how to containerize AI apps with Docker. You learned the mental model of images and containers, built a Dockerfile for a FastAPI + Hugging Face app, ran it, and even added GPU support. You also learned how to compare base images and troubleshoot common issues.

Key takeaways from this lesson:

• Docker packages your AI app, its dependencies, and runtime into a portable container.
• A Dockerfile defines how to build the image; layers are cached, so order matters.
• Pin dependency versions to ensure reproducibility.
• Use GPU-enabled images only when needed; CPU-only is simpler and smaller.
• Common issues include missing dependencies, CUDA mismatches, and large image sizes — each has a known fix.

Next lesson — Now that your AI app is containerized, you can move on to deploying it to a cloud platform like AWS or Google Cloud, or orchestrating multiple containers with Kubernetes. But first, test your skills with the exercise below.

Practice Recap

To solidify your understanding, try this quick exercise:

1. Modify the Dockerfile to use a smaller base image (e.g., python:3.10-alpine) and see how the image size changes. 2. Add a docker-compose.yml file to run your API alongside a simple Redis cache. 3. Build and run your container, then send a test request to verify everything still works.

This hands-on practice will prepare you for the next lesson on deployment.