Install system dependencies in Docker

You finally have a Dockerfile that builds your Python app, but pip install throws gcc: error trying to exec 'cc1': execvp: No such file or directory — or worse, your app crashes at runtime because a shared library is missing. Installing system dependencies in Docker is not as simple as running apt-get install in your base image; every choice adds weight, risk, and complexity. This lesson turns that pain into a repeatable skill — you'll learn where, when, and how to install system packages so your image stays lean, fast, and reliable.

The problem this lesson solves

Docker images are supposed to be minimal, but real applications need system dependencies: compilers, native libraries, database clients, or debugging tools. The default python:3.11-slim image omits many of them to stay small. If you blindly add packages, you bloat the image, increase the attack surface, and waste bandwidth.

Pro tip: Every unnecessary package is a potential vulnerability. A typical apt-get install that you forget to clean up can add 200+ MB to your image.

Without a strategy, developers often:

• Install packages in the wrong layer, breaking caching
• Forget to clean up temporary files (/var/lib/apt/lists/*)
• Use apt-get vs apt inconsistently
• Install build-time deps in the final stage, bloating production images

The goal: install exactly what your app needs, in the right place, without leaving leftovers.

Core concept / mental model

Think of your Docker image as a bread recipe. The base image is the flour — plain python:3.11-slim. The system dependencies are the yeast and salt — necessary for the bread to work but easy to over-measure. The Dockerfile instructions are the kneading steps.

Multi-stage builds are like having a prep kitchen and a serving kitchen. You do all the messy mixing (compilation) in the prep stage, then pass only the final cooked bread (binary / library) to the serving stage — leaving the mixing bowls and extra ingredients behind.

Key definitions:

• Build-time dependencies: packages required during image building (compilers, headers, tools like gcc or build-essential).
• Run-time dependencies: packages your app needs when it actually runs (e.g., libpq5 for PostgreSQL, ffmpeg for media).

Your job is to identify which is which and install accordingly.

How it works step by step

Follow this repeatable sequence every time you need a system dependency:

1. Identify the dependency. Check the error message or the library documentation. For Python: psycopg2 needs libpq-dev at build time, libpq5 at run time.

2. Update and install in one RUN command (no separate apt-get update). This prevents caching stale package lists.

3. Use the --no-install-recommends flag to avoid pulling optional packages.

4. Clean the apt cache in the same layer with rm -rf /var/lib/apt/lists/*.

5. Separate build-time and run-time deps with multi-stage builds if possible.

# Stage 1: build-time deps
FROM python:3.11-slim AS builder
RUN apt-get update && apt-get install --no-install-recommends -y \
    build-essential \
    libpq-dev \
    && rm -rf /var/lib/apt/lists/*

# Stage 2: final image
FROM python:3.11-slim
RUN apt-get update && apt-get install --no-install-recommends -y \
    libpq5 \
    && rm -rf /var/lib/apt/lists/*

Pro tip: Group all system deps in one RUN block. Each RUN creates a new layer; you want one layer for all apt operations combined.

Hands-on walkthrough

Let's build a Python app that connects to PostgreSQL using psycopg2-binary. The binary package contains precompiled libraries, but sometimes you need the C extension for performance or compatibility.

Example 1: Minimal apt-get install

FROM python:3.11-slim

# Install runtime deps only
RUN apt-get update && \
    apt-get install --no-install-recommends -y \
        libpq5 \
        ca-certificates \
    && rm -rf /var/lib/apt/lists/*

# Copy and install Python deps
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .
CMD ["python", "app.py"]

docker build -t myapp:latest .
# Layers shown: only one apt layer, minimal size

Example 2: Build-time deps via multi-stage

# Build stage
FROM python:3.11-slim AS builder

RUN apt-get update && apt-get install --no-install-recommends -y \
        build-essential \
        libpq-dev \
    && rm -rf /var/lib/apt/lists/*

COPY requirements.txt .
RUN pip install --user --no-cache-dir -r requirements.txt

# Final stage
FROM python:3.11-slim AS final

RUN apt-get update && apt-get install --no-install-recommends -y \
        libpq5 \
    && rm -rf /var/lib/apt/lists/*

COPY --from=builder /root/.local /root/.local
ENV PATH=/root/.local/bin:$PATH

WORKDIR /app
COPY . .
CMD ["python", "app.py"]

docker build -t myapp:multistage .
docker images  # Compare size: multistage should be 100-200MB smaller

Example 3: Installing dependencies from a requirements file that includes system packages

Sometimes you need ffmpeg for audio processing in Flask. Use RUN with pip and apt together — but keep them in separate RUN blocks to allow pip's cache independence.

FROM python:3.11-slim

# System deps first
RUN apt-get update && apt-get install --no-install-recommends -y \
        ffmpeg \
        libavcodec-extra \
    && rm -rf /var/lib/apt/lists/*

# Python deps
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY app.py .
CMD ["python", "app.py"]

# Build and run
$ docker build -t audio-app .
$ docker run audio-app python -c "import subprocess; print(subprocess.run(['ffmpeg', '-version'], capture_output=True, text=True).stdout)"
# Output: ffmpeg version 5.1.2 ...

Compare options / when to choose what

Rule of thumb: If you only need run-time deps, stay single-stage. If you need compilers or headers, use multi-stage.

Troubleshooting & edge cases

"Package not found" when using slim images

slim images have minimal repositories. You may need to add --no-install-recommends — but if the package is truly missing, consider switching to python:3.11-buster or a full buildpack-deps variant.

Layer caching breaks your installation

If you run apt-get update in its own RUN block, Docker caches it. Then later when you change requirements.txt, apt won't re-run. Always combine update + install + clean in one RUN.

Error: /var/lib/apt/lists/lock or Could not get lock /var/lib/dpkg/lock

This only happens if you run apt-get during image build on a host that uses the same APT database — unlikely. If it happens, check that you aren't running multiple builds simultaneously.

build-essential adds 200MB+ even after cleaning

build-essential pulls many transient packages. Use multi-stage and copy only the compiled artifact (like a .so from pip install --user).

What you learned & what's next

You can now:

• Explain why and where to install system dependencies in Docker
• Write a single RUN command that updates, installs, and cleans in one layer
• Separate build-time and run-time deps using multi-stage builds
• Avoid common pitfalls like caching problems and unnecessary bloat

Next lesson: Manage environment variables in Docker — you'll inject secrets and configs without leaking them into the image.

Ready to shrink your next Dockerfile? Try refactoring an existing Dockerfile that uses apt-get install in multiple RUN blocks into a single, optimized layer.