Docker Build Cache

You have the Dockerfile written. You run docker build and then wait... and wait. Every build takes minutes, even when you only changed a single line of code. In a team setting, those minutes compound into hours drained from your productivity. The culprit? A cold build cache. Mastering how to use Docker build cache efficiently is the key to cutting waste and keeping your iteration loop tight. Let's turn that waiting into a five-second refresh.

The problem this lesson solves

Docker builds every layer of an image from scratch unless you tell it not to. Without cache awareness:

• Redundant installations: Your Dockerfile re-downloads and compiles the same dependencies every single time.
• Slow feedback loops: Developers push code, wait minutes for CI to rebuild, then discover a typo.
• Blocked workflow: A misplaced line at the top of a Dockerfile can invalidate cache for the entire rest of the build.

The result? Frustrated developers and wasteful resource consumption. The solution is learning how to use Docker build cache efficiently to reuse unchanged layers.

Core concept / mental model

Think of a Docker image as a stack of frozen screenshots (layers). Each command in your Dockerfile (RUN, COPY, CMD) creates a new layer on top of the previous one. When you build a new image, Docker checks if any layer can be reused from the previous build.

Cache hit vs. cache miss

• Cache hit: If the instruction text and all its dependencies (like file contents) are identical, Docker uses the existing layer — instant skip.
• Cache miss: If anything changed in that layer, Docker rebuilds it and every subsequent layer.

Here's the mental rule: order your Dockerfile from least-likely-to-change to most-likely-to-change. That way, cache is rarely invalidated after a small code change.

The layer caching rules

Docker checks these in order for each instruction: 1. Base image 2. Instruction text (RUN pip install flask==2.3 vs RUN pip install flask==2.4) 3. For COPY and ADD: the hash of the file contents (not timestamps) 4. Build arguments (if any)

If any of these differ from the previous build, you get a cache miss — even on the same line.

How it works step by step

Here’s the exact sequence Docker follows when you run docker build with the cache:

1. Fetch base image — Docker checks if the base image (e.g., python:3.11-slim) is already present locally. If yes, cache hit. If no, pull it.
2. Walk the Dockerfile line by line — For each RUN, COPY, or ADD, Docker computes a cache key (instruction + file hashes).
3. Compare against the build cache — If a match is found, the existing layer is tagged and reused.
4. On first mismatch — Docker rebuilds that layer and all following layers, creating a new cache entry.
5. Final image overlay — The newly built layer stack becomes the final image.

The critical insight: cache invalidation propagates forward only. If layer 3 changes, layer 4, 5, and 6 must be rebuilt, but layers 1 and 2 remain cached.

Order of operations matters

Hands-on walkthrough

Let's build a real project and see the difference. Create a simple Python app with this Dockerfile structure:

# BAD ORDER — invalidates cache too early
FROM python:3.11-slim
WORKDIR /app
COPY . .                    # Any file change invalidates everything
RUN pip install -r requirements.txt
EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0"]

Now build it once:

docker build -t myapp:v1 .

Notice the build output — it installs all packages from scratch. Now change a single line in app/main.py and rebuild:

docker build -t myapp:v2 .

You'll see: CACHED for the first layer (base image), then RUN pip install ... runs again — because the COPY . . layer changed. Wasteful.

Fixing the Dockerfile

Reorder to leverage cache:

# GOOD ORDER — cache-friendly
FROM python:3.11-slim
WORKDIR /app

# Install dependencies first (they change rarely)
COPY requirements.txt .
RUN pip install -r requirements.txt

# Then copy the rest (changes frequently)
COPY . .

EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0"]

Build once, then change app/main.py and rebuild:

docker build -t myapp:v3 .

Now only the COPY . . and CMD layers rebuild. The RUN pip install ... layer is cached. Build time drops from 45 seconds to under 2 seconds.

Verify with debug mode

Use buildx for a more detailed view:

DOCKER_BUILDKIT=1 docker build --no-cache-filter=run-code .

This skips cache only for layers after a certain point.

Compare options / when to choose what

For most local development, simply reordering the Dockerfile is 80% of the benefit.

When to avoid heavy reordering

• If your Dockerfile has a single RUN that installs everything together with the code, a refactor into two steps might not work (e.g., npm ci && npx next build combined). In that case, separate explicitly: use a package.json copy before the build step.

Troubleshooting & edge cases

"Cache busted even though I didn't change that line"

Check for: - Build arguments that differ (--build-arg) invalidate all layers that reference them. - Base image tags — python:latest can change silently. - File timestamps across host machines (Docker uses content hash, not timestamps, but some tools like npm install produce different folder structures).

BuildKit doesn't show "CACHED" in plain output

Run with --progress=plain to see full layer IDs and cache status:

docker build --progress=plain -t myapp:v4 .

"My CI always builds from scratch"

Use --cache-from pointing to a remote image that you push after each main branch build:

docker build --cache-from myapp:latest -t myapp:$COMMIT .

Common early mistakes

• Putting COPY . . before RUN pip install — reorders builds from scratch
• Using ADD instead of COPY for local files — ADD not only copies but also resolves URLs, which can change cache keys unpredictably
• Forgetting to add .dockerignore — sending the whole project (including large node_modules or __pycache__) to the daemon, which may cause a spurious cache miss if the folder changes

What you learned & what's next

You now understand how to use Docker build cache efficiently by: - Explaining the layer-based caching mental model - Ordering your Dockerfile from least to most volatile changes - Analyzing build output to spot cache misses - Applying caches in CI/CD with --cache-from

This skill directly reduces build times for every project you containerize. In the next lesson, we'll build on this foundation by exploring multi-stage builds, which combine cache efficiency with final-image size optimization — making your containers both fast to build and small to ship.

Pro tip: Add a comment at the top of every Dockerfile: "ORDER LINES BY CACHE STABILITY."