How Docker Storage Works: Understanding OverlayFS and Layered File Systems

If you’ve ever wondered how a Docker image can be several hundred megabytes, yet starting a new container from that image takes less than a second, you’ve encountered the magic of the Overlay File System.

Docker doesn’t copy the entire file system every time you launch a container. Doing so would be a nightmare for disk space and performance. Instead, it uses a clever architectural trick called layered storage.

The Core Concept: Immutable Layers

To understand Docker's file system, you have to stop thinking of a container as a virtual machine and start thinking of it as a stack of pancakes.

Every instruction in a Dockerfile (like RUN, COPY, or ADD) creates a new read-only layer. These layers are stacked on top of each other. If you install Python in one layer and a Flask app in the next, Docker doesn't merge them into one file; it simply remembers that the Flask layer sits on top of the Python layer.

Why Immutability Matters

Because these image layers are read-only, they can be shared. If you have ten different containers running the same ubuntu:latest image, they all reference the exact same read-only files on your disk. This is why pulling a new image is so fast if you already have the base layers—Docker only downloads the layers it doesn't already possess.

The "Magic" of the OverlayFS

The actual technology that manages these layers is typically OverlayFS (Overlay File System).

OverlayFS allows Docker to "overlay" multiple directories and present them as a single, unified directory. It manages two main types of layers:

1. LowerDir (The Image Layers): These are the read-only layers that make up the image. You cannot change these.
2. UpperDir (The Container Layer): When you start a container, Docker adds a thin, writable layer on the very top. All changes made while the container is running—creating a file, editing a config, or deleting a log—happen here.

The Copy-on-Write (CoW) Strategy

What happens if you want to edit a file that exists in one of the read-only lower layers? Docker uses a strategy called Copy-on-Write.

Here is the step-by-step process: 1. Docker looks for the file in the writable layer. 2. If it’s not there, it searches through the read-only layers from top to bottom. 3. Once the file is found, Docker copies the file up to the writable layer. 4. The container then modifies the copy in the writable layer.

The original file in the image remains untouched. To the user, it looks like the file was edited, but in reality, the writable layer is simply "shading" the original file.

Handling Data: Volumes and Bind Mounts

If the writable layer is the only place where data persists, you have a problem: When a container is deleted, the writable layer is deleted with it.

For databases, logs, or user uploads, you cannot rely on the OverlayFS. This is where Volumes and Bind Mounts come in.

Docker Volumes

Volumes are managed by Docker and stored in a part of the host file system (/var/lib/docker/volumes/ on Linux). They bypass the OverlayFS entirely. When a container writes to a volume, it writes directly to the host disk, meaning the data persists even if the container is destroyed.

Bind Mounts

Bind mounts are similar to volumes, but you specify the exact path on the host machine (e.g., /home/user/project). This is the gold standard for development, allowing you to edit code in your IDE and see the changes instantly inside the container without rebuilding the image.

Summary Table: Layer vs. Volume

By separating the immutable blueprints (images) from the ephemeral changes (writable layer) and the permanent data (volumes), Docker achieves a balance of speed, efficiency, and portability that traditional VMs simply cannot match.