How Docker Works: Namespaces, Cgroups, and the Linux Kernel

Stop thinking of Docker containers as "lightweight virtual machines." While they both isolate applications, calling a container a VM is like calling a motorcycle a "small car"—they both get you from A to B, but the engine under the hood is fundamentally different.

To understand how Docker works, you have to look past the CLI and into the Linux kernel. Docker isn’t actually a "thing" that runs your code; it is a manager that orchestrates several existing Linux features to create an illusion of isolation.

The Core Secret: Shared Kernels

The biggest difference between a VM and a container is the kernel. A Virtual Machine includes a full copy of an operating system, including its own kernel, which runs on top of a hypervisor. This means every VM consumes a massive amount of RAM and CPU just to keep the OS alive.

Docker containers do not have their own kernel. Instead, they share the host machine's Linux kernel.

A container is essentially just a process running on your host OS, but it's been "tricked" into thinking it's running in its own private environment. This is why Docker containers start in milliseconds and use a fraction of the resources a VM would.

The "Magic" Ingredients: Namespaces and Cgroups

If containers share the same kernel, what stops one container from seeing the files or killing the processes of another? This is where two Linux kernel features come in: Namespaces and Control Groups (cgroups).

1. Namespaces (The "Blinders")

Namespaces provide the isolation. Think of them as blinders put on a horse; the process can only see what the namespace allows it to see. Docker uses several types of namespaces:

• PID Namespace: This ensures the container has its own process tree. Inside the container, your app thinks it is "Process ID 1," even though on the host machine, it might be Process ID 4521.
• NET Namespace: This gives the container its own virtual network stack, IP address, and routing table.
• MNT Namespace: This isolates the filesystem. The container thinks its root directory (/) is its own, separate from the host's root.
• UTS Namespace: This allows the container to have its own hostname.

2. Control Groups (The "Budget")

Isolation isn't enough. If one container starts a recursive loop and consumes 100% of the CPU, it could crash the entire host and every other container on it.

Control Groups (cgroups) handle resource management. They act as a governor, allowing Docker to set limits on: * CPU usage (e.g., "this container can only use 0.5 cores"). * Memory (e.g., "limit this container to 512MB of RAM"). * Disk I/O and network bandwidth.

The Layered Filesystem: UnionFS

If every container had a full copy of an OS image, your hard drive would fill up instantly. Docker solves this using a Union File System (UnionFS) and a concept called Copy-on-Write (CoW).

Docker images are built in layers. If you have five different containers all based on ubuntu:22.04, Docker doesn't store that Ubuntu image five times. It stores it once as a series of read-only layers.

When you start a container, Docker adds a thin, writable layer on top of those read-only layers.

• Reading: If the container needs a file, it looks through the layers from top to bottom until it finds it.
• Writing: If the container needs to modify a file existing in a read-only layer, Docker copies that file up to the writable layer first, then modifies it. This is "Copy-on-Write."

This architecture is why you can spin up ten containers from the same image in seconds without duplicating gigabytes of data.

Putting it All Together: The Workflow

When you run docker run python-app, here is what happens behind the scenes:

1. The Docker Daemon pulls the image layers from a registry if they aren't local.
2. The Container Engine creates a new set of Namespaces, isolating the process from the host.
3. Cgroups are applied to ensure the container doesn't hog all the system RAM.
4. UnionFS stacks the read-only image layers and adds a writable layer on top.
5. The Kernel starts the application process inside these boundaries.

The result is a portable, predictable environment that behaves like a separate machine but performs with the speed of a native process.