Use Docker Compose

Manually wiring a web app to a database, a Redis cache, and a background worker with raw docker run commands quickly becomes a nightmare. Every container needs the right network, environment variables, and startup order — one typo and everything breaks. That's exactly why Docker Compose exists: to declare every piece of your multi-container app in a single, version-controlled YAML file and spin it all up with one command.

The problem this lesson solves

Running a modern application — say a Python web API backed by PostgreSQL and a Redis queue — with individual docker run calls is brittle and hard to reproduce. You must manually create networks (docker network create myapp_net), link containers, pass environment variables on each --env flag, and maintain a mental map of startup order. A colleague who clones your repo needs a wiki page or, worse, tribal knowledge to get the app running. Docker Compose eliminates this pain by letting you define the entire application stack in a declarative YAML file (compose.yaml) and manage it as a single unit.

Core concept / mental model

Think of Docker Compose as an application-level orchestrator for a single host. The mental model has three layers:

Services

A service is a container whose configuration you define — the image, ports, environment variables, volumes, and dependencies. In Compose, you don't run a PostgreSQL container; you run a PostgreSQL service that happens to be backed by a container.

Networks

Compose automatically creates a default network for all services in the same file. Services can reach each other by their service name (which acts as DNS). No need to --link or manually specify IPs.

Volumes

Named volumes are declared at the top level and mounted into services. Compose ensures volumes persist across up/down cycles unless you deliberately remove them.

Pro tip: If you've ever used Kubernetes, Compose is like a single-node, developer-friendly subset. You describe the desired state, and Compose reconciles reality to match.

How it works step by step

Here's the flow from zero to running multi-container app:

1. Create a compose.yaml file (or docker-compose.yaml) in your project root.
2. Define services under the services: key. Each service gets a name, an image: (or a build: context), and configuration like ports:, environment:, and volumes:.
3. Declare dependencies with the depends_on: key to control startup order.
4. Add networks (optional) if you need more than one isolated network.
5. Run docker compose up -d — Compose creates networks, pulls/builds images, starts containers in dependency order.
6. Inspect with docker compose ps or docker compose logs -f.
7. Tear down with docker compose down (with -v to remove volumes).

Hands-on walkthrough

Let's build a three-service application: a Python web server, a PostgreSQL database, and a Redis cache.

Project structure

myapp/
├── app/
│   └── server.py
├── Dockerfile
└── compose.yaml

Example 1: Minimal web + database

# compose.yaml
services:
  web:
    build: .
    ports:
      - "8000:8000"
    environment:
      - DATABASE_URL=postgresql://user:pass@db/mydb
    depends_on:
      - db
  db:
    image: postgres:16
    environment:
      POSTGRES_USER: user
      POSTGRES_PASSWORD: pass
      POSTGRES_DB: mydb
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:

Run it:

docker compose up -d

Example 2: Adding Redis and a worker

services:
  web:
    build: .
    ports:
      - "8000:8000"
    environment:
      - DATABASE_URL=postgresql://user:pass@db/mydb
      - REDIS_URL=redis://cache:6379
    depends_on:
      - db
      - cache
  worker:
    build: .
    command: python worker.py
    environment:
      - DATABASE_URL=postgresql://user:pass@db/mydb
      - REDIS_URL=redis://cache:6379
    depends_on:
      - db
      - cache
  db:
    image: postgres:16
    environment:
      POSTGRES_USER: user
      POSTGRES_PASSWORD: pass
      POSTGRES_DB: mydb
    volumes:
      - pgdata:/var/lib/postgresql/data
  cache:
    image: redis:7-alpine

volumes:
  pgdata:

Expected output when running docker compose up -d:

[+] Running 5/5
 ✔ Network myapp_default      Created
 ✔ Volume myapp_pgdata        Created
 ✔ Container myapp-cache-1    Started
 ✔ Container myapp-db-1       Started
 ✔ Container myapp-web-1      Started
 ✔ Container myapp-worker-1   Started

Example 3: Using environment variables from a file

services:
  web:
    build: .
    env_file:
      - .env
    ports:
      - "8000:8000"

With a .env file:

DATABASE_URL=postgresql://prod_user:securepass@prod-db.internal/mydb
SECRET_KEY=supersecret

Pro tip: Never commit .env files to version control. Use .env.example as a template.

Compare options / when to choose what

Choose Docker Compose when your app needs two or more containers that communicate, and you want a reproducible, developer-friendly setup. Skip it for single-container apps (just use docker run) or production clusters (use K8s).

Troubleshooting & edge cases

Service not reachable by name

• Cause: Containers on different networks.
• Fix: Ensure both services are in the same Compose file and network. Don't override networks: unless necessary.

depends_on doesn't wait for database readiness

• Cause: depends_on only waits for container start, not for the process inside to be ready.
• Fix: Use a healthcheck and condition: service_healthy (Compose 2.1+).

services:
  db:
    image: postgres:16
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U user -d mydb"]
      interval: 5s
      timeout: 5s
      retries: 5
  web:
    depends_on:
      db:
        condition: service_healthy

Port conflicts

• Symptom: Error starting userland proxy: listen tcp4 0.0.0.0:5432: bind: address already in use
• Fix: Change the host port mapping (e.g., "5433:5432") or stop the conflicting container.

Volume permissions in Linux

• Issue: Containers write files as root; host user can't read them.
• Fix: Set user ID in the container via user: "1000:1000" (match your host UID).

What you learned & what's next

You now know how to use Docker Compose to define multi‑container apps — declare services, networks, and volumes in a compose.yaml file, manage startup order with depends_on, and run the whole stack with a single command. You saw hands-on examples with a web server, database, cache, and worker, and you learned when Compose beats raw docker run or Kubernetes.

Next step: Connect Docker Compose to Your CI/CD Pipeline — learn how to integrate Compose-based tests into GitHub Actions or GitLab CI.

Key takeaways from this lesson

• ✅ Docker Compose defines multi‑container apps declaratively in YAML.
• ✅ Services communicate over a default network using service names as DNS.
• ✅ depends_on controls startup order; use healthchecks for readiness.
• ✅ Volumes declared at the top level persist data across container restarts.
• ✅ Use docker compose up -d to start and docker compose down -v to clean up.
• ✅ Compose is ideal for local dev and CI, not for production multi-node clusters.