Multi-Service Stacks with Compose

You've mastered single-container Dockerfiles and maybe even a two-service docker-compose.yml. But real applications — an API backed by a database, a message queue, a cache, and a frontend — require you to orchestrate multi-service stacks with Compose as a single, repeatable unit. Without this skill, you'll spend your days manually starting containers in the right order, configuring networks by hand, and wrestling with environment variables across services. Docker Compose does all that for you — this lesson shows you how.

The problem this lesson solves

Running multiple containers by hand is error-prone, slow, and impossible to share with your team. Consider a typical web app stack: an Express API, a PostgreSQL database, and a Redis cache. You need to:

• Start the database before the API (depends_on).
• Connect containers on the same network so they can talk.
• Pass environment variables like database URLs and secret keys without embedding them in images.
• Scale the API to multiple replicas.
• Expose only the API port to the outside world.

Without Compose, you'd write a shell script that chains docker run commands — brittle, hard to debug, and opaque to newcomers. With Compose, you declare the entire stack in a compose.yaml file, then docker compose up brings everything to life. This is orchestration at the developer level — not Kubernetes scale, but exactly the power you need for local development, CI, and small production deployments.

Core concept / mental model

Think of Docker Compose as a declarative blueprint for your multi-service application. Instead of giving Docker a series of step-by-step instructions (imperative), you tell it what the final state should look like: "Run an API container on port 3000, a Postgres container on port 5432, both on a shared network called backend." Docker Compose figures out the order and dependencies.

Definitions

• Service: A container definition in Compose — corresponds to a docker run ... command. Each service has an image (or build context), ports, environment variables, volumes, etc.
• Network: An isolated L2 segment connecting services by service name. No need to use IP addresses — just http://api:3000 from the frontend.
• Volume: Persistent storage shared among containers or mounted from the host.
• Profile: Conditional services you can enable with --profile profilename.

Diagram in words

┌──────────────────────────────────────┐
│            compose.yaml              │
│  services:                           │
│    api:  build: ./api  ports: 3000   │
│    db:   image: postgres:16 volumes  │
│    redis: image: redis:7-alpine      │
│  networks:                           │
│    backend:                           │
└───────────────────────────┬──────────┘
                            │
                            ▼
┌──────────────────────────────────────────┐
│  docker compose up  →  3 containers     │
│  + auto network + volumes               │
│  + start order                          │
└──────────────────────────────────────────┘

How it works step by step

1. Define services in a compose.yaml file (or docker-compose.yaml). Each service gets a name, image or build context, ports, environment, volumes, and networks.
2. Specify dependencies using depends_on. Compose will order startup accordingly, but does not wait for the service to be ready inside the container (that's your health check).
3. Create networks automatically. By default, Compose creates one network per stack — all services can reach each other by service name.
4. Mount volumes for persistence or config. Named volumes (e.g., db-data) survive container restarts. Bind mounts (e.g., ./src:/app) are great for development.
5. Set environment variables in the file or from an .env file. Never hardcode secrets in images.
6. Run the stack with docker compose up. Add -d to detach, --build to rebuild images.
7. Scale services with docker compose up -d --scale api=3 — Compose creates three replicas of the API service.

Pro tip: Use depends_on only for startup order. For readiness, implement a health check (healthcheck section) in the service definition.

Hands-on walkthrough

Let's build a three-service stack: a Python Flask API, a PostgreSQL database, and a Redis cache. The API will increment a counter in Redis and store the logged requests in Postgres.

Step 1: Project structure

myapp/
├── compose.yaml
├── api/
│   ├── Dockerfile
│   ├── requirements.txt
│   └── app.py
└── .env

Step 2: compose.yaml

services:
  api:
    build: ./api
    ports:
      - "5000:5000"
    environment:
      DATABASE_URL: "postgresql://user:pass@db/mydb"
      REDIS_URL: "redis://redis:6379"
    depends_on:
      - db
      - redis
    networks:
      - backend

  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: user
      POSTGRES_PASSWORD: pass
      POSTGRES_DB: mydb
    volumes:
      - pgdata:/var/lib/postgresql/data
    networks:
      - backend

  redis:
    image: redis:7-alpine
    networks:
      - backend

volumes:
  pgdata:

networks:
  backend:

Step 3: The API code (api/app.py)

from flask import Flask, jsonify
import redis
import psycopg2
import os

app = Flask(__name__)
r = redis.Redis.from_url(os.environ["REDIS_URL"])
db_conn = psycopg2.connect(os.environ["DATABASE_URL"])

@app.route("/")
def index():
    # Increment the request counter in Redis
    count = r.incr("hits")
    # Log to Postgres
    cur = db_conn.cursor()
    cur.execute("INSERT INTO hits (timestamp) VALUES (NOW())")
    db_conn.commit()
    cur.close()
    return jsonify({"message": f"Hit #{count} logged"})

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000)

Step 4: Build and run

cd myapp
docker compose up --build

Expected output (trimmed):

[+] Building 2.3s (10/10) FINISHED
[+] Running 4/4
 ✔ Network myapp_backend   Created
 ✔ Volume myapp_pgdata     Created
 ✔ Container myapp-db-1    Started
 ✔ Container myapp-redis-1 Started
 ✔ Container myapp-api-1   Started

Now visit http://localhost:5000. Each request increments the counter — you've orchestrated a multi-service stack.

Compare options / when to choose what

Compromise: Start with Compose. You can later convert the same YAML to a Stacks file for Swarm or use kompose to convert to K8s manifests.

Troubleshooting & edge cases

1. Container can't reach another service by hostname

Symptom: Name or service not known inside the container. Fix: Ensure both services are on the same network. Check compose.yaml — if you define custom networks, list them under both services. Use docker compose exec api ping db to test.

2. Depends_on not waiting for DB to be ready

Symptom: API starts before Postgres is ready to accept connections, causing connection errors. Fix: Use a wait script inside the API's Dockerfile or add a health check and use condition: service_healthy:

services:
  api:
    depends_on:
      db:
        condition: service_healthy
  db:
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U user -d mydb"]
      interval: 5s
      timeout: 5s
      retries: 5

3. Ports already in use

Symptom: Error response from daemon: driver failed programming external connectivity on endpoint ... bind: address already in use. Fix: Change the host port mapping (e.g., "5001:5000") or stop the conflicting container (docker stop <container>).

4. Named volume persists old data

Symptom: Starting fresh stack still has old database records. Fix: Remove the volume with docker compose down -v. This destroys all named volumes, so use with care.

What you learned & what's next

You now understand how to orchestrate multi-service stacks with Compose: how to define services, networks, volumes, and dependencies in a single declarative file. You built a real stack — Flask + Postgres + Redis — and saw how services discover each other by name, start in the right order, and share persistent volumes.

Key takeaways for today: - Docker Compose replaces brittle shell scripts with a declarative blueprint. - Every service in the same default network can reach others by service name. - Use depends_on with health checks for reliable startup ordering. - Volumes keep data alive across container restarts.

Next lesson: We'll explore Docker Compose profiles — how to spin up optional services (like a test database or debug container) without bloating the main stack. That's a critical step toward production-grade Compose files.