StatefulSet for Stateful Apps

Stateless applications are easy to scale: you just add more replicas and route traffic to any of them. But what about databases, caches, or any application that needs to remember its data across restarts? A regular Kubernetes Deployment will happily replace a crashed Pod, but it gives each new Pod a random name and no guarantee of stable storage. That's where StatefulSets come in. They provide ordered, predictable Pods with stable identities for your stateful workloads, making them the correct choice for running databases like PostgreSQL or Redis in production.

The problem this lesson solves

When you use a Deployment to run an application like a MySQL database, every time a Pod restarts or gets rescheduled it gets a new hostname and a new volume. Clients that connected to mysql-0 cannot find it anymore because the new Pod is called something else. Worse, if a Pod moves to a different node you may lose its attached storage entirely, corrupting your data.

Stateful applications need three guarantees that Deployments do not provide:

• Stable network identity — each Pod gets a predictable hostname that survives restarts
• Stable, persistent storage — volumes are not deleted when the Pod is deleted
• Ordered, graceful deployment and scaling — Pods are created one at a time in sequence

StatefulSet is the Kubernetes workload API that delivers these guarantees.

Core concept / mental model

Think of a StatefulSet as an orchestra of named musicians. Each musician has a named seat (violin-0, violin-1, cello-0) and their own sheet music (storage). If a musician leaves, the orchestra keeps an empty chair with the same name and music ready for the next musician who sits there. Compare that to a Deployment, which is more like a bucket of identical balls — when you pull one out and replace it, you cannot tell which ball is which.

A StatefulSet maintains a sticky identity for each Pod via:

• Ordinal index — Pods are named <statefulset-name>-<ordinal> (e.g., my-db-0, my-db-1)
• Stable network identities — each Pod gets a StatefulSet-managed headless Service that maps each ordinal to a specific hostname
• Stable storage — each Pod is bound to its own PersistentVolumeClaim (PVC) that persists across reschedules

Pro tip: StatefulSets use a headless Service (ClusterIP: None) to let you address each Pod by its stable hostname: my-db-0.mydb-service.default.svc.cluster.local.

How it works step by step

1. Define a headless Service

A headless Service has clusterIP: None. It creates DNS records for each Pod so you can reach it directly by name.

apiVersion: v1
kind: Service
metadata:
  name: mydb
spec:
  clusterIP: None
  selector:
    app: mydb
  ports:
  - name: db
    port: 5432

2. Create the StatefulSet manifest

A StatefulSet looks similar to a Deployment but adds serviceName and a volumeClaimTemplates block.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mydb
spec:
  serviceName: mydb
  replicas: 3
  selector:
    matchLabels:
      app: mydb
  template:
    metadata:
      labels:
        app: mydb
    spec:
      containers:
      - name: postgres
        image: postgres:15
        ports:
        - containerPort: 5432
        volumeMounts:
        - name: data
          mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 5Gi

3. Understand the creation order

Kubernetes creates Pods in order from ordinal 0 to N-1. Each Pod must be Running and Ready before the next one starts.

• mydb-0 is created → becomes Ready → mydb-1 starts
• If mydb-0 fails, mydb-1 is not affected but mydb-2 will not be created until mydb-1 is ready

4. Scaling down is also ordered (reverse)

When you scale down, Pods are terminated from highest to lowest ordinal, one at a time. This ensures that mydb-0 (the primary, in many database setups) is the last to be removed.

Hands-on walkthrough

Let's run a real stateful application: a 3-replica PostgreSQL cluster using a StatefulSet. You will need a Kubernetes cluster (minikube or kind works).

Step 1: Apply the headless Service and StatefulSet

Save the manifests above as mydb-service.yaml and mydb-statefulset.yaml, then apply them.

kubectl apply -f mydb-service.yaml
kubectl apply -f mydb-statefulset.yaml

Step 2: Watch Pods come up in order

kubectl get pods -w

Expected output:

NAME    READY   STATUS    RESTARTS   AGE
mydb-0  0/1     Pending   0          1s
mydb-0  1/1     Running   0          8s
mydb-1  0/1     Pending   0          0s
mydb-1  1/1     Running   0          7s
mydb-2  0/1     Pending   0          0s
mydb-2  1/1     Running   0          6s

Step 3: Verify stable network identities

kubectl run -it --rm busybox --image=busybox -- /bin/sh

Inside the pod, resolve one of the names:

nslookup mydb-0.mydb

You will see the Pod's IP and the full DNS name.

Step 4: Check persistent storage

When you delete a Pod, the PVC remains and the new Pod reuses it.

kubectl delete pod mydb-0
kubectl get pvc

Expected output:

NAME               STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS
mydb-data-mydb-0   Bound    pvc-xxxxxxxxxxxx                           5Gi        RWO            standard

Compare options / when to choose what

When to choose StatefulSet:

• You run a database, cache, or queue that needs a stable hostname per replica
• Each Pod requires its own persistent volume (e.g., database data directories)
• You need controlled rollouts, e.g., upgrade the primary last

When to stick with Deployment:

• Your application is stateless and can tolerate random hostnames
• You want fast, parallel scaling
• You do not need per-Pod persistent volumes

Pro tip: For most applications in Kubernetes, start with a Deployment. Only reach for a StatefulSet when you have a concrete need for stable identity or per-Pod storage — they add operational complexity.

Troubleshooting & edge cases

Pod stuck in Pending

Probably a PersistentVolumeClaim that cannot be bound. Check:

kubectl get pvc
kubectl describe pvc mydb-data-mydb-0

Make sure your cluster has a StorageClass that can provision volumes.

Pod fails to start after PVC reuse

If you delete a Pod and the new one fails with FailedMount, the old volume might still be attached to the previous node. In that case, use:

kubectl delete pvc mydb-data-mydb-0

But be careful — this deletes the data! Better to wait or force detach the volume from the previous node.

Services cannot resolve Pod hostnames

Ensure the Service is headless (clusterIP: None) and the StatefulSet's serviceName matches the Service name. Also, kube-dns or CoreDNS must be running.

Scale down with data loss

Scaling down a StatefulSet does not delete PVCs by default. You must manually delete the PVC to release the storage. This is intentional to prevent accidental data loss, but it means you must monitor orphaned PVCs.

What you learned & what's next

You now know:

• The core idea: StatefulSets give stable identity and per-Pod storage to stateful workloads
• How to run a stateful application with StatefulSet using a headless Service and volumeClaimTemplates
• Ordered operations: Pods start and stop in sequence to support database bootstrapping
• When to choose StatefulSet over Deployment for databases, caches, and queues

Next, you should explore Headless Services in depth, then PersistentVolumeClaims retention policies (whenDeleted, whenScaled) to control what happens to storage when you scale down. Then tackle rolling updates for StatefulSets — they behave differently than Deployments.

You are ready to run stateful workloads safely in Kubernetes.