Manage Headless Services StatefulSets

When you deploy a StatefulSet in Kubernetes — say a PostgreSQL cluster, a Cassandra ring, or a ZooKeeper ensemble — every Pod needs a stable, predictable network identity. A regular ClusterIP or NodePort Service won't cut it: they load-balanced across Pods, masking which Pod you're talking to. That's where headless services come in. A headless service gives each Pod its own DNS A or AAAA record, so you can address pod-0, pod-1, pod-2 directly. This lesson shows you how to create and manage headless services for StatefulSets, making stateful workloads reliable and debuggable.

The problem this lesson solves

StatefulSets enforce ordinality — Pods are created, scaled, and updated in a fixed order (0, 1, 2, …). A regular Service would return the IP of any ready Pod, which breaks scenarios where you need to reach a specific instance. For example:

• A database cluster elects a leader that must be Pod-0.
• A replicated queue needs each consumer to connect to its assigned shard.
• A monitoring system scrapes each Pod individually.

Without a headless service, you'd have to hardcode Pod IPs or rely on external service discovery — both are brittle. The headless service solves this by removing the load-balancer and letting DNS resolve to all Pod IPs (or to individual Pod DNS names).

Core concept / mental model

Think of a headless service as a service with clusterIP: None. Normally, a Service allocates a virtual IP (the ClusterIP) and acts as a proxy. Headless means:

• No ClusterIP is assigned.
• kube-proxy does not create routing rules.
• DNS returns A records for every Pod endpoint (instead of a single ClusterIP).
• For StatefulSets, each Pod also gets a DNS entry like <pod-name>.<service-name>.<namespace>.svc.cluster.local.

Blockquote pro tip: A headless service does not provide load-balancing. It exposes each Pod's identity directly. That's exactly what stateful workloads need.

Key vocabulary

• StatefulSet — workload controller for stateful applications; Pods have stable identity and storage.
• Headless Service — a service without ClusterIP; enables direct Pod DNS.
• Ordinal index — Pods are named <statefulset-name>-<ordinal>, e.g., web-0, web-1.
• SRV record — DNS record that maps a service to host and port; headless services can publish SRV records too.

How it works step by step

1. Define the StatefulSet — specify the serviceName field to point to the headless service.
2. Create the headless Service — set clusterIP: None in the Service YAML.
3. DNS resolution flow: - For a regular service, nslookup myservice returns one IP (the ClusterIP). - For a headless service, nslookup myheadless returns all Pod IPs. - For individual Pods, nslookup pod-0.myheadless returns that specific Pod's IP.
4. Pod creation order — StatefulSet creates Pods in order 0 → 1 → 2 …, each Pod registers its IP with the headless service's endpoint set.
5. Stable network identity — if a Pod restarts, it keeps its name and gets the same DNS record.

Visual flow

User/App
  │
  ▼
DNS Query: pod-0.myheadless.default.svc.cluster.local
  │
  ▼
KubeDNS returns IP of Pod-0 (e.g., 10.1.0.10)
  │
  ▼
Direct connection to Pod-0 — no load-balancer

Hands-on walkthrough

Let's build a simple example — a 3-replica Nginx StatefulSet with a headless service.

Step 1: Create the headless service

# headless-svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: nginx-headless
spec:
  clusterIP: None
  selector:
    app: nginx
  ports:
    - port: 80
      targetPort: 80

Apply it:

kubectl apply -f headless-svc.yaml

Verify it's headless:

kubectl get svc nginx-headless

# Expected output:
# NAME             TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
# nginx-headless   ClusterIP   None         <none>        80/TCP    10s

Notice CLUSTER-IP is None.

Step 2: Create the StatefulSet

# sts-nginx.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web
spec:
  serviceName: "nginx-headless"
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:alpine
        ports:
        - containerPort: 80

Apply it:

kubectl apply -f sts-nginx.yaml

Watch Pods come up:

kubectl get pods -l app=nginx -w

# Expected output (abbreviated):
# NAME    READY   STATUS    RESTARTS   AGE
# web-0   1/1     Running   0          10s
# web-1   1/1     Running   0          8s
# web-2   1/1     Running   0          5s

Step 3: Test DNS resolution

Run a temporary DNS debug Pod:

kubectl run -it --rm dnsutils --image=registry.k8s.io/e2e-test-images/jessie-dnsutils:1.3 -- /bin/bash

Inside the Pod, run:

# Query headless service — returns all Pod IPs
nslookup nginx-headless

# Expected output (example):
# Server:    10.96.0.10
# Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
#
# Name:      nginx-headless.default.svc.cluster.local
# Address 1: 10.1.0.10 web-0.nginx-headless.default.svc.cluster.local
# Address 2: 10.1.0.11 web-1.nginx-headless.default.svc.cluster.local
# Address 3: 10.1.0.12 web-2.nginx-headless.default.svc.cluster.local

Now query a specific Pod:

nslookup web-1.nginx-headless

# Expected output:
# Name:    web-1.nginx-headless.default.svc.cluster.local
# Address: 10.1.0.11

Blockquote pro tip: You can curl directly to a Pod by name: curl http://web-1.nginx-headless:80 — no need to look up an IP.

Compare options / when to choose what

When to choose headless:

• You need stable network identity per Pod (databases, queues, caches).
• Your application performs its own leader election or shard discovery.
• You deploy StatefulSets that must scale without breaking client connections.

When to use regular Service:

• Stateless applications behind a load balancer.
• You don't care which Pod responds.
• You need a stable ClusterIP to configure firewalls or external DNS.

Troubleshooting & edge cases

My Pods never get the expected DNS names

Root cause: The StatefulSet's serviceName doesn't match the Service name, or the Service selector doesn't match the Pod labels.

Fix: Verify both:

kubectl get sts web -o jsonpath='{.spec.serviceName}'
kubectl get svc nginx-headless -o jsonpath='{.spec.selector}'
kubectl get pods -l app=nginx    # Should show your Pods

DNS returns no records for headless service

Possible causes:

1. Pods are not ready — headless service only publishes ready Pods.
2. CoreDNS is misconfigured.
3. Network policies block DNS traffic.

Check readiness:

kubectl describe pods web-0 | grep Conditions

If Ready is False, check the Pod's readiness probe or container logs.

Headless service with multiple ports and SRV records

If you define multiple ports, DNS returns SRV records. Example:

# service with multiple ports
ports:
  - name: http
    port: 80
  - name: grpc
    port: 50051

Then query SRV:

dig SRV _http._tcp.nginx-headless.default.svc.cluster.local

Scaling down and orphaned endpoints

When you scale down a StatefulSet (e.g., from 3 to 1), the DNS record for web-1 and web-2 disappears within seconds. However, there can be a small propagation delay in CoreDNS. Use nslookup after scaling to confirm.

What you learned & what's next

In this lesson you learned:

• The problem of stable network identity for stateful workloads and why regular Services fail.
• How a headless service (clusterIP: None) removes load-balancing and exposes per-Pod DNS.
• The step-by-step mechanics: StatefulSet serviceName → headless Service → DNS A/SRV records.
• Hands-on creation of a headless service and StatefulSet, plus DNS validation.
• Troubleshooting missing DNS, readiness issues, and scaling edge cases.

You can now manage headless services for StatefulSets with confidence. Next in the track, we'll explore using StatefulSets with persistent storage — attaching PersistentVolumeClaims to each Pod and handling storage orchestration.

Blockquote pro tip: Before moving on, practice by creating a headless service for a 2-replica Redis StatefulSet and verify you can reach each Pod by name.