HorizontalPodAutoscaler Scaling

In the static world of a Kubernetes Deployment, your application runs with a fixed number of replicas — until a traffic spike hits. Suddenly requests are queued, error rates climb, and users complain of slow load times. The problem is not your deployment; it's your capacity plan. You cannot manually adjust replicas every five minutes. HorizontalPodAutoscaler (HPA) solves this: it automatically scales your pods based on observed CPU, memory, or custom metrics, matching supply to demand in real time.

The problem this lesson solves

Without autoscaling, you face two bad options:

• Over-provision: Run 20 pods all day even during quiet hours — wastes money and cluster resources.
• Under-provision: Run 3 pods and hope the spike doesn't come — risks downtime and poor experience.

Manual scaling via kubectl scale works for a demo, but in production your traffic fluctuates unpredictably. You need a controller that watches metrics, compares them to a target, and adjusts replicas automatically without human intervention. This is exactly what HPA does.

Core concept / mental model

Think of HPA as a thermostat for your workload. You set a target temperature (e.g., "keep average CPU at 50%"). The HPA controller continuously reads the current temperature (current metrics via Metrics Server), compares it to your target, and turns the heat up or down — adds or removes pods — to keep the system near the setpoint.

• Target metric: CPU, memory, or custom metric (e.g., requests-per-second, queue length).
• Current value: average across all running pods, collected by the Metrics Server.
• Desired replicas: current_replicas × (current_metric / target_metric). If current is 80% and target is 50%, you need 1 × (80/50) = 1.6 → 2 replicas.

Pro tip: HPA rounds up the desired replicas to the nearest integer using a ceiling function. It also prevents wild fluctuations with a cooldown mechanism (--horizontal-pod-autoscaler-sync-period, default 15 seconds) and optional stabilization window.

How it works step by step

1. Metrics Server collects resource usage from each pod (CPU/memory) and aggregates them per pod and per workload.
2. HPA controller reads metrics via the metrics.k8s.io API (for CPU/memory) or the custom metrics API (for application-level metrics).
3. Controller computes desired replicas using the formula above.
4. If desired replicas differ from current and the change exceeds a configurable tolerance (default 10%), the controller updates the target resource's .spec.replicas field (e.g., a Deployment, StatefulSet, or ReplicaSet).
5. The Deployment's ReplicaSet creates or terminates pods to match the new desired count.
6. Cooldown avoidance: HPA prevents thrashing by requiring the metric to remain outside the target for a period — usually 2–3 sync cycles.

Required components

• Metrics Server must be installed in the cluster (kubectl top pods works).
• Target resource must have requests set for the metric you want to scale on (e.g., resources.requests.cpu: 250m). Without requests, HPA cannot compute a meaningful utilization percentage.

Hands-on walkthrough

Let's create a deployment that consumes CPU, then attach an HPA that scales on CPU at 50% target.

Step 1: Deploy a sample app

Save the following as stress-deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: stress-generator
spec:
  replicas: 1
  selector:
    matchLabels:
      app: stress
  template:
    metadata:
      labels:
        app: stress
    spec:
      containers:
      - name: stress
        image: polinux/stress
        command: ["/bin/sh", "-c"]
        args: ["stress --cpu 1 &"; "sleep 3600"]
        resources:
          requests:
            cpu: "200m"
          limits:
            cpu: "400m"

Apply it:

kubectl apply -f stress-deployment.yaml

Step 2: Create the HPA

Save as stress-hpa.yaml:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: stress-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: stress-generator
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 50

Apply it:

kubectl apply -f stress-hpa.yaml

Step 3: Watch it scale

kubectl get hpa stress-hpa --watch

Expected output (after ~30 seconds):

NAME         REFERENCE                    TARGETS         MINPODS   MAXPODS   REPLICAS   AGE
stress-hpa   Deployment/stress-generator   <unknown>/50%   1         10        0          10s
stress-hpa   Deployment/stress-generator   0%/50%          1         10        1          30s
stress-hpa   Deployment/stress-generator   80%/50%         1         10        2          60s

The HPA detected CPU above 50% and scaled from 1 to 2 replicas. Each new pod shares the load, bringing average CPU down closer to target.

Step 4: Simulate load increase (optional)

If you want to trigger more scaling, you can increase the stress:

kubectl exec deploy/stress-generator -- stress --cpu 2 &  # adds more CPU load

After a few minutes, you'll see replicas climb.

Compare options / when to choose what

When to use which: - CPU is the simplest and works for most stateless services. - Memory is useful for workloads that cache data in RAM but must stay within limits. - Custom/external metrics are for advanced scenarios where resource metrics are insufficient — e.g., scale on HTTP requests per second or Kafka lag.

Troubleshooting & edge cases

• HPA shows <unknown>/50% — Metrics Server is not running or pods missing requests. Run kubectl top pods to verify metrics. If empty, install Metrics Server.
• HPA never scales down — Metrics Server might report no data for a while. Check kubectl describe hpa <name> for conditions like FailedGetResourceMetric.
• Pod flapping (scales up then down repeatedly) — Enable the stabilization window in autoscaling/v2: yaml behavior: scaleDown: stabilizationWindowSeconds: 300
• HPA ignored for some pods — If pods have different resource requests, HPA averages across all ready pods. Ensure homogeneous request sizes for predictable scaling.
• Race conditions — When HPA and kubectl scale are used simultaneously, the HPA controller overwrites the manual scale. Avoid manual scaling on HPA-managed resources.

What you learned & what's next

You now understand:

• How HPA detects load and computes desired replicas using a simple formula.
• The difference between resource metrics (CPU, memory) and custom/external metrics.
• How to create an HPA, watch it scale, and interpret the output.
• How to avoid common pitfalls like missing Metrics Server or missing resource requests.

HPA is the foundation of elastic Kubernetes. Next, you will combine HPA with a Cluster Autoscaler to also scale your node pool when the cluster runs out of capacity — true cloud-native auto-scaling.