Configure AKS Auto-scaling

You've deployed a workload to Azure Kubernetes Service (AKS), and it works — until traffic spikes and your pods start timing out, or you're paying for idle nodes at 3 AM. Manually scaling your cluster is like trying to fill a bathtub while reading a book: you're always either overflowing or running dry. This lesson solves that pain by showing you exactly how to configure AKS auto-scaling — both the Kubernetes Horizontal Pod Autoscaler (HPA) and the AKS cluster autoscaler — so your cluster scales itself based on real demand, not guesswork. By the end, you'll be able to set up, test, and troubleshoot both autoscalers with confidence.

The problem this lesson solves

Without auto-scaling, every AKS cluster is a ticking time bomb or a money pit. When traffic surges, your pods saturate CPU and start returning 503s. When traffic drops, you're still paying for nodes that sit idle. You could scale manually, but that means someone has to be on call 24/7, watching dashboards and running kubectl scale at 2 AM.

Auto-scaling solves this by making your cluster self-adjusting. The Horizontal Pod Autoscaler keeps your pods at the right count based on metrics like CPU or memory. The cluster autoscaler keeps your nodes at the right count based on unschedulable pods. Together, they form a two-layer system that reacts to demand automatically.

But getting them to work together isn't automatic — you need to configure them correctly, set sensible limits, and know what each one can and can't do. Misconfigure them, and you'll either scale wildly or never scale at all.

Core concept / mental model

Think of your AKS cluster as a restaurant. The pod autoscaler is the manager who adds more waiters (pods) when the restaurant gets busy and sends some home when it's quiet. The cluster autoscaler is the owner who decides whether to rent more space (nodes) when the waiters run out of room.

The pod autoscaler (HPA) works on the replica set level. It monitors a metric — usually CPU or memory — and adjusts the replicas field in your deployment. It's fast, reacting in seconds to minutes.

The cluster autoscaler works on the node pool level. It watches for pods that can't be scheduled because there aren't enough available CPU or memory across existing nodes. When it finds such pods, it adds a node (scales out). When nodes are underutilized and all pods could fit on fewer nodes, it removes nodes (scales in). This process is slower, taking minutes because it involves provisioning new VMs.

The two autoscalers are complementary, not redundant. The HPA needs enough nodes to schedule new pods; the cluster autoscaler needs to see unschedulable pods to kick in. They work as a chain: HPA scales pods up, pods can't fit, cluster autoscaler adds nodes. On the way down, the reverse happens.

How it works step by step

To configure AKS auto-scaling, you follow these logical stages, each building on the last:

1. Define resource requests — The foundation. The HPA can't calculate CPU utilization if your pods don't specify requests. Without requests, the autoscaler has no baseline to measure against. Always set requests and limits on your container specs.

2. Create the pod autoscaler — Define an HorizontalPodAutoscaler object that targets your deployment, specifies a metric (like CPU), and sets min/max replica counts. Apply it with kubectl apply.

3. Enable the cluster autoscaler — When creating your AKS cluster, you can enable it with the --enable-cluster-autoscaler flag. For existing clusters, use az aks update with the same flag. Configure the node pool's min and max node counts.

4. Generate load to test — Apply a deployment that outputs autoscaler-loadgen or similar, then run a load generator in another pod or from your local machine. Watch your HPA scale up pods.

5. Watch and verify — Use kubectl get hpa to see current CPU utilization and replica count. Use kubectl get nodes to see if the cluster autoscaler added nodes. Finally, stop the load and watch both scale back down.

Hands-on walkthrough

Let's put theory into practice. We'll create a simple web server deployment, set up the HPA, enable the cluster autoscaler, and test the whole thing end-to-end.

Step 1: Create a deployment with resource requests

First, create a file named deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
spec:
  replicas: 1
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
      - name: myapp
        image: nginx:1.25
        resources:
          requests:
            cpu: "250m"
            memory: "128Mi"
          limits:
            cpu: "500m"
            memory: "256Mi"

Apply it:

kubectl apply -f deployment.yaml

Step 2: Create the Horizontal Pod Autoscaler

Create hpa.yaml:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: myapp-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: myapp
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 50

Apply and check:

kubectl apply -f hpa.yaml
kubectl get hpa myapp-hpa

Expected output (after a minute or two):

NAME         REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
myapp-hpa    Deployment/myapp   0%/50%    1         10        1          1m

Step 3: Enable the cluster autoscaler

If you created your cluster without it, update it now. If your cluster is named myAKSCluster in resource group myResourceGroup:

az aks update \
  --resource-group myResourceGroup \
  --name myAKSCluster \
  --enable-cluster-autoscaler \
  --min-count 1 \
  --max-count 5

Step 4: Generate load and watch auto-scaling

Run a load generator pod that hammers your service:

kubectl run loadgen --image=busybox -- /bin/sh -c "while true; do wget -q -O- http://myapp-service; done"

Pro tip: Use a service pointing to your deployment, or use the deployment's service if you have one. If not, create a simple ClusterIP service first.

After 2-3 minutes, check the HPA:

kubectl get hpa myapp-hpa

You'll see the target percentage and replica count climb:

NAME         REFERENCE          TARGETS    MINPODS   MAXPODS   REPLICAS   AGE
myapp-hpa    Deployment/myapp   120%/50%   1         10        3          3m

Then check nodes:

kubectl get nodes

If your cluster was small, you'll see a new node appear after the cluster autoscaler provisions it. This confirms both autoscalers are working together.

Step 5: Stop the load and scale down

Delete the load generator:

kubectl delete pod loadgen

Wait 5-10 minutes, then check again:

kubectl get hpa myapp-hpa
kubectl get nodes

You'll see the replica count drop back to 1, and eventually nodes will scale in if they're underutilized.

Compare options / when to choose what

Choose HPA when your workload is stateless and can tolerate pods being added/removed. Choose cluster autoscaler when you want to avoid paying for idle nodes. In practice, you'll almost always want both — they solve different layers of the same problem.

Troubleshooting & edge cases

HPA shows <unknown> or 0%/50% but doesn't scale

Cause: Your pods don't have requests set. The HPA can't compute utilization without a baseline. Fix: Add resources.requests.cpu and resources.requests.memory to your deployment spec and re-apply. Restart your deployment if needed.

HPA scales up but never scales down

Cause: The default scale-down stabilization window is 5 minutes to avoid flapping. For cluster autoscaler, the default scale-down delay is 10 minutes. Fix: Wait longer, or tune the --scale-down-delay-after-add and --scale-down-unneeded-time flags on the cluster autoscaler. For HPA, you can adjust behavior in the HPA spec, but be conservative.

Cluster autoscaler never adds nodes

Cause: The node pool's max count is set too low, or the cluster autoscaler is missing required IAM permissions (if you created the cluster manually). Fix: Verify with az aks show --resource-group myResourceGroup --name myCluster --query autoScalerProfile. Ensure max-count is higher than current node count. For permissions, enable the autoscaler through az aks update which sets up managed identity correctly.

Two types of autoscalers fight each other

Scenario: HPA scales pods down while cluster autoscaler scales nodes down simultaneously, causing flapping. Fix: Set conservative minReplicas and min-count values, and use stabilization windows. A good rule of thumb: keep HPA minReplicas aligned with your ability to absorb traffic, and set cluster autoscaler min-count low enough to save cost but high enough to handle minimum load.

Node pool stuck at max count

Cause: You have many pods with high resource requests, or max-count is too low for worst-case load. Fix: Review pod CPU/memory requests — reducing over-provisioning helps. Alternatively, increase max-count but be aware of cost.

What you learned & what's next

You now understand the core idea behind AKS auto-scaling: two complementary autoscalers — HPA for pods and cluster autoscaler for nodes — that work together to match capacity with demand. You completed a hands-on exercise that created a deployment, applied an HPA, enabled the cluster autoscaler, generated load, and verified that both scaled up and down correctly. You also learned to troubleshoot common issues like missing requests, slow scale-down, and autoscaler conflicts.

With auto-scaling configured, your next logical step in the Azure Tutorial path is to optimize costs further by using spot node pools or Azure Policy for cost governance. These topics build on the autoscaling foundation you just mastered.

Remember these key facts moving forward: - HPA scales pods; cluster autoscaler scales nodes — use both for true auto-scaling. - Always set requests on your containers. - Test with a load generator to see scaling in action. - Watch for stabilization windows — scaling down takes time by design. - Tune min/max values to balance cost and performance.