Configure VerticalPodAutoscaler

You're running a dozen microservices and constantly guessing CPU and memory requests — over-provisioning to stay safe, yet still hitting OOMKilled errors or throttled containers. Manually tuning every Deployment's resource envelopes is tedious, error-prone, and never quite right. The VerticalPodAutoscaler (VPA) automates this: it observes your pod usage, calculates the optimal request and limit, and adjusts without you touching YAML by hand.

The problem this lesson solves

Kubernetes relies on requests and limits to schedule pods and enforce resource boundaries. When these values are off, you pay the price:

• Under-requesting → pods get scheduled on overcommitted nodes, leading to CPU throttling or, worse, the kubelet kills them with OOMKilled.
• Over-requesting → nodes waste capacity, your cluster costs skyrocket, and you can't pack as many workloads.
• Static tuning doesn't work at scale — traffic patterns shift, code changes, and seasonal spikes invalidate yesterday's perfect numbers.

Manually editing a Deployment every week is not sustainable. VPA acts as a resource recommendation engine that runs in your cluster, learns from live usage, and either suggests or applies corrections automatically.

Core concept / mental model

Think of VPA as a thermostat for pod resources. Just as a thermostat measures room temperature and adjusts the HVAC to reach your target, VPA measures pod resource consumption and adjusts CPU/memory requests (and optionally limits) to match actual demand.

VPA components

A VPA deployment consists of three components that form a feedback loop:

VPA modes

VPA operates in three distinct modes that control how much automation you want:

• Off – VPA only computes recommendations (stored in a ConfigMap / status). You must manually adjust the Deployment. Safe for production when you want advice but no automated changes.
• Auto – VPA both recommends and applies resources. The Updater evicts running pods when recommendations change, and new pods get the updated values via the admission webhook. Experimental; use with caution – eviction can cause disruption.
• Recreate – Similar to Auto, but the Updater always recreates pods (by evicting them). No in-place updates. Suitable for workloads that tolerate brief restarts (e.g., batch jobs).

Pro tip: Start with mode: Off to inspect the recommendations before switching to Auto in a staging environment.

How it works step by step

When you create a VPA object targeting a Deployment or StatefulSet, here is the lifecycle:

1. Observation phase: The Recommender queries the Metrics Server every minute (configurable) to collect CPU and memory usage percentiles for each container in the target pods.
2. Recommendation phase: Using a sliding window of up to 8 days of data, the Recommender calculates the target, lowerBound, and upperBound for each resource. The target is the recommended value; the bounds define safe limits.
3. Update (if mode allows): If the current pod's resources differ from the target by more than a threshold (default 0.1% for CPU, 1% for memory — effectively immediate drift triggers), the Updater evicts the pod.
4. Admission: When a new pod is created (either from a scale-up, rollout, or eviction), the VPA Admission Controller mutates the pod spec to set the container's requests (and optionally limits) to the recommended values.

Request vs limit behavior

VPA only sets requests by default. If you want it to also set limits, you must configure containerResourcePolicy per container. This is intentional — setting limits differently can cause thrashing and unexpected OOM kills.

What VPA does NOT do

• It does not set limits automatically (unless you explicitly opt in).
• It does not scale pods horizontally (use HPA for that — but VPA and HPA can coexist with care).
• It does not update the original Deployment/StatefulSet spec — it mutates pods only. To make changes persistent, you must copy the recommendation back to your Deployment YAML.

Hands-on walkthrough

Let's configure a VPA for a sample Deployment called web-server. You'll need a running Kubernetes cluster with Metrics Server installed and the VPA components deployed (we'll use the VPA manifests from the vertical-pod-autoscaler repo).

Prerequisites

# Install Metrics Server (if not present)
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

# Clone VPA repo and deploy components
cd /tmp
git clone https://github.com/kubernetes/autoscaler.git
cd autoscaler/vertical-pod-autoscaler
./hack/vpa-up.sh

Step 1: Create a sample Deployment

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-server
spec:
  replicas: 1
  selector:
    matchLabels:
      app: web-server
  template:
    metadata:
      labels:
        app: web-server
    spec:
      containers:
        - name: nginx
          image: nginx:latest
          resources:
            requests:
              cpu: 50m
              memory: 100Mi
            limits:
              cpu: 100m
              memory: 200Mi

Apply it:

kubectl apply -f deployment.yaml

Step 2: Create a VPA object in Off mode

# vpa-off.yaml
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: web-server-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-server
  updatePolicy:
    updateMode: "Off"
  resourcePolicy:
    containerPolicies:
      - containerName: nginx
        minAllowed:
          cpu: 10m
          memory: 50Mi
        maxAllowed:
          cpu: 1000m
          memory: 1Gi

Apply the VPA:

kubectl apply -f vpa-off.yaml

Wait 1–2 minutes for the Recommender to collect metrics, then check the recommendations:

kubectl describe vpa web-server-vpa

You'll see output like:

Status:
  Conditions:
    Last Transition Time:  2025-03-01T10:00:00Z
    Status:                True
    Type:                  RecommendationReady
  Recommendation:
    Container Recommendations:
      Container Name:  nginx
      Lower Bound:
        Cpu:     5m
        Memory:  65536k
      Target:
        Cpu:     15m
        Memory:  128Mi
      Upper Bound:
        Cpu:     30m
        Memory:  256Mi

The Target values are what VPA suggests. Your current Deployment uses 50m CPU and 100Mi memory requests — VPA says you can lower CPU to 15m and raise memory to 128Mi. Cool, right?

Step 3: Apply recommendation manually (Off mode)

Edit the Deployment and set requests to the target values from the VPA status:

kubectl edit deployment web-server

Change resources.requests.cpu to 15m and resources.requests.memory to 128Mi. Save and exit.

Step 4: Switch to Auto mode (staging only)

# vpa-auto.yaml (same as above but updateMode: "Auto")
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: web-server-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-server
  updatePolicy:
    updateMode: "Auto"
  resourcePolicy:
    containerPolicies:
      - containerName: nginx
        minAllowed:
          cpu: 10m
          memory: 50Mi
        maxAllowed:
          cpu: 1000m
          memory: 1Gi

kubectl apply -f vpa-auto.yaml
# Pod will be evicted and recreated with new resources
kubectl get pods -w

Notice the old pod disappears and a new one appears with adjusted resources. Verify:

kubectl describe pod <new-pod-name> | grep -A5 Requests

Compare options / when to choose what

When to choose which: * For exploratory analysis → Off * For automated tuning with acceptable disruption → Auto on non-production * For jobs or canary deployments → Recreate

VPA + HPA: can they co-exist?

Yes, but only if HPA scales based on custom/external metrics (not CPU/memory), because VPA adjusts resource requests while HPA adjusts replicas. If both target CPU, they fight — VPA changes the request, which changes the HPA's utilization calculation, causing oscillation. Rule of thumb: use HPA on CPU/memory only without VPA, or use VPA with HPA on custom metrics.

Troubleshooting & edge cases

VPA recommendations not appearing

Conditions:
  Type:    RecommendationReady
  Status:  False
  Reason:  MissingResourceRecommendation

• Cause: Metrics Server not reporting data, or target pod has no containers with resources.
• Fix: Ensure Metrics Server is running (kubectl top pods works). Verify the target Deployment has resources defined in its container spec. Wait 1–2 minutes after pod creation.

Pod stuck in Pending after VPA Auto update

• Cause: VPA set a request larger than available node capacity.
• Fix: Set maxAllowed in the VPA spec to cap requests. Use kubectl describe pod to check Events: FailedScheduling. Adjust maxAllowed or add more nodes.

VPA recomputes too often, causing constant evictions

• Cause: Default minReplicas policy is not set; VPA evicts even singleton pods, causing downtime.
• Fix: For critical workloads, use updateMode: Off or implement a PodDisruptionBudget (PDB) to limit concurrent evictions.

OOMKilled even with VPA

• Cause: VPA only sets requests, not limits. Limits may still be too low.
• Fix: Enable limit recommendation in containerPolicy:

containerPolicies:
  - containerName: nginx
    controlledResources: ["cpu", "memory"]
    controlledValues: RequestsAndLimits

VPA and DaemonSets or Jobs

VPA does not support DaemonSet (because pods cannot be evicted independently) and generally should not be used with Jobs that run once. Focus on Deployments and StatefulSets.

What you learned & what's next

You now know how to configure VerticalPodAutoscaler for resources — you understand the three modes (Off, Auto, Recreate), how to read recommendations, and how to apply them manually or automatically. You also learned the pitfalls: co-existing with HPA, setting safe bounds, and avoiding pod churn.

Next: Move to [[Configure PodDisruptionBudget for availability]] — learn how to protect your VPA-managed workloads from excessive evictions during voluntary disruptions.