Persistent Volumes & Claims

You have a stateless web app running in Kubernetes, but what happens when you need to persist critical data — like user uploads or a database state? Without persistent storage, restarting a Pod wipes everything. This is the pain point that Persistent Volumes (PVs) and Persistent Volume Claims (PVCs) solve: they decouple storage management from Pod lifecycle, giving you durable, cluster-wide storage that survives Pod restarts or rescheduling.

The problem this lesson solves

In Kubernetes, Pods are ephemeral by design — they can be killed, rescheduled, or moved to another node at any time. When a Pod terminates, its local ephemeral storage (like emptyDir) is deleted. This is fine for cache or temporary data but breaks any application that needs to retain state: databases, file upload systems, or logging pipelines.

Without PVs and PVCs, you would have to manually provision storage on each node, embed storage details into your Pod specs, and manually clean up when scaling. This approach is brittle, insecure, and impossible to manage at scale. Persistent Volumes and Claims solve this by providing an abstract, cluster-wide storage layer that can be provisioned automatically or manually, then consumed by your Pods via a simple claim.

Core concept / mental model

Think of a Persistent Volume (PV) as a storage resource in your cluster — similar to a node's CPU or memory. PVs exist independently of any Pod. They can be backed by many types of storage: cloud disks (EBS, GCE PD), NFS, hostPath, or CSI drivers.

A Persistent Volume Claim (PVC) is a request for storage by a user, much like a Pod requests CPU. The PVC specifies the desired size, access mode (ReadWriteOnce, ReadOnlyMany, etc.), and optionally a storage class. Kubernetes then binds the PVC to a matching PV that satisfies the request.

Pro Tip: PVs and PVCs separate storage provisioning (cluster admin) from storage consumption (developer). Developers only need to know the claim name — they never touch the underlying storage provider.

The binding is one-to-one: each PVC binds to exactly one PV. Once bound, the Pod can mount the PVC as a volume. When the Pod is deleted, the PVC can remain, keeping the data safe for the next Pod.

How it works step by step

Here's the typical lifecycle of a PV and PVC in Kubernetes:

1. Provision a Persistent Volume – A cluster admin creates a PV object pointing to actual storage (e.g., a Google Compute Engine disk). Alternatively, you can use dynamic provisioning via a StorageClass — Kubernetes automatically creates a PV when a PVC requests one.

2. Create a Persistent Volume Claim – A developer creates a PVC specifying size, access mode, and storage class (if any). The Kubernetes control plane looks for a PV that matches the claim.

3. Binding – If a suitable PV exists, the PVC is bound to it. The PV becomes bound and cannot be used by other claims. If no PV matches, the PVC stays pending until a matching PV appears or dynamic provisioning kicks in.

4. Use in a Pod – The Pod's volume definition references the PVC by name. Kubernetes mounts the underlying storage into the Pod's filesystem.

5. Reclaim policy – When a PV is no longer needed (the PVC is deleted), the reclaim policy decides what happens: Retain (keep data), Recycle (wipe data and re-use), or Delete (delete the backing storage).

Hands-on walkthrough

Let's create a simple example using a hostPath PV (for development — not production) and a PVC.

Step 1: Create a Persistent Volume

Create a directory on your node (e.g., /tmp/dev-pv-data), then define the PV:

# pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: dev-pv
spec:
  capacity:
    storage: 1Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  hostPath:
    path: /tmp/dev-pv-data

Apply it:

kubectl apply -f pv.yaml
kubectl get pv

Expected output:

NAME     CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   REASON   AGE
dev-pv   1Gi        RWO            Retain           Available                                   5s

The PV is Available and waiting for a claim.

Step 2: Create a Persistent Volume Claim

# pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: dev-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

Apply and check:

kubectl apply -f pvc.yaml
kubectl get pvc

Expected output:

NAME      STATUS   VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
dev-pvc   Bound    dev-pv   1Gi        RWO                           5s

The PVC is Bound to dev-pv. Check the PV status again — it should show Bound with a claim reference.

Step 3: Use the PVC in a Pod

# pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: pvc-demo-pod
spec:
  containers:
  - name: app
    image: alpine
    command: ["sleep", "3600"]
    volumeMounts:
    - mountPath: /data
      name: app-storage
  volumes:
  - name: app-storage
    persistentVolumeClaim:
      claimName: dev-pvc

Apply and verify the mount:

kubectl apply -f pod.yaml
kubectl exec pvc-demo-pod -- ls -la /data

You should see an empty directory — ready to store data.

Step 4: Test persistence

Write a file inside the Pod:

kubectl exec pvc-demo-pod -- touch /data/testfile.txt
kubectl exec pvc-demo-pod -- ls /data

Now delete the Pod:

kubectl delete pod pvc-demo-pod

Re-create it with the same PVC:

kubectl apply -f pod.yaml
kubectl exec pvc-demo-pod -- ls /data

You will see testfile.txt — the data survived the Pod deletion!

Compare options / when to choose what

Pro Tip: In production, use Retain for critical data and Delete with dynamic provisioning for ephemeral workloads. Avoid hostPath in production — use cloud or CSI-backed storage for reliability.

Troubleshooting & edge cases

• PVC stuck in Pending – No matching PV exists and no StorageClass configured. Check PV size and access modes, or ensure a StorageClass is available.
• Pod fails with MountVolume.SetUp failed – The PVC might be deleted or the underlying storage unavailable (e.g., disk detached). Verify the PVC status and the storage backend health.
• Multiple Pods trying to use the same RWO PVC – Only one node can mount RWO at a time. Use RWX or distribute Pods across different PVCs.
• Data lost after PVC deletion – If the PV's reclaim policy is Delete, the data is gone. Always use Retain for important data.
• Volume not released after PVC deletion – The PV goes to Released state. To reuse, manually delete the PV's claimRef or the PV itself.

What you learned & what's next

You now understand the core idea behind Persistent Volumes and Volume Claims: they abstract storage provisioning from Pod consumption, offering durable, independent storage that survives Pod restarts. You completed a hands-on exercise, learned about access modes and reclaim policies, and saw how to troubleshoot common binding issues.

Next step: Learn about StatefulSets — the workload API designed for stateful applications that use PVCs to maintain identity and storage per Pod. You'll combine StatefulSets with PVCs to deploy databases and stateful services reliably.