Use kubectl apply

Picture this: you've hand-crafted a perfect Deployment manifest for your application, but when you run kubectl create, Kubernetes throws an error because a resource with that name already exists. Or worse — someone on your team manually scaled a ReplicaSet, and now your YAML is out of sync with the cluster's actual state. These frictions vanish when you adopt declarative management with kubectl apply. Instead of issuing imperative commands (create, replace, delete), you write your desired state in YAML or JSON and let Kubernetes compute the diff to make the cluster match. This approach, powered by server-side apply, brings GitOps workflows, rollback safety, and collision detection to every resource you manage.

The problem this lesson solves

Manual kubectl operations work for experiments, but they break down fast in team environments. Here's what happens when you rely on kubectl create or kubectl replace:

• Idempotency failure – Running kubectl create twice raises an error. Your deployment scripts must handle exits, increasing complexity.
• State drift – When you run kubectl scale or kubectl edit, those changes live only in the cluster. Your local YAML files become stale, making rollbacks or restores dangerous.
• No change lineage – kubectl create overwrites fields without recording who set them or why. Debugging mysterious overrides becomes a guessing game.

Declarative config with kubectl apply solves all three. You store the intended state in version control (Git), and kubectl apply reconciles the cluster to that state, every time — no matter what happened before.

Core concept / mental model

Think of kubectl apply as a two-way diff engine. It compares:

1. The manifest you provide (local file or stdin).
2. The current live object in the cluster.

For every field, it calculates the minimal change to make the cluster match your manifest. Fields not present in your file are left untouched (unless removed via kubectl apply --prune). This is fundamentally different from kubectl create (which rejects existing objects) or kubectl replace (which overwrites the entire object, potentially wiping out fields managed by other controllers).

Declarative vs. imperative: a quick grid

Server-side apply (Kubernetes 1.18+, GA in 1.22)

Since Kubernetes 1.18, kubectl apply uses server-side apply by default (K8s 1.22+ with kubectl ≥1.24). The server computes the diff and stores ownership metadata in the object's managedFields. This means:

• Two team members can apply different fields to the same Deployment without clobbering each other.
• Field ownership is tracked per field, per user/agent.
• Remove a field by moving its ownership to another manager.

How it works step by step

1. Write a manifest (YAML preferred) with your desired state — apiVersion, kind, metadata, and spec.
2. Run kubectl apply -f <file> – the client sends the whole manifest to the API server.
3. API server compares your manifest against the current object (if it exists).
4. Server-side apply calculates the patch: fields in your manifest are updated; fields absent are left alone unless you explicitly delete them.
5. Kubernetes stores the merged result and updates managedFields with a new manager entry (by default, kubectl-client-side-apply for older versions, or kubectl for server-side).
6. Return success – the updated object is returned.

If the object does not exist, kubectl apply creates it. That's why you can run the same kubectl apply command in CI/CD every deployment — it's idempotent.

The imperative trap illustrated

# Imperative: creates NGINX Deployment
kubectl create deployment nginx --image=nginx:1.21 --replicas=3

# Later, apply a YAML with replicas: 5 and image: nginx:1.25
kubectl apply -f nginx-deploy.yaml

With kubectl create, the second command would fail unless you first delete the object. With kubectl apply, it works seamlessly — the cluster updates to replicas: 5 and image: nginx:1.25 without manual cleanup.

Hands-on walkthrough

Let's build a real Deployment manifest and apply it twice to see idempotency in action.

Step 1: Create your first declarative manifest

# nginx-deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-declarative
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.21
        ports:
        - containerPort: 80

Step 2: Apply the manifest

kubectl apply -f nginx-deploy.yaml

Expected output:

deployment.apps/nginx-declarative created

Check the pods are running:

kubectl get pods -l app=nginx

NAME                                 READY   STATUS    RESTARTS   AGE
nginx-declarative-7f9c7c8b7f-abc12   1/1     Running   0          10s
nginx-declarative-7f9c7c8b7f-xyz34   1/1     Running   0          10s

Pro tip: Always include --- at the top of a multi-resource YAML file. It helps viewers (and linters) parse multiple documents.

Step 3: Modify and re-apply (idempotent update)

Update the YAML to change the image and scale:

spec:
  replicas: 3
  template:
    spec:
      containers:
      - image: nginx:1.25

Apply again:

kubectl apply -f nginx-deploy.yaml

Expected output:

deployment.apps/nginx-declarative configured

Verify the changes:

kubectl get deploy nginx-declarative -o jsonpath='{.spec.replicas}'
3

kubectl get deploy nginx-declarative -o jsonpath='{.spec.template.spec.containers[0].image}'
ginx:1.25

Step 4: Apply an identical manifest (no-op)

Apply the same file again:

kubectl apply -f nginx-deploy.yaml

Expected output:

deployment.apps/nginx-declarative unchanged

This is the power of declarative config: Kubernetes detects no diff and does nothing. Your CI/CD pipeline can safely run kubectl apply hundreds of times.

Step 5: (Bonus) View managed fields

kubectl get deploy nginx-declarative -o yaml | head -50 | grep -A5 managedFields

You'll see entries like:

  managedFields:
  - manager: kubectl
    operation: Update
    apiVersion: apps/v1
    fieldsType: FieldsV1
    fieldsV1: {...}

This metadata enables advanced GitOps tools (ArgoCD, Flux) to detect drift and resolve conflicts.

Compare options / when to choose what

When to choose kubectl apply: - You want pure YAML management without templating. - You don't need Helm's chart packaging or lifecycle hooks. - Your team prefers GitOps with tools like ArgoCD (which uses kubectl apply under the hood).

When to choose Helm or Kustomize: - You need environment-specific overrides (staging vs. prod). - You're distributing applications as packages. - You need to manage release history and rollbacks (Helm).

Pro tip: Even if you use Helm, Helm itself calls kubectl apply internally. Understanding how kubectl apply behaves — including the no-op for unchanged resources — helps you debug Helm releases faster.

Troubleshooting & edge cases

1. kubectl apply reports "NotFound" when you expect an update

Cause: The manifest's metadata.name is wrong or the namespace doesn't exist.

Fix:

# Check exact name
kubectl get deployments --all-namespaces | grep your-app

# Ensure the namespace exists
kubectl get ns

If the namespace is missing, create it imperatively first (kubectl create ns my-ns) or add a Namespace resource to your YAML.

2. Fields are not being updated

Symptom: You modify a field like replicas: 5, but after apply, replicas stays at 3.

Common cause: The field is being overridden by a mutating admission controller (e.g., VPA, Istio, or a custom webhook).

Diagnosis:

kubectl get deploy nginx-declarative -o yaml | grep -A2 managedFields

Look for a manager other than kubectl. If you see horizontal-pod-autoscaler or custom-webhook, that controller is setting replicas.

Solution: - For HPA/VPA: Add the field to the .spec.ignoreOverrides (if supported) or accept its value. - For custom webhooks: Check admission logs.

3. Unexpected resource deletion after kubectl apply --prune

Symptom: Running kubectl apply --prune -f . deletes a Service you intended to keep.

Cause: The prune command removes resources that are not listed in the applied files. If you forgot to include a resource in your file, prune deletes it.

Fix: Always dry-run first:

kubectl apply --prune -f . --dry-run=client

Review the output for resources that would be pruned. Add any missing YAML files.

4. Server-side apply conflict error

Error: Apply failed with 1 conflict: conflict with "another-agent"

Cause: Two different kubectl clients (or controllers) have managed different fields, and your apply tries to set a field already claimed by another manager.

Solution:

# Force overwrite
kubectl apply --force-conflicts -f file.yaml

Use --force-conflicts carefully – it overrides ownership and may cause other managers to lose track of their fields.

What you learned & what's next

You now understand the core power of declarative configuration with kubectl apply:

• Idempotency – run the same command in CI/CD without error.
• State reconciliation – Kubernetes automatically computes diffs.
• Ownership tracking – managedFields prevents accidental overwrites.
• Safe updates – use --prune and --force-conflicts with understanding.

This model is the foundation for GitOps tools and production-grade deployments. In the next lesson, "Manage Kubernetes resources with ConfigMaps and Secrets," you'll apply this same declarative pattern to configuration data — storing environment variables, TLS certificates, and config files outside of your container images.

Master kubectl apply, and you've mastered the sustainable Kubernetes workflow.