kubectl describe and logs debug

Your pod is stuck in CrashLoopBackOff, a deployment is showing zero ready replicas, or an Ingress returns 503 seemingly at random. Without a systematic way to inspect what's happening inside the cluster, you're left guessing — and guessing wastes time. In this lesson, you'll learn to use kubectl describe and kubectl logs, the two commands that turn Kubernetes black boxes into transparent systems you can debug with confidence.

The problem this lesson solves

Kubernetes abstracts away the complexity of scheduling and running containers. But when something goes wrong — a pod fails to start, a service maps to no endpoints, or a persistent volume claim stays Pending — that abstraction becomes a wall. Standard commands like kubectl get pods only show status, not why. Without digging deeper, you’re flying blind.

kubectl describe fetches the full object spec, status conditions, events, and error messages. kubectl logs gives you the container's stdout/stderr. Together they replace guesswork with data. This lesson teaches you exactly how to use them to pinpoint failures — from image pull errors to liveness probe failures.

Core concept / mental model

Think of Kubernetes objects as state machines. Every object — Pod, Deployment, Service, PVC — has:

• Spec: the desired state you declared in YAML
• Status: the current state as observed by the control plane
• Events: a chronological log of operations and failures on that object

kubectl describe prints all three. It is the single source of truth for why an object is in its current state.

kubectl logs is your runtime telemetry. While describe tells you what Kubernetes thinks is wrong, logs tell you what your application says is wrong — errors from your code, connection failures, missing config files.

Pro tip: Think of describe as the crash report and logs as the black box flight recorder. Use describe first to understand the cluster's view; then use logs to inspect the application's behaviour.

Key definitions

• Events: Pod-level or object-level messages such as FailedPullImage, BackOff, Liveness probe failed. They appear at the bottom of kubectl describe pod <name>.
• Container states: Waiting, Running, Terminated. describe shows the reason and message for each (e.g., CrashLoopBackOff with “back-off 5s restarting failed container”).

How it works step by step

Step 1: Get a list of objects

Always start with kubectl get to identify the name and namespace of the object you need to debug.

kubectl get pods -n my-app

Output:

NAME                     READY   STATUS             RESTARTS   AGE
backend-7d4c9f8c-4k2mz   0/1     CrashLoopBackOff   5          3m

Step 2: Describe the failing pod

kubectl describe pod backend-7d4c9f8c-4k2mz -n my-app

Focus on two sections: - Status → Container States → Terminated / Waiting (the Reason and Message) - Events (usually at the bottom): look for Failed, Error, BackOff, Unhealthy.

Example snippet:

Events:
  Type     Reason     Age                 From               Message
  ----     ------     ----                ----               -------
  Normal   Killing    6m38s               kubelet            Container backend failed liveness probe, will be restarted
  Warning  Unhealthy  6m38s (x3 over 7m)  kubelet            Liveness probe failed: Get "http://10.1.0.25:8080/health": dial tcp 10.1.0.25:8080: connect: connection refused

Step 3: Check logs (if container ever started)

kubectl logs backend-7d4c9f8c-4k2mz -n my-app

If the pod restarted multiple times, use --previous to see the logs of the last terminated container:

kubectl logs backend-7d4c9f8c-4k2mz -n my-app --previous

Hands-on walkthrough

Scenario 1: Image pull failure

First, create a pod with a wrong image name:

# bad-image.yaml
apiVersion: v1
kind: Pod
metadata:
  name: bad-image
spec:
  containers:
  - name: app
    image: nginx:nonexistent

kubectl apply -f bad-image.yaml
kubectl describe pod bad-image

Output snippet:

Events:
  Type     Reason     Age   From               Message
  ----     ------     ----  ----               -------
  Normal   Scheduled  12s   default-scheduler  Successfully assigned default/bad-image to node-1
  Warning  Failed     10s   kubelet            Failed to pull image "nginx:nonexistent": rpc error: code = NotFound desc = manifest for nginx:nonexistent not found: manifest unknown: manifest unknown
  Warning  Failed     10s   kubelet            Error: ErrImagePull
  Normal   BackOff    4s    kubelet            Back-off pulling image "nginx:nonexistent"

Diagnosis: The image tag is wrong. Fix the YAML and re-apply.

Scenario 2: CrashLoopBackOff from a runtime error

Create a pod that runs a command that fails immediately:

# crash.yaml
apiVersion: v1
kind: Pod
metadata:
  name: crashy
spec:
  containers:
  - name: fail
    image: busybox
    command: ["sh", "-c", "exit 1"]

kubectl apply -f crash.yaml
kubectl describe pod crashy

Notice the container state: Terminated with Reason: Error and Exit Code: 1.

Now check the logs (even a failed container may produce stdout before exiting):

kubectl logs crashy

Likely empty — the application exited immediately. Use kubectl logs --previous if the container restarted.

Scenario 3: Liveness probe failure

Deploy a pod with a liveness probe that points to a non-existent endpoint:

# probe-fail.yaml
apiVersion: v1
kind: Pod
metadata:
  name: probe-fail
spec:
  containers:
  - name: web
    image: nginx
    livenessProbe:
      httpGet:
        path: /healthz
        port: 80
      initialDelaySeconds: 2
      periodSeconds: 3

kubectl apply -f probe-fail.yaml
sleep 15
kubectl describe pod probe-fail

Look for:

Events:
  Type     Reason     Age   From               Message
  ----     ------     ----  ----               -------
  Warning  Unhealthy  12s   kubelet            Liveness probe failed: HTTP probe failed with statuscode: 404
  Normal   Killing    12s   kubelet            Container web failed liveness probe, will be restarted

Fix: Either create the /healthz endpoint in your app, or change the probe path to / or a valid route.

Compare options / when to choose what

Rule of thumb: 1. Start with kubectl describe on the failing resource. 2. If the container shows CrashLoopBackOff or Error, run kubectl logs (with --previous if needed). 3. For probe or network issues, check both the pod and the service endpoints (kubectl describe endpoints).

Troubleshooting & edge cases

Edge case 1: Pod in Pending state

kubectl describe might show:

Events:
  Warning  FailedScheduling  15s   default-scheduler  0/1 nodes are available: 1 Insufficient cpu.

Solution: Increase node resources or reduce resource requests in the pod spec.

Edge case 2: Init container failure

If you have init containers, describe the pod and look for the init container's container state:

  Init Containers:
    init-myservice:
      State:          Terminated
        Reason:       Error
        Exit Code:    1

Use kubectl logs pod-name -c init-myservice to see the init container’s logs.

Edge case 3: Multi-container pod

Always specify the container name with -c <name>:

kubectl logs my-pod -c sidecar -n my-ns
kubectl describe pod my-pod | grep -A 5 "Container ID"

Common mistakes

• Forgetting --previous: After a container restarts, current logs are empty. Always check kubectl logs --previous when debugging restarts.
• Describing a Deployment instead of a Pod: kubectl describe deployment gives the deployment spec, not the running pod's events. Describe the pod directly.
• Ignoring Events: Many developers only look at the status field. The Events section often contains the exact error message from kubelet.
• Using kubectl logs on a pod that never started: If the pod is Pending or ErrImagePull, there are no logs. Use describe first.

Variations

• Using kubectl logs -f to tail logs in real time during debugging.
• Using stern or kubetail for streaming logs from multiple pods matching a label selector.
• Using kubectl describe on other objects: kubectl describe svc, kubectl describe pvc, kubectl describe ingress — all expose events that may reveal misconfigurations.

What you learned & what's next

You now have a repeatable debugging workflow for Kubernetes. You can: - Use kubectl describe to inspect an object's full state, see container reasons, and read the event timeline. - Use kubectl logs (with --previous) to extract application output from running or crashed containers. - Combine both to diagnose common failures: image pull errors, probe failures, scheduling issues, and init container crashes.

In the next lesson, you'll extend this diagnostic technique to Services and Ingress — debugging why your app is reachable inside the cluster but not outside, or why a load balancer returns 502s. These same describe and logs commands will be your first tools there too.

Debugging Kubernetes isn't magic — it's knowing where to look. Start with describe, check logs, and you'll solve 90% of issues in minutes.