Create a Kubernetes Job

You've deployed a web app that runs 24/7, but now you need to process a massive CSV file, generate thumbnails for ten thousand images, or run a database migration — tasks that should start, finish, and then disappear. Leaving a long-running Deployment alive for a single batch job wastes cluster resources and complicates cost tracking. Kubernetes Jobs solve exactly this: they run a container to completion and then stop automatically, giving you a clean, safe way to execute short-lived, finite workloads.

The problem this lesson solves

Standard Kubernetes Deployments assume your containers run forever. If you use a Deployment for a one-off task, the Pod restarts endlessly, burning CPU and memory for nothing. Worse, you have to manually scale it to zero or delete it afterward, which is error-prone in automation. Batch processing demands a different resource — one that knows when the work is done. That resource is a Job.

Jobs handle three core scenarios that Deployments cannot: - Run once and stop: A single Pod starts, does its work, and exits with a zero exit code. The Job marks itself complete. - Run many times in parallel: Process a queue in parallel (e.g., 10 workers pulling from a message queue). - Retry on failure: Specify a backoff limit so the Job retries failed Pods automatically.

Without Jobs, your CI/CD pipeline would either leave zombie Pods or force you to build a custom babysitter. Let's fix that.

Core concept / mental model

Think of a Job as a task runner with a completion guarantee. It's like cron but event-driven and cluster-aware. The Job controller watches the Pod's exit code: - If the container exits with 0 (success), the Job records success and stops. - If it exits non-zero (failure), the Job restarts the Pod (up to a limit) until it succeeds or hits the backoff limit. - If you need exactly N successful completions, set .spec.completions. For N parallel workers, set .spec.parallelism.

Pro tip: Jobs are not for daemons. Use Deployments or StatefulSets if your process should run continuously. Use CronJobs (next lesson) if you need scheduled, recurring batch jobs.

Key definitions

• Job: The top-level Kubernetes resource. Defines the Pod template and completion criteria.
• Pod template: Just like a Deployment but without a restart policy of Always. For Jobs, the restart policy must be OnFailure or Never.
• Completions: How many successful Pod runs must occur before the Job is complete (default: 1).
• Parallelism: How many Pods can run concurrently (default: 1).
• Backoff limit: How many retries before marking the Job as failed (default: 6).

How it works step by step

Here's the lifecycle when you kubectl apply a Job manifest:

1. API Server receives the Job YAML. The Job object is stored in etcd.
2. Job controller (part of kube-controller-manager) picks it up. It creates the appropriate number of Pods based on .spec.parallelism.
3. Each Pod runs the container(s). The Pod is scheduled onto a worker node and executes the command.
4. Pod exits. If the exit code is 0, the Job increments .status.succeeded. If non-zero, the Job increments .status.failed and (if below .spec.backoffLimit) creates a new Pod.
5. Job reaches .spec.completions. The Job status becomes Complete. No new Pods are created. The old Pods remain (for log debugging) but are not restarted.
6. Cleanup (optional). You can delete the Job with kubectl delete job <name> to remove the Pods. Or use a TTL controller (.spec.ttlSecondsAfterFinished) to auto-clean.

Pro tip: Default behavior is exactly one Pod, exactly one success. For fan-out processing (e.g., 10 independent tasks), set completions: 10 and parallelism: 3 to run three at a time until all ten are done.

Hands-on walkthrough

Let's create a Job that calculates the sum of numbers from 1 to 1000 and exits.

Prerequisites

• A running Kubernetes cluster (minikube or kind works fine).
• kubectl installed and configured.

Step 1: Write the Job manifest

Create a file named sum-job.yaml:

apiVersion: batch/v1
kind: Job
metadata:
  name: sum-calculator
spec:
  completions: 1
  parallelism: 1
  backoffLimit: 4
  template:
    spec:
      containers:
      - name: calculator
        image: alpine:3.20
        command: ["/bin/sh", "-c"]
        args:
          - |
            total=0
            for i in $(seq 1 1000); do
              total=$((total + i))
            done
            echo "The sum is $total"
      restartPolicy: Never

Key points: - restartPolicy: Never — the Job will not restart the container if it exits; it will create a brand new Pod. - backoffLimit: 4 — if this fails, we retry up to 4 times.

Step 2: Apply the Job

kubectl apply -f sum-job.yaml

Output:

job.batch/sum-calculator created

Step 3: Watch the Job status

kubectl get jobs --watch

You'll see something like:

NAME            COMPLETIONS   DURATION   AGE
sum-calculator  0/1           0s         2s
sum-calculator  0/1           1s         3s
sum-calculator  1/1           2s         4s

Once COMPLETIONS shows 1/1, the Job is done.

Step 4: View logs from the completed Pod

First, find the Pod that ran:

kubectl get pods --selector=job-name=sum-calculator

Output:

NAME                  READY   STATUS      RESTARTS   AGE
sum-calculator-7z4v4  0/1     Completed   0          2m

Then get its logs:

kubectl logs sum-calculator-7z4v4

Output:

The sum is 500500

Step 5: Clean up

kubectl delete job sum-calculator

This removes the Job and its completed Pods.

Compare options / when to choose what

When to choose a Job over alternatives: - You need exactly one successful run (or N runs). - The task is finite and you don't want to waste resources. - You want automatic retry without a custom watchdog.

When NOT to use a Job: - The task must run periodically → use CronJob. - The task is long-lived (a daemon) → use Deployment. - The task needs persistent identity (e.g., database master election) → use StatefulSet.

Troubleshooting & edge cases

The Job never completes (stuck at 0/1)

Symptom: kubectl get jobs shows 0/1 for a long time. kubectl describe job shows no Pods.

Cause 1: No worker node resources (CPU/memory).

kubectl describe node <node-name>

Look for Allocatable vs Allocated. If full, either scale cluster or request less resources (set resources.requests).

Cause 2: The image pull fails.

kubectl describe pod <pod-name>

Look for ImagePullBackOff or ErrImageNeverPull. Fix the image name or tag.

The Job fails repeatedly (shows failed status)

Symptom: kubectl get jobs eventually shows 1/1 but with failures first, or kubectl describe job shows Failed.

Common fix: Check logs of the last failed Pod:

kubectl logs <pod-name>

Common issues: - Missing environment variables (use env in Pod spec) - Script syntax error in args - Command not found (use absolute path like /bin/sh)

kubectl describe job sum-calculator

Look at Conditions and Pod Statuses:

Conditions:
  Type     Reason            Age   From            Message
  ----     ------            ----  ----            -------
  Failed   BackoffLimitExceeded  2m   job-controller  Job has reached the specified backoff limit

Pods stay after Job completes (runaway logs)

By default, completed Pods linger. If they accumulate, use .spec.ttlSecondsAfterFinished to auto-delete:

spec:
  ttlSecondsAfterFinished: 3600  # delete after 1 hour

Pro tip: For CI/CD pipelines, always set ttlSecondsAfterFinished to avoid filling up etcd with completed Pod metadata.

What you learned & what's next

You now know how to create a Kubernetes Job for batch processing: you understand the lifecycle (Pod starts, does work, exits with 0), how to configure parallelism and retries, and how to debug common failures. You can confidently replace fragile scripts that curl the kube-apiserver with a proper Job resource.

Key takeaways from this lesson: - Jobs run a Pod to completion then stop; use them for one-off workloads. - restartPolicy must be Never or OnFailure for Jobs. - .spec.completions and .spec.parallelism control the fan-out pattern. - .spec.backoffLimit sets retry count; default is 6. - Completed Pods remain by default; use ttlSecondsAfterFinished for auto-cleanup.

What's next: CronJobs — the natural extension of Jobs for recurring batch processing (e.g., nightly reports, weekly cleanup). The next lesson shows how to replace cron with Kubernetes-native scheduling.

Pro tip: Before moving on, try creating a Job that runs 5 parallel Pods, each computing a Fibonacci number. That simple exercise cements the concurrency model.