Schedule Python Batch Jobs

Your Python ETL pipeline crunches data every night at 2 AM. But if you're still setting up cron jobs on a lone server, you're one disk failure or missed schedule away from silence. Kubernetes CronJobs give you the same time-based scheduling, but with the resilience, observability, and declarative power of your cluster. Stop babysitting cron entries on a disappearing box — learn to schedule Python batch jobs with CronJobs today.

The problem this lesson solves

Traditional cron is a stubborn relic. It runs on a single machine, its environment isn't versioned, and if that machine disappears, so does your entire batch pipeline. Imagine this: your DataDog alert fires at 3 AM because last night's report never ran — but you're asleep. Even if you wake up, you have to SSH into the box, squint at logs, and restart the job manually. That's a losing game.

Kubernetes CronJobs solve this by turning scheduled work into declarative, self-healing batch processes. Your Python batch job runs as a Pod, gets restarted if it fails, and enjoys the same ConfigMaps, Secrets, and resource limits as your web services. The schedule is versioned alongside your code in YAML, so a colleague can review it, and you can roll it back if needed.

If your task is a short, idempotent batch job (database cleanup, data export, email digest), a CronJob is the idiomatic answer. You're moving from a foggy, fire-and-forget model to a clean, auditable, retryable one.

Core concept / mental model

Think of a CronJob as a timetable for your batch work. It's not a job itself — it's a controller that, at the moment your cron expression fires, creates a Job object. That Job then spins up a Pod (containing your Python code), waits for it to exit with a zero status, and considers itself successful.

The hierarchy is simple:

1. CronJob — the timer. Has a schedule string (cron syntax) and a jobTemplate.
2. Job — the unit of work. It ensures your Pod runs to completion and retries if needed.
3. Pod — the actual execution. Contains your Python image and runs your script.

Why the extra layer? Because a bare schedule can't handle retries or failure. A Job guarantees your Python script runs to completion, even if the node crashes mid-run. And a CronJob guarantees the Job spawns at the right time (with a tolerance, configurable via startingDeadlineSeconds).

Pro tip: In Kubernetes, a CronJob is not a Pod scheduler — it's a Job factory. Understanding this distinction saves you hours of debugging when your Python job doesn't run exactly at the second you expected.

Your Python code makes the contract simple:

• Exit code 0 → Job is successful, no retries.
• Any non-zero exit code → Job failed, and depending on restartPolicy and backoffLimit, it may retry.

To make the most of this, design your batch scripts to be idempotent — re-running them shouldn't duplicate side effects. Use a timestamp or a unique key in your database writes to make them safe.

How it works step by step

The lifecycle of a CronJob event looks like this:

1. Create the CronJob — you write a YAML manifest with a schedule (e.g., 0 2 * * * for every night at 2 AM). You apply it with kubectl apply -f cronjob.yaml.
2. The scheduler ticks — the Kubernetes controller-manager watches the clock. When the cron expression matches the current time, it creates a Job object from the jobTemplate.
3. The Job starts — the Job controller sees the new Job and creates a Pod from the Pod template inside the Job. The Pod pulls your Python image (e.g., python:3.11-slim) and runs your entrypoint command.
4. Your Python script runs — it processes data, writes to a database, sends emails, or whatever you designed it to do. It finishes and exits with code 0.
5. Job is marked complete — the Pod goes to Completed state, and the Job shows Complete. The CronJob controller cleans up old Jobs according to successfulJobsHistoryLimit and failedJobsHistoryLimit.

That's it! Periodically, your Python batch work runs — but now it's on ephemeral, reproducible infrastructure. You can scale it, monitor it with Prometheus, and ship it like any other deployment.

Hands-on walkthrough

Let's build a real cron job. We'll create a simple Python script that fetches data (simulated) and writes a summary to a file — classic batch work.

First, create a work directory:

mkdir python-cronjob && cd python-cronjob

Now create a Python script that simulates an ETL task:

# batch.py
import datetime
import json

def run_etl():
    now = datetime.datetime.now()
    records = [
        {"id": i, "value": i * 10, "ts": now.isoformat()}
        for i in range(100)
    ]
    # Simulate writing to a database or file
    with open("/tmp/output.json", "w") as f:
        json.dump(records, f)
    print(f"ETL complete: {len(records)} records processed at {now}")

if __name__ == "__main__":
    run_etl()

Build a tiny Docker image for it:

# Dockerfile
FROM python:3.11-slim
WORKDIR /app
COPY batch.py .
CMD ["python", "batch.py"]

If you have Docker locally, build and push it to a registry. For simplicity, we'll use your-dockerhub-id/python-batch:latest. (If you're on minikube, you can use minikube image build -t python-batch:latest ..)

Now write the CronJob manifest:

# cronjob.yaml
apiVersion: batch/v1
kind: CronJob
metadata:
  name: nightly-etl
spec:
  schedule: "*/2 * * * *"   # Every 2 minutes, for demo. Use "0 2 * * *" for daily at 2 AM.
  concurrencyPolicy: Forbid
  startingDeadlineSeconds: 120
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: OnFailure
          containers:
            - name: etl
              image: python-batch:latest
              resources:
                requests:
                  cpu: "100m"
                  memory: "128Mi"
                limits:
                  cpu: "500m"
                  memory: "512Mi"
      backoffLimit: 3
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 1

Pro tip: restartPolicy must be OnFailure or Never — not Always — because a batch job is expected to complete.

Apply it and watch the magic:

kubectl apply -f cronjob.yaml
kubectl get cronjob nightly-etl

Expected output:

NAME          SCHEDULE     TIMEZONE   SUSPEND   ACTIVE   LAST SCHEDULE   AGE
nightly-etl   */2 * * * *  <none>     False     0        42s              2m

After a couple of minutes, check the Jobs and Pods:

kubectl get jobs
kubectl get pods

You should see a new Job and a Pod that ran your Python script. To see the logs:

kubectl logs job/nightly-etl-<job-id>

Expected output:

ETL complete: 100 records processed at 2025-05-15 14:03:00.123456

And, as the CronJob cleans up, kubectl get jobs will show only the latest few.

Compare options / when to choose what

You may wonder: shouldn't I just use a Deployment with a sidecar that sleeps? No — that wastes resources. Let's compare the idiomatic options:

When to choose CronJob: You already have a Kubernetes cluster, your batch is short (minutes, not hours), and you want it to live alongside your other services. If your job takes hours or needs to process a queue, consider a Deployment with a work queue instead.

A note on scheduling precision: CronJobs can start up to a few seconds late (by design, to avoid thundering herds). If you need sub-second accuracy, use a time scheduler inside your Python app and run it as a Deployment.

Troubleshooting & edge cases

1. The job never shows up

• Check the CronJob schedule: 0 * * * runs every hour at minute 0. /5 * * * * runs every 5 minutes. Use kubectl get cronjob and look at LAST SCHEDULE.
• Is the CronJob suspended? SUSPEND column shows True if you set .spec.suspend. Unsuspend with kubectl patch cronjob nightly-etl -p '{"spec":{"suspend":false}}'.
• Check controller logs: kubectl logs -n kube-system deployment/controller-manager | grep cron to see scheduling decisions.

2. Pod is stuck in ImagePullBackOff

• If your image is local only (e.g., built with Docker but not pushed), the node can't pull it. Use kubectl describe pod to see the error. Fix: push to a registry or use imagePullPolicy: IfNotPresent when the image exists on the node.
• In minikube: eval $(minikube docker-env) before building, or use minikube image load.

3. Job fails with a non-zero exit code but backoffLimit is 0

• The Job won't retry. Set backoffLimit: 3 (or more) to allow retries. Also check restartPolicy — with OnFailure, the pod restarts inside the same Job; with Never, a new Pod is created on each attempt.

4. Timezone surprises

• By default, a CronJob runs in UTC, not your local time. If you expect 2 AM local, you might get 2 AM UTC — often wrong. Use .spec.timeZone (Kubernetes 1.27+ stable) to set a named zone, e.g., "America/New_York".

spec:
  timeZone: "Europe/Berlin"
  schedule: "0 2 * * *"

• For older clusters, do the timezone math yourself or use an environment variable inside the pod to set TZ.

5. Overlapping runs

• If your pipeline takes longer than the schedule interval, you'll get overlapping Jobs. Set concurrencyPolicy: Forbid (prevent) or Replace (run the new one, killing the old). Default is Allow — dangerous for idempotent-less code.

6. Too many old Jobs piling up

• Use successfulJobsHistoryLimit: 3 and failedJobsHistoryLimit: 1 to keep history small. Otherwise, your kubectl get jobs list will blow up.

What you learned & what's next

You now know how to schedule Python batch jobs with CronJobs: you learned the core concept (CronJob → Job → Pod), the manual step-by-step flow, and a hands-on exercise that ran a Python script on a schedule. You can troubleshoot the most common pitfalls — image pulls, retry policies, timezones, and concurrency — and you know when a CronJob beats a Deployment or native cron.

As a next step, think about making your batch jobs more robust: add resource limits (you already did), but also consider using volumes for persistent output, or sending results to external systems like S3. And when your batch job needs to process a queue of messages, you'll want a Deployment with multiple replicas and a message broker — that's exactly what the next lesson in this track covers.