Schedule Python Scripts with Cron

You've spent hours perfecting a Python script that collects metrics, cleans up temp files, or pings an API — but who runs it at 3 AM? Leaving a terminal open or relying on a colleague to run it manually is a recipe for disaster. The answer is cron, the Unix job scheduler that's been quietly powering automation for decades. In this lesson, you'll learn how to schedule Python scripts with cron, including the critical gotchas around environment variables and logging, so your automation runs reliably even when you're asleep.

The problem this lesson solves

Manual script execution is a bandwidth killer. If a script must run "every hour" or "every Monday at 9 AM," someone has to remember to do it — and humans forget. Worse, a missed run can cascade into stale dashboards, unfilled backups, or unrotated logs. Cron exists to make scheduled execution deterministic and OS-native. It's the DevOps equivalent of setting a fire-and-forget timer for your Python code.

But cron isn't just about "running a script." It's about running it with the right environment, capturing output, and failing loudly when something breaks. Without those safeguards, a cron job that silently fails is worse than no job at all. This lesson fills that gap.

Core concept / mental model

Think of cron as a tiny scheduler daemon that wakes up every minute to check a list of instructions. The list — your crontab — is a simple text file with one line per job. Each line tells cron two things: when to run and what to run.

The when part is a compressed timeline: five fields that represent minute, hour, day of month, month, and day of week. For example, 0 3 * * * means “run at 3:00 AM every day.” Each asterisk acts as a wildcard.

Here's the mental shortcut: read the five fields left-to-right as a sentence. 15 4 * * 1 translates to “at minute 15 of hour 4, on any day of month, in any month, on Monday.” Once you internalize that, you can read any cron schedule in seconds.

The what part is simply the shell command to execute. When that command is python /path/to/script.py, you've got a scheduled Python job. But here's the nuance: cron runs with a minimal environment, so you often need to specify the full path to the Python interpreter and set environment variables yourself.

Pro tip: Always use the absolute path to your script and your Python executable in cron. A relative path that works in your terminal will break under cron because the working directory is often / or your home directory, not your project folder.

How it works step by step

1. Write your Python script that does the work — e.g., fetch_metrics.py.
2. Open your user's crontab with crontab -e. This launches your default editor (usually Vim or Nano).
3. Add a line with the schedule and the command. Example: */5 * * * * /usr/bin/python3 /home/dev/fetch_metrics.py.
4. Save and exit. Cron validates the syntax; if you made a mistake, it may reject the entry.
5. Verify with crontab -l to list your active jobs.
6. Check output — by default cron emails the output, but on modern systems you'll often write logs to a file instead.

The syntax allows special shortcuts too: @hourly, @daily, @reboot (runs at system start). These are perfect for maintenance tasks like log rotation or starting a long-lived Python watcher.

Cron also supports ranges (1-5), lists (1,15), and steps (*/10). For example, 0 0 * * 5 runs Friday midnight, while */30 9-17 * * * runs every half hour during business hours.

Hands-on walkthrough

Let's put theory into practice. First, create a simple Python script that logs its execution and writes a status file. We'll schedule it to run every 5 minutes.

#!/usr/bin/env python3
# report.py — writes a timestamped report to a log file
import datetime

LOG_PATH = "/home/dev/reports.log"
with open(LOG_PATH, "a") as f:
    now = datetime.datetime.now().isoformat()
    f.write(f"Report generated at {now}\n")
print("Report written.")

Before scheduling, test the script manually:

python3 /home/dev/report.py
cat /home/dev/reports.log
# Expected output:
# Report generated at 2025-01-01T12:00:00.123456

Now open your crontab:

crontab -e

Add this line (adjusting paths to match your system):

*/5 * * * * /usr/bin/python3 /home/dev/report.py >> /home/dev/report_cron.log 2>&1

Save and exit. The >> /home/dev/report_cron.log 2>&1 redirects both standard output and standard error to a log file so you can debug later.

After five minutes, check that it ran:

cat /home/dev/report_cron.log
# Should contain the same output as your manual run

If nothing appears, first check your Python path:

which python3
# Usually /usr/bin/python3 or /usr/local/bin/python3

Also note that if your script needs environment variables (like API keys), they won't be available by default. You can source a .env-style file or export variables directly in the crontab line:

*/5 * * * * export API_KEY=abc123; /usr/bin/python3 /home/dev/report.py

Pro tip: A good practice is to create a wrapper shell script that activates a virtualenv and sets environment variables, then call that from cron. It keeps your crontab clean and testable.

#!/bin/bash
# /home/dev/run_reports.sh
cd /home/dev
source venv/bin/activate
export API_KEY=abc123
python report.py

Then your cron line simplifies to:

*/5 * * * * /home/dev/run_reports.sh >> /home/dev/job.log 2>&1

Compare options / when to choose what

Cron isn't the only game in town. Here's how it stacks up against common alternatives for scheduling Python scripts:

When to choose cron: your script is self-contained, runs on one server, and you need zero ceremony. Cron is the Swiss Army knife of scheduling — every Unix admin knows it, and it's debugged by decades of usage.

When might you pick a systemd timer instead? If you want that same no-frills approach but need better integration with services, logging via journald, or the ability to restart on failure. For anything distributed or needing retries, move to Celery or Airflow.

Pro tip: Start with cron for single-server tasks. If you later need retries or monitoring, wrap your script in a loop with time.sleep() and launch it under cron at @reboot — a simple, effective hybrid.

Troubleshooting & edge cases

Cron errors usually fall into a few buckets. Here's how to diagnose them fast.

Job didn't run at all — First, check the crontab syntax with crontab -l. Ensure the line uses the correct time fields. A common mistake is adding a 6th field for the year (not allowed). Verify the cron service is running: systemctl status cron (Debian/Ubuntu) or sudo cron -l (some distros).

Script runs but no output — This is almost always an environment problem. Cron uses a minimal PATH (often just /usr/bin:/bin). If your script calls other executables outside those directories, they'll fail silently. Solution: use absolute paths or set PATH at the top of your crontab.

Python module not found — You might have installed a package with pip install, but cron is using the system Python, not your virtualenv. Always point directly to the Python interpreter inside your venv, e.g., /home/dev/venv/bin/python. For safety, add a shebang line and make the script executable.

Spaces in paths break the line — Cron treats spaces as field separators. If your script path contains a space, escape it with \ or use quotes:

0 0 * * * "/path/with space/script.py"

Output email spam — By default, cron emails each job's stdout/stderr. On a busy server this can flood your inbox. Redirect output to a log file (as shown earlier) or set MAILTO="" in your crontab to disable emailing.

Timezone confusion — Cron uses the system timezone, not whatever your Python process might assume. If your script relies on UTC, ensure the server is set to UTC, or handle timezone conversion inside Python using pytz or zoneinfo.

Job still doesn't run after fixes — Add debug logging to a file, then check the file permissions. Cron may be running as a different user, so ensure the script and the log file are writable by the crontab owner. When in doubt, run the cron line manually and inspect the error.

What you learned & what's next

You now have the power to schedule Python scripts with cron — from writing a crontab entry to troubleshooting environment pitfalls. You know how to specify schedules with the five-field syntax, redirect output for later review, and handle virtualenv and environment variable quirks. This clears the way for hands-free automation in your DevOps toolkit.

As a next step, you'll build on this by packaging your scheduled scripts as systemd services or adding monitoring with tools like health checks. That's the bridge from “it runs” to “it runs and we know it runs.” Stay tuned for the next lesson in the track, where we'll explore how to make your automation self-aware with alerts and retries.

Remember: cron is only as reliable as the environment you give it. Lock down paths, logs, and variables — then let it run forever.