Containerize Python Automation

Your Python automation script works perfectly on your laptop — until the day it doesn't. A teammate upgrades their system Python, a library pins a different dependency version, or the cron host has no requests installed. Suddenly your precious automation is the source of a production incident, and 'it works on my machine' is not an acceptable DevOps answer. The fix is to containerize Python automation: wrap your script, its interpreter, and every dependency into a single, immutable artifact that behaves identically anywhere Docker runs. This lesson teaches you the core mental model, a step-by-step build and run workflow, and the troubleshooting you'll need to keep your automation deterministic in real pipelines.

The problem this lesson solves

If you've been building automation with Python, you know the pain of environment drift. A script that runs today may break tomorrow when pip install grabs a new version of a transitive dependency. Virtual environments help, but they're tied to a specific base OS and often require manual activation. In a DevOps context, you need your automation to be reproducible — the same input should produce the same output, every time, regardless of where it runs.

Containerization solves this by bundling your application with its entire runtime: the OS libraries, the Python interpreter, and every pip package. This isn't just about avoiding the 'works on my machine' problem; it's about making your automation a first-class citizen in CI/CD, scheduled jobs, and serverless-like execution. Containerize Python automation so you can ship changes with confidence, roll back quickly, and scale your scripts horizontally without rewriting them.

Pro tip: When a bug appears in a containerized script, you can debug the exact same environment rather than guessing which dependency version caused the issue. That alone saves hours of DevOps debugging.

Core concept / mental model

Think of a Docker image as a frozen snapshot of your automation's entire universe — the OS, the libraries, the code, and the configuration. A container is that snapshot, running. The image is defined by a Dockerfile, a declarative recipe that Docker reads line by line to build your environment. Every line creates a layer, and layers are cached. This layer-based design is the key to efficient builds and small images.

The mental model: your script is the heart, but the image is the whole body. Without the proper runtime, your script is just a dead file. With a container, you give it a lightweight, self-contained body that can run on any Linux machine with Docker. You don't need to install Python or pip on the host — the container carries its own.

Why this matters for DevOps

In DevOps, automation must be deterministic. You might run the same script in a cron job, a CI pipeline, and a Kubernetes pod. Without containers, each environment introduces variables: different OS package versions, different Python minor versions, or missing system libraries. By containerizing, you eliminate that variance and make your automation a reliable building block in your infrastructure.

How it works step by step

Let's walk through the process of containerizing a Python automation script. We'll use a simple example that reads a file, processes data, and writes a result — a common pattern in ETL or log analysis.

Step 1: Start with a base image

The first line of your Dockerfile specifies the base image. For Python automation, you want a small, official image like python:3.10-slim. The slim variant removes common bloat to reduce size. Avoid python:latest because it changes unpredictably; pin a specific minor version for reproducibility.

FROM python:3.10-slim

Step 2: Set a working directory and copy dependencies

Create a directory inside the container, copy your requirements.txt, and install dependencies before copying your source code. This leverages Docker's layer caching: if your dependencies don't change, Docker reuses the cached layer instead of reinstalling everything, making builds much faster.

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

Step 3: Copy your script and set the entry point

Now copy your actual code and define what runs when the container starts. Use CMD or ENTRYPOINT — ENTRYPOINT is preferred for automation because you can append arguments from the command line.

COPY . .
ENTRYPOINT ["python", "./main.py"]

Step 4: Build and run

docker build -t my-automation .
docker run --rm -v $(pwd)/data:/data my-automation

The -v flag mounts a local directory into the container so your script can read input and write output to the host — essential for file-based automation.

Pro tip: Use --rm to automatically remove the container after it exits. For scheduled jobs, this keeps your system clean of stopped containers.

Hands-on walkthrough

Let's put the theory into practice. Create a working directory with these files.

The automation script

main.py — a simple script that reads data/input.txt, counts words, and writes the result to data/output.txt.

# main.py
import sys
from pathlib import Path

input_path = Path("/data/input.txt")
output_path = Path("/data/output.txt")

def count_words(text: str) -> int:
    return len(text.split())

def main() -> None:
    if not input_path.exists():
        print(f"Error: {input_path} not found", file=sys.stderr)
        sys.exit(1)
    text = input_path.read_text(encoding="utf-8")
    word_count = count_words(text)
    output_path.write_text(f"Word count: {word_count}\n", encoding="utf-8")
    print(f"Wrote word count {word_count} to {output_path}")

if __name__ == "__main__":
    main()

The requirements.txt

Even a script with no external dependencies should declare them — it documents your environment and prepares for future needs.

# requirements.txt
# No external dependencies for this example

The Dockerfile

FROM python:3.10-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .
ENTRYPOINT ["python", "./main.py"]

Build and run

echo "Hello world from your automation" > data/input.txt
mkdir -p data
docker build -t word-counter .
docker run --rm -v $(pwd)/data:/data word-counter

cat data/output.txt

Expected output:

Wrote word count 6 to /data/output.txt
Word count: 6

The script ran in a clean, isolated environment and produced deterministic output. Notice we didn't install Python on the host or activate a venv — Docker handled everything.

Compare options / when to choose what

Containerization isn't the only way to package Python automation. Let's compare it with common alternatives.

For DevOps automation that must run reliably in varied environments, Docker containers are the clear winner. They give you the strongest guarantees with minimal effort. Use virtual environments for local dev speed, and consider serverless when you don't want to manage any containers at all.

Pro tip: Even if you're not using containers in production yet, containerizing during development catches environment issues early — the 'it works on my machine' problem disappears before it ever reaches your pipeline.

Troubleshooting & edge cases

You'll hit a few common issues when containerizing Python automation. Here's how to diagnose and fix them.

docker: command not found

Docker isn't installed or isn't in your PATH. Install Docker Desktop (macOS/Windows) or the Docker Engine package (Linux). Ensure the Docker daemon is running.

pip install fails with network errors

Your base image may not have internet access in a restricted network. Consider using --network=host during build if you trust the network, or build with a --build-arg for a proxy. For production, use a private registry mirror.

The script runs, but can't write files outside the container

By default, containers are isolated. If your script tries to write to /data, but you didn't mount a volume, the data won't persist. Always use -v $(pwd)/data:/data (or a named volume) for I/O. For read-only scripts, you can skip the volume and just capture stdout.

Critical: Image size explosion

Every COPY and RUN adds a layer. If you copy the entire project directory (including data/, __pycache__, or .venv), your image balloons. Use a .dockerignore file:

__pycache__/
*.pyc
.venv/
data/
.git/

This keeps the image lean and avoids accidentally leaking secrets like .env files.

No such file or directory for python

Your ENTRYPOINT may reference an interpreter that isn't in the PATH inside the container. Use the absolute path like /usr/local/bin/python or rely on the official image's python alias which is already in PATH. Also, verify the line endings of main.py — CRLF breaks the shebang line in Linux containers.

What you learned & what's next

You've learned how to containerize Python automation: you understand the mental model of images and containers, you can write a Dockerfile with layered caching, you can build and run your script with mounted volumes for data persistence, and you can troubleshoot the most common pitfalls. You also know how containerization compares to other packaging methods and when to choose Docker for your DevOps automation.

This is a foundational skill that unlocks the next lesson: orchestrating your containerized automation. In the next step, you'll learn how to run multiple containers together with Docker Compose, schedule them, and integrate them into CI/CD pipelines. But first, practice what you've learned in the exercise below.