Files, Paths & Error Handling

Ever written a script that worked flawlessly on your laptop, only to crash in production with a cryptic FileNotFoundError or PermissionError? You're not alone. For DevOps engineers, file operations and path handling are the backbone of automation — from reading config files to writing logs and deploying artifacts. But without a solid grasp of Python's pathlib and exception handling, your automation can fail in ways that are hard to debug and even harder to recover from. This lesson gives you the tools to handle files, paths, and errors like a pro.

The problem this lesson solves

Automation scripts often need to:

• Read configuration files (e.g., JSON, YAML, INI)
• Write logs or output files
• Copy, move, or delete artifacts during deployments
• Validate paths before acting on them

In DevOps, these tasks are non-negotiable. If your script assumes a file exists but it doesn't — or if a directory path is wrong — you get crashes. Worse, in a pipeline, a single unhandled exception can halt the entire deployment. The pain is real: silent failures, unreadable tracebacks, and scripts that only work on the author's machine.

Python's standard library gives you two main tools: the classic os.path and the modern pathlib. Pair them with robust exception handling using try/except blocks, and you can build automation that's both deterministic and resilient. This lesson covers what you need to know to stop fighting filesystem issues and start writing automation that just works.

Core concept / mental model

Think of the filesystem as a tree. Each branch is a directory, each leaf is a file. A path is like a set of directions to a specific leaf: either absolute (starting from the root) or relative (starting from your current location).

pathlib is Python's object-oriented interface to the filesystem. Instead of juggling strings, you work with Path objects that have intuitive methods like .exists(), .read_text(), and .write_text(). It's the modern way — recommended since Python 3.6 and the de facto standard for new code.

Error handling is your safety net. When something goes wrong — a file is missing, a directory is unwritable — Python raises an exception. If you don't catch it, your script crashes. If you do catch it, you can log a friendly message, skip the step, or retry.

Here's a mental model that ties them together:

• Path: a Path object that may or may not point to something real.
• Operation: you attempt a file operation (read, write, list, etc.).
• Exception: Python interrupts your program and hands you an error object.
• Handler: your except block that decides what to do — log, recover, or fail gracefully.

You don't need to know everything that can go wrong — you need to expect that something will, and plan for it.

How it works step by step

Let's walk through the logical flow for any file operation in your automation:

1. Construct the path — Use Path() to build a file or directory path. Prefer absolute paths or be explicit about the current working directory.
2. Check existence — Call .exists() to see if the file or directory is there. Optionally check .is_file() or .is_dir() to verify the type.
3. Perform the operation — Read, write, list, copy, move — whatever your automation needs. Always wrap this in a try block.
4. Handle exceptions — Catch specific exceptions like FileNotFoundError, PermissionError, or IsADirectoryError. Handle each with an appropriate action: log, create the directory, or abort with a clear message.
5. Clean up — Use with statements to close files automatically, even if an error occurs.

Cause → effect works like this: if you try to open a file that doesn't exist, Python raises FileNotFoundError. If you don't catch it, the script stops and the traceback points to the exact line. If you do catch it, you control the outcome.

Hands-on walkthrough

Let's put this into practice. We'll build a small script that reads a config file, writes a log, and handles common errors.

Example 1: Basic file reading with error handling

from pathlib import Path

config_path = Path("/etc/myapp/config.json")

try:
    content = config_path.read_text()
    print(f"Config loaded: {content}")
except FileNotFoundError:
    print(f"ERROR: Config file not found at {config_path}")
except PermissionError:
    print(f"ERROR: No permission to read {config_path}")

Expected output (if missing):

ERROR: Config file not found at /etc/myapp/config.json

Example 2: Writing files safely

from pathlib import Path
import datetime

log_dir = Path("/var/log/myapp")
log_file = log_dir / f"app-{datetime.date.today()}.log"

# Create directory if it doesn't exist
log_dir.mkdir(parents=True, exist_ok=True)

try:
    with log_file.open("a") as f:
        f.write("Deployment started\n")
    print(f"Logged to {log_file}")
except PermissionError:
    print(f"ERROR: Cannot write to {log_file}. Check permissions.")

Expected output:

Logged to /var/log/myapp/app-2025-03-14.log

Example 3: Walking a directory tree

from pathlib import Path

def list_artifacts(base_dir: Path, pattern: str):
    artifacts = list(base_dir.rglob(pattern))
    if not artifacts:
        print(f"No artifacts matching '{pattern}' found in {base_dir}")
    else:
        for artifact in artifacts:
            print(f"Found: {artifact}")

# Use current directory as an example
list_artifacts(Path("."), "*.yml")

Expected output (depending on your files):

Found: config/deploy.yml
...

These examples show the pattern: build a Path, wrap operations in try/except, and use with for automatic cleanup. You can adapt this to any file-based automation task.

Compare options / when to choose what

When to use what: If you're writing new code, always prefer pathlib. It's the standard and makes your intent clear. If you're maintaining legacy code that already uses os.path, stick with it to avoid mixing styles — but plan to migrate.

For error handling, you have another choice: try/except vs. if checks. Checking before you operate (e.g., .exists()) is good for flow control, but it's not a substitute for exception handling — race conditions can still occur. Use both: pre-check for clarity, and wrap in try for safety.

Troubleshooting & edge cases

Common errors and how to fix them

• FileNotFoundError — The file or directory doesn't exist. Fix: use .exists() before reading, or catch the exception and create the file if needed.
• PermissionError — You don't have the right to read/write. Fix: check file permissions, run as a different user, or catch and log a meaningful message.
• IsADirectoryError — You tried to open a directory as a file. Fix: use .is_file() to verify before opening.
• NotADirectoryError — You tried to treat a file as a directory, e.g., Path('/path/to/file') / 'subdir'. Fix: validate the path with .is_dir().
• Path.exists() on a broken symlink — Returns False. Use .exists() with follow_symlinks=False if you need to detect symlinks.

Edge cases in path handling

• Relative vs absolute paths — Relative paths depend on the current working directory, which can change in pipelines. Use Path(__file__).resolve().parent to get the script's directory and build absolute paths from there.
• Special characters — Spaces or Unicode in filenames are handled fine by pathlib, but be careful when passing paths to shell commands — use subprocess with a list, not a string.
• Trailing slashes on directories — pathlib normalizes them, but os.path can trip up.

Pro tip: When writing automation, always log the exact path and the action you're attempting. A good error message is worth a hundred lines of debugging.

What you learned & what's next

Now you can:

• Use pathlib to construct, check, and manipulate paths in a readable, object-oriented way.
• Handle errors gracefully with try/except, catch specific exceptions, and apply the right fallback.
• Read and write files safely using with blocks, ensuring clean resource management.
• Walk directories with rglob() to find files recursively — essential for artifact collection and log management.

These skills are the foundation for DevOps automation: you'll rely on them for config management, log analysis, and deployment scripts. Next in the track, you'll learn how to work with external commands and subprocesses — taking your automation beyond file operations to interact with the shell, tooling, and system services.

Practice this lesson by writing a short script that reads a JSON config, validates a required field, and writes a log entry — handle both missing files and bad JSON. You'll build confidence and prepare for more advanced automation patterns.