Deploy Code with systemd

You've written a solid Python script that automates a critical task, tested it locally, and pushed it to your repository. But the moment it needs to run continuously in production, you hit the classic DevOps wall: the script dies, nothing restarts it, and it never survives a reboot. Manually babysitting processes with nohup and & is a recipe for silent failures, and using a cron job that starts a new process every minute creates duplicates and race conditions. This lesson shows you how to deploy Python code with systemd, the standard init system on modern Linux distributions, turning your scripts into first-class, managed services that are resilient, observable, and production-ready.

The problem this lesson solves

Unmanaged Python scripts are a liability in any serious deployment. When you run a script with python app.py, it runs in your terminal's process group. Close the terminal, and the process gets a SIGHUP signal and dies. Even if you use nohup to ignore that signal, a crash in the script — an unhandled exception, an out-of-memory kill, or a segfault — leaves nothing behind but a dead process and an angry support ticket.

Cron jobs add their own set of problems. A cron entry that runs your script every minute will happily start a new instance even if the previous one is still running, leading to overlapping executions and corrupted state. Cron gives you no visibility into the script's health, no automatic restart policy, and no proper logging beyond mail-to-root. It's a scheduling tool, not a process manager.

What you need is a robust, declarative way to tell the OS: "Run this Python script, keep it alive, restart it if it fails, start it at boot, and capture its output." That's exactly what systemd provides.

Core concept / mental model

Think of systemd as the Linux operating system's factory floor supervisor. When your machine boots, systemd is the first process (PID 1) and is responsible for starting everything else. It reads unit files — declarative configuration files with [Unit], [Service], and [Install] sections — to learn how to manage a service. A unit file is like a work order: it specifies the command to run, the working directory, the user to run as, and the policy for restarts.

Your Python script, by contrast, is the worker. It doesn't need to know how to daemonize itself, write PID files, or handle logging — systemd takes care of those concerns. This separation of concerns is key: the unit file defines the deployment contract, and the Python code focuses on business logic.

Here's a word diagram of the flow:

System boot / systemctl start myapp
        |
        v
[systemd] --> reads /etc/systemd/system/myapp.service
        |
        |-- defines: ExecStart, Restart, Environment, User
        |
        v
[Python script] --> runs as a child of systemd
        |
        |-- logs to stdout/stderr
        |
        v
[systemd journal] --> captures logs, manages restarts

The key insight is that your script should run in the foreground (not daemonize itself) and write logs to stdout/stderr. systemd will handle backgrounding and log collection through the journal.

How it works step by step

Deploying a Python script with systemd involves a systematic sequence of steps. Each step builds on the previous one, so it's important to get the basics right.

1. Write a production-ready Python script - The script should be executable, have a proper shebang (#!/usr/bin/env python3), and run in a loop if it's a long-running service. For a one-shot task, it can exit normally. - Use if __name__ == "__main__": to allow the script to be run directly or imported. - Make the script respect environment variables for configuration, so systemd can inject them via Environment=.

2. Create a dedicated system user (optional but recommended) - Running services as root is bad practice. Create a service account with sudo useradd -r -s /usr/sbin/nologin myappuser so the service runs with minimal privileges.

3. Place the script in a standard location - Install the script to an application directory like /opt/myapp/myapp.py and make it executable with chmod +x.

4. Create the systemd unit file - The unit file tells systemd what to run, how to run it, and when to restart it. Start with a [Unit] section to describe the service and its dependencies, then [Service] with the execution details, and finally [Install] to enable start-at-boot.

5. Reload systemd and enable the service - Run systemctl daemon-reload to make systemd aware of the new unit file, then systemctl enable myapp.service to create symlinks for boot start, and systemctl start myapp.service to launch it.

6. Check status and logs - Use systemctl status myapp.service for a quick health check and journalctl -u myapp.service -f to follow logs and verify the script is behaving as expected.

7. Verification and iteration - Test that the service survives a crash by killing its PID and watching systemd restart it. Check that it starts at boot by rebooting a test instance.

Hands-on walkthrough

Let's walk through a complete example. We'll create a simple Python web server using Flask that responds to requests and logs them.

1. The Python script — myapp.py:

#!/usr/bin/env python3
"""A minimal Flask web service designed to run under systemd."""
import os
import sys
from flask import Flask, jsonify

# Use an environment variable for port configuration (set in the unit file)
PORT = int(os.environ.get("MYAPP_PORT", "5000"))

app = Flask(__name__)

@app.route("/health")
def health():
    """Health check endpoint that returns 200 OK."""
    return jsonify({"status": "healthy"})

@app.route("/")
def index():
    """Return a simple greeting."""
    app.logger.info("Root endpoint hit")
    return jsonify({"service": "myapp", "port": PORT})

if __name__ == "__main__":
    # Run in foreground; systemd sends signals when it wants to stop us
    app.run(host="0.0.0.0", port=PORT)

2. Place it in the app directory and make it executable:

sudo mkdir -p /opt/myapp
sudo cp myapp.py /opt/myapp/myapp.py
sudo chmod +x /opt/myapp/myapp.py

3. Create the systemd unit file at /etc/systemd/system/myapp.service:

[Unit]
Description=My Python Flask Service
After=network.target

[Service]
User=myappuser
Group=myappuser
WorkingDirectory=/opt/myapp
Environment=MYAPP_PORT=8080
ExecStart=/usr/bin/python3 /opt/myapp/myapp.py
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

4. Enable and start the service:

sudo systemctl daemon-reload
sudo systemctl enable myapp.service
sudo systemctl start myapp.service

5. Verify it works:

sudo systemctl status myapp.service
curl http://localhost:8080/health
journalctl -u myapp.service -f

Expected output from status starts with:

● myapp.service - My Python Flask Service
     Loaded: loaded (/etc/systemd/system/myapp.service; enabled; vendor preset: enabled)
     Active: active (running) since ...
   Main PID: 12345 (python3)
     Memory: 15.2M

And the health check returns {"status": "healthy"}.

6. Test the restart policy:

Kill the main process and watch systemd revive it:

sudo kill $(systemctl show -p MainPID --value myapp.service)
sleep 6  # Wait for RestartSec
systemctl is-active myapp.service  # Should print 'active'

This proves the service restarts automatically under the on-failure policy.

Compare options / when to choose what

While systemd is the standard, it's not the only option. The table below compares systemd with other common approaches to running Python services.

When to choose systemd: - Your deployment target is a bare-metal or virtual machine running a modern Linux distribution. - You need the service to start at boot and auto-restart on failure. - You want tight integration with the OS ecosystem (users, permissions, resource limits).

When to look elsewhere: - If you're deploying to containerized environments (Kubernetes, Docker Swarm), obviously the container orchestrator takes over this role. - For legacy systems without systemd (e.g., older SysV init), consider supervisor or upstart.

Troubleshooting & edge cases

Even with a clean setup, things can go wrong. Here are the most common pitfalls and how to fix them.

• Service starts then immediately exits with code 1 — Your Python script likely threw an exception. Check the logs with journalctl -u myapp.service -e. If the error is a missing module, ensure the Python environment (e.g., a virtualenv) is correctly specified in ExecStart. For a virtualenv, set ExecStart=/opt/myapp/venv/bin/python /opt/myapp/myapp.py.

• Permission denied even though the script is executable — The service runs under the user specified by User=. That user must have read and execute permissions on the script and its directory. If you plan to write files, the WorkingDirectory and any output paths must be writable by that user.

• The service doesn't start at boot — You probably forgot to run systemctl enable. Enabling creates the WantedBy symlinks. Another cause is the After=network.target dependency not being met if network is not ready; tune with After=network-online.target and Wants=network-online.target.

• Overlapping runs of a one-shot script — If you need a periodic task that must not overlap with a previous run, add a lock file. systemd doesn't manage this natively. Use a file lock in Python or use systemd.timer (a cron-like unit) which can be configured to avoid overlaps with Persistent=true.

• Environment variables not visible to the script — The Environment= line in the unit file only sets variables. They are not exported to child processes unless you start your script via a shell. If your script (or its child processes) needs them, use EnvironmentFile= to load from a file, or invoke the script via /bin/sh -c 'set -a; source /etc/default/myapp; exec python myapp.py' — though better to simply let your Python script parse a config file.

• systemctl status shows activating (auto-restart) repeatedly — This means the service is failing too quickly. Check logs; often it's a port already in use. Use RestartSec to add a delay and StartLimitIntervalSec/StartLimitBurst to limit retries.

• The script doesn't read stdin — Not a problem for services, but if your script expects input, systemd by default has no stdin. Pass configuration via environment variables or a config file, not interactive input.

What you learned & what's next

You've learned the core concept: deploying Python code with systemd means declaring a service contract in a unit file that the OS manages on your behalf. You now can:

• Write a Python script that runs in the foreground and logs to stdout.
• Create a systemd unit file with proper [Unit], [Service], and [Install] sections.
• Start, enable, and monitor the service using systemctl and journalctl.
• Configure auto-restart with Restart=on-failure.
• Verify that the service survives crashes and starts at boot.

These skills form the foundation of robust deployments in any Linux environment. Your next step in this Python for DevOps automation track is to explore how to automate the deployment process itself — perhaps using Ansible to push unit files and scripts to many hosts, or diving into containerization with Docker later on. Each lesson builds on the last, so keep practicing by creating your own unit files for different kinds of scripts.

Now, go deploy something that stays up!