Running External Commands with Python's Subprocess

Okay, here is the article for PythonSkillset.com.

Running External Commands Like a Pro with Python's Subprocess

So you’re writing a Python script, and you hit a wall. You need to do something that Python can’t easily do by itself, like compress a folder, run a system ping test, or call a specialized tool like ffmpeg. The obvious next thought is, "Can I just run a command from my terminal inside my script?"

You absolutely can, and Python’s subprocess module is your ticket. It’s the modern, reliable way to talk to your operating system. Let’s break it down without the fluff.

The Old Way vs. The Right Way

If you’ve been around Python for a while, you might have seen os.system() or os.popen(). These work, but they have problems. They’re not secure for user input, and they give you very little control.

The subprocess module, however, is the Swiss Army knife. It’s designed to replace all those older methods with one consistent tool.

The star of the show is subprocess.run(). Think of it as the "fire and forget" command. You tell it what to run, and it waits politely for the command to finish, then hands you a result object.

Your First Command: subprocess.run()

Let’s start simple. Imagine you want to list the files in your current directory. In your terminal, you’d type ls on macOS/Linux or dir on Windows. Here’s how you do it in Python:

import subprocess

result = subprocess.run(["ls", "-l"], capture_output=True, text=True)

Let’s unpack that.

• The first argument is a list of strings. The first item is the command (ls), and every following item is an argument (-l). This is the safest way to pass commands because it avoids shell injection issues.
• capture_output=True tells Python to grab the text that the command would normally print to the screen.
• text=True ensures the output is treated as a string, not a bunch of bytes.

Now, what’s in that result object? It has a few key pieces:

• result.returncode: A zero usually means success. Anything else? An error happened.
• result.stdout: The printed output of the command (a string).
• result.stderr: Any error messages the command spit out.

Let’s print out what ls gave us:

print("Files in current directory:")
print(result.stdout)

Simple, right? You now have the power of your operating system inside your Python script.

A Real-World Example: Processing a Video File

A common task for us at PythonSkillset is automating media processing. Let’s say you want to get the duration of a video file using ffmpeg. You could run a command like this manually, but automating it is way more powerful.

import subprocess

video_file = "my_vacation.mp4"
# ffprobe comes with ffmpeg and gets file info
command = [
    "ffprobe", 
    "-v", "error", 
    "-show_entries", "format=duration", 
    "-of", "default=noprint_wrappers=1:nokey=1", 
    video_file
]

result = subprocess.run(command, capture_output=True, text=True)

if result.returncode == 0:
    duration_seconds = float(result.stdout.strip())
    print(f"The video is {duration_seconds:.2f} seconds long.")
else:
    print(f"Error: {result.stderr}")

See what happened? We built a clean list of arguments. subprocess.run executed the command, and we safely parsed the output. This is a perfect example of why using a list is safer than a string. If your video_file variable was named my_vacation.mp4; rm -rf /, using a string could be catastrophic. Using a list prevents that entirely.

When You Need a Shell

Sometimes, you really do need the shell. For example, if you want to use a pipe (|) or a wildcard (*.txt). You can do this, but be careful.

# Dangerous: Only use if you trust the input completely!
command_string = "ls -la | grep .py"
result = subprocess.run(command_string, shell=True, capture_output=True, text=True)
print(result.stdout)

The shell=True flag passes your command to the system shell. It’s powerful, but it opens the door to shell injection vulnerabilities. As a general rule at PythonSkillset, we recommend avoiding shell=True unless you really, really need it, and you control every part of the input string.

Beyond run(): Spawning a Process

What if you want to run a command and not wait for it? Or interact with it while it’s still running? That’s where subprocess.Popen comes in. It’s the lower-level building block.

# Start a long-running process and let your script do other things
process = subprocess.Popen(["sleep", "10"])
print("The sleepy process is running in the background.")
# ... do other stuff ...
process.wait()  # Wait for it to finish when you're ready
print("The process is done.")

Popen gives you total control. You can write to its stdin, read from its stdout line by line, and manage it as a live process. For 95% of your daily tasks, subprocess.run() is all you need. But it’s good to know Popen is there for those tricky situations.

Wrapping Up

The subprocess module isn't just for system administrators. Every Python developer at PythonSkillset uses it to glue tools together, automate system tasks, and build powerful pipelines. Start with subprocess.run() and a list of arguments. Use capture_output to grab results. Only use the shell (shell=True) when you have to, and treat external input as untrusted.

You’ll find that the barrier between your Python script and the rest of your computer just melts away. Go ahead, give it a try.