Parse Ansible Output

You've just run a 200-task Ansible playbook, and the terminal scrolls past faster than you can read. Buried in that wall of text are hostnames, changed flags, failure messages, and return values—the exact data you need to report status, trigger alerts, or feed a dashboard. Manually grepping for ok: or failed: is brittle and misses the structured truth Ansible already produces. In this lesson, you'll learn to parse and transform Ansible output with Python so you can turn noisy terminal logs into clean, actionable JSON, CSV, or custom summaries.

The problem this lesson solves

When you run ansible-playbook in a CI pipeline, operational dashboard, or audit script, the raw output is designed for human eyes—not for programmatic consumption. Consider the typical output:

PLAY [webservers] ***********************************************
TASK [Gathering Facts] *****************************************
ok: [web-01]
ok: [web-02]
TASK [Install nginx] *******************************************
changed: [web-01]
changed: [web-02]
PLAY RECAP ******************************************************
web-01 : ok=2    changed=1    unreachable=0    failed=0
web-02 : ok=2    changed=1    unreachable=0    failed=0

If you're automating a notification, generating a compliance report, or tracking drift over time, copy-pasting that text is useless. You need structured data: each host, which tasks succeeded, which failed, and what values the tasks returned. This is the core pain—parsing and transforming Ansible output—and Python, with its standard library and a few key flags, makes it painless.

Beyond the immediate chaos, the deeper problem is repeatability. A human can skim and understand the recap, but a script must be deterministic. One syntax change in Ansible's output format can quietly break your monitoring feed. The solution is to stop parsing text at all: Ansible can emit JSON, and Python was born to consume JSON.

Core concept / mental model

Think of Ansible as having two output modes:

1. Human mode: colorized, indented, with task names and host lines—iot for eyeballs.
2. Machine mode: structured JSON with every event, result, and play recap in a predictable schema.

Your mental model: Ansible is an API that prints. When you run ansible-playbook, the process is really producing a stream of events. Without the --json flag, you lose that structure. With it, each line (or chunk) can be parsed by Python’s json module into dictionaries and lists.

Visualize the flow:

ansible-playbook --json → stdout (JSON) → Python subprocess → json.loads() → dict → transform/filter → CSV/JSON/Notification

This lesson focuses on the parse step (turning raw bytes into Python objects) and the transform step (reshaping those objects into something your automation needs).

How it works step by step

Step 1: Get structured output

The modern Ansible CLI (2.14+) supports a native JSON callback plugin. Run:

ansible-playbook -i inventory.yml site.yml --json

This emits a single JSON object to stdout, which includes keys like plays (a list), stats (per-host play recap), and custom_stats. If your Ansible version is older or you use ansible ad-hoc commands, you might need -o for a single-line JSON per task, but --json is the cleanest.

Step 2: Capture with subprocess

Use Python's subprocess.run() to execute the playbook and capture stdout and stderr separately. Always set capture_output=True and text=True so you get strings, not bytes.

Step 3: Parse with json.loads()

Feed stdout to json.loads(). If the output is not valid JSON (due to warnings or errors), you'll get a JSONDecodeError—so first check the return code and stderr.

Step 4: Transform

Now that you have a dictionary, you can iterate through plays, each containing tasks and hosts. For every task, you can access task.name, hosts[host].status, hosts[host].result, etc. Transform this into a flat list of records:

records = []
for play in data["plays"]:
    for task in play["tasks"]:
        task_name = task["task"]["name"]
        for host, host_data in task["hosts"].items():
            records.append({
                "play": play.get("play", {}).get("name"),
                "task": task_name,
                "host": host,
                "status": host_data.get("status"),
                "changed": host_data.get("changed", False),
            })

Step 5: Output to your target

Finally, write to CSV, print a summary, or post to a webhook. The data is now yours.

Hands-on walkthrough

Example 1: Parse a playbook’s JSON and list failures

Let’s create a small playbook and a Python script that runs it and extracts all failed hosts and tasks.

playbook.yml:

- name: Test play
  hosts: localhost
  tasks:
    - name: always passes
      debug:
        msg: "Hello"
    - name: might fail
      command: /bin/false
      ignore_errors: true

parse_ansible.py:

import subprocess
import json
import sys

result = subprocess.run(
    ["ansible-playbook", "--json", "-i", "localhost,", "playbook.yml"],
    capture_output=True,
    text=True
)

if result.returncode != 0:
    print("Playbook failed with return code", result.returncode, file=sys.stderr)
    print(result.stderr, file=sys.stderr)
    sys.exit(1)

try:
    data = json.loads(result.stdout)
except json.JSONDecodeError as e:
    print("Failed to parse JSON:", e, file=sys.stderr)
    print("Raw stdout:", result.stdout[:500], file=sys.stderr)
    sys.exit(1)

failed = []
for play in data.get("plays", []):
    for task in play.get("tasks", []):
        task_name = task.get("task", {}).get("name", "?")
        for host, host_data in task.get("hosts", {}).items():
            if host_data.get("status") == "failed":
                failed.append({"host": host, "task": task_name, "msg": host_data.get("result", {}).get("msg")})

print("Failures:", len(failed))
for f in failed:
    print(f"- {f['host']}: {f['task']} -> {f['msg']}")

Expected output (when run on your machine with the playbook above):

Failures: 1
- localhost: might fail -> failed

Example 2: Transform into CSV for reporting

Now let’s expand to produce a CSV file for every host/task result.

parse_to_csv.py:

import subprocess
import json
import csv
import sys

result = subprocess.run(
    ["ansible-playbook", "--json", "-i", "localhost,", "playbook.yml"],
    capture_output=True, text=True
)
if result.returncode != 0:
    sys.exit("Playbook failed")

data = json.loads(result.stdout)

with open("report.csv", "w", newline="") as f:
    writer = csv.DictWriter(f, fieldnames=["play", "task", "host", "status", "changed"])
    writer.writeheader()
    for play in data.get("plays", []):
        play_name = play.get("play", {}).get("name")
        for task in play.get("tasks", []):
            task_name = task.get("task", {}).get("name")
            for host, host_data in task.get("hosts", {}).items():
                writer.writerow({
                    "play": play_name,
                    "task": task_name,
                    "host": host,
                    "status": host_data.get("status"),
                    "changed": host_data.get("changed", False),
                })

print("Wrote report.csv")

Run it and inspect report.csv. You’ll see something like:

play,task,host,status,changed
Test play,always passes,localhost,ok,False
Test play,might fail,localhost,failed,False

Example 3: Using ansible-doc -j to parse module docs

Sometimes the output to parse isn’t a playbook run but ansible-doc. Let’s extract the description from a module’s JSON.

parse_doc.py:

import subprocess
import json

result = subprocess.run(
    ["ansible-doc", "-j", "copy"],
    capture_output=True, text=True
)
if result.returncode == 0:
    data = json.loads(result.stdout)
    module_name = list(data.keys())[0]
    doc = data[module_name].get("doc", {})
    print(f"Module: {module_name}")
    print(f"Description: {doc.get('description')}")
else:
    print("Error fetching docs", result.stderr)

Output:

Module: copy
Description: ['Copy files to remote locations']

Compare options / when to choose what

Recommendation: for most automation scripts, start with --json + subprocess. It’s zero-dependency and covers 80% of use cases. If you need to handle thousands of hosts or real-time streaming, invest in ansible-runner.

Troubleshooting & edge cases

Problem: JSONDecodeError even with --json

Cause: Ansible may print warnings to stdout (e.g., "DEPRECATION WARNING") or the callback plugin isn’t properly loaded. Fix: Capture stderr separately and check result.stderr first. Use --json with the default callback plugin, or explicitly set ANSIBLE_STDOUT_CALLBACK=default.

Problem: The --json flag isn’t recognized

Cause: Your Ansible version is older (pre-2.14) or you’re using an ad-hoc command (which doesn’t support --json for all modules). Fix: Upgrade Ansible, or fall back to -o output and parse line-by-line (but this is fragile).

Problem: Output has non-JSON content before the JSON object

Cause: Some ansible plugins print banners. Fix: Use json.JSONDecoder to find the first { and parse from there:

decoder = json.JSONDecoder()
# assume result.stdout starts with some noise
for i, char in enumerate(result.stdout):
    if char == '{':
        data, idx = decoder.raw_decode(result.stdout[i:])
        break

Problem: Missing keys like changed for some hosts

Cause: Not every task has a changed attribute (e.g., debug tasks). Fix: Use .get() with defaults, as in the examples above.

Problem: Unicode/encoding errors

Cause: Remote hosts with non-UTF-8 output. Fix: Set text=True and encoding='utf-8' in subprocess.run(), and sanitize bytes if needed.

What you learned & what's next

You’ve learned to parse and transform Ansible output with Python: you can now capture a playbook’s JSON, navigate its nested structure, extract failures, and produce a CSV summary. You also know how to compare CLI parsing against ansible-runner and when to use which. This skill directly feeds into your DevOps automation toolbox—imagine writing a script that sends a Slack alert every time a playbook fails, or a scheduler that stores execution times in a database.

In the next lesson, you’ll build on this by orchestrating multiple Ansible runs and processing their combined output—turning single-run parsers into powerful multi-step automation pipelines. Keep your JSON parser handy; you’ll reuse it soon.