How to Log Prompts and Completions as JSONL Audit Files in Python

Read a JSONL file of LLM prompt–completion pairs, compute totals and averages, then write an audit summary with timestamps.

Easy Python 3.9+ Aug 9, 2026 AI & LLM integration patterns 15 views 0 copies

jsonl audit llm logging

Python code

51 lines
Python 3.9+
Run it 
import json
from pathlib import Path
from datetime import datetime


def audit_jsonl(filepath):
    logs = []
    with open(filepath, encoding="utf-8") as f:
        for line in f:
            line = line.strip()
            if not line:
                continue
            entry = json.loads(line)
            logs.append(entry)

    total_prompts = len(logs)
    completions = [e.get("completions", 0) for e in logs]
    total_completions = sum(completions)
    avg_completions = total_completions / total_prompts if total_prompts else 0

    with open("audit_summary.json", "w", encoding="utf-8") as out:
        json.dump(
            {
                "generated_at": datetime.now().isoformat(),
                "total_prompts": total_prompts,
                "total_completions": total_completions,
                "avg_completions_per_prompt": round(avg_completions, 2),
                "entries": logs,
            },
            out,
            indent=2,
        )

    return {
        "total_prompts": total_prompts,
        "total_completions": total_completions,
        "avg_completions_per_prompt": round(avg_completions, 2),
    }


if __name__ == "__main__":
    sample = [
        {"id": 1, "prompt": "What is Python?", "completions": 3},
        {"id": 2, "prompt": "How to sort a list?", "completions": 5},
        {"id": 3, "prompt": "Explain decorators", "completions": 2},
    ]
    Path("sample_prompts.jsonl").write_text(
        "\n".join(json.dumps(e) for e in sample), encoding="utf-8"
    )
    result = audit_jsonl("sample_prompts.jsonl")
    print(result)

Output

stdout 
{'total_prompts': 3, 'total_completions': 10, 'avg_completions_per_prompt': 3.33}

How it works

The script reads a JSONL file line by line, skipping blank lines, and parses each entry with the stdlib json module. It collects all entries into a list, then computes total prompts and total completions using a list comprehension and sum(). The average is calculated defensively to avoid division by zero. A new JSON file audit_summary.json is written with the current timestamp and the full entry list, making it easy to audit LLM usage over time. Returning the summary dict lets you reuse the function programmatically.

Common mistakes

  • Using `json.load()` on a JSONL file instead of iterating lines with `json.loads()`
  • Stripping whitespace incorrectly and treating empty lines as valid entries
  • Not guarding against division by zero when the file is empty
  • Forgetting to encode output files with `utf-8` for cross-platform portability

Variations

  1. Use `pathlib.Path.read_text` and `splitlines()` to load all lines at once, then filter blanks
  2. Extend to write CSV or append each entry to a single log file instead of a summary JSON

Real-world use cases

  • Track LLM API usage across a team to monitor prompt counts and token consumption for cost analysis.
  • Maintain an audit trail of AI interactions in a support chatbot for regulatory compliance.
  • Analyze prompt–completion patterns in production to identify quality issues and common user intents.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from AI & LLM integration patterns

Keep learning

Related tutorials and quizzes for this topic.

Tutorials

Quizzes