How to Log Errors with Structured Fields in Python

Logs error details as structured dictionary fields using Python's logging module with extra parameters.

Medium Python 3.9+ Aug 9, 2026 Errors & debugging 15 views 0 copies

Python code

25 lines
Python 3.9+
import logging
import sys

def log_structured_error(operation: str, user_id: int, status_code: int, error_msg: str):
    """Log an error with structured fields using a dictionary."""
    logger = logging.getLogger("structured_logger")
    logger.setLevel(logging.ERROR)
    
    # Create console handler if not already present
    if not logger.handlers:
        handler = logging.StreamHandler(sys.stdout)
        handler.setFormatter(logging.Formatter("%(message)s"))
        logger.addHandler(handler)
    
    structured_data = {
        "operation": operation,
        "user_id": user_id,
        "status_code": status_code,
        "error": error_msg,
    }
    logger.error("Operation failed: %s", operation, extra={"structured": structured_data})

if __name__ == "__main__":
    log_structured_error("checkout", 12345, 500, "Database connection timeout")
    log_structured_error("login", 67890, 401, "Invalid credentials")

Output

stdout
Operation failed: checkout
Operation failed: login

How it works

The logging module allows custom fields via the extra parameter, which passes a dictionary to the log record. The formatter is set to %(message)s so only the message is displayed, but the structured data remains available programmatically. The logger.setLevel(logging.ERROR) ensures only error-level and above are processed. Adding a handler only if none exist prevents duplicate logs on repeated calls.

Common mistakes

  • Forgetting the 'extra' dictionary keys in the formatter when not using a simple message formatter
  • Adding multiple handlers causing duplicate log output
  • Not setting the logger level, so lower-level messages get filtered unexpectedly

Variations

  1. Use `logger.error(json.dumps(structured_data))` to log JSON directly
  2. Define a custom formatter that includes structured fields in the output text

Real-world use cases

  • Capturing operation context (user ID, endpoint) when an API call fails for later audit trails.
  • Feeding structured logs into Elasticsearch or Loki for faster error searching and aggregation.
  • Parsing structured error logs in CI/CD pipelines to trigger alerts or auto-tagging systems.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Errors & debugging

Related tutorials and quizzes for this topic.