How to Log Errors with Structured Fields in Python
Logs error details as structured dictionary fields using Python's logging module with extra parameters.
Python code
25 linesimport logging
import sys
def log_structured_error(operation: str, user_id: int, status_code: int, error_msg: str):
"""Log an error with structured fields using a dictionary."""
logger = logging.getLogger("structured_logger")
logger.setLevel(logging.ERROR)
# Create console handler if not already present
if not logger.handlers:
handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(logging.Formatter("%(message)s"))
logger.addHandler(handler)
structured_data = {
"operation": operation,
"user_id": user_id,
"status_code": status_code,
"error": error_msg,
}
logger.error("Operation failed: %s", operation, extra={"structured": structured_data})
if __name__ == "__main__":
log_structured_error("checkout", 12345, 500, "Database connection timeout")
log_structured_error("login", 67890, 401, "Invalid credentials")
Output
Operation failed: checkout
Operation failed: login
How it works
The logging module allows custom fields via the extra parameter, which passes a dictionary to the log record. The formatter is set to %(message)s so only the message is displayed, but the structured data remains available programmatically. The logger.setLevel(logging.ERROR) ensures only error-level and above are processed. Adding a handler only if none exist prevents duplicate logs on repeated calls.
Common mistakes
- Forgetting the 'extra' dictionary keys in the formatter when not using a simple message formatter
- Adding multiple handlers causing duplicate log output
- Not setting the logger level, so lower-level messages get filtered unexpectedly
Variations
- Use `logger.error(json.dumps(structured_data))` to log JSON directly
- Define a custom formatter that includes structured fields in the output text
Real-world use cases
- Capturing operation context (user ID, endpoint) when an API call fails for later audit trails.
- Feeding structured logs into Elasticsearch or Loki for faster error searching and aggregation.
- Parsing structured error logs in CI/CD pipelines to trigger alerts or auto-tagging systems.
Sponsored
More from Errors & debugging
- Catch RecursionError and Fail Gracefully in Python easy
- Catch ValueError and print friendly message in Python easy
- Collect Multiple Validation Errors in Python Before Raising medium
- Handle ValueError and ZeroDivisionError in Python with try except easy
- How to Add a Correlation ID to Logging Records in Python medium
- How to Assert Preconditions with Descriptive Messages in Python easy
Keep learning
Related tutorials and quizzes for this topic.