Python Observability Data Helper for Beginners
A beginner-friendly Python helper to log events, record metrics, summarize observability data, and export it as JSON.
Python code
67 linesimport json
from datetime import datetime
from collections import defaultdict
class ObservabilityDataHelper:
"""Helper for exploring basic observability data patterns."""
def __init__(self):
self.events = []
self.metrics = defaultdict(list)
def log_event(self, service, level, message):
"""Record an event with timestamp."""
self.events.append({
"timestamp": datetime.utcnow().isoformat(),
"service": service,
"level": level,
"message": message
})
def record_metric(self, name, value):
"""Record a metric sample."""
self.metrics[name].append(value)
def summarize(self):
"""Return a simple summary for SRE review."""
levels = defaultdict(int)
for event in self.events:
levels[event["level"]] += 1
metrics_summary = {}
for name, values in self.metrics.items():
if values:
metrics_summary[name] = {
"count": len(values),
"min": min(values),
"max": max(values),
"avg": round(sum(values) / len(values), 2)
}
return {
"total_events": len(self.events),
"events_by_level": dict(levels),
"metrics_summary": metrics_summary
}
def export_json(self):
"""Export observability data as JSON."""
return json.dumps({
"events": self.events,
"metrics": {k: v for k, v in self.metrics.items()}
}, indent=2)
if __name__ == "__main__":
helper = ObservabilityDataHelper()
helper.log_event("api-gateway", "error", "Timeout connecting to auth service")
helper.log_event("api-gateway", "info", "Request completed successfully")
helper.log_event("auth-service", "warn", "Token refresh rate high")
helper.record_metric("request_latency_ms", 120)
helper.record_metric("request_latency_ms", 95)
helper.record_metric("request_latency_ms", 210)
helper.record_metric("error_rate", 0.03)
print(json.dumps(helper.summarize(), indent=2))
print(helper.export_json())
Output
{
"total_events": 3,
"events_by_level": {
"error": 1,
"info": 1,
"warn": 1
},
"metrics_summary": {
"request_latency_ms": {
"count": 3,
"min": 95,
"max": 210,
"avg": 141.67
},
"error_rate": {
"count": 1,
"min": 0.03,
"max": 0.03,
"avg": 0.03
}
}
}
{
"events": [
{
"timestamp": "2025-01-01T12:00:00.000000",
"service": "api-gateway",
"level": "error",
"message": "Timeout connecting to auth service"
},
{
"timestamp": "2025-01-01T12:00:00.000001",
"service": "api-gateway",
"level": "info",
"message": "Request completed successfully"
},
{
"timestamp": "2025-01-01T12:00:00.000002",
"service": "auth-service",
"level": "warn",
"message": "Token refresh rate high"
}
],
"metrics": {
"request_latency_ms": [
120,
95,
210
],
"error_rate": [
0.03
]
}
}
How it works
This helper uses defaultdict for metrics so you can append samples without pre-initializing lists. The log_event method stamps each event with UTC time via datetime.utcnow().isoformat(), giving a consistent timestamp format. The summarize method aggregates event counts by level and computes basic stats (count, min, max, avg) for each metric — a common pattern for SRE dashboards. Finally, export_json serializes everything to readable JSON with indent=2 for easy sharing or debugging.
Common mistakes
- Using `datetime.utcnow()` which is deprecated in Python 3.12; prefer `datetime.now(timezone.utc)`
- Assuming all metrics have values before calling `summarize` — the code guards against empty lists
- Forgetting that `events` could grow unbounded in production; this helper is for learning, not production logging
Variations
- Use `dataclasses` to define Event and Metric types for cleaner structure
- Queue events to a background worker or log file instead of storing in memory
Real-world use cases
- Teaching new team members how structured logging and metric collection fit into an SRE workflow.
- Prototyping a lightweight local monitor that aggregates event counts and latency stats for a microservice.
- Testing JSON export logic before integrating with a real observability backend like Prometheus or ELK.
Sponsored
More from Observability & SRE
- Adding a Correlation ID to Log Context in Python medium
- Calculate Error Rate from Log Stream in Python easy
- Check if a Timestamp Falls in a Daily Maintenance Window in Python easy
- Export Metrics with OTLP Mock in Python medium
- Generate Mock CPU and Memory Metrics in Python easy
- Generate Prometheus Text Exposition Format in Python easy
Keep learning
Related tutorials and quizzes for this topic.