How to Compute SRE Metrics Like Error Rate and Availability in Python
Tracks log events in a sliding time window and calculates error rate per second and availability percentage using an easy-to-follow class.
Python code
56 linesfrom collections import deque
from datetime import datetime, timedelta
from typing import Dict, Deque
class LogMetrics:
"""Simple observability helper to track log events and calculate SRE metrics."""
def __init__(self, window_seconds: int = 60):
self.window_seconds = window_seconds
self.events: Deque[tuple] = deque()
self.counts: Dict[str, int] = {}
def add_event(self, level: str) -> None:
"""Record a log event at the current timestamp."""
now = datetime.now()
self.events.append((now, level))
self.counts[level] = self.counts.get(level, 0) + 1
self._prune_expired(now)
def _prune_expired(self, current_time: datetime) -> None:
"""Remove events older than the sliding window."""
cutoff = current_time - timedelta(seconds=self.window_seconds)
while self.events and self.events[0][0] < cutoff:
_, level = self.events.popleft()
self.counts[level] = max(0, self.counts[level] - 1)
if self.counts[level] == 0:
del self.counts[level]
def error_rate_per_second(self) -> float:
"""Calculate error rate (errors per second) in the current window."""
if not self.events:
return 0.0
total_errors = self.counts.get("ERROR", 0)
elapsed = (self.events[-1][0] - self.events[0][0]).total_seconds()
return total_errors / elapsed if elapsed > 0 else 0.0
def availability(self) -> float:
"""Calculate availability percentage (non-error events / total events)."""
total = max(1, sum(self.counts.values()))
errors = self.counts.get("ERROR", 0)
return (total - errors) / total * 100
if __name__ == "__main__":
metrics = LogMetrics(window_seconds=10)
# Simulate a series of events: mostly INFO, with a few ERRORs
for _ in range(25):
metrics.add_event("INFO")
for _ in range(5):
metrics.add_event("ERROR")
print(f"Events in window: {sum(metrics.counts.values())}")
print(f"Error rate (errors/sec): {metrics.error_rate_per_second():.2f}")
print(f"Availability: {metrics.availability():.1f}%")
Output
Events in window: 30
Error rate (errors/sec): 0.50
Availability: 83.3%
How it works
The LogMetrics class records each log event with its timestamp in a deque. As new events are added, the _prune_expired method removes events older than the sliding window, keeping metrics fresh. The error_rate_per_second method divides the count of ERROR events by the elapsed time of the window, giving a rate. Availability is computed as the percentage of non-error events among all events in the window. Using a deque ensures efficient popleft operations for pruning.
Common mistakes
- Not pruning expired events, causing the window to become stale.
- Dividing by zero when there is no elapsed time, leading to a ZeroDivisionError.
- Forgetting to use `max(0, ...)` when decrementing counts, which can drop below zero.
Variations
- Use a library like `prometheus_client` to expose these metrics to a monitoring system.
- Store log events in a database instead of memory if you need longer historical analysis.
Real-world use cases
- Monitoring error rates in real time for a web application to trigger alerts.
- Tracking availability of a service over a rolling minute to report on Service Level Objectives.
- Debugging intermittent issues by analyzing recent error frequency after a deploy.
Sponsored
More from Observability & SRE
- Adding a Correlation ID to Log Context in Python medium
- Calculate Error Rate from Log Stream in Python easy
- Check if a Timestamp Falls in a Daily Maintenance Window in Python easy
- Export Metrics with OTLP Mock in Python medium
- Generate Mock CPU and Memory Metrics in Python easy
- Generate Prometheus Text Exposition Format in Python easy
Keep learning
Related tutorials and quizzes for this topic.