Observability & SRE
Structured logging, metrics, tracing, health checks, and SLO-friendly instrumentation.
Check if a Timestamp Falls in a Daily Maintenance Window in Python
A small Python function that returns True when a datetime falls inside a daily maintenance window, and a demo printing yes/no for sample timestamps.
from datetime import datetime, timedelta
from zoneinfo import ZoneInfo
def in_maintenance_window(now: datetime, start_hour: int = 2, duration_hours: int = 4) -> bool:
"""Return True if 'now' falls inside the daily maintenance window."""
day_start = now.replace(hour=start_hour, minute=0, second=0, microsecond…Generate Prometheus Text Exposition Format in Python
Mock a Prometheus metrics endpoint by formatting metrics into the text exposition format with HELP, TYPE, and sample lines.
import time
from random import randint
# Mock a Prometheus metrics endpoint output
metrics = {
"http_requests_total": {
"help": "Total number of HTTP requests",
"type": "counter",
"samples": [
{"labels": {"method": "get", "code": "200"}, "value": randint(1000, 9999)},
…Generate Synthetic CPU Utilization Metrics in Python
Creates realistic time-series CPU utilization samples with timestamps, noise, and output as structured JSON for observability demos and testing.
from datetime import datetime, timedelta
import random
import json
def generate_metric_samples(base_value, noise, count=60, interval_minutes=1):
"""Generate realistic CPU utilization samples for a given time window."""
timestamps = []
values = []
now = datetime.utcnow()
start_time = now - timede…Generate Synthetic SRE Metrics and Calculate Availability in Python
Create realistic service metrics with random latency, error rate, and request counts, then compute availability and summarize the stream for SLO checks.
from datetime import datetime, timedelta
import random
def generate_service_metrics(service_name: str, minutes: int = 30) -> list[dict]:
"""Generate synthetic SRE metrics for a service across recent minutes."""
metrics = []
now = datetime.now()
for i in range(minutes):
timestamp = now - t…How to Build a Consumer Lag Gauge in Python
Simulate Kafka consumer lag with a Python class that tracks lag over time and reports health and averages.
import time
import random
from collections import deque
class ConsumerLagGauge:
"""Mock consumer lag gauge measuring how far behind a consumer is."""
def __init__(self, producer_rate=10, consumer_rate=7, initial_lag=0):
self.producer_rate = producer_rate
self.consumer_rate = consumer_rate
…How to Calculate Apdex Score from Latency Data in Python
Generate simulated latency samples and compute the Apdex score to gauge user satisfaction with an application's performance.
import random
import statistics
def generate_latencies(count=100, base=100, stddev=30):
return [max(0, random.gauss(base, stddev)) for _ in range(count)]
def apdex(latencies, threshold=200):
satisfied = sum(1 for lat in latencies if lat < threshold)
tolerating = sum(1 for lat in latencies if lat >= thres…How to Calculate Percentile Latency in Python
Generate mock latency samples with occasional spikes and compute 50th, 90th, 95th, and 99th percentile values in milliseconds.
import random
import statistics
def generate_latency_samples(n=1000):
"""Generate realistic mock latency data (ms) with occasional spikes."""
samples = []
for _ in range(n):
# Normal case: ~50ms with jitter
base = random.gauss(50, 5)
# 2% spike chance: slow downstream or GC pause
…How to Calculate SLO Error Budget in Python
Simulate an SLO error budget by computing allowed downtime from a target availability percentage and mocking monthly incidents.
```python
import random
def calculate_error_budget(total_seconds: int, target_availability: float) -> float:
return (1.0 - target_availability) * total_seconds
def simulate_monthly_availability(seconds_in_month: int, budget_seconds: float) -> float:
# Mock: randomly consume a fraction of the error budget i…How to Implement Tail Sampling in Python
Sample the slowest subset of calls (tail) for latency analysis using a deque with a random ratio gate.
import random
import time
from collections import deque
class TailSampler:
def __init__(self, tail_ratio=0.1, max_samples=100):
self.tail_ratio = tail_ratio
self.max_samples = max_samples
self.samples = deque(maxlen=max_samples)
self.total_calls = 0
def record(self, latency_ms…How to Link Parent and Child Span Elements in Python
This code defines a lightweight mock element class and a function that links child elements to a parent when their ranges are nested within the parent's range.
class MockElement:
def __init__(self, name, start, end, children=None):
self.name = name
self.start = start
self.end = end
self.children = children or []
def __repr__(self):
return f"MockElement({self.name}, {self.start}-{self.end})"
def link_parent_child(parent, chil…How to Mock Database Query Duration in Python
Simulate realistic database query durations with random jitter for testing dashboards, alerts, and SLO calculations.
import random
import time
def mock_query_duration(db_name, avg_ms, jitter_ms=5, runs=3):
"""Simulate database query durations with realistic variation."""
durations = []
for _ in range(runs):
# Base duration plus random jitter (can be negative)
duration = avg_ms + random.uniform(-jitter_m…How to Mock Service Resource Attributes in Python
Temporarily override service name, version, and other resource attributes with a context manager, then restore them automatically.
from contextlib import contextmanager
import random
_SERVICE_ATTRIBUTES = {
"service.name": "payment-api",
"service.version": "1.4.2",
"service.instance.id": str(random.randint(10000, 99999)),
"service.namespace": "production",
}
@contextmanager
def mock_service_attributes(**overrides):
"""Tempor…How to Model Span Events in Python
Define a Span class with timestamped milestone events and a completion marker to track operation lifecycle.
import time
from dataclasses import dataclass, field
from enum import Enum
from typing import List
class SpanStatus(Enum):
STARTED = "started"
COMPLETED = "completed"
@dataclass
class SpanEvent:
name: str
timestamp: float = field(default_factory=time.time)
attributes: dict = field(default_facto…How to Process System Metrics (RSS, CPU) in Python
Simulate and aggregate RSS and CPU system metrics to compute averages and maximums for monitoring dashboards.
import random
import time
from collections import namedtuple
Metric = namedtuple("Metric", ["name", "value", "unit"])
def generate_metrics(num_metrics: int = 5) -> list:
"""Simulate a batch of system metrics."""
metrics = []
for i in range(num_metrics):
rss = random.randint(50, 500) # MB
…How to Redact Secrets from Log Messages in Python
Build a lightweight RedactingFormatter class that replaces sensitive tokens like passwords and API keys with [REDACTED] before log messages are printed.
class RedactingFormatter:
def __init__(self, secrets):
self.secrets = secrets
def redact(self, message):
for secret in self.secrets:
message = message.replace(secret, "[REDACTED]")
return message
def format(self, record):
message = record["message"]
ret…How to Simulate Trace Sampling Head in Python
Simulate head-based probabilistic trace sampling on mock trace data with a configurable sample rate and optional seed for reproducibility.
import random
def trace_sampling_head(mock_traces, sample_rate=0.5, seed=None):
"""Simulate probabilistic trace sampling (head-based) on mock data.
Args:
mock_traces: list of trace dictionaries with a unique 'trace_id'
sample_rate: float 0.0-1.0, probability of keeping a trace
see…How to Simulate a Queue Depth Gauge in Python
Simulate a queue depth over time using a random enqueue/dequeue process, returning depth values that can be used for monitoring or testing dashboards.
import collections
import random
import time
def simulate_queue_depth(max_depth=10, steps=20):
queue = collections.deque()
depth_history = []
for _ in range(steps):
# Randomly enqueue or dequeue
if random.random() < 0.6 and len(queue) < max_depth:
queue.append("task")
…How to mock Prometheus alert rule thresholds in Python
Simulate a Prometheus alert rule with a configurable threshold and duration window, firing only when the metric exceeds the threshold long enough.
import time
import random
class MetricsStore:
def __init__(self):
self.metrics = {}
def set_metric(self, name, value, labels=None):
key = (name, tuple(sorted((labels or {}).items())))
self.metrics[key] = value
def get_metric(self, name, labels=None):
key = (name, tuple(s…How to mock SLI availability success ratio in Python
Simulate request outcomes with deterministic randomness and compute the SLI availability success ratio to check if a target is met.
import random
from collections import defaultdict
def mock_availability(num_requests=1000, target_ratio=0.995):
"""
Simulate request outcomes and compute the SLI availability success ratio.
Args:
num_requests: Total number of requests to simulate
target_ratio: Target availability rati…Mocking a Metrics Gauge's set_value Method in Python
Demonstrates using unittest.mock.Mock with wraps to intercept a gauge's set_value call while verifying arguments and preserving real behavior.
from unittest.mock import Mock
class MetricsGauge:
def __init__(self, name):
self.name = name
self.value = 0.0
def set_value(self, new_value):
self.value = float(new_value)
return self.value
# Usage demonstration with a mock
gauge = MetricsGauge("cpu_usage")
gauge_mock = Mock…Rotate Log Files by Size in Python
A mock log rotation script that renames log files exceeding a size threshold, appending numbered backups.
import os
from pathlib import Path
def rotate_logs(directory: str, max_size: int = 100) -> None:
"""Rotate log files that exceed max_size bytes."""
log_dir = Path(directory)
for log_file in sorted(log_dir.glob("*.log"), key=lambda p: str(p)):
if log_file.stat().st_size > max_size:
for …Browse by section
Each section groups closely related Python snippets.
Observability & SRE — Python code examples
What you will find here
This page collects observability & sre snippets — short, copy-ready Python you can paste into our free online IDE and run without installing anything. Each sample includes a plain-English explanation and the full source code.
Samples vs tutorials and challenges
Samples are quick reference — one concept per page. For step-by-step teaching, use our Python tutorials. To test yourself, try quizzes or coding challenges. Clean up style with the Python formatter.