Observability & SRE
Structured logging, metrics, tracing, health checks, and SLO-friendly instrumentation.
Adding a Correlation ID to Log Context in Python
Injects a correlation ID into the logging context using a context manager and a custom log record factory so every log line includes the ID.
import logging
import uuid
from contextlib import contextmanager
logging.basicConfig(level=logging.INFO, format='%(levelname)s | %(correlation_id)s | %(message)s')
@contextmanager
def correlation_id_context(correlation_id):
"""Temporarily inject a correlation_id into the logging context."""
extra = {'correl…Check if a Timestamp Falls in a Daily Maintenance Window in Python
A small Python function that returns True when a datetime falls inside a daily maintenance window, and a demo printing yes/no for sample timestamps.
from datetime import datetime, timedelta
from zoneinfo import ZoneInfo
def in_maintenance_window(now: datetime, start_hour: int = 2, duration_hours: int = 4) -> bool:
"""Return True if 'now' falls inside the daily maintenance window."""
day_start = now.replace(hour=start_hour, minute=0, second=0, microsecond…Export Metrics with OTLP Mock in Python
Simulates system metric collection and exports them as an OTLP-like JSON payload using only Python's standard library.
from dataclasses import dataclass, asdict
import json
import random
import time
@dataclass
class Metric:
name: str
value: float
timestamp: int
unit: str = "1"
def collect_system_metrics() -> list[Metric]:
"""Mock metric collection for OTLP export simulation."""
now = int(time.time())
re…Generate Prometheus Text Exposition Format in Python
Mock a Prometheus metrics endpoint by formatting metrics into the text exposition format with HELP, TYPE, and sample lines.
import time
from random import randint
# Mock a Prometheus metrics endpoint output
metrics = {
"http_requests_total": {
"help": "Total number of HTTP requests",
"type": "counter",
"samples": [
{"labels": {"method": "get", "code": "200"}, "value": randint(1000, 9999)},
…Generate Synthetic CPU Utilization Metrics in Python
Creates realistic time-series CPU utilization samples with timestamps, noise, and output as structured JSON for observability demos and testing.
from datetime import datetime, timedelta
import random
import json
def generate_metric_samples(base_value, noise, count=60, interval_minutes=1):
"""Generate realistic CPU utilization samples for a given time window."""
timestamps = []
values = []
now = datetime.utcnow()
start_time = now - timede…Generate Synthetic SRE Metrics and Calculate Availability in Python
Create realistic service metrics with random latency, error rate, and request counts, then compute availability and summarize the stream for SLO checks.
from datetime import datetime, timedelta
import random
def generate_service_metrics(service_name: str, minutes: int = 30) -> list[dict]:
"""Generate synthetic SRE metrics for a service across recent minutes."""
metrics = []
now = datetime.now()
for i in range(minutes):
timestamp = now - t…How to Build a Burn Rate Alert with Multiple Time Windows in Python
Track token consumption and trigger alerts when the burn rate exceeds a threshold across multiple time windows using deque and time-based sliding windows.
import time
from collections import deque
class BurnRateAlert:
def __init__(self, windows_seconds=(60, 300, 900), threshold_rate=0.8):
self.windows = {w: deque() for w in windows_seconds}
self.threshold_rate = threshold_rate
self.previous_tokens = None
def record_sample(self, current_…How to Build a Consumer Lag Gauge in Python
Simulate Kafka consumer lag with a Python class that tracks lag over time and reports health and averages.
import time
import random
from collections import deque
class ConsumerLagGauge:
"""Mock consumer lag gauge measuring how far behind a consumer is."""
def __init__(self, producer_rate=10, consumer_rate=7, initial_lag=0):
self.producer_rate = producer_rate
self.consumer_rate = consumer_rate
…How to Build a Python Latency Histogram with Mock Buckets
This code implements a mock latency histogram that records request durations into configurable buckets and outputs counts, total, and average latency.
import time
import random
from collections import Counter
class LatencyHistogram:
def __init__(self, buckets):
self.buckets = sorted(buckets)
self.counts = Counter()
self.total = 0
self.sum_latency = 0
def record(self, latency_ms):
for i, boundary in enumerate(self.bu…How to Build an HTTP Server Request Duration Histogram in Python
Create a small HTTP server that times each GET request, buckets the duration, and prints a histogram on shutdown.
import time
import random
from collections import Counter
from http.server import HTTPServer, BaseHTTPRequestHandler
class HistogramHandler(BaseHTTPRequestHandler):
response_times = Counter()
def do_GET(self):
start = time.perf_counter()
time.sleep(random.uniform(0.001, 0.1))
duratio…How to Calculate Apdex Score from Latency Data in Python
Generate simulated latency samples and compute the Apdex score to gauge user satisfaction with an application's performance.
import random
import statistics
def generate_latencies(count=100, base=100, stddev=30):
return [max(0, random.gauss(base, stddev)) for _ in range(count)]
def apdex(latencies, threshold=200):
satisfied = sum(1 for lat in latencies if lat < threshold)
tolerating = sum(1 for lat in latencies if lat >= thres…How to Calculate Percentile Latency in Python
Generate mock latency samples with occasional spikes and compute 50th, 90th, 95th, and 99th percentile values in milliseconds.
import random
import statistics
def generate_latency_samples(n=1000):
"""Generate realistic mock latency data (ms) with occasional spikes."""
samples = []
for _ in range(n):
# Normal case: ~50ms with jitter
base = random.gauss(50, 5)
# 2% spike chance: slow downstream or GC pause
…How to Calculate SLO Error Budget in Python
Simulate an SLO error budget by computing allowed downtime from a target availability percentage and mocking monthly incidents.
```python
import random
def calculate_error_budget(total_seconds: int, target_availability: float) -> float:
return (1.0 - target_availability) * total_seconds
def simulate_monthly_availability(seconds_in_month: int, budget_seconds: float) -> float:
# Mock: randomly consume a fraction of the error budget i…How to Create a Mock OpenTelemetry Trace in Python
Create a mock OpenTelemetry trace in memory to test span creation, attributes, and parent-child relationships without exporting to a backend.
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter
def create_mock_trace():
tracer_provider = TracerProvider()
span_exporter =…How to Create a TCP DNS Mock Server in Python
This code creates a mock TCP DNS server that listens on a specified port, accepts probe connections, and returns a fixed DNS response header to simulate a live DNS service for testing and observability.
import socket
import threading
def handle_client(client_socket, address):
print(f"[+] Connection from {address}")
try:
while True:
data = client_socket.recv(1024)
if not data:
break
print(f"[*] Received {len(data)} bytes (TCP DNS probe)")
…How to Implement Tail Sampling in Python
Sample the slowest subset of calls (tail) for latency analysis using a deque with a random ratio gate.
import random
import time
from collections import deque
class TailSampler:
def __init__(self, tail_ratio=0.1, max_samples=100):
self.tail_ratio = tail_ratio
self.max_samples = max_samples
self.samples = deque(maxlen=max_samples)
self.total_calls = 0
def record(self, latency_ms…How to Link Parent and Child Span Elements in Python
This code defines a lightweight mock element class and a function that links child elements to a parent when their ranges are nested within the parent's range.
class MockElement:
def __init__(self, name, start, end, children=None):
self.name = name
self.start = start
self.end = end
self.children = children or []
def __repr__(self):
return f"MockElement({self.name}, {self.start}-{self.end})"
def link_parent_child(parent, chil…How to Mock Database Query Duration in Python
Simulate realistic database query durations with random jitter for testing dashboards, alerts, and SLO calculations.
import random
import time
def mock_query_duration(db_name, avg_ms, jitter_ms=5, runs=3):
"""Simulate database query durations with realistic variation."""
durations = []
for _ in range(runs):
# Base duration plus random jitter (can be negative)
duration = avg_ms + random.uniform(-jitter_m…How to Mock Service Resource Attributes in Python
Temporarily override service name, version, and other resource attributes with a context manager, then restore them automatically.
from contextlib import contextmanager
import random
_SERVICE_ATTRIBUTES = {
"service.name": "payment-api",
"service.version": "1.4.2",
"service.instance.id": str(random.randint(10000, 99999)),
"service.namespace": "production",
}
@contextmanager
def mock_service_attributes(**overrides):
"""Tempor…How to Model Span Events in Python
Define a Span class with timestamped milestone events and a completion marker to track operation lifecycle.
import time
from dataclasses import dataclass, field
from enum import Enum
from typing import List
class SpanStatus(Enum):
STARTED = "started"
COMPLETED = "completed"
@dataclass
class SpanEvent:
name: str
timestamp: float = field(default_factory=time.time)
attributes: dict = field(default_facto…How to Process System Metrics (RSS, CPU) in Python
Simulate and aggregate RSS and CPU system metrics to compute averages and maximums for monitoring dashboards.
import random
import time
from collections import namedtuple
Metric = namedtuple("Metric", ["name", "value", "unit"])
def generate_metrics(num_metrics: int = 5) -> list:
"""Simulate a batch of system metrics."""
metrics = []
for i in range(num_metrics):
rss = random.randint(50, 500) # MB
…How to Redact Secrets from Log Messages in Python
Build a lightweight RedactingFormatter class that replaces sensitive tokens like passwords and API keys with [REDACTED] before log messages are printed.
class RedactingFormatter:
def __init__(self, secrets):
self.secrets = secrets
def redact(self, message):
for secret in self.secrets:
message = message.replace(secret, "[REDACTED]")
return message
def format(self, record):
message = record["message"]
ret…How to Simulate Trace Sampling Head in Python
Simulate head-based probabilistic trace sampling on mock trace data with a configurable sample rate and optional seed for reproducibility.
import random
def trace_sampling_head(mock_traces, sample_rate=0.5, seed=None):
"""Simulate probabilistic trace sampling (head-based) on mock data.
Args:
mock_traces: list of trace dictionaries with a unique 'trace_id'
sample_rate: float 0.0-1.0, probability of keeping a trace
see…How to Simulate a Queue Depth Gauge in Python
Simulate a queue depth over time using a random enqueue/dequeue process, returning depth values that can be used for monitoring or testing dashboards.
import collections
import random
import time
def simulate_queue_depth(max_depth=10, steps=20):
queue = collections.deque()
depth_history = []
for _ in range(steps):
# Randomly enqueue or dequeue
if random.random() < 0.6 and len(queue) < max_depth:
queue.append("task")
…Browse by section
Each section groups closely related Python snippets.
Observability & SRE — Python code examples
What you will find here
This page collects observability & sre snippets — short, copy-ready Python you can paste into our free online IDE and run without installing anything. Each sample includes a plain-English explanation and the full source code.
Samples vs tutorials and challenges
Samples are quick reference — one concept per page. For step-by-step teaching, use our Python tutorials. To test yourself, try quizzes or coding challenges. Clean up style with the Python formatter.