Observability & SRE
Structured logging, metrics, tracing, health checks, and SLO-friendly instrumentation.
How to Mock Service Resource Attributes in Python
Temporarily override service name, version, and other resource attributes with a context manager, then restore them automatically.
from contextlib import contextmanager
import random
_SERVICE_ATTRIBUTES = {
"service.name": "payment-api",
"service.version": "1.4.2",
"service.instance.id": str(random.randint(10000, 99999)),
"service.namespace": "production",
}
@contextmanager
def mock_service_attributes(**overrides):
"""Tempor…How to Mock a Baggage Context (Key-Value Store) in Python
This code implements an in-memory key-value mock of a baggage context, letting you set, get, check, and delete keys for tracing-style metadata.
class BaggageContext:
def __init__(self):
self._store = {}
def set(self, key, value):
self._store[key] = value
return value
def get(self, key, default=None):
return self._store.get(key, default)
def has(self, key):
return key in self._store
def delete(sel…How to Mock an OTLP HTTP Endpoint in Python
This code implements a lightweight HTTP server that accepts OTLP/HTTP trace exports, stores spans by trace ID, and exposes them via a simple GET endpoint for debugging.
import json
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from collections import defaultdict
class TraceHandler(BaseHTTPRequestHandler):
traces = defaultdict(list)
def do_POST(self):
if self.path == "/v1/traces":
length = int(self.headers.get("Content-Length", 0))
…How to Model Span Events in Python
Define a Span class with timestamped milestone events and a completion marker to track operation lifecycle.
import time
from dataclasses import dataclass, field
from enum import Enum
from typing import List
class SpanStatus(Enum):
STARTED = "started"
COMPLETED = "completed"
@dataclass
class SpanEvent:
name: str
timestamp: float = field(default_factory=time.time)
attributes: dict = field(default_facto…How to Parse Log Lines with Regex in Python
Extracts timestamp, log level, service name, and message from a log line using compiled regex named groups.
import re
LOG_PATTERN = re.compile(
r'^(?P<timestamp>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}) '
r'\[(?P<level>\w+)\] '
r'\((?P<service>[^)]+)\) '
r'(?P<message>.*)$'
)
def parse_log_line(line: str) -> dict:
match = LOG_PATTERN.match(line)
if not match:
return {"error": "invalid log format…How to Process System Metrics (RSS, CPU) in Python
Simulate and aggregate RSS and CPU system metrics to compute averages and maximums for monitoring dashboards.
import random
import time
from collections import namedtuple
Metric = namedtuple("Metric", ["name", "value", "unit"])
def generate_metrics(num_metrics: int = 5) -> list:
"""Simulate a batch of system metrics."""
metrics = []
for i in range(num_metrics):
rss = random.randint(50, 500) # MB
…How to Redact Secrets from Log Messages in Python
Build a lightweight RedactingFormatter class that replaces sensitive tokens like passwords and API keys with [REDACTED] before log messages are printed.
class RedactingFormatter:
def __init__(self, secrets):
self.secrets = secrets
def redact(self, message):
for secret in self.secrets:
message = message.replace(secret, "[REDACTED]")
return message
def format(self, record):
message = record["message"]
ret…How to Route Alerts by Severity in Python
Map alert severity levels to routing targets and simulate dispatching alerts to on-call pages, email, Slack, or logs.
def main():
# Severity levels with corresponding alert routing targets
routing_map = {
"critical": "call_page",
"high": "call_page",
"medium": "email_team",
"low": "slack_channel",
"info": "log_only"
}
# Simulated alerts with severity
alerts = [
{"na…How to Ship Logs to an Aggregator Endpoint in Python
Ship batched log entries to a mock HTTP aggregator endpoint with proper error handling and response status.
import json
import requests
from datetime import datetime, timezone
LOG_ENTRIES = [
{"timestamp": "2024-01-15T10:00:00Z", "level": "INFO", "message": "Server started"},
{"timestamp": "2024-01-15T10:00:05Z", "level": "WARN", "message": "High memory usage"},
{"timestamp": "2024-01-15T10:00:10Z", "level": "E…How to Simulate Trace Sampling Head in Python
Simulate head-based probabilistic trace sampling on mock trace data with a configurable sample rate and optional seed for reproducibility.
import random
def trace_sampling_head(mock_traces, sample_rate=0.5, seed=None):
"""Simulate probabilistic trace sampling (head-based) on mock data.
Args:
mock_traces: list of trace dictionaries with a unique 'trace_id'
sample_rate: float 0.0-1.0, probability of keeping a trace
see…How to Simulate a Queue Depth Gauge in Python
Simulate a queue depth over time using a random enqueue/dequeue process, returning depth values that can be used for monitoring or testing dashboards.
import collections
import random
import time
def simulate_queue_depth(max_depth=10, steps=20):
queue = collections.deque()
depth_history = []
for _ in range(steps):
# Randomly enqueue or dequeue
if random.random() < 0.6 and len(queue) < max_depth:
queue.append("task")
…How to Use Log Levels DEBUG INFO WARNING ERROR in Python
Demonstrates Python's logging levels (DEBUG, INFO, WARNING, ERROR) with basicConfig and a logger, showing how severity filtering controls output.
import logging
# Configure a mock logger to demonstrate log levels
logging.basicConfig(level=logging.DEBUG, format="%(levelname)s: %(message)s")
logger = logging.getLogger("mock_logger")
# Simulate events at each severity level
logger.debug("Detailed diagnostic info")
logger.info("General system operation")
logger.w…How to mock Prometheus alert rule thresholds in Python
Simulate a Prometheus alert rule with a configurable threshold and duration window, firing only when the metric exceeds the threshold long enough.
import time
import random
class MetricsStore:
def __init__(self):
self.metrics = {}
def set_metric(self, name, value, labels=None):
key = (name, tuple(sorted((labels or {}).items())))
self.metrics[key] = value
def get_metric(self, name, labels=None):
key = (name, tuple(s…How to mock SLI availability success ratio in Python
Simulate request outcomes with deterministic randomness and compute the SLI availability success ratio to check if a target is met.
import random
from collections import defaultdict
def mock_availability(num_requests=1000, target_ratio=0.995):
"""
Simulate request outcomes and compute the SLI availability success ratio.
Args:
num_requests: Total number of requests to simulate
target_ratio: Target availability rati…Mock Health Endpoint Liveness Check in Python
Simulate a liveness endpoint that reports service health with a configurable failure rate and uptime.
import time
import random
def liveness_check(service_name: str, failure_rate: float = 0.1) -> dict:
"""Mock health check that returns liveness status with a configurable failure rate."""
healthy = random.random() > failure_rate
response = {
"service": service_name,
"status": "alive" if he…Mocking a Metrics Gauge's set_value Method in Python
Demonstrates using unittest.mock.Mock with wraps to intercept a gauge's set_value call while verifying arguments and preserving real behavior.
from unittest.mock import Mock
class MetricsGauge:
def __init__(self, name):
self.name = name
self.value = 0.0
def set_value(self, new_value):
self.value = float(new_value)
return self.value
# Usage demonstration with a mock
gauge = MetricsGauge("cpu_usage")
gauge_mock = Mock…Python Observability Data Helper for Beginners
A beginner-friendly Python helper to log events, record metrics, summarize observability data, and export it as JSON.
import json
from datetime import datetime
from collections import defaultdict
class ObservabilityDataHelper:
"""Helper for exploring basic observability data patterns."""
def __init__(self):
self.events = []
self.metrics = defaultdict(list)
def log_event(self, service, level, message):
…Rotate Log Files by Size in Python
A mock log rotation script that renames log files exceeding a size threshold, appending numbered backups.
import os
from pathlib import Path
def rotate_logs(directory: str, max_size: int = 100) -> None:
"""Rotate log files that exceed max_size bytes."""
log_dir = Path(directory)
for log_file in sorted(log_dir.glob("*.log"), key=lambda p: str(p)):
if log_file.stat().st_size > max_size:
for …Track Success Rates and Latency in Python: SRE Metrics Helper
A beginner-friendly Python class to record request outcomes and latencies, then report success rate, average latency, and p99.
import random
import time
from collections import defaultdict
class MetricsTracker:
"""Simple helper to track success rates and latencies for SRE beginners."""
def __init__(self):
self.successes = 0
self.failures = 0
self.latencies = []
def record(self, success, latency_ms):
…Browse by section
Each section groups closely related Python snippets.
Observability & SRE — Python code examples
What you will find here
This page collects observability & sre snippets — short, copy-ready Python you can paste into our free online IDE and run without installing anything. Each sample includes a plain-English explanation and the full source code.
Samples vs tutorials and challenges
Samples are quick reference — one concept per page. For step-by-step teaching, use our Python tutorials. To test yourself, try quizzes or coding challenges. Clean up style with the Python formatter.