How to Build a Health Check System with Instance Up and Down Status in Python

Track instance health by marking them up or down and simulating health checks with a mock class in Python.

Easy Python 3.9+ Aug 9, 2026 System design patterns 14 views 0 copies

Python code

48 lines
Python 3.9+
from datetime import datetime
import random

class HealthChecker:
    def __init__(self):
        self.status = {}
    
    def mark_up(self, instance_id):
        self.status[instance_id] = {
            "state": "up",
            "last_check": datetime.now().isoformat(),
            "healthy": True
        }
    
    def mark_down(self, instance_id):
        self.status[instance_id] = {
            "state": "down",
            "last_check": datetime.now().isoformat(),
            "healthy": False
        }
    
    def mock_health_check(self, instance_id):
        # Simulate a real health check with some randomness
        is_healthy = random.random() > 0.3
        if is_healthy:
            self.mark_up(instance_id)
        else:
            self.mark_down(instance_id)
        return self.status[instance_id]

    def get_status(self, instance_id):
        return self.status.get(instance_id, {"state": "unknown", "healthy": False})

if __name__ == "__main__":
    checker = HealthChecker()
    
    # Mock health checks for instances
    for instance in ["web-01", "web-02", "db-01"]:
        result = checker.mock_health_check(instance)
        print(f"{instance}: {result['state']} (healthy={result['healthy']})")
    
    # Mark down explicitly
    checker.mark_down("web-02")
    print(f"web-02: {checker.get_status('web-02')['state']}")
    
    # Mark up explicitly
    checker.mark_up("web-02")
    print(f"web-02: {checker.get_status('web-02')['state']}")

Output

stdout
web-01: up (healthy=True)
web-02: down (healthy=False)
db-01: up (healthy=True)
web-02: down
web-02: up

How it works

The HealthChecker class maintains a dictionary of instance statuses with state and timestamp fields. mark_up and mark_down update the status dictionary, while mock_health_check simulates a real health check with random probability. The get_status method provides a safe fallback for unknown instances. This pattern mirrors production health monitor registries where each service's liveness is tracked and queried.

Common mistakes

  • Not using a default value in the get_status method leading to KeyError on unknown instances
  • Storing mutable objects without copying them, causing accidental status corruption
  • Ignoring thread safety when marking instances up or down from multiple workers

Variations

  1. Add a timeout field to detect stale health checks
  2. Use a thread-safe dict like `collections.OrderedDict` for concurrent updates

Real-world use cases

  • Load balancers tracking backend server health to route traffic away from down instances.
  • Monitoring systems like Prometheus or Datadog recording service liveness for alerting and dashboards.
  • Kubernetes-style controllers marking pods ready or not ready based on liveness probes.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from System design patterns

Related tutorials and quizzes for this topic.