Add Readiness Probes for Python

Your Python service is running in Kubernetes, the Deployment shows 3/3 replicas ready, but users are hitting 502 errors. You spend an hour debugging, only to discover that your database connection pool was still warming up, and the container was accepting traffic before it could actually serve a single request. This is the classic problem that readiness probes solve. Without them, Kubernetes assumes a container is ready the moment it starts — even if your Python app needs a few extra seconds to load models, connect to external services, or warm caches. In this lesson, you'll learn how to add readiness probes for Python traffic readiness, using practical examples you can apply immediately to your own deployments.

The problem this lesson solves

Kubernetes is an orchestrator, not a mind reader. When a Pod is created, it schedules the container and marks the Pod as Running. But running doesn't mean ready. Your Python app might be up but still initializing: connecting to PostgreSQL, loading a machine learning model, or syncing with a message queue. During that window, if Kubernetes sends traffic to the Pod, your service will fail or return errors.

The pain is real: traffic is lost, users see 5xx errors, and your on-call pager goes off at 3 AM. You might even introduce a sleep(10) at startup to hack around it — a common but fragile workaround that slows down rolling deployments and doesn't adapt to variable startup times.

The solution is a readiness probe, a health check that tells Kubernetes exactly when your container is capable of handling requests. When the probe fails, Kubernetes stops sending traffic to that Pod and, if you've configured it, restarts the container. When it succeeds, the Pod is marked Ready and added to the load balancer. This is your defense against releasing an app that isn't actually ready to serve.

Core concept / mental model

Think of your Python service as a new employee on their first day. They arrive at the office (container started), but they're not productive until they've logged into the system, gotten their badge, and opened the necessary tools. Would you start assigning them customer calls before they're ready? No — you'd wait for them to say, "I'm ready." A readiness probe is that exact signal in Kubernetes.

In more formal terms, a readiness probe is a periodic check performed by the kubelet on your container to determine whether it's ready to accept traffic. The probe can be one of three types:

• HTTP GET — Kubernetes makes an HTTP request to a URL in your container (e.g., /health/ready). A response code between 200 and 399 means success.
• TCP socket — Kubernetes tries to open a TCP connection to a port. If the connection succeeds, the container is ready.
• Exec command — Kubernetes runs a command inside the container (e.g., a Python script that checks dependencies). Exit code 0 means success.

Here's the mental model: the probe acts as a gatekeeper. While the probe fails, the Pod is not added to the Endpoints of a Service, so no traffic is routed to it. During rolling updates, the old Pods continue serving until the new ones pass their probes. If a probe starts failing after the Pod was ready, Kubernetes removes it from the Endpoints immediately, preventing traffic from being sent to a broken instance.

Probes are configured in the container spec of a Pod template under livenessProbe and readinessProbe. They share similar settings, but have different purposes: liveness tells Kubernetes when to restart the container (e.g., deadlock in Python), while readiness tells when to stop sending traffic. For this lesson, we focus exclusively on readiness.

How it works step by step

Let's walk through the lifecycle of a Pod with a readiness probe configured:

1. Container starts — The kubelet starts the container and begins executing the probe at the initialDelaySeconds interval.
2. Probe fires — The kubelet runs the probe (HTTP GET, TCP, or exec) every periodSeconds (default 10 seconds).
3. Success or failure — Each probe returns success or failure. If it fails, the probe increments a failure counter.
4. Marking ready — After successThreshold consecutive successes (default 1), the Pod is marked Ready. Kubernetes then adds its IP to the Service's endpoints, and traffic flows.
5. Failure handling — If the probe fails failureThreshold consecutive times (default 3), the Pod is marked NotReady and removed from endpoints. No traffic is sent until the probe succeeds again.
6. Self-healing — Unlike liveness probes, a failed readiness probe does not restart the container. The container keeps running, giving your app time to recover (e.g., a transient database outage).

The key timing parameters are:

• initialDelaySeconds — How long after container start to wait before the first probe. Use this to give your app time for CPU-intense imports or connections.
• periodSeconds — How often to run the probe.
• timeoutSeconds — How long the probe can take before it's considered failed.
• failureThreshold — How many consecutive failures before the Pod is marked NotReady.
• successThreshold — How many consecutive successes before the Pod is marked Ready (for higher availability, set to 2 in blue-green deployments).

In a typical Python web app, you'll want an HTTP readiness endpoint that checks all critical dependencies. This is more reliable than a simple TCP check because it verifies your app's logic, not just that the port is open.

Hands-on walkthrough

1. Add a readiness endpoint in your FastAPI app

First, add a /health/ready endpoint to your Python app. It should return HTTP 200 only when your app is truly ready. For example, if you use SQLAlchemy, check that you can run a simple query; if you use model files, verify they're loaded.

# main.py
from fastapi import FastAPI, Response
from sqlalchemy import text
from sqlalchemy.exc import SQLAlchemyError
from .db import engine_pool, model_state

app = FastAPI()

@app.get("/health/ready")
async def readiness_check(response: Response):
    """Return 200 only when the app can handle traffic."""
    try:
        with engine_pool.connect() as conn:
            conn.execute(text("SELECT 1"))
    except SQLAlchemyError:
        response.status_code = 503
        return {"status": "not ready"}

    if not model_state["loaded"]:
        response.status_code = 503
        return {"status": "not ready", "reason": "model not loaded"}

    return {"status": "ready"}

Expected behavior: when the database is down or the model isn't loaded, the endpoint returns 503. When everything is ready, it returns 200.

2. Define the readiness probe in your Deployment manifest

Now, create or update your Kubernetes Deployment YAML to include a readiness probe that hits /health/ready.

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: python-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: python-app
  template:
    metadata:
      labels:
        app: python-app
    spec:
      containers:
      - name: app
        image: python-app:1.2.3
        ports:
        - containerPort: 8000
        readinessProbe:
          httpGet:
            path: /health/ready
            port: 8000
          initialDelaySeconds: 5
          periodSeconds: 10
          timeoutSeconds: 2
          failureThreshold: 3
          successThreshold: 1

Apply the manifest:

kubectl apply -f deployment.yaml

Verify readiness:

kubectl get pods

During startup, you'll see 0/1 in the READY column. After the probe succeeds, it changes to 1/1. If the probe keeps failing, the Pod stays NotReady, and no traffic is routed to it.

3. Watch readiness work during a rolling update

When you update the image, Kubernetes rolls out new Pods. During the rollout, old Pods keep serving until new ones pass their readiness probes. Watch the deployment status:

kubectl rollout status deployment/python-app

This shows how readiness prevents downtime: unless the probe fails, the rollout won't complete.

Compare options / when to choose what

For most Python services, HTTP GET is the primary choice. It gives the most accurate signal. TCP is a quick fallback if you're using a plain TCP server, but it can mark a container ready when it's not. Exec is useful when you want to use e.g., python -c "import os; sys.exit(0 if some_check else 1)", but avoid overly complex scripts that slow down probe execution.

A common hybrid: use a startup probe (available since Kubernetes 1.16) for apps with slow, unpredictable startup, and readiness for ongoing readiness. The startup probe runs once at startup and only after it succeeds do the readiness probes begin. This prevents premature traffic before your app finishes initializing.

Troubleshooting & edge cases

Probe returns 503 but app seems fine

Check your endpoint's response code. Kubernetes only considers codes 200–399 as success. A common mistake is a redirect (301/302) or a 500 error. Make sure your readiness endpoint returns exactly 200 (or another 2xx/3xx) when ready.

Readiness never flips to Ready

Use kubectl describe pod <pod-name> to see probe events. Look for messages like Readiness probe failed: HTTP probe failed with statuscode: 503. Then inspect your endpoint inside the container:

kubectl exec -it <pod-name> -- curl -v localhost:8000/health/ready

Fix the underlying issue and the probe will pass.

Initial delay too short

The first probe might fire while your app is still importing heavy libraries. Increase initialDelaySeconds to match your app's worst-case startup time. Watch kubectl logs to see when your app actually begins serving.

Probe URL is wrong

Double-check the path. A trailing slash can cause a 404. Also ensure your app binds to 0.0.0.0 inside the container — otherwise the probe will only work if you hardcode the IP.

Health endpoint expensive

An HTTP readiness check that queries the database every 10 seconds across 10 replicas can add load. Cache the result of expensive checks for a few seconds, or use a lighter check — but remember, if the dependency fails, you want it to fail fast.

Readiness probes and init containers

Use init containers for tasks that must finish before the main app starts (e.g., DB migration). Then your readiness probe only needs to verify runtime readiness, not deployment steps. For slow migrations, combine init containers with the startup probe to avoid blocking the pod.

What you learned & what's next

In this lesson, you learned how to add readiness probes for Python traffic readiness. You now understand:

• How Kubernetes uses readiness probes to decide if a Pod can receive traffic.
• How to configure HTTP, TCP, and exec probes with timing parameters.
• How to implement a readiness endpoint in a FastAPI app that checks DB connectivity and model state.
• How to compare probe types and choose the right one for your Python service.
• How to troubleshoot common probe failures.

The next lesson will build on this foundation by exploring liveness probes — how to detect and restart deadlocked or crashed Python processes. You'll learn how to combine readiness and liveness for robust self-healing apps.