Python Container Resource Limits

Your Python service is fast in development, but in production it's eating 2 GB of RAM and pegging the CPU — and your cluster is charging you for every millicore. Without resource limits, one misbehaving container can starve its neighbors, trigger OOM kills, and turn a smooth deploy into a fire drill. In this lesson, you'll learn how to configure resource limits for Python containers in Kubernetes so your workloads run predictably and your cluster stays healthy. You'll understand the difference between requests and limits, see how they apply to CPU and memory, and get hands-on with realistic Python examples — including a common memory leak gotcha you'll want to avoid.

The problem this lesson solves

Imagine you're running a Django API and a Celery worker in the same node. Your Celery worker is processing a huge batch and suddenly consumes all available memory. The kubelet detects the pressure and kills the Django API pod to reclaim memory — even though it did nothing wrong. In Kubernetes, resource limits are your first line of defense against this kind of noisy-neighbor chaos.

Without requests and limits:

• Pods get scheduled anywhere, which can overcommit the node CPU and memory.
• A single container can OOM-kill other containers on the same node (the kernel OOM killer doesn't care about fairness).
• You're over-provisioning hardware 'just in case,' which wastes cloud spend.
• Autoscaling (HPA, cluster autoscaler) has no signal for how much headroom each pod actually needs.

When you configure resource limits for Python containers, you tell the scheduler how much CPU and memory to reserve for a pod (requests) and the hard ceiling it can't cross (limits). This gives you predictable performance, better utilization, and a much happier production environment.

Core concept / mental model

Think of requests and limits like a hotel room reservation:

• Requests are your guaranteed room and board — the scheduler books that much capacity for your pod, and if it's not available, the pod won't be placed.
• Limits are the maximum room service you can order — beyond that, the manager (kubelet) steps in: CPU gets throttled, memory gets the pod killed.

For a Python developer, the practical translation:

• CPU requests measure the share of a core a container needs — e.g., 500m means half a core. The scheduler uses this for placement, and the kernel's CFS (Completely Fair Scheduler) makes sure the container gets that share if it needs it.
• CPU limits set the ceiling — the container can burst above its request, but never above its limit. If it exceeds the limit for a sustained period, the kernel throttles it.
• Memory requests are the guaranteed amount of RAM the pod is promised. The scheduler ensures the node has that much free memory for the pod.
• Memory limits are the hard cap — if a container exceeds this, the kernel triggers an OOM kill and restarts the container (with backoff). Unlike CPU, memory limits are not throttled; they're absolute.

Pro tip: The kubelet treats memory limits as a firm boundary. Once your Python process allocates beyond the limit, the kernel kills it with SIGKILL — no warning, no graceful shutdown. That's why you should always set memory limits above your realistic steady-state usage, but below the node's physical capacity.

How to translate Python's memory footprint into resource values

A Python container's memory usage has a few components:

• The Python interpreter itself: python:3.12 images typically idle at 20–40 MB.
• Your libraries and loaded modules: NumPy, pandas, and ML frameworks can add hundreds of MB.
• Runtime data structures: lists, dicts, and object graphs grow with your workload.
• Garbage collector overhead: Python frees memory lazily; RSS (resident set size) can appear higher than the live heap.

So a simple FastAPI health-check container might need only 50m CPU and 128Mi memory, while a Celery worker processing large payloads could easily request 500m CPU and 512Mi memory. The key is to measure before you set — use kubectl top or a metrics server to see real usage.

How it works step by step

Let's walk through what happens when you submit a pod with resource spec:

1. You write the pod spec with resources.requests and resources.limits for each container (and optionally for init containers).
2. The API server validates the spec — e.g., limits must be lower than the node's allocatable resources, and requests can't exceed limits (if both are set).
3. The scheduler uses the requests to decide which node to place the pod on. It sums up all requests on each node and picks a node where the sum is ≤ the node's allocatable capacity.
4. The kubelet on that node starts the container and creates a cgroup for it, applying the limits.
5. At runtime, the kernel enforces limits: - CPU: the CFS quota throttles the container's threads when they exceed the limit. - Memory: if the container exceeds its limit, the kernel invokes the OOM killer, which may kill the container process (or other processes in the same cgroup).
6. The kubelet restarts the container based on the restartPolicy (default Always for Deployments). Each restart adds a backoff, so the pod might go CrashLoopBackOff if it keeps OOM-killing.

Where to put resources in your YAML

In a Deployment, resources are defined at the container level — inside spec.template.spec.containers. You can also set them on init containers, but for most Python apps you'll only need the main container.

A typical snippet for a Python web service:

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: flask-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: flask
  template:
    metadata:
      labels:
        app: flask
    spec:
      containers:
      - name: app
        image: myregistry/flask-app:latest
        ports:
        - containerPort: 8000
        resources:
          requests:
            cpu: 250m
            memory: 256Mi
          limits:
            cpu: 500m
            memory: 512Mi

Pro tip: Setting only a request with no limit is often fine for CPU (you just give that pod priority), but for memory you should almost always set a limit — otherwise a single container can eat the entire node's RAM.

Hands-on walkthrough

Let's get practical. You'll create a small Python app that logs its PID and sleeps for a while, apply resource limits, and then verify with kubectl top and describe.

Step 1: Create a simple Python container image

Create a Dockerfile:

# Dockerfile
FROM python:3.12-slim
WORKDIR /app
COPY app.py .
ENTRYPOINT ["python", "app.py"]

And the app script:

# app.py
import time
import os

print(f"PID: {os.getpid()}", flush=True)
while True:
    time.sleep(10)

Build and push to your registry (e.g., yourname/flask-app:limits-demo). Or, if you're using a local cluster like kind, you can load it directly with kind load docker-image yourname/flask-app. For brevity, we'll assume an image exists.

Step 2: Apply the Deployment and set resource limits

Create deployment.yaml with resource requests and limits as shown above. Then apply it:

kubectl apply -f deployment.yaml
kubectl get pods

Expected output (something like):

NAME                        READY   STATUS    RESTARTS   AGE
flask-app-6b9c7d9d5d-2kd9j   1/1     Running   0          15s

Step 3: Verify the requests and limits are set

Use kubectl describe on a pod to check the resource section:

kubectl describe pod flask-app-6b9c7d9d5d-2kd9j | grep -A 5 "Limits:"

Expected output:

    Limits:
      cpu:     500m
      memory:  512Mi
    Requests:
      cpu:      250m
      memory:   256Mi

Step 4: Observe actual usage

If you have the metrics server installed, run kubectl top pods:

kubectl top pods

Expected output (your numbers will vary):

NAME                        CPU(cores)   MEMORY(bytes)
flask-app-6b9c7d9d5d-2kd9j   2m           30Mi

Notice the idle memory is well under the request — that's fine. The request is a reservation, not a target.

Step 5: Test the memory limit (the OOM demo)

Let's see what happens when a Python container tries to allocate more than its limit. Modify app.py to allocate a large list:

# app.py (OOM version)
import time

print("Allocating 1 GiB...", flush=True)
# This will try to allocate beyond the 512Mi limit
big_list = [b"x" * 1_000_000] * 1100  # roughly 1.1 GiB
print("done", flush=True)
while True:
    time.sleep(10)

Build and deploy. Wait a few seconds, then check the pod status:

kubectl get pods

You'll likely see a restart or a CrashLoopBackOff:

NAME                        READY   STATUS             RESTARTS   AGE
flask-app-6b9c7d9d5d-8f3k2   0/1     CrashLoopBackOff   3          40s

Then inspect the events:

kubectl describe pod flask-app-6b9c7d9d5d-8f3k2

Look for a line like:

  Warning:  OOMKilling  3s   kubelet  Memory cgroup out of memory: Killed process 1234 (python) total-vm:123456kB

Pro tip: If you see CrashLoopBackOff, your app is repeatedly crossing the memory limit. Use kubectl logs on the previous container (--previous) to see your app's last prints before the kill.

Compare options / when to choose what

There are a few ways to approach resource management; each has trade-offs. Here's a comparison table:

When to choose what

• For a stateless web API (FastAPI, Flask): set cpu: 250m requests and 500m limits, memory: 256Mi requests and 512Mi limits — start there and tune with load tests.
• For a Celery worker that processes memory-heavy tasks: set memory request at 75% of your typical peak, and limit at 1.5× that — giving headroom for temporary spikes (like loading a big CSV).
• For a batch job (e.g., data pipeline using pandas): set limits carefully to match the peak RAM you measured — you don't want an OOM kill mid-job, but you also don't want to overprovision.

Troubleshooting & edge cases

Error: Invalid value: 512Mi: memory limit must be greater than or equal to request

If you set a limit lower than a request, the API server rejects it. Fix: ensure limits.memory >= requests.memory and limits.cpu >= requests.cpu.

Error: OOMKilled / CrashLoopBackOff

Your container is exceeding the memory limit. Solutions:

• Increase the memory limit (but keep it under the node's capacity).
• Profile your Python app (e.g., tracemalloc, objgraph) to find leaks — famously, singletons or caches that never release references.
• Set PYTHONUNBUFFERED=1 to avoid output buffering masking crash logs.
• Consider a memoryHigh and memorySwap for cgroup v2 — but most people just bump the limit.

Error: Failed for reason PodExceedsFreeCPU or memory

The node doesn't have enough requests available. Options:

• Scale down other pods, or add nodes.
• Reduce your request values (if you've over-reserved).
• Use priorityClassName to schedule critical pods first.

Edge case: Python's garbage collector doesn't return memory to the OS

Python's gc frees references, but the allocator (pymalloc) keeps arenas cached, so RSS can stay high even after gc.collect(). This can cause OOM kills even if your actual live set is small. Practical fix: run your app in a subprocess and restart it periodically (common pattern for ML serving), or use resource.setrlimit in your app to pre-set a limit.

Counterintuitive: CPU limits can cause latency spikes

Because CPU is throttled, if your app hits the CPU limit, it gets scheduled less often — leading to higher tail latency. For latency-sensitive services, consider setting CPU requests only (no limit) and rely on node guarantees instead.

What you learned & what's next

You now know how to configure resource limits for Python containers like a pro. You can set requests and limits, interpret kubectl describe, diagnose OOM kills, and choose the right strategy for your Python service. You also learned the subtle differences between CPU and memory — CPU throttles, memory kills.

Up next, you'll dive into horizontal pod autoscaling — using metrics like CPU and memory request utilization to automatically scale your Python deployments. That's where your resource requests become the foundation for scalable, cost-efficient Python services. Keep going!