Enable tracing with Jaeger or OpenTelemetry

When a user reports a slow page load, you can't just guess which microservice is at fault. In a Kubernetes cluster with dozens of pods, a single request might touch an Ingress, a service mesh sidecar, a backend API, a database, and a queue — all before returning a response. Without distributed tracing, you're debugging blind. This lesson shows you how to enable tracing with Jaeger or OpenTelemetry in your cluster so you can follow a request's entire journey and pinpoint latency bottlenecks in seconds.

The problem this lesson solves

Traditional monitoring tools (Prometheus metrics, logging with Loki or ELK) tell you what happened — high error rates, slow p99 response times — but they don't tell you why a single request is slow across service boundaries. When a call chain spans five services, a 50ms slowdown in one pod cascades into a 2-second user experience. You need to correlate events across microservices, not just inside one container.

Blockquote: A real-world pain "We saw 500 errors in our payment service, but the root cause was a slow DNS lookup in the order service — the trace showed it in seconds." — Senior SRE at a fintech startup

Distributed tracing solves this by attaching a unique trace ID to every request and collecting spans (timed units of work) from each service it passes through. With Jaeger or OpenTelemetry in your cluster, you get a flame graph that visualizes exactly where time is spent.

Core concept / mental model

Think of a trace as a shopping receipt. Each service you visit is a line item (span), showing what was purchased, how much it cost (duration), and who rang it up (service name). OpenTelemetry is the standard way to generate those line items; Jaeger is the tool that prints the receipt so you can read it.

Definitions

• Trace: The complete path of a single request as it travels through your system. Identified by a unique trace ID.
• Span: A single unit of work inside a trace. Each span has a start time, end time, status, and optional tags. For example, an HTTP request to /api/orders is a span.
• Context propagation: The mechanism that passes the trace ID from one service to the next (via HTTP headers like traceparent). Without it, spans are orphaned and the trace breaks.

OpenTelemetry provides the SDK and instrumentation to create spans automatically. Jaeger provides the backend to store, query, and visualize those spans.

How it works step by step

Enabling tracing in Kubernetes follows a logical pipeline:

1. Install OpenTelemetry Collector — a vendor-agnostic agent that receives spans from instrumented services and exports them to Jaeger (or any backend).
2. Instrument your services — either automatically (via OpenTelemetry auto-instrumentation for Java, Python, Node.js, etc.) or manually by adding SDK calls.
3. Configure exporters — tell the Collector where to send spans (e.g., Jaeger endpoint).
4. Deploy Jaeger — all-in-one (single pod for dev) or production mode (Elasticsearch backend, multiple components).
5. Trigger traffic and view traces in the Jaeger UI.

Context propagation flow

A trace starts when a request enters your system (e.g., at the Ingress). The first service generates a trace ID and propagates it to downstream services via headers. Each service creates child spans and attaches them to the same trace ID. At the end, all spans are collected and stitched together by the Jaeger backend.

Hands-on walkthrough

Let's enable tracing in a demo Kubernetes cluster. We'll deploy Jaeger (all-in-one), the OpenTelemetry Collector, and a simple microservice app (Python + Flask) with auto-instrumentation.

Step 1: Deploy Jaeger all-in-one

Apply the following manifest to your cluster:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: jaeger
  namespace: tracing
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jaeger
  template:
    metadata:
      labels:
        app: jaeger
    spec:
      containers:
      - name: jaeger
        image: jaegertracing/all-in-one:latest
        ports:
        - containerPort: 16686  # UI
        - containerPort: 4318   # OTLP HTTP
        - containerPort: 4317   # OTLP gRPC

Pro tip: For production, use the Jaeger Operator and an Elasticsearch backend. The all-in-one image is for development only — it stores traces in memory and will lose data on restart.

Step 2: Deploy OpenTelemetry Collector

The Collector acts as a pipeline: it receives OTLP traces (OpenTelemetry Protocol), processes them, and exports to Jaeger.

apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: otel-collector
  namespace: tracing
spec:
  config: |
    receivers:
      otlp:
        protocols:
          grpc:
            endpoint: 0.0.0.0:4317
          http:
            endpoint: 0.0.0.0:4318
    exporters:
      jaeger:
        endpoint: jaeger-collector.tracing.svc.cluster.local:14250
        tls:
          insecure: true
    service:
      pipelines:
        traces:
          receivers: [otlp]
          exporters: [jaeger]

Step 3: Auto-instrument a Python service

Add the OpenTelemetry Python auto-instrumentation package to your Docker image and run it via an init container or sidecar. Here's a minimal Flask app that sends traces:

from flask import Flask
from opentelemetry import trace
from opentelemetry.instrumentation.flask import FlaskInstrumentor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor

app = Flask(__name__)

# Set up OpenTelemetry
provider = TracerProvider()
exporter = OTLPSpanExporter(endpoint="otel-collector.tracing.svc.cluster.local:4317", insecure=True)
provider.add_span_processor(BatchSpanProcessor(exporter))
trace.set_tracer_provider(provider)

# Auto-instrument Flask
FlaskInstrumentor().instrument_app(app)

@app.route("/")
def hello():
    return "Hello, traces!"

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8080)

Expected output:

After deploying and sending a few requests to the Flask service, open the Jaeger UI (port-forward 16686):

kubectl port-forward -n tracing svc/jaeger-query 16686:16686

Visit http://localhost:16686. You should see a service list with your Flask app. Click "Find Traces" and you'll see a flame graph of each request, showing the total duration and any internal spans (like HTTP calls or DB queries if instrumented).

Compare options / when to choose what

When to choose Jaeger directly: Small clusters, quick experiments, or if you're already using Jaeger agents. Use the Jaeger Operator for production. When to choose OpenTelemetry: New projects, multi-vendor strategy, or need auto-instrumentation for multiple languages. OTel is the CNCF incubating standard.

Troubleshooting & edge cases

Missing traces in Jaeger UI - Verify the Collector is receiving spans: check its logs for incoming OTLP requests. - Ensure the service can reach the Collector endpoint. Use kubectl exec to test connectivity. - Confirm the Jaeger query service can reach the storage backend (if not all-in-one).

Span context not propagating - Check HTTP headers: your services must forward traceparent header to downstream services. - If using an Ingress controller, ensure it preserves headers (some Ingresses drop unknown headers by default).

High memory usage in all-in-one Jaeger - All-in-one stores traces in memory. Set the --memory.max-traces flag to limit trace count, or switch to production mode with Elasticsearch.

Auto-instrumentation fails in a container - Use an init container to copy the auto-instrumentation agent (e.g., Java agent JAR) into the application container, or use OpenTelemetry Operator's automatic instrumentation injection.

Common mistakes

• Forgetting to propagate the trace context via HTTP headers (services get orphan spans).
• Using the Jaeger all-in-one in production without setting memory limits — it will OOM kill the pod.
• Not configuring the OpenTelemetry Collector to export to the correct Jaeger endpoint (e.g., using port 14250 for gRPC but Jaeger expects 14268 for HTTP).
• Assuming auto-instrumentation works for all languages — you still need the correct SDK and environment variables.

Variations

• Instead of Jaeger, export traces to Zipkin (change the Collector's exporter to zipkin).
• Use the OpenTelemetry Operator for automatic injection of instrumentation into pods without modifying images.
• Deploy Jaeger in production with the Jaeger Operator and Elasticsearch for persistent storage and scalability.

Real-world use cases

• An e-commerce platform traces orders from the frontend to payment, inventory, and shipping services to identify which microservice causes checkout timeouts.
• A SaaS company uses OpenTelemetry to monitor a polyglot microservices architecture (Java, Go, Python) with Jaeger as the central visualization backend.
• A fintech startup traces a high-volume trading pipeline end-to-end, pinpointing a 2-second delay in a C++ analytics service that was hidden by normal monitoring.

Key takeaways

• Distributed tracing uses trace IDs to correlate spans across microservices, solving the "which service is slow" problem.
• OpenTelemetry is the industry standard for generating and exporting traces; Jaeger is a leading backend for storage and visualization.
• The OpenTelemetry Collector acts as a middleware to receive, process, and export traces to Jaeger (or other backends).
• Auto-instrumentation captures common spans (HTTP, gRPC, DB calls) without code changes, but manual instrumentation gives finer control.
• Always propagate the traceparent header downstream — broken context leads to orphan spans.
• For production, use the Jaeger Operator and a persistent storage backend (Elasticsearch) instead of the all-in-one image.

What you learned & what's next

You now understand how to enable tracing with Jaeger or OpenTelemetry in a Kubernetes cluster — from installing the Collector and Jaeger to instrumenting a sample Python service. You can identify latency bottlenecks across services and choose the right deployment strategy (all-in-one for dev, full production setup for prod).

In the next lesson, we'll explore monitoring with Prometheus, where you'll learn to collect metrics from your instrumented services and set up dashboards and alerts. Tracing tells you where a problem occurs; metrics tell you how often and how many.

Practice recap

Deploy a simple two-service app (e.g., a frontend that calls a backend API) in your cluster, instrumented with OpenTelemetry. Send at least 50 requests, then open the Jaeger UI and examine the flame graph. Identify the service that contributes the most latency. Try adding a simulated delay inside one service and verify the trace shows the spike.