Visualize Telemetry in Prometheus

You’ve got your Linux servers humming, your networking stack is stable, and you’re finally collecting telemetry—but the metrics are piling up in a dashboard that’s about as informative as a wall of numbers. You’ve heard Prometheus is the go‑to for monitoring, but when you open the expression browser, you see raw counters and gauges with no context. You need to visualize that telemetry — to turn raw metric names into graphs that tell a story. That’s exactly what this lesson tackles: taking the Prometheus basics you’ve set up and turning them into meaningful visualizations that help you spot anomalies, trends, and capacity issues before they become incidents.

The problem this lesson solves

You’ve already installed Prometheus and scraped some metrics from a target—maybe node_exporter or a custom app. But staring at the Prometheus expression browser, you’re probably wondering: What do I do with all these numbers? A list of node_cpu_seconds_total values for every CPU core is not a dashboard; it’s data soup.

Without proper visualization, telemetry is just noise. You can’t easily answer questions like “Is my CPU spiking at 2 PM?” or “Which service has the highest memory usage?” You end up doing mental math on raw counters, missing trends, and reacting to alerts only after things break. The pain is real: teams with powerful monitoring stacks but poor visualization often have worse incident response because they can’t quickly interpret the signals.

That’s why mastering Prometheus visualization is a core skill for anyone working on Linux and networking telemetry. Once you can create, edit, and save graphs, you move from “collecting data” to “actually seeing what’s happening.” You’ll be able to spot a memory leak, detect a network bottleneck, and explain to a colleague why latency is spiking—all by glancing at a graph.

Core concept / mental model

Think of Prometheus as your telemetry warehouse. It stores time‑series data—each metric has a name, labels (key‑value pairs), and a timestamped value. The PromQL (Prometheus Query Language) is the tool you use to pull specific slices of that data. But raw query results are just tables of numbers. Visualization is the layer that turns those numbers into lines, bars, and heatmaps that your human brain can parse instantly.

Here’s the mental model: Data → Query → Graph. You have a huge pile of data points; a PromQL expression slices and aggregates them; the graph renders that slice as a picture. The art of visualization is choosing the right query and the right graph type to highlight what matters.

• Counter: a cumulative value that only goes up (e.g., http_requests_total). You rarely display a counter as‑is; you’d use rate() to show per‑second increase.
• Gauge: a value that goes up and down (e.g., memory_used_bytes). You can display it directly.
• Histogram: a set of buckets that count observations (e.g., request latency). You’d often use histogram_quantile() to get percentiles.

In Prometheus’s own UI, you can run a query and see a table of values, or you can switch to the Graph tab—but that’s just a quick glance. The real power comes from creating dashboard panels that persist and refresh, combining multiple graphs on one screen. You can do that with Prometheus’s built‑in dashboard feature (if you’re using the Prometheus UI) or with a dedicated tool like Grafana, which is the de facto standard.

For this lesson, we’ll focus on the Prometheus web UI—it’s already there, and it’s perfect for learning the fundamentals. Once you’re comfortable, moving to Grafana is a natural next step.

How it works step by step

Let’s break down how visualization works in Prometheus, step by step. You don’t need any extra configuration—everything is in the Prometheus binary you already have.

1. Access the Prometheus UI – By default, Prometheus runs on port 9090. Open http://your-server:9090 in your browser. You’ll see the classic purple interface with tabs: Alerts, Graph, Status, and Help.
2. Explore the Graph tab – Click on Graph. This is your visualization playground. You have an input box for PromQL expressions, a “Execute” button, and two tabs below: Table and Graph.
3. Write a PromQL expression – Start with a simple metric like up (which tells you if a target is currently scraped successfully). Type up in the box and press Enter or click “Execute”. The default view is Table, which shows the metric name, labels, and the value 1 (up) or 0 (down).
4. Switch to Graph – Click the Graph tab right below the execute button. You’ll see a time‑series plot of the metric over the last hour by default. You can change the time range using the time picker (e.g., “Last 1 hour”, “Last 6 hours”, or a custom range).
5. Refine your query – For a counter like node_cpu_seconds_total, plotting the raw counter gives you a flat line that jumps up by 1 or 2 every few seconds—hard to read. Instead, use rate(node_cpu_seconds_total[5m]) to get the per‑second increase averaged over 5 minutes. That shows you CPU usage as a percentage (if you divide by the number of CPU cores, but we’ll cover that later).
6. Save your graph – The Prometheus UI doesn’t let you save named dashboards (that’s Grafana’s job), but you can bookmark the URL with the query, or copy the query for later. More importantly, you’ll learn to compose and reuse these queries.

The core workflow is: Type query → Execute → Switch to Graph → Analyze. It’s iterative—you’ll try different queries, adjust time ranges, and zoom in on anomalies.

Hands-on walkthrough

Let’s do a practical exercise. You should already have Prometheus running and scraping at least one target—if not, start Prometheus with a minimal config that scrapes itself (scrape_interval: 15s).

Step 1: Confirm your setup

First, make sure Prometheus is scraping itself. Open http://localhost:9090 (or your server’s IP). Go to Status → Targets, and you should see one target with state “UP”.

Step 2: Basic graph with up

Go to Graph, type up, and click Execute. You’ll see a table showing up{instance="localhost:9090", job="prometheus"} 1. Now click the Graph tab. You’ll see a flat line at 1—meaning your target is up. Change the time range to Last 15 minutes to see a longer trend.

Step 3: Visualize a counter with rate()

Now let’s visualize HTTP requests. If you’re scraping a sample app, use http_requests_total; if not, use the Prometheus internal metric prometheus_tsdb_head_samples_appended_total (a counter of samples appended). Type:

rate(prometheus_tsdb_head_samples_appended_total[5m])

Click Execute, then switch to Graph. You’ll see a line that goes up and down – that’s the rate of samples being appended per second. It’s a classic example of turning a monotonically increasing counter into a useful metric.

Step 4: Combine and aggregate

Let’s get a bit more advanced. To see the total rate across all instances, use sum():

sum(rate(prometheus_tsdb_head_samples_appended_total[5m]))

You’ll see a single line above. This is the foundation of dashboard panels: you aggregate raw metrics into a meaningful signal.

Step 5: Use labels to split the graph

If you have multiple targets, you can split by labels. Use by (job) to group the sum by job:

sum by (job) (rate(prometheus_tsdb_head_samples_appended_total[5m]))

You’ll see multiple lines, one per job—like a legend. This is how you compare services.

Step 6: Export a grafana-style view (optional)

If you’re ready to move beyond the Prometheus UI, install Grafana (it’s a single binary too). Add Prometheus as a data source, then create a new panel and paste the same PromQL expression. Grafana gives you richer graph types (heatmaps, gauge, bar) and allows you to save dashboards.

Here’s a complete example of a PromQL query you might use in a real dashboard:

# CPU usage as a percentage across all cores on a node
100 - (avg by (instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)

That gives you the non‑idle CPU percentage per instance—a classic “CPU usage” panel.

Expected output: In the Prometheus graph, you’ll see a line (or multiple lines if you have multiple instances) fluctuating between 0% and 100%—typically a sawtooth pattern as load varies.

Compare options / when to choose what

Prometheus’s built‑in UI is fine for quick checks, but for serious dashboards you’ll likely use Grafana. Here’s a comparison:

When to use Prometheus UI: When you’re debugging a metric or trying out a PromQL query—it’s instant and zero‑setup. When to use Grafana: When you need to build a monitoring dashboard for your team, or you want to visualize multiple metrics together.

Variations: You can also use alternatives like Thanos for long‑term storage and cross‑cluster queries, or Loki for logs—but they aren’t necessary for visualisation basics. Another variation is using Prometheus’s built‑in “Console templates” which let you write HTML templates, but that’s a legacy feature. For most people, stick with the UI and Grafana.

Troubleshooting & edge cases

Graph is empty – The most common issue. Check if your query returns data in the Table view. If not, you might have a typo, the metric name is wrong, or the time range doesn’t cover any data. Also, counters reset after a restart, so you’ll see drop‑offs in graphs—that’s normal; use increase() or rate() to smooth them.

Rate() gives multiples – If you have many targets, rate() returns multiple series. Use sum() or by to aggregate.

Graph shows spikes or gaps – This usually happens when a target is temporarily down or the scrape interval changes. Check Status → Targets to ensure all targets are UP. Prometheus scrapes at intervals (default 15s), so your graph resolution is limited to that interval; you can’t see sub‑second changes.

Empty legend names – If you don’t use by, the legend shows the full label set. That can be verbose. Use by (job) to make it cleaner.

Time range too short – If you only have 15 minutes of data, you won’t see daily patterns. Change the time range to Last 24 hours or a custom span.

Using rate() on a gauge – This is a classic mistake. rate() works only on counters; on a gauge it gives nonsense. For gauges, just plot the value directly.

What you learned & what's next

You’ve learned the core idea behind visualizing telemetry in Prometheus: you start with a PromQL query, run it in the Graph tab, and interpret the resulting time‑series plot. You can now turn a raw counter into a meaningful rate, aggregate multiple series, and compare different groups using labels. You also know the difference between the Prometheus UI and Grafana, and when to use each.

That covers the learning objectives: you can explain how visualization fits into the telemetry pipeline, and you completed a hands‑on exercise that took you from a raw metric to a clear graph.

Your next step is to combine these visualizations with alerts—using Prometheus’s alerting rules to notify you when a metric crosses a threshold. That’s the natural evolution: see a problem, then get proactively warned. In the next lesson, you’ll dive into writing alert rules and integrating them with Alertmanager. That’s where you’ll turn your new visualization skills into an automated monitoring response.