SLOs for Releases

Your latest release passed every test, the pipeline was green, and you hit deploy — but within minutes, users start reporting slow load times and errors. The build succeeded; the service didn't. That’s the gap this lesson closes: define service level objectives for releases so you move beyond did the build pass? to did the service stay healthy? Without explicit SLOs, every release is a gamble — you only know it broke after someone complains. By the end of this lesson, you’ll set measurable targets that gate your pipeline and protect your users.

The problem this lesson solves

Teams often treat CI/CD as a binary: if the pipeline is green, the release is good. But a green pipeline only proves the code built and unit tests passed — it says nothing about whether the service meets user expectations in production. The result is a recurring cycle: deploy, wait, react to alerts, roll back, and then ask why didn’t we catch this earlier?

Consider a typical incident: your checkout service deployed at 10:00 AM. By 10:15, API latency doubles due to a new database query. The deploy succeeded, but the service failed. Without a predefined SLO, you discover this through customer complaints, not automated checks. That’s the pain — you’re flying blind after each release, and every rollback costs time, trust, and money.

The solution is to define service level objectives before you release — explicit, measurable thresholds that your pipeline can verify. These are not just dashboards; they become release gates: if the SLO is breached, the release is blocked or rolled back automatically. This lesson shows you how to define those targets and wire them into your CI/CD workflow, so a green pipeline means the service meets user expectations, not just that the code compiled.

If you’re new to SLOs, you’re not alone — many teams treat them as an afterthought. But as you’ll see, they’re as fundamental to release safety as tests, and they pay off immediately in reduced incidents and faster recovery.

Core concept / mental model

Think of an SLO (Service Level Objective) as a contract between your team and your users, expressed in numbers. It answers three questions:

• What does good look like? (e.g., 99.9% of requests succeed)
• Over what window? (e.g., the last 30 days)
• Who decides? (You, based on user expectations, not arbitrary targets)

A mental model: imagine you’re a flight operator. Your SLI (Service Level Indicator) is on-time departure — the metric you measure. Your SLO is a target like 99.5% of flights depart within 15 minutes of schedule. The SLA (Service Level Agreement) is the contract with a passenger where you promise a refund if you miss the SLO. In releases, you care about SLIs and SLOs: they’re measurable and actionable.

Why does this matter for releases specifically? Because every deploy changes your SLIs. Your pipeline can compare the live SLO with the target after the deploy and decide automatically: keep the change or revert. That’s the key mental shift — SLOs become a deterministic release gate, not a retrospective report.

There are three critical components:

• Service Level Indicator (SLI): a metric that reflects user experience, e.g., request success rate, latency, error rate.
• Service Level Objective (SLO): a target value for that SLI, e.g., “99.9% of requests succeed in a 30-day window.”
• Error budget: the margin you can afford to fail — 100% minus the SLO. If SLO is 99.9%, your error budget is 0.1% (8.76 hours per year). Deployments that consume the error budget without warning force you to slow down or fix before next release.

Here’s a visual of the flow:

User request → SLI (latency, success) → compare to SLO (target) → error budget remaining → release decision

Key definitions to internalize:

SLI: a measurable metric like “HTTP 200 rate.” SLO: a target threshold like “99.9% of requests return 200.” SLA: a contract with consequences. For releases, you only need SLIs and SLOs.

How it works step by step

Defining SLOs for releases isn’t a one-time task — it’s a process you repeat as your service evolves. Here’s the logical sequence, cause → effect:

1. Identify your critical user journeys (CUJs). What do users do constantly? For an e-commerce site: browse products, add to cart, checkout. For an API: authenticate, fetch data, save data. These journeys become the basis of your SLIs.

2. Choose one or two SLIs per journey. Focus on the ones that matter most — usually availability (success rate) and latency. Avoid vanity metrics like CPU usage; they don’t reflect user experience.

3. Set a target SLO for each SLI. Base it on what you can realistically achieve — not a guess. Start with current performance plus a small buffer, e.g., if you’re at 99.95% availability, set 99.9% to allow room for errors.

4. Define the measurement window. Common choices: rolling 30 days, 7 days, or a calendar month. Shorter windows react faster but have more variance; longer windows are stable but hide spikes.

5. Turn SLOs into release gates. In your CI/CD pipeline, after deploying to canary or production, run checks that compare live SLIs against the SLO. If the SLO is exhausted (error budget is zero or negative), block further promotion or auto-rollback.

6. Monitor, review, and adjust. SLOs aren’t static. Revisit monthly — if you’re always exceeding the target, raise it; if you’re always burning, lower it or fix the root cause.

The cause and effect: setting a realistic SLO upfront means your pipeline only blocks releases when users are actually affected. That reduces flaky gates while catching real regressions early.

Hands-on walkthrough

Let’s apply this with a concrete example. You have a Python FastAPI service that serves a GET /products endpoint. You want to define two SLOs:

• Availability: 99.9% of requests succeed over 30 days
• Latency: 95% of requests complete under 300ms over 30 days

You’ll store these in a config file (slo.yaml) and use a Python script to compute error budgets from metrics, then integrate with your pipeline.

Step 1: Define SLOs in a YAML file

Create slo.yaml:

slo_name: products_service
slis:
  - name: availability
    sli_type: success_rate
    metric_target: 99.9
    window: 30d
  - name: latency
    sli_type: percentile
    percentile: 95
    threshold_ms: 300
    window: 30d

Step 2: Compute error budget from metrics

Assume you have metrics in Prometheus format. Install prometheus-api-client and write a Python script to calculate your SLO status:

from prometheus_api_client import PrometheusConnect
import yaml

prom = PrometheusConnect(url="http://localhost:9090", disable_ssl=True)

with open("slo.yaml") as f:
    config = yaml.safe_load(f)

# Example: fetch success rate over the last 5 minutes (simplified)
query = 'sum(rate(http_requests_total{status=~"2.."}[5m])) / sum(rate(http_requests_total[5m])) * 100'
success_rate = prom.custom_query(query)[0]['value'][1]

availability_slo = config['slis'][0]['metric_target']
error_budget = float(success_rate) - availability_slo
print(f"Current success rate: {success_rate:.2f}%")
print(f"SLO target: {availability_slo}%")
print(f"Error budget remaining: {error_budget:.2f}%")

Expected output (simulated):

Current success rate: 99.95%
SLO target: 99.9%
Error budget remaining: 0.05%

Step 3: Make a release decision

Put this logic into a Python script that your CI can call to decide whether to promote a release:

import sys

def should_release(success_rate, sla_target):
    # Return True if error budget is positive, else False
    return float(success_rate) >= sla_target

if __name__ == "__main__":
    # This would come from your metrics query
    success_rate = 99.99  # example
    sla_target = 99.9
    if should_release(success_rate, sla_target):
        print("SLO met — proceed with release")
        sys.exit(0)
    else:
        print("SLO violated — block release and alert")
        sys.exit(1)

Run it in your pipeline after canary deployment:

python check_slo.py

If the script exits non-zero, the pipeline stops and your release is blocked — that’s your gate.

Compare options / when to choose what

Not all SLOs are equal — you need to choose the right metrics and targets. Here’s a comparison of common approaches:

For release gating, you typically start with availability + one latency percentile. Over-engineering with too many SLIs makes gates noisy and slows delivery — pick the top two that matter to your users.

Pro tip: If you’re just starting, set a single availability SLO first. You can always add latency later without redesigning your pipeline.

Troubleshooting & edge cases

Here are common issues you’ll hit when defining SLOs for releases, with concrete fixes:

• SLO is always violated after deploy even when users are fine. Your SLI is wrong — you’re measuring internal metrics like database CPU instead of user-facing success. Fix: switch to request success rate from your ingress or API gateway.
• Error budget is constantly oscillating between zero and positive, causing flaky gates. Your measurement window is too short (e.g., 5 minutes). Fix: use a rolling 30-day window, but for release gates, use a shorter window (e.g., 15 minutes) to catch immediate regressions, then combine with the long-term budget.
• You hit the SLO but your latency is terrible. You only measured availability. Fix: add a latency SLI as a separate gate, as shown above.
• The pipeline blocks a release due to SLO, but the issue was a pre-existing problem, not the new release. You need a baseline. Fix: compare post-deploy SLO with pre-deploy values—if the degradation existed before, don’t block this release; alert instead.
• No metrics available in staging/CI environment. Many teams only have production telemetry. Fix: generate synthetic traffic to your staging environment after deploy, or use a smoke-test that mimics user requests and measures response time.

What you learned & what's next

You now understand the difference between SLI, SLO, and SLA, and how to define service level objectives for releases. You’ve learned why SLOs are critical release gates — they protect users from regressions that unit tests miss. You’ve seen how to compute error budgets and use them to block or allow a release in your pipeline. You’ve also practiced a hands-on exercise with a Python script that evaluates an SLO and decides release readiness.

Every learning objective is covered: you can explain the core idea behind define service level objectives for releases, and you completed a practical exercise that checks an SLO in a pipeline.

Next in the CI/CD foundations track, you’ll learn how to monitor and alert on SLOs — turning these static targets into live dashboards and automated alerts. That’s the natural next step after defining them, and it will close the loop on release safety.