Test Coverage & Quality Gates

You've written tests for your FastAPI app, and they all pass — but do you actually know what they're covering? Without a clear picture of code coverage, you might be shipping critical bugs that your tests never touch. This lesson shows you how to measure test coverage with pytest-cov and coverage.py, then enforce a minimum quality bar with quality gates that block bad merges automatically.

The problem this lesson solves

Every test suite has blind spots. You might test the happy path of every endpoint but miss the error branches, or test your routes thoroughly while your service layer sits untouched. When you deploy to production, that's when the gaps get discovered — by your users. Manually guessing which parts of your code are exercised is error-prone and slow.

Coverage measurement solves this by telling you exactly which lines and branches your tests execute. But numbers alone aren't enough. You need to turn that measurement into action — a quality gate that fails a build when coverage drops below an agreed threshold. Without gates, coverage reports end up as PDFs that nobody reads.

This lesson teaches you to measure test coverage and set quality gates in a FastAPI project, so you merge code with confidence.

Core concept / mental model

Think of coverage as a map of your codebase. Each line and branch is a territory, and your tests are explorers. Coverage tools identify which territories have been visited and which remain uncharted.

• Statement coverage — the percentage of executable lines that ran during tests.
• Branch coverage — the percentage of if/else and similar branch outcomes that were exercised.

A quick analogy: statement coverage is like knowing you visited 10 out of 20 rooms in a house. Branch coverage also tells you whether you opened both doors in each room.

A quality gate is a guardrail — it checks that every new change maintains or improves the coverage map. If a change drops coverage below the threshold, the gate rejects it, blocking a merge until the developer adds more tests.

How it works step by step

1. Install tooling — Add pytest-cov (which wraps coverage.py) to your project's dev dependencies.
2. Run coverage — Execute pytest --cov=yourapp to run your tests and collect coverage data.
3. Read the report — The terminal shows a table with per-file percentages and overall totals.
4. Set a threshold — Use --cov-fail-under=80 to make pytest exit non-zero if coverage falls below 80%.
5. Integrate with CI — Your quality gate runs these checks automatically on every pull request.
6. Review and iterate — Developers see the report, add tests to cover gaps, and the gate passes.

Hands-on walkthrough

Step 1: Install coverage tools

Make sure your requirements-dev.txt includes:

pytest
pytest-cov

Install them:

pip install pytest pytest-cov

Step 2: Write a sample FastAPI endpoint

# app/main.py
from fastapi import FastAPI, HTTPException

app = FastAPI()

items_db = {1: "apple", 2: "banana"}

@app.get("/items/{item_id}")
def get_item(item_id: int):
    if item_id not in items_db:
        raise HTTPException(status_code=404, detail="Item not found")
    return {"item": items_db[item_id]}

Step 3: Test the happy path only

# tests/test_main.py
from fastapi.testclient import TestClient
from app.main import app

client = TestClient(app)

def test_get_item_happy_path():
    response = client.get("/items/1")
    assert response.status_code == 200
    assert response.json() == {"item": "apple"}

Step 4: Measure coverage

Run:

pytest --cov=app --cov-report=term-missing

Output (simplified):

Name         Stmts   Miss  Cover   Missing
------------------------------------------
app/main.py       8      2    75%   5, 8
------------------------------------------
TOTAL             8      2    75%

The Missing column lists the unexecuted lines — here, the raise HTTPException and the return after it. Your coverage is 75%, below a typical 80% gate.

Pro tip: Use --cov-report=html to generate an interactive HTML report. Open it in your browser to visually spot uncovered lines highlighted in red.

Step 5: Add a failing test for the 404 branch

# tests/test_main.py
def test_get_item_not_found():
    response = client.get("/items/999")
    assert response.status_code == 404

Re-run coverage:

pytest --cov=app --cov-report=term-missing

Now both branches execute, and coverage jumps to 100%.

Step 6: Set the quality gate

Run pytest with the failure threshold:

pytest --cov=app --cov-report=term --cov-fail-under=80

If coverage drops below 80%, pytest exits with a non-zero code and the command fails. In CI, that failure blocks the merge.

Compare options / when to choose what

If you need just a pass/fail gate, --cov-fail-under is enough. If you want team visibility and trending, integrate a coverage service that comments on pull requests and tracks changes over time.

Troubleshooting & edge cases

• Coverage shows 0% for your app — You forgot to pass --cov=app. If your package is named differently, adjust the path.
• Tests pass but --cov-fail-under says failure — The threshold is lower than actual coverage? No, that's fine. The error usually appears when you add new code without tests. Run with --cov-report=term-missing to see which lines are uncovered.
• Coverage looks too high — You might have excluded your __init__.py or test files incorrectly. Use .coveragerc to filter what counts.
• CI says pytest not found — Make sure you install requirements-dev.txt in the CI image, not just production packages.
• conftest.py is marked as uncovered — That's normal; dedicated test infrastructure doesn't need to count.

What you learned & what's next

You now know how to measure test coverage in a FastAPI project with pytest-cov, interpret terminal reports to spot untested branches, and enforce a minimum quality threshold with --cov-fail-under. You've also seen how to compare tools and integrate coverage into CI quality gates.

You're ready for the next lesson in the FastAPI track, which builds on this foundation to explore deployment strategies. Continuous integration and quality gates are a natural stepping stone — next you'll learn how to package and deploy your application reliably.

Keep practicing: add coverage to an existing project, set a realistic threshold, and watch your confidence rise with every merge.