Publish Test Reports & Artifacts

Your tests passed locally, but the CI log scrolls for a thousand lines and your teammates can't tell if anything actually failed. Worse, that compiled binary, the one QA needs for manual testing, is trapped inside the runner's virtual machine and will be deleted the second the job finishes. This is the silent killer of every CI/CD pipeline: test results and build artifacts exist, but nobody can see them. This lesson teaches you how to publish test reports and artifacts so your pipeline's output becomes a compact, shareable, and permanent part of your delivery process — step 10 in your CI/CD foundations journey.

The Problem: Invisible Test Results and Lost Build Output

Every time you run a build or a test suite in CI, the runner does a bunch of work and then … discards it. Let's look at what actually happens without publishing:

• Test reports — By default, CI systems capture raw log output. A single pytest run can produce megabytes of text, interleaved with warnings, coverage prints, and dependency download messages. Finding the one assertion that failed is like finding a needle in a stack of needles.
• Artifacts — Compiled binaries, wheel files, Docker images, or even coverage HTML reports are generated inside the runner's file system. The moment the job finishes, that entire environment is destroyed. No one can download that binary, no one can inspect the report, and the only proof of your work is a log timestamp.

Why does this matter now? In earlier lessons, you learned to build, test, and deploy. But without publishing, your pipeline is a black box. When a test fails, the only thing you know is something failed. When a build succeeds, you have nothing to show for it. Publishing changes that: it turns raw output into structured, searchable, and persistent information that you and your team can act on immediately.

Pro tip: In the 90s, CI was about running tests. Today, CI is about understanding what the tests say. Publishing your results is what separates a build machine from a diagnostic tool.

Core Concept: Two Kinds of Output You Must Publish

The mental model is simple: a successful pipeline produces two kinds of valuable output, and each must be published separately.

1. Test Reports (the 'What')

A test report is a structured summary of your test run: how many tests passed, failed, were skipped, and why something failed. Formats like JUnit XML, HTML, or JSON make this machine-readable and human-friendly. When you publish a test report, CI can:

• Show a pass/fail badge on your repo.
• Display a trend chart of test results over time.
• Allow a developer to click into a failed test and see the exact assertion error without scrolling logs.

2. Artifacts (the 'Produced')

An artifact is any file your pipeline generates that you want to keep: a compiled .jar, a .whl Python package, a coverage report, a deployment bundle, or even a screenshot from a UI test. Publishing an artifact means storing it in a persistent location attached to the pipeline run, where anyone with access can download it.

Think of it like this: Your pipeline is a chef. The test report is the menu description of what was cooked (and if it burned). The artifact is the actual dish you can taste. Both need to be placed on the counter — not left behind in the kitchen.

Defining terms

• Publish — The act of making pipeline output available outside the runner's ephemeral environment.
• Report — A structured file (e.g., JUnit XML) that summarizes test results.
• Artifact — Any binary or file produced by the build that you want to persist (e.g., a wheel, an executable).

How It Works Step by Step

The process of publishing is surprisingly uniform across CI systems. Whether you use GitHub Actions, GitLab CI, or Jenkins, the steps are always the same. Let's walk through the general workflow, then dive into a hands-on example with GitHub Actions.

Step 1: Generate the report or artifact inside your build

Your test framework must be configured to output a machine-readable report. For Python's pytest, that means installing pytest-junit or using the built-in --junitxml flag. For JavaScript's Jest, you'd use the jest-junit reporter. Your build tool (e.g., setuptools, cargo, docker build) already produces a binary; you just need to know where it lands.

Step 2: Upload it as an artifact

Most CI systems provide a built-in action or command to upload a file or directory. In GitHub Actions, that's actions/upload-artifact. In GitLab CI, it's the artifacts keyword in your .gitlab-ci.yml. The upload process stores the file with a name and expiration policy.

Step 3: Attach the report to the summary or annotate it

Uploading artifacts doesn't automatically make them readable in the PR view. Many CI systems let you attach reports to the run summary or even annotate the code with results. GitHub Actions, for example, can display JUnit XML reports in an annotations panel on the Checks tab.

Step 4: Set retention policies

Artifacts accumulate fast. Every commit could produce hundreds of files. Best practice is to set a retention period — GitHub defaults to 90 days, but you can configure it. Some artifacts (like release binaries) need longer retention; others (like intermediate build files) can be deleted after a day.

Pro tip: You don't have to publish everything. A common anti-pattern is publishing huge logs. Instead, publish only the summary of the log, and keep the raw log in the CI system's own storage.

Hands-On Walkthrough

Let's put this into practice with a simple Python project and GitHub Actions. You'll learn to publish test reports and a wheel artifact.

Prerequisites

• A GitHub repository with a basic Python package (e.g., a pyproject.toml and a tests folder).
• A GitHub Actions workflow file (.github/workflows/ci.yml).

Example 1: Publish a JUnit test report

First, configure pytest to output JUnit XML. In your repository root, run:

pip install pytest pytest-junit
pytest --junitxml=reports/junit.xml

Now create a workflow that runs the tests and uploads the report:

name: CI
on: [push]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - run: pip install -e . pytest pytest-junit
      - run: pytest --junitxml=reports/junit.xml
      - name: Publish test report
        uses: actions/upload-artifact@v4
        with:
          name: test-report
          path: reports/
          retention-days: 30

Expected output: After a successful run, you'll see a downloadable artifact named test-report on the workflow run's page containing junit.xml.

Example 2: Publish a build artifact (wheel)

Now build a Python wheel and publish it. Add a build step:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - run: pip install build
      - run: python -m build
      - name: Publish wheel
        uses: actions/upload-artifact@v4
        with:
          name: dist
          path: dist/
          retention-days: 90

Expected output: A dist artifact containing your_package-0.1.0-py3-none-any.whl and a source tarball. Anyone can download it from the run page.

Example 3: Combine both in one job

Often you want both in the same run. Here's a compact workflow that runs tests, then builds, then uploads both:

name: CI
on: [push]

jobs:
  build-and-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - run: pip install -e . pytest pytest-junit build
      - run: pytest --junitxml=reports/junit.xml
      - run: python -m build
      - name: Upload test report
        uses: actions/upload-artifact@v4
        with:
          name: test-report
          path: reports/
      - name: Upload build artifacts
        uses: actions/upload-artifact@v4
        with:
          name: dist
          path: dist/

Expected output: Two separate artifacts, test-report and dist, each with the appropriate files. Note that upload actions are idempotent; you can upload multiple times with the same name if you want to merge.

Pro tip: In GitHub Actions, you can view JUnit XML directly on the Checks tab if you also use the dorny/test-reporter action, which renders failures beautifully. But upload-artifact is sufficient for raw downloads.

Compare Options / When to Choose What

Not all CI systems handle publishing the same way. Here's a comparison to help you choose:

When to choose what: - Small projects: Use GitHub Actions with upload-artifact; it's zero-config and deeply integrated. - Multi-branch pipelines: GitLab's artifacts are automatically available to subsequent jobs in the same pipeline, making it easy to pass test reports to a deploy job. - Enterprise with many tools: Jenkins offers the most flexibility, but with higher maintenance overhead.

Variations worth knowing

• S3/GCS as artifact storage: For very large artifacts, you might upload directly to cloud storage instead of the CI system's built-in artifact store. CI then just references the URL. This is useful for assets > 2GB or when you need immutable storage.
• CI cache vs. artifacts: Caches speed up future runs by storing dependencies; artifacts preserve current run output. They serve different purposes — don't mix them up. Caches are not meant for human download.
• Publish test reports as PR comments: Tools like pytest-github-actions-reporter can post a summary comment on pull requests. This is more interactive than an artifact but less persistent.

Troubleshooting & Edge Cases

Here are common pitfalls you'll encounter when publishing test reports and artifacts — and how to fix them.

1. The artifact is empty or missing files

Symptom: You download the artifact and it's empty, or only has logs.

Cause: You pointed path at a directory that doesn't exist, or the build created files in a different location than you expected.

Fix: Always verify the path exists before upload. Run a quick ls -la step in the workflow to see the file structure. For example:

ls -la reports/

If it's still empty, check that your test command actually generates the report. Many test frameworks only write XML if you pass the correct flag.

2. Upload fails due to permissions or large size

Symptom: The upload step errors with Resource not accessible by integration or Artifact size exceeds max.

Cause: The default GITHUB_TOKEN may lack write permission to the artifact store, or the artifact is above the 2GB limit.

Fix: - Grant actions: write permission to the workflow (e.g., permissions: contents: read, actions: write). - Split large artifacts into smaller chunks or use external storage.

3. Test report is not visible in the PR checks

Symptom: You uploaded the JUnit XML, but the Checks tab still just shows a log.

Cause: Uploading an artifact doesn't automatically render reports. You need a dedicated test reporter action/plugin.

Fix: Use dorny/test-reporter with your JUnit XML, or switch to a CI system that natively renders reports.

4. Artifact retention too short for release deliverables

Symptom: You need a release binary from months ago, but it's gone.

Fix: Set retention-days to 365 or more for release artifacts. For intermediate files, keep the default 30 days. Better yet: promote final artifacts to a permanent release storage like GitHub Releases or an artifact registry.

What You Learned & What's Next

You've just learned the core idea behind publishing test reports and artifacts: turning ephemeral pipeline output into persistent, inspectable, and shareable assets. You can now:

• Explain the difference between test reports and artifacts.
• Upload both using GitHub Actions' upload-artifact.
• Set retention policies and troubleshoot common issues.
• Choose the right tool for your CI environment.

That covers the learning objective of this lesson — you've completed a practical exercise that directly applies this concept.

The next lesson in your CI/CD foundations path is about approvals and promotions — how to gate your pipeline when human sign-off is required before deploying a build. You'll learn how to use the artifacts you just published as the gateway to release. With your new ability to publish test reports and artifacts, you'll be able to see exactly what a developer approved or rejected. Ready to add a human touch to your automated delivery? Let's move forward.