Add Integration Tests to Deployment Jobs
Learn how to add integration tests to deployment jobs in CI/CD pipelines. This hands-on tutorial covers core concepts, step-by-step implementation, troubleshooting, and what to study next.
Focus: add integration tests to deployment jobs
You’ve automated your builds, your unit tests are green, and your deployment job runs like clockwork. But when your latest release reaches staging, the database schema is wrong, the API can’t reach the message queue, and the frontend can’t talk to the backend. Unit tests never caught it, because they don’t test the integration — the way your services actually connect and behave together. That’s the gap this lesson closes: adding integration tests to your deployment jobs so you catch real-world failures before they hit production, not after.
The problem this lesson solves
Unit tests check a single function, class, or module in isolation — but they don’t verify that your application works as a whole. When you deploy, you’re wiring together services, databases, caches, queues, and third-party APIs. If any of those connections break, the whole deployment fails.
Integration tests exercise those connections. By adding them to your deployment job, you validate that your freshly built artifact actually works with the infrastructure it needs. The pain points this solves:
- Happy-path deployments that break in production: Your code works on your laptop because you have different versions of dependencies or services.
- Schema mismatches: Your migration ran, but the application expects a column that doesn’t exist yet.
- Networking issues: Your service can’t reach the database because credentials or endpoints changed.
- Missing configuration: Environment variables, secrets, or feature flags that aren’t set correctly in the deployment environment.
Without integration tests, you’re in a poker game where you only see the cards after the money’s already bet. With them, you fold before production sees the mess.
Core concept / mental model
Think of your deployment pipeline like a plane’s pre-flight checklist. You don’t just check the engines fire individually — you test that the fuel system, hydraulics, and avionics work together. Integration tests are that systems check.
In CI/CD, the mental model is:
- Build → produce a deployable artifact (container image, binary, package).
- Unit test → verify individual components in isolation.
- Deploy to environment → temporarily place the artifact into a controlled staging environment.
- Integration test → test the artifact with its dependencies (database, services, network) to confirm it works as a whole.
- Promote/rollback → if integration tests pass, promote to production; if they fail, rollback.
Pro tip: Integration tests aren’t a replacement for unit tests — they complement them. Unit tests are fast and cheap; integration tests are slower and more expensive, but they catch the stuff unit tests can’t.
Think of it as a trust ladder: the higher you climb (from unit → integration → end-to-end), the more trust you have in the deployment, but also the more time and complexity each rung costs.
How it works step by step
Adding integration tests to your deployment job follows a repeatable sequence. This is the cause-and-effect order you’ll use every time:
- Deploy your application to a test environment (staging, preview, or a dedicated integration namespace). This is the setup phase.
- Run integration test suite — a separate set of tests that hit the running application through its real interfaces (HTTP, database, message queues). This is the verification phase.
- Collect results — check exit code and test report.
- Automate the decision — if tests pass, continue to next stage (e.g., production promotion); if they fail, stop the pipeline and roll back.
Each step feeds into the next. The key is that integration tests run against the live artifact you’re about to deploy — not a separately built test instance that doesn’t match production.
Key detail: In GitHub Actions, you can use the
needskeyword to control job dependencies. Your integration-test jobneedsthe deployment job, and your production-promotion jobneedsthe integration-test job (with a check that it succeeded). This creates a gate.
Hands-on walkthrough
Let’s make this concrete with GitHub Actions. We’ll add an integration test job to a pipeline that deploys to a staging environment, then only promotes to production if those tests pass.
Example 1: Simple integration test job in GitHub Actions
name: Deploy with integration tests
on:
push:
branches: [ main ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: docker build -t myapp:${{ github.sha }} .
- run: docker push registry.example.com/myapp:${{ github.sha }}
deploy-staging:
needs: build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: ./scripts/deploy.sh staging ${{ github.sha }}
integration-test:
needs: deploy-staging
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Wait for staging to be ready
run: ./scripts/wait-for-it.sh https://staging.example.com --timeout=60
- name: Run integration tests
run: ./scripts/integration-tests.sh https://staging.example.com
deploy-production:
needs: integration-test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: ./scripts/deploy.sh production ${{ github.sha }}
Expected output: The pipeline runs build → deploy-staging → integration-test → deploy-production. If integration-test fails, deploy-production is skipped.
Example 2: Python integration test script
Here’s a simple integration test script in Python using requests and pytest:
# tests/integration/test_api.py
import os
import requests
def test_health_endpoint():
base_url = os.environ["APP_URL"]
response = requests.get(f"{base_url}/health", timeout=10)
assert response.status_code == 200
def test_user_creation_flow():
base_url = os.environ["APP_URL"]
payload = {"email": "test@example.com", "password": "secret"}
response = requests.post(f"{base_url}/users", json=payload, timeout=10)
assert response.status_code == 201
user_id = response.json()["id"]
fetch = requests.get(f"{base_url}/users/{user_id}", timeout=10)
assert fetch.status_code == 200
And the shell script that runs it:
#!/usr/bin/env bash
set -euxo pipefail
APP_URL="$1"
IMPORTANT: use the same environment as the deployed artifact
export APP_URL
pytest tests/integration --junitxml=integration-report.xml
If any assertion fails, pytest exits non-zero, and GitHub Actions marks the job as failed.
Example 3: Separate test environment (preview deployment)
For pull requests, you can deploy a preview environment and run integration tests against it:
name: PR preview
on:
pull_request:
jobs:
deploy-preview:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: ./scripts/deploy-preview.sh ${{ github.event.pull_request.head.sha }}
integration-test:
needs: deploy-preview
runs-on: ubuntu-latest
steps:
- run: ./scripts/wait-for-it.sh https://preview-${{ github.event.number }}.example.com --timeout=60
- run: pytest tests/integration
comment-status:
needs: integration-test
runs-on: ubuntu-latest
if: always()
steps:
- uses: actions/github-script@v7
with:
script: |
const outcome = needs.integration-test.result;
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: `Integration tests: ${outcome}`
});
Here, the comment job runs even if tests fail, so you get feedback on every PR.
Pro tip: Always add a
wait-for-it(or health-check) step before running integration tests. Your app may take a few seconds to boot, and tests run too early will fail even though the app is fine.
Compare options / when to choose what
You have several ways to add integration tests to deployment jobs. Here’s how to choose:
| Approach | Pros | Cons | Best for |
|---|---|---|---|
| Same job, after deploy step | Simple, fast, single job | Couples deploy and test logic; hard to reuse | Small projects, quick checks |
Separate job that needs deploy |
Clean separation, reusable, can reuse in other pipelines | Larger pipeline, more jobs | Most projects |
| Preview environment per PR | Same environment for everyone, great feedback | Cost per PR, more infrastructure | Large teams, high change frequency |
| In-cluster test run (e.g., k8s job) | Tests the actual deployment, with quotas | Complex setup, requires cluster access | Kubernetes-heavy projects |
When to choose:
- Small app, low stakes: same job is fine.
- Medium/large app: separate job with
needs. - Many PRs, want parallel testing: preview environments.
- Already using k8s: inline test job is natural.
Troubleshooting & edge cases
Even with the right setup, things break. Here are the most common issues:
- Tests fail because the app isn’t ready yet — you hit a race condition. Fix: add health check with retry, not just a sleep.
- Database isolation: Your integration tests share a database with other tests and corrupt state. Fix: use a dedicated schema or spin up a test database per run.
- Credentials not available in test job: The integration test job doesn’t have the same secrets as the deploy job. Fix: explicitly pass secrets using
env:orsecrets:in the job. - Test environment differs from production: A different database version, different environment variables, or different network config. Fix: keep the test environment as close to production as possible (use the same images, same config templates).
Pro tip: When a deployment fails an integration test, rollback to the previous version rather than trying to fix forward. It’s faster and safer.
What you learned & what's next
You now know how to add integration tests to deployment jobs: you understand the purpose, the mental model, the step-by-step flow, how to implement it in GitHub Actions, and when to use different approaches.
You can now:
- Explain the core concept behind integration tests in deployment jobs.
- Complete a hands-on exercise (like the ones above) to add integration tests to your own pipeline.
- Diagnose common failures and avoid them.
Next lesson: In the next step of the CI/CD foundations track, you’ll learn to handle deployment failures gracefully — how to rollback, alert, and debug when a deployment goes wrong despite your tests passing. That’s the natural follow-up to making your pipeline safer at the gate.
Go ahead and add an integration test job to one of your own pipelines — even a simple health check. You’ll immediately see the safety it adds.
Practice recap
Add an integration-test job to one of your existing GitHub Actions workflows that deploys to a staging environment. Write at least one Python test (e.g., a health check or a simple API flow), and make sure the production deployment is only triggered if that test passes. Then deliberately break a dependency (like changing a database URL) and confirm the pipeline fails before production.
Common mistakes
- Running integration tests as part of the same job as the build, before the app is deployed — they test the artifact but not the running service.
- Forgetting to wait for the application to be healthy before starting integration tests, causing failures due to startup latency.
- Not isolating test data or databases between test runs, leading to flaky tests from state pollution.
- Not passing the correct environment variables or secrets to the test job, so the application connects to the wrong services.
- Assuming that unit tests passing means integration tests will pass — they test different layers.
Variations
- Use a separate staging environment (e.g., a review app) for integration tests instead of the main deployment target.
- Run integration tests in a Docker container that includes the app and its dependencies for a more realistic environment.
- In Kubernetes, use a job in the cluster to run integration tests against the deployed service, giving tighter integration.
Real-world use cases
- E-commerce platform: run integration tests on a staging deploy that exercises checkout flow (cart, payment, order) before production promotion.
- REST API service: test that the API can talk to the database and message queue after a new schema migration is deployed.
- Microservices saga: deploy a new version of a service and run integration tests that verify cross-service calls still work with the updated contract.
Key takeaways
- Integration tests in deployment jobs verify that your built artifact actually works with its dependencies, catching issues unit tests miss.
- The pattern is: deploy to a test environment → wait for it to be healthy → run integration tests → only promote if they pass.
- Use the
needskeyword in GitHub Actions to make the integration-test job depend on the deployment job and gate the production deployment. - Keep a health-check/wait-for-ready step before running integration tests to avoid race conditions.
- The environment for integration tests should mirror production as closely as possible to avoid surprises after deployment.
- Decide between inline tests, separate jobs, preview environments, or in-cluster test runs based on your project size and complexity.
Keep learning
Related tutorials, quizzes, and articles for this topic.
Discussion
Questions, corrections, and tips help everyone reading this page.
0 comments
Add a comment
No comments yet — start the thread.