Job Dependencies and Concurrency

You've got a pipeline with a build job and a test job, but they're running at the same time, racing each other, or worse — tests are failing because the build hasn't finished. And when two developers push to main at once, you're suddenly running duplicate deployments that step all over each other. The fix? You need to add job dependencies and concurrency to your pipeline: needs to sequence jobs that depend on each other, and concurrency to keep parallel runs from colliding. In this tutorial, you'll learn how to control the order and the flow of your GitHub Actions workflows so your CI/CD pipeline runs predictably every time.

The problem this lesson solves

Without explicit dependencies, every job in a GitHub Actions workflow starts at the same moment. That's fine if jobs are totally independent, but the moment one job uses the output of another — a compiled artifact, a test report, or a pushed image — you hit a race. Tests run against stale code, deployments ship half-built software, and failures become impossible to trace.

Concurrency issues are the second pain. Suppose two commits land on main seconds apart. Without limits, your pipeline runs the deploy job twice at the same time, and the last write wins — or worse, the first write overwrites the second. Production gets a mix of both versions, and no one can say what's actually running.

You don't need an orchestrator like Airflow to fix this. In GitHub Actions, two keywords — needs and concurrency — give you complete control over job order and parallel execution limits. Once you master them, your pipeline becomes a well-ordered relay race instead of a chaotic free-for-all.

Core concept / mental model

Think of a workflow as a dependency graph. Each job is a node, and needs draws a directed arrow from one job to another. A job only starts when all the jobs it needs have completed successfully. No arrow means no dependency — jobs running in parallel by default.

Pro tip: The graph is a DAG — a directed acyclic graph. That means you can't have circular dependencies like job A needs B and job B needs A. GitHub Actions will reject that with an error, so always think in one direction: from foundational jobs upstream to dependent jobs downstream.

Concurrency is a separate concept: it puts a gate on the whole run. You define a concurrency block with a key (often a branch or a tag) and a value like cancel-in-progress: true. When a new run arrives with the same key, GitHub Actions either cancels the older run or queues the new one, depending on how you set it. This prevents two runs of the same branch or environment from stepping on each other.

Here's the mental model in a sentence: needs controls order between jobs; concurrency controls how many runs of the same group can be active at once.

How it works step by step

1. Declare dependencies with needs

The needs key goes in any job you want to make dependent. You list the job IDs it depends on. A job waits until those complete with a success status. If any parent job fails, the dependent job is skipped — it won't even start.

2. Use job outputs to pass data

Dependencies are more useful when data flows between jobs. Each job can define outputs, and a dependent job can read them through the needs context. This is a clean way to share values like a version number or a build path without writing artifacts to disk.

3. Group runs with concurrency

The concurrency block sits at the workflow level (or inside a job). You give it a key — typically ${{ github.workflow }}-${{ github.ref }} — so each branch or tag has its own group. Then you set cancel-in-progress to either true (kill the older run) or false (queue the new one).

4. Combine both for predictable pipelines

You'll often use needs to build a sequence: build → test → deploy. On top of that, you add a concurrency group for the deployment environment, so only one deployment runs per environment at a time. Together, they give you both order and mutual exclusion.

Hands-on walkthrough

Let's start with a minimal workflow that builds, then runs tests, then deploys. You'll add outputs to share the build version, and a concurrency group to prevent deploy collisions.

Example 1: Build → Test → Deploy with needs

name: CI Pipeline

on:
  push:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    outputs:
      app_version: ${{ steps.version.outputs.app_version }}
    steps:
      - uses: actions/checkout@v4
      - name: Set version
        id: version
        run: echo "app_version=$(date +%s)" >> "$GITHUB_OUTPUT"
      - name: Build
        run: |
          echo "Building version ${{ steps.version.outputs.app_version }}"
          mkdir -p dist
          echo "app-${{ steps.version.outputs.app_version }}" > dist/version.txt

  test:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run tests
        run: echo "Running tests against app ${{ needs.build.outputs.app_version }}"

  deploy:
    needs: [build, test]
    runs-on: ubuntu-latest
    steps:
      - name: Deploy
        run: echo "Deploying app ${{ needs.build.outputs.app_version }}"

Expected behavior: The build job runs first. Once it succeeds, test runs. When both build and test succeed, deploy starts. If any parent fails, deploy is skipped.

Example 2: Add concurrency to prevent deploy collisions

name: Deploy to Production

on:
  push:
    branches: [main]

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: false

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - run: echo "Building..."
  deploy:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - run: echo "Deploying..."

Expected behavior: If two pushes hit main, the first run gets the deploy job started. The second run is queued because cancel-in-progress: false. It waits until the first completes. With cancel-in-progress: true, the first run would be cancelled the moment the second starts — useful for fast iteration on PRs.

The github.ref in the group key means each branch or tag gets its own queue. For production, you might use just production as the group to serialise all deploys regardless of branch.

Example 3: Keep PR previews lean with cancel-in-progress

name: PR Check

on:
  pull_request:

concurrency:
  group: pr-${{ github.event.pull_request.number }}
  cancel-in-progress: true

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: echo "Running linter..."
  test:
    needs: lint
    runs-on: ubuntu-latest
    steps:
      - run: echo "Running tests..."

Expected behavior: Each PR number gets its own concurrency group. When you push the second commit, the first run is cancelled — you don't waste minutes waiting for stale tests. Only the latest commit gets a full run.

Compare options / when to choose what

When to choose what: - needs is for order — always use when a job consumes the output of another. - concurrency with cancel-in-progress: true is for speed — use in PR checks where old runs are useless. - concurrency with false is for safety — use in production deployments where you want a queue, not a cancellation.

Consider alternatives: If you're using GitLab CI, you'd use stages (implicit ordering) and resource_group for concurrency. In Jenkins, you'd use build steps and locks. But if you're on GitHub, needs + concurrency is the idiomatic, fastest-to-implement solution.

Troubleshooting & edge cases

“Job is skipped — no matching version of node found” — This happens when a parent job fails due to a missing dependency or setup error. Check the parent job's logs first; skipping is deliberate, not a bug.

“No such property: needs” — You're trying to access needs.job_id.outputs in a job that doesn't declare needs: job_id. Always add the needs key to the dependent job, or use jobs.<job_id>.outputs with fromJSON.

eg., in a push to main, you might see:

Run echo "BUILD_ARTIFACT=${{ needs.build.outputs.artifact_path }}" >> "$GITHUB_ENV"
Error: Unrecognized named-value: 'needs'

Fix:

jobs:
  build:
    runs-on: ubuntu-latest
    outputs:
      artifact_path: ${{ steps.output_artifact.outputs.path }}
    steps:
      - id: output_artifact
        run: echo "path=dist/" >> "$GITHUB_OUTPUT"
  deploy:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - run: echo "Deploying ${{ needs.build.outputs.artifact_path }}"

Concurrency never allows more than one run per group — that's correct. If you want to allow two concurrent runs (like for a rolling deploy), you'll need max-parallel (GitLab) — GitHub Actions doesn't support >1 per group. Work around it with separate groups or a staggered schedule.

cancel-in-progress cancels the older run, not the new one. If you want to cancel the new one and keep the old, use cancel-in-progress: false — that queues the new run, but doesn't kill it; it waits. There's no built-in “don't start” — you'd add a first step that checks a lock file.

Edge case: Job failure stops the chain. If test fails, deploy won't run — that's the default. To run deploy even on failure, you'd need if: always() — but use that sparingly, as it can hide real issues.

What you learned & what's next

You now know how to add job dependencies and concurrency to your CI/CD pipelines:

• Use needs to sequence jobs — a job starts only after its dependencies complete successfully.
• Share data between jobs with outputs and the needs context.
• Use concurrency with a meaningful group key to either cancel or queue duplicate runs.
• Combine both to create a predictable, safe deployment pipeline — order and mutual exclusion.

You've taken a major step from “jobs that run in parallel” to “jobs that run in a controlled graph.” The next lesson in this track will show you how to add environment approvals and promotions, so you can gate a deploy before it touches production — a natural extension of the control you just gained.

Now put it into practice: modify a simple workflow you use today, add needs between its build and test jobs, and add a concurrency group for the deploy. Watch the workflow run and see the order enforced. That hands-on feel will make these concepts stick.