Integrate GitHub with CodePipeline

You've got your Python application running beautifully on a dev machine, but the moment a teammate pushes a commit, the build breaks in production and nobody noticed until a customer did. Manually triggering builds, SSH-ing into servers, and chasing deployment artifacts is a recipe for burnout. That's the pain: software delivery is only as fast and reliable as your automation. In this lesson, you'll solve that by integrating GitHub with AWS CodePipeline, creating a fully automated CI/CD pipeline that builds, tests, and deploys your Python app every time you push code. By the end, you'll not only have a working pipeline but also the mental model to extend it to staging, production, and beyond.

The problem this lesson solves

Manually deploying your application is slow, error-prone, and leaves your team in the dark. Every push to GitHub should trigger a predictable, repeatable pipeline that runs your tests, builds an artifact, and deploys it — without human intervention. Without this integration, you face:

• No audit trail — Who deployed what, and when? Good luck answering that.
• Environment drift — Production and staging slowly become different beasts, and 'it works on my machine' becomes a real excuse.
• Delayed feedback — A bug introduced at 9 AM isn't caught until 5 PM, when users report it.

AWS CodePipeline is a fully managed continuous delivery service that orchestrates the build, test, and deploy phases. When you connect it to your GitHub repository, every code change automatically moves through your pipeline, giving you instant feedback and a deployable release at all times.

Core concept / mental model

Think of CodePipeline as a conveyor belt for your software. Each stage is a station on that belt:

• Source — Your GitHub repository is the raw material. Any code change (push, pull request merge) triggers the belt.
• Build — AWS CodeBuild (or your chosen build provider) compiles, runs tests, and packages your Python app into an artifact (e.g., a zip file or a Docker image).
• Deploy — AWS CodeDeploy (or Elastic Beanstalk, ECS, EKS) takes that artifact and installs it on your target environment (EC2, Lambda, etc.).

Key terms you'll see:

• Pipeline — the end-to-end automated workflow.
• Stage — a logical phase (Source, Build, Deploy).
• Action — a specific task within a stage (e.g., 'GitHub Source' or 'Run tests').
• Artifact — the output of a build stage (e.g., a deployment package).
• Webhook — the mechanism GitHub uses to notify CodePipeline of changes.

The magic is that CodePipeline uses AWS CloudFormation-style declarative configuration (or the AWS Management Console) to wire these actions together, and the whole thing is event-driven: no cron jobs, no manual triggers.

How it works step by step

Let's trace a single push to GitHub through the integrated pipeline:

1. Code push — A developer runs git push origin main.
2. GitHub webhook — GitHub sends an HTTP POST to an AWS API Gateway endpoint (or directly to CodePipeline if you use the CodeStar-style connection) with the commit metadata.
3. Pipeline trigger — CodePipeline receives the event and starts a new pipeline execution.
4. Source stage — The source action pulls the latest commit from your GitHub repository (usually a zipped archive) and stores it in an artifact bucket (S3).
5. Build stage — CodeBuild spins up a container, runs pip install, executes your tests (e.g., pytest), and packages the app. If any step fails, the pipeline stops and sends you a notification (e.g., SNS email).
6. Deploy stage — CodeDeploy (or your chosen compute) pulls the artifact (e.g., from S3) and deploys it to your Ec2 instance or Lambda function, following your appspec.yml instructions.
7. Success — The pipeline emits a success event, and you (or your monitoring system) receives a confirmation.

This entire flow is idempotent — running it twice on the same commit yields the same result — which is exactly what you want for reliable deployments.

Pro tip: Use AWS CodePipeline with GitHub Actions? Don't confuse the two. CodePipeline is AWS-native orchestration; GitHub Actions lives inside GitHub. You can even use a GitHub Action as a deployment target, but this lesson focuses on CodePipeline as the orchestrator.

Hands-on walkthrough

Let's build a real pipeline for a simple Python Flask app. We'll use the AWS Management Console, but you can also do this via the AWS CLI or CloudFormation (I'll show you that at the end).

Prerequisites

• An AWS account (you can use the Free Tier).
• A GitHub repository with a Python app (e.g., my-python-app). The repo should have a buildspec.yml (for CodeBuild) and an appspec.yml (for CodeDeploy) if you're targeting EC2. We'll create them.

1. Create your pipeline (console)

1. Open the CodePipeline console.
2. Click Create pipeline.
3. Pipeline name: my-python-app-pipeline
4. Service role: Choose New service role (or use an existing one).
5. Artifact store: Default S3 bucket works.
6. Source provider: Select GitHub (Version 2).
7. Click Connect to GitHub and authenticate. Choose your repository and branch (e.g., main).
8. Change detection: Select Start pipeline on push (uses webhooks).
9. Build provider: AWS CodeBuild. Click Create project and configure: - Environment: Managed image, python:3.10. - Buildspec: Use the buildspec.yml from your repo.
10. Deploy provider: AWS CodeDeploy. Choose or create a deployment group. For simplicity, you can skip deploy and just have a build stage; pipeline will end after build.
11. Click Create pipeline.

2. The buildspec.yml

Create this file in your repo root:

version: 0.2

phases:
  install:
    runtime-versions:
      python: 3.10
    commands:
      - pip install -r requirements.txt
  build:
    commands:
      - python -m pytest tests/
  post_build:
    commands:
      - echo "Build succeeded"

artifacts:
  files:
    - app.py
    - requirements.txt
    - templates/**/*

This will run your tests and package the app for deployment.

3. Trigger the pipeline

Push a simple change to your GitHub repo (e.g., update a README). The webhook will fire, and you should see a new execution in the pipeline's history.

Check the pipeline status:

aws codepipeline get-pipeline-state --name my-python-app-pipeline

Expected output (abridged):

{
  "pipelineName": "my-python-app-pipeline",
  "stageStates": [
    {
      "stageName": "Source",
      "latestExecution": {
        "status": "Succeeded"
      }
    },
    {
      "stageName": "Build",
      "latestExecution": {
        "status": "Succeeded"
      }
    }
  ]
}

4. Automate with Python (Boto3)

The whole thing can be scripted using Boto3. Here's a snippet to create a pipeline programmatically:

import boto3

client = boto3.client('codepipeline', region_name='us-east-1')

response = client.create_pipeline(
    pipeline={
        'name': 'my-python-app-pipeline',
        'roleArn': 'arn:aws:iam::123456789012:role/CodePipelineServiceRole',
        'artifactStore': {
            'type': 'S3',
            'location': 'codepipeline-us-east-1-123456789012'
        },
        'stages': [
            {
                'name': 'Source',
                'actions': [
                    {
                        'name': 'Source',
                        'actionTypeId': {
                            'category': 'Source',
                            'owner': 'AWS',
                            'provider': 'CodeStarSourceConnection',
                            'version': '1'
                        },
                        'configuration': {
                            'ConnectionArn': 'arn:aws:codestar-connections:us-east-1:123456789012:connection/your-connection-id',
                            'FullRepositoryId': 'your-org/my-python-app',
                            'BranchName': 'main'
                        },
                        'outputArtifacts': [{'name': 'SourceArtifact'}],
                        'runOrder': 1
                    }
                ]
            },
            {
                'name': 'Build',
                'actions': [
                    {
                        'name': 'Build',
                        'actionTypeId': {
                            'category': 'Build',
                            'owner': 'AWS',
                            'provider': 'CodeBuild',
                            'version': '1'
                        },
                        'configuration': {
                            'ProjectName': 'my-python-app-build'
                        },
                        'inputArtifacts': [{'name': 'SourceArtifact'}],
                        'outputArtifacts': [{'name': 'BuildArtifact'}],
                        'runOrder': 1
                    }
                ]
            }
        ]
    }
)
print(response['pipeline']['name'])

This script requires you to have created the CodeStar connection (via console or CLI) and the IAM role. You can create the connection with:

aws codestar-connections create-connection --provider-type GitHub --connection-name my-github-connection

Then complete the handshake in the console (it’s a one-time OAuth flow). Note the connection ARN to use in the script.

Compare options / when to choose what

You have several ways to connect GitHub to your delivery pipeline. Here’s a table to help you decide:

When to choose what?

• Use CodeStar connection if you're going all-in on AWS and want a fully managed, secure integration. That's our recommendation.
• Use GitHub Actions if your team is GitHub-centric and you want minimal vendor lock-in beyond GitHub — you can still deploy to AWS using the aws-actions/configure-aws-credentials action.

For most learners, CodeStar connections is the right call: it’s the current AWS best practice.

Troubleshooting & edge cases

Pipeline doesn't trigger on push

• Check that your CodeStar connection is in AVAILABLE status (not PENDING). If it’s pending, go to codestar-connections in the console, click the connection, and complete the authorization.
• Verify the webhook in GitHub: the App or webhook should be installed on the repo.

Build fails with 'ModuleNotFoundError'

• Your requirements.txt might not include a package. Add it, commit, and let the pipeline rebuild.
• In CodeBuild, ensure the install phase runs pip install -r requirements.txt before the build phase.

Permission errors during build or deploy

• The CodeBuild service role must have permissions to read from S3 (for artifacts) and write to CloudWatch Logs. Attach AmazonS3ReadOnlyAccess and CloudWatchLogsFullAccess (or better, scoped down).
• The CodeDeploy service role needs permissions to codedeploy:CreateDeployment and ec2:DescribeInstances.

Pipeline hangs at 'Source' stage

• This often means the artifact bucket is wrong or the source artifact isn't being generated. Verify your artifact storage bucket is in the same region and exists.

GitHub push doesn't match the branch

• Make sure the branch in the pipeline configuration is the actual default branch (e.g., main not master).

What you learned & what's next

You just connected GitHub to AWS CodePipeline, turning every push into an automated build and test cycle. You learned the mental model of stages and actions, built a buildspec.yml, and even scripted the pipeline with Boto3. You now have a repeatable, auditable delivery process — a core DevOps superpower.

Next in the track: Extending this pipeline to production environments with approval gates and multi-environment deployment (e.g., staging → production). You'll also learn to add CloudWatch alarms to monitor deployment health. That will make your pipeline truly production-grade.