CI/CD with CodePipeline & CodeBuild
Build end-to-end CI/CD pipelines on AWS with CodePipeline, CodeBuild, and CodeDeploy, including blue/green and canary deployments.
Prerequisites
- Basic understanding of CI/CD concepts
- Familiarity with Git version control
- AWS account with IAM permissions for CodePipeline, CodeBuild, and CodeDeploy
- Understanding of AWS CloudFormation or CDK
CI/CD on AWS Overview
Continuous Integration and Continuous Delivery (CI/CD) is the practice of automating the build, test, and deployment of software. On AWS, the native CI/CD toolchain consists of four services: CodeCommit (Git repository hosting),CodeBuild (managed build service), CodeDeploy (automated deployment service), and CodePipeline (pipeline orchestration). Together, they provide a fully managed, end-to-end pipeline from code commit to production deployment.
While many teams use third-party tools like GitHub Actions, GitLab CI, or Jenkins, the AWS-native CI/CD services offer deep integration with AWS resources. Deploying to ECS, Lambda, EC2, or S3 is a first-class operation rather than a custom script. They also run entirely within your AWS account, which simplifies IAM permissions and eliminates the need to share AWS credentials with external services.
The choice between AWS-native CI/CD and third-party tools is not binary. Many production environments use a hybrid approach: GitHub for source control, GitHub Actions or CodeBuild for building and testing, and CodeDeploy for safe production deployments with blue/green and canary strategies.
CodeCommit Deprecation Notice
As of July 2024, AWS has stopped accepting new CodeCommit customers and recommends migrating existing repositories to other Git hosting providers (GitHub, GitLab, Bitbucket). CodeCommit will continue to work for existing customers, but AWS is not adding new features. This guide focuses on CodePipeline, CodeBuild, and CodeDeploy, which integrate with any Git provider. For source control, use GitHub, GitLab, or Bitbucket.
CodePipeline Architecture
AWS CodePipeline is a continuous delivery service that models and visualizes your software release process. A pipeline consists of stages, and each stage contains one or more actions. Actions run sequentially or in parallel within a stage, and stages run sequentially. The pipeline is triggered automatically when the source changes (e.g., a new commit is pushed to GitHub).
A typical pipeline has three to five stages: Source (pull code from Git),Build (compile, run unit tests, create artifacts), Test(integration tests, security scans), Staging (deploy to a staging environment), and Production (deploy to production, often with a manual approval gate).
Pipeline Components
| Component | Role | Examples |
|---|---|---|
| Source action | Retrieves source code | GitHub (v2), S3, ECR, CodeCommit |
| Build action | Compiles and creates artifacts | CodeBuild, Jenkins, custom action |
| Test action | Runs tests and quality checks | CodeBuild, third-party testing tools |
| Deploy action | Deploys artifacts to target | CodeDeploy, CloudFormation, ECS, S3, Lambda |
| Approval action | Manual gate requiring human approval | SNS notification with approve/reject |
| Invoke action | Run a Lambda function as a pipeline step | Custom validation, notifications, data migration |
# CloudFormation - Complete CI/CD pipeline
Resources:
Pipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
Name: my-app-pipeline
RoleArn: !GetAtt PipelineRole.Arn
ArtifactStore:
Type: S3
Location: !Ref ArtifactBucket
Stages:
# Stage 1: Source (GitHub)
- Name: Source
Actions:
- Name: GitHub
ActionTypeId:
Category: Source
Owner: AWS
Provider: CodeStarSourceConnection
Version: "1"
Configuration:
ConnectionArn: !Ref GitHubConnection
FullRepositoryId: "my-org/my-app"
BranchName: main
OutputArtifactFormat: CODE_ZIP
OutputArtifacts:
- Name: SourceOutput
# Stage 2: Build and Test
- Name: Build
Actions:
- Name: BuildAndTest
ActionTypeId:
Category: Build
Owner: AWS
Provider: CodeBuild
Version: "1"
Configuration:
ProjectName: !Ref BuildProject
InputArtifacts:
- Name: SourceOutput
OutputArtifacts:
- Name: BuildOutput
# Stage 3: Deploy to Staging
- Name: Staging
Actions:
- Name: DeployToStaging
ActionTypeId:
Category: Deploy
Owner: AWS
Provider: ECS
Version: "1"
Configuration:
ClusterName: !Ref StagingCluster
ServiceName: !Ref StagingService
FileName: imagedefinitions.json
InputArtifacts:
- Name: BuildOutput
# Stage 4: Manual Approval
- Name: Approval
Actions:
- Name: ManualApproval
ActionTypeId:
Category: Approval
Owner: AWS
Provider: Manual
Version: "1"
Configuration:
NotificationArn: !Ref ApprovalSNSTopic
CustomData: "Review staging deployment and approve for production"
# Stage 5: Deploy to Production
- Name: Production
Actions:
- Name: DeployToProduction
ActionTypeId:
Category: Deploy
Owner: AWS
Provider: CodeDeploy
Version: "1"
Configuration:
ApplicationName: !Ref CodeDeployApp
DeploymentGroupName: !Ref ProductionDeploymentGroup
InputArtifacts:
- Name: BuildOutputCodeBuild Configuration
AWS CodeBuild is a fully managed build service that compiles source code, runs tests, and produces deployable artifacts. CodeBuild scales automatically: it spins up a new build environment for each build, runs your build commands, and tears it down when complete. You never have to manage build servers, and you only pay for the compute time used during builds.
CodeBuild uses a buildspec.yml file in your repository root to define build commands. The buildspec is divided into phases: install (install dependencies), pre_build (login to registries, run pre-build checks),build (compile and test), and post_build (create artifacts, push images).
version: 0.2
env:
variables:
NODE_ENV: "production"
AWS_DEFAULT_REGION: "us-east-1"
parameter-store:
DATABASE_URL: "/myapp/prod/database-url"
secrets-manager:
GITHUB_TOKEN: "prod/github-token:token"
phases:
install:
runtime-versions:
nodejs: 20
commands:
- echo "Installing dependencies..."
- npm ci --production=false
pre_build:
commands:
- echo "Running linting and type checks..."
- npm run lint
- npm run type-check
- echo "Logging in to ECR..."
- aws ecr get-login-password --region $AWS_DEFAULT_REGION |
docker login --username AWS --password-stdin
$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com
build:
commands:
- echo "Running tests..."
- npm test -- --coverage --ci
- echo "Building application..."
- npm run build
- echo "Building Docker image..."
- docker build -t $ECR_REPO_URI:$CODEBUILD_RESOLVED_SOURCE_VERSION .
- docker tag $ECR_REPO_URI:$CODEBUILD_RESOLVED_SOURCE_VERSION $ECR_REPO_URI:latest
post_build:
commands:
- echo "Pushing Docker image to ECR..."
- docker push $ECR_REPO_URI:$CODEBUILD_RESOLVED_SOURCE_VERSION
- docker push $ECR_REPO_URI:latest
- echo "Generating deployment artifacts..."
- printf '[{"name":"app","imageUri":"%s"}]'
$ECR_REPO_URI:$CODEBUILD_RESOLVED_SOURCE_VERSION > imagedefinitions.json
artifacts:
files:
- imagedefinitions.json
- appspec.yml
- scripts/**/*
discard-paths: no
reports:
jest-reports:
files:
- "coverage/clover.xml"
file-format: CLOVERXML
cache:
paths:
- "node_modules/**/*"
- "/root/.npm/**/*"CodeBuild Compute Types
| Compute Type | vCPU | Memory | Cost (per minute) | Best For |
|---|---|---|---|---|
| BUILD_GENERAL1_SMALL | 2 | 3 GB | $0.005 | Simple builds, linting |
| BUILD_GENERAL1_MEDIUM | 4 | 7 GB | $0.010 | Standard application builds |
| BUILD_GENERAL1_LARGE | 8 | 15 GB | $0.020 | Large projects, Docker builds |
| BUILD_GENERAL1_2XLARGE | 72 | 145 GB | $0.200 | Massive builds, parallel testing |
| BUILD_LAMBDA_1GB | 2 | 1 GB | $0.00375 | Quick builds (Lambda-based compute) |
CodeBuild Lambda Compute
CodeBuild supports Lambda-based compute environments that start in under 2 seconds (compared to 30-60 seconds for container-based environments). Lambda compute is ideal for fast builds like linting, running unit tests, or deploying Lambda functions. It does not support Docker builds or custom Docker images, so use container-based compute for builds that need Docker-in-Docker or custom build tools.
CodeDeploy Strategies
AWS CodeDeploy automates application deployments to EC2 instances, on-premises servers, Lambda functions, and ECS services. Its key value is providing safe deployment strategies (blue/green, canary, and rolling deployments) with automatic rollback if health checks fail. CodeDeploy uses an appspec.yml file to define deployment lifecycle hooks.
Deployment Strategies Comparison
| Strategy | How It Works | Risk | Rollback Speed |
|---|---|---|---|
| In-place (rolling) | Update instances one at a time | Moderate (partial exposure) | Slow (must redeploy old version) |
| Blue/Green (EC2) | Create new instances, swap traffic | Low (old instances preserved) | Fast (swap back to original instances) |
| Blue/Green (ECS) | Create new task set, shift traffic | Low | Fast (route traffic back to original task set) |
| Canary (Lambda) | Shift X% of traffic, wait, then shift 100% | Very low (limited exposure) | Instant (shift traffic back to original) |
| Linear (Lambda) | Shift X% every N minutes until 100% | Very low (gradual exposure) | Instant |
# appspec.yml for EC2/on-premises deployment
version: 0.0
os: linux
files:
- source: /
destination: /opt/myapp
overwrite: true
permissions:
- object: /opt/myapp
owner: ec2-user
group: ec2-user
mode: "755"
hooks:
BeforeInstall:
- location: scripts/before-install.sh
timeout: 300
runas: root
AfterInstall:
- location: scripts/after-install.sh
timeout: 300
runas: root
ApplicationStart:
- location: scripts/start-app.sh
timeout: 300
runas: ec2-user
ValidateService:
- location: scripts/validate.sh
timeout: 300
runas: ec2-user# appspec.yml for ECS blue/green deployment
version: 0.0
Resources:
- TargetService:
Type: AWS::ECS::Service
Properties:
TaskDefinition: "arn:aws:ecs:us-east-1:123456789012:task-definition/my-app:42"
LoadBalancerInfo:
ContainerName: "app"
ContainerPort: 8080
PlatformVersion: "LATEST"
Hooks:
- BeforeInstall: "LambdaFunctionToValidateBeforeInstall"
- AfterInstall: "LambdaFunctionToValidateAfterInstall"
- AfterAllowTestTraffic: "LambdaFunctionToRunIntegrationTests"
- BeforeAllowTraffic: "LambdaFunctionToRunSmokeTests"
- AfterAllowTraffic: "LambdaFunctionToValidateProductionTraffic"Building Your First Pipeline
Let us walk through creating a complete CI/CD pipeline for a Node.js application deployed to ECS. This pipeline pulls from GitHub, builds a Docker image, pushes it to ECR, deploys to a staging ECS service, waits for manual approval, and then deploys to production using CodeDeploy blue/green deployment.
Resources:
# GitHub connection (created once via console or CLI)
GitHubConnection:
Type: AWS::CodeStarConnections::Connection
Properties:
ConnectionName: github-connection
ProviderType: GitHub
# ECR repository for Docker images
AppECRRepo:
Type: AWS::ECR::Repository
Properties:
RepositoryName: my-app
ImageScanningConfiguration:
ScanOnPush: true
LifecyclePolicy:
LifecyclePolicyText: |
{
"rules": [
{
"rulePriority": 1,
"description": "Keep last 10 images",
"selection": {
"tagStatus": "any",
"countType": "imageCountMoreThan",
"countNumber": 10
},
"action": { "type": "expire" }
}
]
}
# CodeBuild project
BuildProject:
Type: AWS::CodeBuild::Project
Properties:
Name: my-app-build
ServiceRole: !GetAtt BuildRole.Arn
Source:
Type: CODEPIPELINE
BuildSpec: buildspec.yml
Environment:
Type: LINUX_CONTAINER
ComputeType: BUILD_GENERAL1_MEDIUM
Image: aws/codebuild/amazonlinux2-x86_64-standard:5.0
PrivilegedMode: true # Required for Docker builds
EnvironmentVariables:
- Name: ECR_REPO_URI
Value: !Sub "${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/my-app"
- Name: AWS_ACCOUNT_ID
Value: !Ref AWS::AccountId
Artifacts:
Type: CODEPIPELINE
Cache:
Type: S3
Location: !Sub "${ArtifactBucket}/build-cache"
LogsConfig:
CloudWatchLogs:
Status: ENABLED
GroupName: /codebuild/my-app
StreamName: buildGitHub Connection Requires Console Activation
The AWS::CodeStarConnections::Connection resource creates the connection in a "PENDING" state. You must go to the CodePipeline console, find the connection under Settings > Connections, and complete the OAuth handshake with GitHub. This is a one-time manual step per GitHub connection. After activation, CodePipeline can access your repositories without GitHub personal access tokens.
Blue/Green & Canary Deployments
Safe deployment strategies are essential for production applications. Instead of updating all instances at once (big-bang deployment), modern strategies gradually shift traffic to the new version while monitoring health metrics. If something goes wrong, traffic is automatically shifted back to the old version.
ECS Blue/Green with CodeDeploy
For ECS services, CodeDeploy manages blue/green deployments by creating a new task set (green) alongside the existing one (blue). Traffic is shifted from blue to green using the Application Load Balancer's target groups. You can configure test traffic first (on a separate port) to validate the new version before shifting production traffic.
Resources:
# CodeDeploy application
CodeDeployApp:
Type: AWS::CodeDeploy::Application
Properties:
ApplicationName: my-app
ComputePlatform: ECS
# Deployment group with blue/green configuration
ProductionDeploymentGroup:
Type: AWS::CodeDeploy::DeploymentGroup
Properties:
ApplicationName: !Ref CodeDeployApp
DeploymentGroupName: production
ServiceRoleArn: !GetAtt CodeDeployRole.Arn
DeploymentConfigName: CodeDeployDefault.ECSCanary10Percent5Minutes
DeploymentStyle:
DeploymentType: BLUE_GREEN
DeploymentOption: WITH_TRAFFIC_CONTROL
BlueGreenDeploymentConfiguration:
TerminateBlueInstancesOnDeploymentSuccess:
Action: TERMINATE
TerminationWaitTimeInMinutes: 60 # Keep blue for 1 hour (rollback window)
DeploymentReadyOption:
ActionOnTimeout: CONTINUE_DEPLOYMENT
WaitTimeInMinutes: 0
AutoRollbackConfiguration:
Enabled: true
Events:
- DEPLOYMENT_FAILURE
- DEPLOYMENT_STOP_ON_ALARM
AlarmConfiguration:
Enabled: true
Alarms:
- Name: !Ref HighErrorRateAlarm
- Name: !Ref HighLatencyAlarm
ECSServices:
- ClusterName: !Ref ProductionCluster
ServiceName: !Ref ProductionService
LoadBalancerInfo:
TargetGroupPairInfoList:
- TargetGroups:
- Name: !GetAtt BlueTargetGroup.TargetGroupName
- Name: !GetAtt GreenTargetGroup.TargetGroupName
ProdTrafficRoute:
ListenerArns:
- !Ref ProductionListener
TestTrafficRoute:
ListenerArns:
- !Ref TestListenerLambda Deployment Preferences
For Lambda functions deployed through SAM or CloudFormation, you can define deployment preferences that control how traffic shifts between the old and new function versions. AWS provides built-in deployment configurations:
| Configuration | Behavior |
|---|---|
Canary10Percent5Minutes | 10% of traffic for 5 minutes, then 100% |
Canary10Percent30Minutes | 10% of traffic for 30 minutes, then 100% |
Linear10PercentEvery1Minute | 10% more traffic every minute over 10 minutes |
Linear10PercentEvery10Minutes | 10% more traffic every 10 minutes over 100 minutes |
AllAtOnce | Shift all traffic immediately |
Pipeline Integrations (GitHub Actions etc)
While CodePipeline is a capable orchestrator, many teams prefer to use GitHub Actions or GitLab CI for CI and only use AWS services for deployment. This hybrid approach leverages the developer experience of modern CI platforms while using CodeDeploy's deployment safety features.
name: Build and Deploy
on:
push:
branches: [main]
permissions:
id-token: write # Required for OIDC
contents: read
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Configure AWS Credentials (OIDC)
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::123456789012:role/GitHubActionsRole
aws-region: us-east-1
- name: Login to ECR
id: ecr-login
uses: aws-actions/amazon-ecr-login@v2
- name: Build and Push Docker Image
env:
ECR_REGISTRY: ${{ steps.ecr-login.outputs.registry }}
IMAGE_TAG: ${{ github.sha }}
run: |
docker build -t $ECR_REGISTRY/my-app:$IMAGE_TAG .
docker push $ECR_REGISTRY/my-app:$IMAGE_TAG
- name: Deploy to ECS
uses: aws-actions/amazon-ecs-deploy-task-definition@v2
with:
task-definition: task-definition.json
service: my-app-service
cluster: production
wait-for-service-stability: trueUse OIDC for GitHub Actions Authentication
Instead of storing long-lived AWS access keys in GitHub secrets, use OpenID Connect (OIDC) federation. This creates a trust relationship between GitHub and your AWS account, allowing GitHub Actions to assume an IAM role with temporary credentials. No access keys to rotate, no secrets to manage. Configure an IAM OIDC identity provider fortoken.actions.githubusercontent.com and create a role with a trust policy that restricts access to your specific repository and branch.
Infrastructure as Code Pipelines
CI/CD is not just for application code. Your infrastructure definitions (CloudFormation templates, CDK apps, Terraform configurations) should also be deployed through automated pipelines. This ensures infrastructure changes are reviewed, tested, and deployed consistently.
version: 0.2
phases:
install:
commands:
- npm install -g aws-cdk@latest
- npm ci
pre_build:
commands:
- echo "Synthesizing CDK app..."
- npx cdk synth
- echo "Running CDK diff against current deployment..."
- npx cdk diff 2>&1 | tee cdk-diff-output.txt
- echo "Running cfn-lint on generated templates..."
- pip install cfn-lint
- cfn-lint cdk.out/*.template.json
build:
commands:
- echo "Deploying CDK stack..."
- npx cdk deploy --all --require-approval never --ci
- echo "Running post-deployment validation..."
- node scripts/validate-deployment.js
artifacts:
files:
- cdk-diff-output.txt
- cdk.out/*.template.json
reports:
cfn-lint-report:
files:
- cfn-lint-results.json
file-format: GENERICXMLCDK Pipelines
AWS CDK Pipelines is a higher-level construct that creates self-mutating CI/CD pipelines. When you push a change to the CDK pipeline definition itself, the pipeline updates itself before deploying your infrastructure. This means the pipeline and the infrastructure it deploys are defined in the same codebase and deployed through the same process.
Testing & Quality Gates
A CI/CD pipeline is only as reliable as its testing stages. Quality gates are checkpoints in the pipeline that prevent bad code from reaching production. They include unit tests, integration tests, security scans, code coverage thresholds, and performance benchmarks.
Types of Tests in the Pipeline
| Test Type | Pipeline Stage | Tool | Duration |
|---|---|---|---|
| Linting & type checking | Build (pre_build) | ESLint, TypeScript, Pylint | 10-30 seconds |
| Unit tests | Build | Jest, pytest, JUnit | 30-120 seconds |
| Security scanning | Build (post_build) | Snyk, Trivy, CodeGuru | 30-120 seconds |
| Integration tests | Test (after staging deploy) | Custom test suite | 2-10 minutes |
| End-to-end tests | Test (after staging deploy) | Playwright, Cypress, Selenium | 5-30 minutes |
| Load tests | Test (optional) | k6, Artillery, Locust | 10-60 minutes |
| Smoke tests | Production (after deploy) | Custom health checks | 30-60 seconds |
Do Not Skip Tests to Speed Up Deploys
It is tempting to skip slow tests (integration, E2E) to reduce pipeline duration. Instead, run fast tests (lint, unit) as a gate and run slower tests in parallel. CodeBuild supports batch builds that run multiple buildspec files concurrently. Use this to run unit tests, security scans, and integration tests in parallel rather than sequentially. A 10-minute pipeline with comprehensive tests is better than a 2-minute pipeline that lets bugs through.
Best Practices & Troubleshooting
Effective CI/CD is about more than just automating deployments. It requires discipline around branching strategies, artifact management, secret handling, and incident response.
Pipeline Best Practices
Immutable artifacts: Build once, deploy everywhere. The same Docker image or zip file that was tested in staging should be deployed to production. Never rebuild for production. Environment-specific configuration should come from environment variables or parameter store, not from the build process.
Fast feedback: Optimize for pipeline speed. Use build caching (npm cache, Docker layer cache), parallel test execution, and Lambda-based CodeBuild compute for fast steps. Developers should get feedback on their changes within 5-10 minutes.
Secrets management: Never store secrets in source code, buildspec files, or environment variables visible in the console. Use AWS Systems Manager Parameter Store (for non-sensitive config) or Secrets Manager (for passwords, API keys, database credentials). CodeBuild natively integrates with both.
Pipeline as code: Define your pipeline in CloudFormation, CDK, or Terraform, not through the console. This makes pipelines reproducible, version- controlled, and reviewable through the same code review process as application code.
Rollback strategy: Always have a rollback plan. CodeDeploy supports automatic rollback on deployment failure or CloudWatch alarm. For CloudFormation deployments, enable automatic rollback on stack update failure. Test your rollback process regularly because an untested rollback is not a rollback.
Common Troubleshooting Issues
Pipeline stuck in InProgress: Check the CodeBuild logs for the currently running action. Common causes include tests that hang, Docker builds that time out, or missing IAM permissions.
CodeBuild IAM errors: CodeBuild runs with the IAM role you specify. If the build fails with "Access Denied," the role is missing permissions for the AWS services your build accesses (ECR, S3, Secrets Manager, etc.). Check the build logs for the specific API call that failed and add the permission to the build role.
Artifact size limits: CodePipeline artifacts stored in S3 have a 5 GB limit. If your build produces large artifacts, consider storing them directly in S3 or ECR and passing only a reference (like an image URI) through the pipeline.
Pipeline Notifications
Set up pipeline notifications to stay informed about build and deployment status. CodePipeline integrates with Amazon SNS and AWS Chatbot for Slack notifications. At minimum, notify on pipeline failures and successful production deployments. UseAWS::CodeStarNotifications::NotificationRule in CloudFormation to define notification rules for pipeline events.
Key Takeaways
- 1CodePipeline orchestrates the CI/CD workflow with source, build, test, and deploy stages.
- 2CodeBuild provides managed build environments with support for custom Docker images.
- 3CodeDeploy supports in-place, blue/green, and canary deployment strategies.
- 4Blue/green deployments eliminate downtime by switching traffic between two environments.
- 5GitHub Actions can integrate with AWS CI/CD services via OIDC for keyless authentication.
- 6Infrastructure as Code pipelines enable safe, automated infrastructure changes.
Frequently Asked Questions
What is the difference between CodePipeline and CodeBuild?
Can I use GitHub instead of CodeCommit?
How do blue/green deployments work on AWS?
How much does CodePipeline cost?
Can I integrate third-party tools with CodePipeline?
Written by CloudToolStack Team
Cloud engineers and architects with hands-on experience across AWS, Azure, and GCP. We write guides based on real-world production patterns, not just documentation rewrites.
Disclaimer: This guide is for educational purposes. Cloud services change frequently; always refer to official documentation for the latest information. AWS, Azure, and GCP are trademarks of their respective owners.