Skip to content

CI/CD

Automate building, testing, and deploying your application.

Philosophy

Continuous Integration

Every code change should:

  1. Build — Verify code compiles/bundles
  2. Test — Run automated tests
  3. Lint — Check code quality
  4. Scan — Check for vulnerabilities

Continuous Deployment

After CI passes:

  1. Stage — Deploy to staging environment
  2. Verify — Run smoke tests
  3. Approve — Manual or automatic gate
  4. Deploy — Release to production

Pipeline Overview

┌─────────────────────────────────────────────────────────────────┐
│                        CI Pipeline                               │
├─────────┬─────────┬─────────┬─────────┬─────────┬──────────────┤
│  Lint   │  Test   │  Build  │  Scan   │  Push   │   Deploy     │
│         │         │         │         │         │              │
│ ruff    │ pytest  │ docker  │ trivy   │ ECR/GCR │ staging →    │
│ eslint  │ vitest  │ build   │ snyk    │         │ production   │
│ mypy    │         │         │         │         │              │
└─────────┴─────────┴─────────┴─────────┴─────────┴──────────────┘

Section Contents

Topic Description
GitHub Actions Workflow syntax and patterns
Testing Pipeline Test parallelization and strategies
Build Pipeline Docker builds and optimization
Deployment Deployment strategies
Environments Environment management
Deploy Monitoring Observability during deploys
Troubleshooting Common issues and fixes

Quick Start

Basic GitHub Actions Workflow

# .github/workflows/ci.yml
name: CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Lint
        run: |
          pip install ruff mypy
          ruff check .
          mypy src/

  test:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: postgres
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
        ports:
          - 5432:5432

    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: "pip"

      - name: Install dependencies
        run: pip install -e ".[test]"

      - name: Run tests
        run: pytest --cov=src --cov-report=xml
        env:
          DATABASE_URL: postgresql://postgres:postgres@localhost:5432/test

      - name: Upload coverage
        uses: codecov/codecov-action@v4

  build:
    runs-on: ubuntu-latest
    needs: [lint, test]
    steps:
      - uses: actions/checkout@v4

      - name: Build Docker image
        run: docker build -t myapp:${{ github.sha }} .

      - name: Push to registry
        if: github.ref == 'refs/heads/main'
        run: |
          echo ${{ secrets.DOCKER_PASSWORD }} | docker login -u ${{ secrets.DOCKER_USERNAME }} --password-stdin
          docker push myapp:${{ github.sha }}

Key Concepts

Jobs vs Steps

jobs:
  # Jobs run in parallel by default
  job1:
    runs-on: ubuntu-latest
    steps:
      # Steps run sequentially within a job
      - name: Step 1
        run: echo "First"
      - name: Step 2
        run: echo "Second"

  job2:
    needs: job1  # Runs after job1
    runs-on: ubuntu-latest
    steps:
      - run: echo "After job1"

Caching

- name: Cache pip packages
  uses: actions/cache@v4
  with:
    path: ~/.cache/pip
    key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
    restore-keys: |
      ${{ runner.os }}-pip-

Secrets

env:
  DATABASE_URL: ${{ secrets.DATABASE_URL }}
  API_KEY: ${{ secrets.API_KEY }}

Artifacts

- name: Upload test results
  uses: actions/upload-artifact@v4
  with:
    name: test-results
    path: test-results/

Pipeline Stages

1. Validation

Fast checks that fail early:

validate:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4

    - name: Check formatting
      run: ruff format --check .

    - name: Lint
      run: ruff check .

    - name: Type check
      run: mypy src/

2. Test

Comprehensive testing:

test:
  runs-on: ubuntu-latest
  strategy:
    matrix:
      python-version: ["3.11", "3.12"]
  steps:
    - name: Run tests
      run: pytest

3. Build

Create deployable artifacts:

build:
  runs-on: ubuntu-latest
  needs: [validate, test]
  steps:
    - name: Build Docker image
      run: docker build -t app:${{ github.sha }} .

4. Deploy

Release to environments:

deploy-staging:
  needs: build
  environment: staging
  steps:
    - name: Deploy to staging
      run: ./deploy.sh staging

deploy-production:
  needs: deploy-staging
  environment: production
  steps:
    - name: Deploy to production
      run: ./deploy.sh production

Best Practices

Practice Benefit
Fail fast Run quick checks first
Parallelize Speed up pipeline
Cache dependencies Reduce build time
Use matrix builds Test multiple versions
Pin action versions Reproducible builds
Use environments Control deployments
Store secrets securely Never in code
Monitor pipelines Alert on failures

Common Patterns

Monorepo with Path Filters

on:
  push:
    paths:
      - 'backend/**'
      - '.github/workflows/backend.yml'

Skip CI

git commit -m "docs: update readme [skip ci]"

Manual Approval

deploy-production:
  environment:
    name: production
    url: https://myapp.com
  # Requires approval in GitHub settings