CI/CD Pipelines

A practitioner's guide to continuous integration and continuous delivery for production systems. This skill covers pipeline design, GitHub Actions workflows, deployment strategies, and the operational patterns that keep software shipping safely at speed. The emphasis is on when to apply each pattern and why it matters, not just the YAML syntax.

CI/CD is not a tool configuration problem - it is a software delivery discipline. The pipeline is the product team's contract with production: every commit that passes is a candidate release, and the pipeline enforces that contract automatically.

When to use this skill

Trigger this skill when the user:

• Creates or modifies a GitHub Actions, GitLab CI, or Jenkins pipeline
• Implements PR checks, branch protection rules, or required status checks
• Sets up deployment environments (staging, production) with promotion gates
• Implements blue-green, canary, rolling, or recreate deployment strategies
• Configures caching for dependencies or build artifacts to speed up pipelines
• Sets up matrix builds to test across multiple Node versions or operating systems
• Automates secrets injection, environment promotion, or rollback procedures
• Diagnoses a slow pipeline and needs to find what to parallelize or cache

Do NOT trigger this skill for:

• Infrastructure provisioning from scratch (use a Terraform/Kubernetes skill instead)
• Application-level testing strategies unrelated to pipeline structure

Key principles

1. Fail fast - The pipeline should surface errors as early as possible. Run linting and type-checking before tests. Run unit tests before integration tests. A 30-second lint failure beats a 10-minute test run that tells you the same thing.

2. Cache aggressively - node_modules, Maven .m2, pip wheels, and Docker layer caches can turn a 12-minute pipeline into a 3-minute one. Cache by the lockfile hash so the cache busts exactly when dependencies change.

3. Keep pipelines under 10 minutes - Pipelines longer than 10 minutes cause developers to stop watching them, batch commits to avoid waiting, and skip running them locally. Parallelize jobs, split slow test suites, and move heavy analysis to scheduled runs.

4. Trunk-based development - Short-lived branches merged frequently (at least daily) are the prerequisite for effective CI. Long-lived branches turn CI into a lie - the code integrates in CI but not in reality.

5. Immutable artifacts - Build once, deploy everywhere. The same Docker image or archive that passed staging must be the thing that goes to production. Never rebuild from source at deploy time.

Core concepts

Pipeline stages run in order and each must pass before the next begins:

build -> test -> deploy:staging -> approve -> deploy:production

Triggers determine when a pipeline runs:

• push on any branch - run build and test
• pull_request - run full check suite for the PR
• schedule (cron) - run security scans or long test suites nightly
• workflow_dispatch - manual trigger with optional inputs for on-demand deploys

Environments are named targets (staging, production) with their own secrets, protection rules, and deployment history. GitHub Environments let you require manual approvals before promoting to production.

Secrets management - secrets live in GitHub Secrets or an external vault (Vault, AWS Secrets Manager). They are injected as environment variables at runtime. Never print them in logs. Rotate them on a schedule.

Artifact storage - build outputs (compiled code, Docker images, test reports) are stored in GitHub Artifacts or a registry (GHCR, ECR, Docker Hub). Artifacts have a retention window; images are tagged with the commit SHA.

Common tasks

Set up GitHub Actions for Node.js

A standard Node.js pipeline with lint, test, and build, using dependency caching:

# .github/workflows/ci.yml
name: CI

on:
  push:
    branches: [main]
  pull_request:

jobs:
  ci:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm            # caches ~/.npm by package-lock.json hash

      - run: npm ci             # clean install from lockfile

      - run: npm run lint

      - run: npm test -- --coverage

      - run: npm run build

      - uses: actions/upload-artifact@v4
        with:
          name: dist
          path: dist/
          retention-days: 7

Use npm ci instead of npm install in CI. It is faster, deterministic, and will fail if package-lock.json is out of sync with package.json.

Implement PR checks

Require the CI workflow to pass before merging. Configure in GitHub Settings > Branches > Branch protection rules:

• Enable "Require status checks to pass before merging"
• Add the job name (ci) as a required check
• Enable "Require branches to be up to date before merging"

# .github/workflows/pr-check.yml
name: PR Check

on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci
      - run: npm run lint

  test:
    runs-on: ubuntu-latest
    needs: lint           # only run tests if lint passes
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci
      - run: npm test

  typecheck:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci
      - run: npm run typecheck

Set up deployment environments with approvals

Use GitHub Environments to gate production deploys behind a manual approval:

# .github/workflows/deploy.yml
name: Deploy

on:
  push:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    outputs:
      image-tag: ${{ steps.tag.outputs.tag }}
    steps:
      - uses: actions/checkout@v4
      - id: tag
        run: echo "tag=${{ github.sha }}" >> $GITHUB_OUTPUT
      - run: docker build -t myapp:${{ github.sha }} .
      - run: docker push ghcr.io/org/myapp:${{ github.sha }}
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

  deploy-staging:
    needs: build
    runs-on: ubuntu-latest
    environment: staging        # uses staging secrets + URL
    steps:
      - run: ./scripts/deploy.sh
        env:
          IMAGE_TAG: ${{ needs.build.outputs.image-tag }}
          DEPLOY_URL: ${{ vars.DEPLOY_URL }}
          API_KEY: ${{ secrets.DEPLOY_API_KEY }}

  deploy-production:
    needs: deploy-staging
    runs-on: ubuntu-latest
    environment: production     # requires manual approval in GitHub UI
    steps:
      - run: ./scripts/deploy.sh
        env:
          IMAGE_TAG: ${{ needs.build.outputs.image-tag }}
          DEPLOY_URL: ${{ vars.DEPLOY_URL }}
          API_KEY: ${{ secrets.DEPLOY_API_KEY }}

Configure environment protection rules in GitHub Settings > Environments > production > Required reviewers.

Implement blue-green deployment

Route traffic between two identical environments. Switch instantly; roll back by switching back:

  deploy-blue-green:
    runs-on: ubuntu-latest
    environment: production
    env:
      IMAGE_TAG: ${{ needs.build.outputs.image-tag }}
    steps:
      - uses: actions/checkout@v4

      - name: Determine inactive slot
        id: slot
        run: |
          ACTIVE=$(curl -s https://api.example.com/active-slot)
          if [ "$ACTIVE" = "blue" ]; then
            echo "target=green" >> $GITHUB_OUTPUT
          else
            echo "target=blue" >> $GITHUB_OUTPUT
          fi

      - name: Deploy to inactive slot
        run: ./scripts/deploy-slot.sh ${{ steps.slot.outputs.target }} $IMAGE_TAG

      - name: Run smoke tests against inactive slot
        run: ./scripts/smoke-test.sh ${{ steps.slot.outputs.target }}

      - name: Switch traffic to new slot
        run: ./scripts/switch-slot.sh ${{ steps.slot.outputs.target }}

      - name: Verify production is healthy
        run: ./scripts/health-check.sh production

      - name: Roll back on failure
        if: failure()
        run: ./scripts/switch-slot.sh ${{ steps.slot.outputs.target == 'blue' && 'green' || 'blue' }}

See references/deployment-strategies.md for a detailed comparison of blue-green vs canary vs rolling vs recreate.

Implement canary release with rollback

Route a small percentage of traffic to the new version before full rollout:

  deploy-canary:
    runs-on: ubuntu-latest
    environment: production
    steps:
      - uses: actions/checkout@v4

      - name: Deploy canary (10% traffic)
        run: ./scripts/deploy-canary.sh ${{ env.IMAGE_TAG }} 10

      - name: Monitor canary for 5 minutes
        run: |
          for i in $(seq 1 10); do
            sleep 30
            ERROR_RATE=$(./scripts/get-error-rate.sh canary)
            echo "Canary error rate: $ERROR_RATE%"
            if (( $(echo "$ERROR_RATE > 1.0" | bc -l) )); then
              echo "Error rate too high. Rolling back canary."
              ./scripts/rollback-canary.sh
              exit 1
            fi
          done

      - name: Promote canary to 100%
        run: ./scripts/promote-canary.sh ${{ env.IMAGE_TAG }}

      - name: Roll back on any failure
        if: failure()
        run: ./scripts/rollback-canary.sh

Cache dependencies and build artifacts

Cache node_modules by lockfile hash. Always restore-then-save so partial installs don't get cached:

      - name: Cache node_modules
        id: cache-node-modules
        uses: actions/cache@v4
        with:
          path: node_modules
          key: node-modules-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
          restore-keys: |
            node-modules-${{ runner.os }}-

      - name: Install dependencies
        if: steps.cache-node-modules.outputs.cache-hit != 'true'
        run: npm ci

      - name: Cache Next.js build
        uses: actions/cache@v4
        with:
          path: |
            .next/cache
          key: nextjs-${{ runner.os }}-${{ hashFiles('package-lock.json') }}-${{ hashFiles('**/*.ts', '**/*.tsx') }}
          restore-keys: |
            nextjs-${{ runner.os }}-${{ hashFiles('package-lock.json') }}-
            nextjs-${{ runner.os }}-

Cache keys should go from most-specific to least-specific in restore-keys. A partial cache restore is almost always faster than a cold install.

Set up matrix builds

Test across multiple Node versions and operating systems in parallel:

  test-matrix:
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false        # don't cancel other jobs if one fails
      matrix:
        node-version: [18, 20, 22]
        os: [ubuntu-latest, windows-latest, macos-latest]
        exclude:
          - os: windows-latest
            node-version: 18  # don't test EOL Node on Windows

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
          cache: npm
      - run: npm ci
      - run: npm test

Set fail-fast: false when the matrix combinations are independent. Use fail-fast: true (default) when any failure means the whole build is broken.

Error handling

Gotchas

1. Secrets not available in fork PRs - GitHub Actions does not expose repository secrets to workflows triggered by pull requests from forks (security boundary). Using pull_request_target to work around this requires careful vetting because it runs in the base repo context with full secret access - a malicious PR can exfiltrate them. Default to requiring manual approval for external PRs instead.

2. Required status check name must match exactly - If the workflow or job name changes (even a typo, case difference, or rename), GitHub branch protection silently stops enforcing the old required check. Always verify the status check name in branch protection settings after any workflow rename.

3. Cache poisoning via restore-keys - A restore-keys partial hit restores an older cache, then the job installs on top. If the old cache has a broken or security-patched package that npm ci doesn't replace (because it wasn't in the lockfile update), the cached bad state persists. Use npm ci (not npm install) so it always installs exactly what the lockfile specifies.

4. Immutable artifact drift - Building from source at deploy time (instead of promoting the artifact that passed staging) is a subtle but critical violation of the immutable artifact principle. A second build from the same SHA can produce different outputs if external dependencies (base images, npm packages without lockfiles) changed between builds.

5. Matrix fail-fast: true masking failures - With fail-fast: true (the default), a single failing job cancels all others, which can hide that multiple matrix combinations are broken. Use fail-fast: false when you want a full picture of which combinations fail, then switch back to true for normal CI.

References

For detailed implementation guidance on specific deployment strategies:

• references/deployment-strategies.md - blue-green, canary, rolling, recreate, A/B, and shadow deployments with ASCII diagrams and decision framework

Only load the references file when choosing or implementing a specific deployment strategy - it is detailed and will consume context.