# Automation, CI/CD & GitOps — Part 4: GitHub & GitHub Actions > **Series:** Automation, CI/CD & GitOps (4 of 8) > **Part 1:** `01-cicd-fundamentals.md` — CI/CD Fundamentals > **Part 2:** `02-infrastructure-as-code.md` — Infrastructure as Code > **Part 3:** `03-gitops.md` — GitOps > **Part 4:** This file — GitHub & GitHub Actions > **Part 5:** `05-github-security-governance.md` — GitHub Security & Governance > **Part 6:** `06-gitlab-cicd.md` — GitLab & GitLab CI/CD > **Part 7:** `07-bitbucket-pipelines.md` — Bitbucket & Bitbucket Pipelines > **Part 8:** `08-azure-devops.md` — Azure DevOps > **Questions:** `questions.md` ## Table of Contents 1. [Where GitHub Fits in This Series](#where-github-fits-in-this-series) 2. [GitHub Plans and Actions Pricing at a Glance](#github-plans-and-actions-pricing-at-a-glance) 3. [GitHub as a Git Host — What's Actually Different](#github-as-a-git-host--whats-actually-different) 4. [The `.github/` Directory](#the-github-directory) 5. [Anatomy of a Workflow File](#anatomy-of-a-workflow-file) 6. [Events — What Triggers a Workflow](#events--what-triggers-a-workflow) 7. [Jobs, Steps, and Runners](#jobs-steps-and-runners) 8. [A Minimal Workflow, Built Up Step by Step](#a-minimal-workflow-built-up-step-by-step) 9. [Matrix Builds — Testing Across Many Configurations](#matrix-builds--testing-across-many-configurations) 10. [Artifacts and Caching](#artifacts-and-caching) 11. [Composite Actions — Reusable Steps](#composite-actions--reusable-steps) 12. [Reusable Workflows — Reusable Jobs](#reusable-workflows--reusable-jobs) 13. [Composite Action vs Reusable Workflow — When to Use Which](#composite-action-vs-reusable-workflow--when-to-use-which) 14. [Environments and Manual Approval Gates](#environments-and-manual-approval-gates) 15. [Self-Hosted Runners](#self-hosted-runners) 16. [Contexts, Expressions, and Conditional Steps](#contexts-expressions-and-conditional-steps) 17. [Job Outputs — Passing Small Data Between Jobs](#job-outputs--passing-small-data-between-jobs) 18. [Service Containers — Databases for Integration Tests](#service-containers--databases-for-integration-tests) 19. [Concurrency Control](#concurrency-control) 20. [A Full Realistic Multi-Stage Pipeline](#a-full-realistic-multi-stage-pipeline) 21. [Common Mistakes](#common-mistakes) 22. [Worked Practice Problems](#worked-practice-problems) 23. [Summary and What's Next](#summary-and-whats-next) --- ## Where GitHub Fits in This Series Part 1 of this series built a tool-agnostic mental model of CI/CD: pipeline anatomy, deployment strategies, the DORA metrics. Part 2 covered Infrastructure as Code, and Part 3 covered GitOps — again, tool-agnostic. Starting with this chapter, the series gets concrete: **how does a real team, on a real platform, actually build the pipeline Part 1 described?** GitHub is the natural starting point because it's the single most widely used Git hosting platform in the industry, and — critically for this series — it ships its own first-party CI/CD system, **GitHub Actions**, built directly into the same product. You don't add a separate CI tool on top of GitHub the way older stacks bolted Jenkins onto a GitHub repo; the pipeline definition lives in the same repository as the code, triggered natively by the same events (a push, a pull request, a release) that GitHub already tracks. ```mermaid graph TD OldWay["OLD PATTERN:
Git host (GitHub) is
SEPARATE from the
CI system (e.g. Jenkins) —
webhooks connect them,
credentials must be
shared between two
separate systems"] --> OldProb["Extra infrastructure to
run and secure, extra
integration surface,
pipeline config often
lives OUTSIDE the repo
it builds"] NewWay["GITHUB ACTIONS:
the git host and the
CI/CD system are the
SAME product — pipeline
YAML lives IN the repo,
at .github/workflows/"] --> NewGood["No separate system to
run, no webhook
plumbing, pipeline
changes are reviewed
in the exact same PR
as the code they build"] ``` This chapter and the next (Part 5) cover GitHub end to end: this one focuses on the mechanics of building a working pipeline — workflow syntax, reusability, matrix builds, environments. Part 5 focuses on securing and governing that pipeline once it exists — token permissions, OIDC, branch protection, and GitHub's built-in security tooling. Splitting it this way mirrors a real adoption curve: teams get a pipeline running first, then harden it. --- ## GitHub Plans and Actions Pricing at a Glance Before writing a single workflow, it's worth understanding what a team is actually signing up for — GitHub Actions billing is usage-based on top of whichever GitHub plan a team is already on, not a separate product with its own subscription. | Plan | Included Actions minutes/month (Linux) | Included storage | Concurrent jobs (hosted) | Typical fit | |---|---|---|---|---| | **Free** | 2,000 | 500 MB | Up to 20 | Personal projects, open-source (public repos get **unlimited** minutes on standard runners) | | **Team** | 3,000 | 2 GB | Up to 60 | Small-to-mid paid teams needing private-repo CI | | **Enterprise** | 50,000 | 50 GB | Up to 180 (higher on request) | Large orgs, needs SSO/SAML, audit log, custom runner policies | **Two multipliers that catch teams off guard when a bill arrives higher than expected:** 1. **Minutes are billed per-OS at different rates on private repos** — a Linux minute costs 1×, a Windows minute costs 2×, and a macOS minute costs **10×** the same wall-clock minute, because GitHub's underlying hosted-runner costs differ that much across operating systems. A macOS matrix leg that seems "just one more OS" in a workflow file can dominate a team's entire monthly Actions bill. 2. **Public repositories get genuinely unlimited Actions minutes on standard GitHub-hosted runners** — this is a major reason so much of the open-source ecosystem runs GitHub Actions specifically, versus a competing platform with a hard included-minutes cap even for public projects. **Larger, more powerful hosted runners** (more vCPU/RAM, up to 96-core, and GPU-enabled runners) are available as a separate paid add-on on top of any plan, billed per-minute at a higher rate than the standard 2-core runner — relevant for the self-hosted-runner tradeoff discussed later in this chapter, since a larger hosted runner is often a simpler first step than standing up self-hosted infrastructure. --- ## GitHub as a Git Host — What's Actually Different Everything this course has assumed about Git so far (branches, commits, pull requests) is generic — GitHub didn't invent any of it. What GitHub actually adds on top of plain Git is a set of **platform features that CI/CD pipelines hook into directly**: | Feature | What it is | Why it matters for CI/CD | |---|---|---| | **Pull Requests** | A reviewable proposal to merge one branch into another | Workflows can trigger on PR open/update, post status checks back onto the PR, and block merge until checks pass | | **Branch protection / rulesets** | Rules that restrict what can happen to a branch (e.g. no direct pushes to `main`) | Pipelines are often the *enforcement mechanism* for these rules — a required status check IS a pipeline job | | **Environments** | Named deployment targets (`staging`, `production`) with their own secrets and approval rules | This is GitHub's implementation of the "manual approval gate" from Part 1's pipeline diagram | | **Releases** | A tagged, versioned snapshot of the repo, with attached binary artifacts | Workflows commonly trigger on release creation to build and publish final packages | | **Issues / Projects** | Native issue tracking and lightweight project boards | Not CI/CD directly, but workflows can auto-comment, auto-label, or auto-close issues as part of a pipeline | **A concrete example of the difference this makes:** with a bolted-on CI tool, "block merge until tests pass" requires configuring the CI tool to post a webhook back to GitHub's status API, and configuring GitHub's branch protection to require that named status check. With GitHub Actions, the workflow run *is already* a native GitHub check — no webhook plumbing required, it's a first-class citizen of the platform from the start. --- ## The `.github/` Directory GitHub reserves a specific top-level directory in every repository for platform configuration. The parts relevant to this chapter: ``` .github/ ├── workflows/ # Every CI/CD pipeline definition lives here │ ├── ci.yml │ ├── deploy-prod.yml │ └── nightly-scan.yml ├── actions/ # Local composite actions (repo-private, not published) │ └── setup-env/ │ └── action.yml ├── CODEOWNERS # Covered in Part 5 — enforces review ownership ├── dependabot.yml # Covered in Part 5 — automated dependency updates └── ISSUE_TEMPLATE/ ``` **The one rule that matters most right now:** every file directly inside `.github/workflows/` ending in `.yml` or `.yaml` is treated as an independent, top-level pipeline definition. There is no single master pipeline file the way some other platforms use one `.gitlab-ci.yml` — a GitHub repo commonly has many small, focused workflow files (one for CI on every PR, one for a nightly security scan, one for production deploys), each triggered by its own set of events. --- ## Anatomy of a Workflow File A workflow file has four structural layers, and understanding the nesting is the single biggest unlock for reading (and writing) GitHub Actions YAML correctly: ```mermaid graph TD Workflow["WORKFLOW
(the whole .yml file)"] --> Trigger["on: — WHEN this
workflow runs"] Workflow --> Jobs["jobs: — one or more
named jobs"] Jobs --> Job1["job 'build'"] Jobs --> Job2["job 'deploy'"] Job1 --> Steps1["steps: — an ORDERED
list of individual
actions/commands"] Job2 --> Steps2["steps: — its own
ordered list"] ``` - **Workflow** — the entire file. Has a `name:`, an `on:` trigger definition, and one or more jobs. - **Job** — a unit of work that runs on its own fresh runner (virtual machine or container), in parallel with other jobs by default, unless you explicitly declare a dependency with `needs:`. - **Step** — an ordered instruction inside a job. A step is either `run:` (a raw shell command) or `uses:` (invoke a reusable "Action" — a packaged, shareable unit someone else, or you, already wrote). - **Action** — the smallest reusable unit; a piece of packaged automation referenced by `uses:`, either from the public GitHub Marketplace (`actions/checkout@v4`), a private repo, or a local path (`./.github/actions/setup-env`). --- ## Events — What Triggers a Workflow The `on:` key defines what makes a workflow run at all — this is GitHub's version of the generic "code committed/pushed" trigger from Part 1's pipeline diagram, but with far more event types than a simple push: ```yaml on: push: branches: [main] paths: ['src/**'] # only run if files under src/ changed pull_request: branches: [main] release: types: [published] schedule: - cron: '0 2 * * *' # nightly at 02:00 UTC workflow_dispatch: # manual "Run workflow" button in the UI inputs: environment: description: 'Target environment' required: true type: choice options: [staging, production] workflow_call: # THIS workflow can be called BY another workflow ``` **Why `paths:` filtering matters at scale:** in a monorepo with a frontend and a backend, without `paths:` filtering, *every* commit — even a docs typo fix in the backend — triggers the (possibly expensive) frontend test suite too. Scoping triggers to the paths a workflow actually cares about is a direct, practical application of the "fail fast, don't waste time/money on irrelevant work" principle already established in this series. **`workflow_dispatch` is worth calling out specifically** — it's what turns a workflow into something a human can trigger on demand from the GitHub UI or API, with typed inputs, rather than only ever firing automatically off a Git event. This is commonly how a manual production deploy button is implemented in GitHub Actions. --- ## Jobs, Steps, and Runners ```yaml jobs: build: runs-on: ubuntu-latest # GitHub-hosted runner: a fresh Ubuntu VM steps: - name: Check out the repo uses: actions/checkout@v4 # Action: clones the repo onto the runner - name: Set up Node uses: actions/setup-node@v4 with: node-version: '20' - name: Install dependencies run: npm ci # raw shell command - name: Run tests run: npm test ``` Two facts about jobs that trip people up constantly: 1. **Every job starts from a completely clean runner.** There is no filesystem state shared between jobs by default — a file created in job `build` does not exist in job `deploy` unless it's explicitly passed via `actions/upload-artifact` / `actions/download-artifact` (covered below), or the two steps are inside the *same* job. 2. **Jobs run in parallel unless you say otherwise.** To force one job to wait for another — e.g. `deploy` must not start until `build` and `test` both succeed — use `needs:`: ```yaml jobs: build: runs-on: ubuntu-latest steps: [...] test: runs-on: ubuntu-latest steps: [...] deploy: needs: [build, test] # waits for BOTH to succeed first runs-on: ubuntu-latest steps: [...] ``` ```mermaid flowchart LR Build["build"] --> Deploy["deploy"] Test["test"] --> Deploy Build -.parallel with.-> Test ``` This `needs:` graph is directly how GitHub Actions implements the layered pipeline-stage ordering from Part 1 — except unlike a single linear pipeline, jobs without a `needs:` relationship genuinely run concurrently, which is why a real-world workflow's `build` and `lint` jobs typically run side by side rather than one after another. --- ## A Minimal Workflow, Built Up Step by Step Starting from the smallest possible useful workflow and adding one real-world concern at a time: **Step 1 — the bare minimum that does something:** ```yaml name: CI on: push jobs: test: runs-on: ubuntu-latest steps: - run: echo "hello" ``` **Step 2 — actually checking out and testing code:** ```yaml name: CI on: push jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: { node-version: '20' } - run: npm ci - run: npm test ``` **Step 3 — scoping the trigger and adding least-privilege permissions** (permissions are covered in depth in Part 5, but every workflow should set this from day one): ```yaml name: CI on: push: branches: [main] pull_request: branches: [main] permissions: contents: read jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: { node-version: '20' } - run: npm ci - run: npm test ``` **Step 4 — failing fast with a lint job that runs in parallel, plus caching** (caching covered next section): ```yaml name: CI on: push: branches: [main] pull_request: branches: [main] permissions: contents: read jobs: lint: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: { node-version: '20', cache: 'npm' } - run: npm ci - run: npm run lint test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: { node-version: '20', cache: 'npm' } - run: npm ci - run: npm test ``` This progression is deliberately incremental — a real team almost never designs a workflow file in one sitting; it grows exactly like this as new requirements (linting, caching, deployment) show up. --- ## Matrix Builds — Testing Across Many Configurations A **matrix** runs the *same* job definition multiple times, once per combination of variables you supply — the direct GitHub Actions implementation of "test this across every Node version / OS / database version we support" without hand-writing a separate job for each combination. ```yaml jobs: test: runs-on: ${{ matrix.os }} strategy: matrix: os: [ubuntu-latest, windows-latest, macos-latest] node-version: ['18', '20', '22'] fail-fast: false # let ALL combinations finish, don't cancel siblings on first failure steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: { node-version: '${{ matrix.node-version }}' } - run: npm ci - run: npm test ``` This single job definition expands into **9 real jobs** (3 operating systems × 3 Node versions), running in parallel, each reported back as its own separate check. ```mermaid graph TD Matrix["strategy.matrix:
os × node-version"] --> J1["ubuntu / node18"] Matrix --> J2["ubuntu / node20"] Matrix --> J3["ubuntu / node22"] Matrix --> J4["windows / node18"] Matrix --> J5["windows / node20"] Matrix --> J6["windows / node22"] Matrix --> J7["macos / node18"] Matrix --> J8["macos / node20"] Matrix --> J9["macos / node22"] ``` **`fail-fast: false` is a genuinely important, easy-to-miss setting.** The default (`true`) cancels every other matrix combination the instant *any one* combination fails — which sounds efficient, but means a failure specific to, say, Windows + Node 18 might cancel the macOS + Node 22 job before it ever reports its own, potentially different, result. For a compatibility matrix specifically, you almost always want `fail-fast: false` so you see the *complete* picture of what's broken where in a single run. You can also exclude specific, known-bad combinations without removing an entire row or column: ```yaml strategy: matrix: os: [ubuntu-latest, windows-latest] node-version: ['18', '20', '22'] exclude: - os: windows-latest node-version: '18' # this one combination is skipped ``` --- ## Artifacts and Caching Two related but distinct mechanisms for moving data between jobs and speeding up repeated runs — mixing them up is a common source of confusion. | | **Cache** | **Artifact** | |---|---|---| | **Purpose** | Speed up future runs (skip re-downloading dependencies) | Pass a file from one job to another, or preserve a build output | | **Lifetime** | Best-effort — GitHub may evict old caches; treat as disposable | Explicit retention period (default 90 days), guaranteed available for that window | | **Typical content** | `node_modules`, pip/Maven/Go module caches | Compiled binaries, test reports, coverage output, build logs | | **Action** | `actions/cache@v4` | `actions/upload-artifact@v4` + `actions/download-artifact@v4` | ```yaml # Caching dependencies (many setup-* actions like setup-node have this built in via `cache:`) - uses: actions/cache@v4 with: path: ~/.npm key: npm-${{ runner.os }}-${{ hashFiles('package-lock.json') }} restore-keys: npm-${{ runner.os }}- # Passing a build output from one job to another # In job "build": - uses: actions/upload-artifact@v4 with: name: dist path: dist/ # In job "deploy" (which has `needs: build`): - uses: actions/download-artifact@v4 with: name: dist path: dist/ ``` **The cache key pattern above (`hashFiles('package-lock.json')`) is worth understanding, not just copying:** the cache key changes automatically whenever the lockfile changes, so a dependency bump correctly invalidates the old cache instead of silently reusing stale, now-wrong dependencies — the same "invalidate on real change, reuse otherwise" idea underlying Terraform's plan/apply model from Part 2. --- ## Composite Actions — Reusable Steps A **composite action** packages a *sequence of steps* into one reusable, named unit — the fix for the same 5-10 lines of setup steps being copy-pasted at the top of every workflow file in a repo. ```yaml # .github/actions/setup-env/action.yml name: 'Set up environment' description: 'Checks out code and installs Node dependencies with caching' inputs: node-version: description: 'Node version to install' default: '20' runs: using: 'composite' steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: ${{ inputs.node-version }} cache: 'npm' - run: npm ci shell: bash # required for every `run:` step inside a composite action ``` Used from any workflow in the same repo: ```yaml jobs: test: runs-on: ubuntu-latest steps: - uses: ./.github/actions/setup-env with: node-version: '20' - run: npm test ``` **The one syntax detail that catches people out:** every `run:` step *inside* a composite action must explicitly declare `shell: bash` (or whichever shell) — unlike a normal workflow step, composite actions don't infer a default shell. --- ## Reusable Workflows — Reusable Jobs Where a composite action reuses *steps within one job*, a **reusable workflow** reuses an *entire job definition* — including its own `runs-on`, its own `strategy.matrix`, and multiple jobs if needed. It's invoked with `uses:` at the *job* level (not inside `steps:`), and the source workflow must declare `workflow_call` as one of its triggers. ```yaml # .github/workflows/reusable-deploy.yml name: Reusable Deploy on: workflow_call: inputs: environment: required: true type: string secrets: DEPLOY_TOKEN: required: true jobs: deploy: runs-on: ubuntu-latest environment: ${{ inputs.environment }} steps: - uses: actions/checkout@v4 - run: ./deploy.sh --env ${{ inputs.environment }} env: TOKEN: ${{ secrets.DEPLOY_TOKEN }} ``` Called from another workflow, once per target environment via a matrix — the same "deploy across many environments" pattern this series has already covered conceptually in Part 1's deployment-strategies discussion, now expressed concretely: ```yaml # .github/workflows/deploy-all.yml name: Deploy All Environments on: push: branches: [main] jobs: deploy: strategy: matrix: target: [staging, production] uses: ./.github/workflows/reusable-deploy.yml with: environment: ${{ matrix.target }} secrets: DEPLOY_TOKEN: ${{ secrets.DEPLOY_TOKEN }} ``` ```mermaid graph TD Caller["deploy-all.yml
(caller)"] --> Reusable["reusable-deploy.yml
(workflow_call)"] Reusable --> Staging["deploy job:
environment=staging"] Reusable --> Prod["deploy job:
environment=production"] ``` A reusable workflow can also be called from a completely **different repository** (`uses: my-org/shared-workflows/.github/workflows/reusable-deploy.yml@v1`), which is how larger organizations centralize a "golden path" deploy pipeline that every product repo calls into, rather than each team maintaining its own copy. GitHub caps this at **10 levels of nesting** (the top-level caller plus up to 9 reusable workflows) and **50 unique reusable workflows** callable from a single workflow file — generous limits that only matter at real organizational scale. --- ## Composite Action vs Reusable Workflow — When to Use Which A table worth memorizing, because this exact question ("should this be a composite action or a reusable workflow?") comes up in every non-trivial GitHub Actions codebase: | | **Composite Action** | **Reusable Workflow** | |---|---|---| | **Reuses** | A sequence of steps | One or more entire jobs | | **Invoked from** | Inside a job's `steps:` list | A job's `uses:` key directly (no `steps:` needed) | | **Can define its own `runs-on`?** | No — runs on whatever runner the calling job already picked | Yes — each reusable job has its own `runs-on` | | **Can use `strategy.matrix`?** | Only the calling job's matrix (it has none of its own) | Yes, independently | | **Typical use case** | "Set up my environment" boilerplate shared across many jobs | "Deploy to an environment" — a full, self-contained unit of work | | **Secrets** | Inherits whatever the calling job already has | Must be explicitly passed via `secrets:` (or `secrets: inherit`) | **Rule of thumb:** if what you're deduplicating is *setup steps that still run inside a bigger job*, use a composite action. If what you're deduplicating is *a self-contained unit of work with its own runner and possibly its own matrix* (most commonly: a deploy job), use a reusable workflow. --- ## Environments and Manual Approval Gates GitHub **Environments** (`staging`, `production`, etc.) are GitHub's concrete implementation of the "manual approval?" decision diamond from Part 1's pipeline-anatomy diagram — and this is also where GitHub-native Continuous Delivery (Part 1's distinction from Continuous Deployment) actually gets enforced, not just described. An environment, configured in repo Settings → Environments, can carry: - **Required reviewers** — one or more specific people (or teams) who must click "Approve" before a job referencing this environment proceeds — the workflow run **pauses**, genuinely blocked, until approval. - **Wait timer** — a mandatory delay (e.g. 10 minutes) before the job runs, even with no human involved — useful as a "cool-down" window to catch an obviously-bad deploy just landed elsewhere. - **Deployment branch/tag rules** — restrict which branches are even allowed to deploy to this environment (e.g. only `main` can deploy to `production`). - **Environment-scoped secrets** — a secret defined on the `production` environment is invisible to a job running against `staging`, even in the same workflow file. ```yaml jobs: deploy-prod: runs-on: ubuntu-latest environment: name: production url: https://app.example.com steps: - run: ./deploy.sh env: API_KEY: ${{ secrets.PROD_API_KEY }} # only visible because job targets "production" ``` ```mermaid sequenceDiagram participant CI as CI pipeline participant GH as GitHub Environment participant R as Required reviewer CI->>GH: Job targets "production" environment GH->>GH: Check required reviewers + wait timer GH->>R: Request approval R-->>GH: Approve (or reject) GH-->>CI: Unblock job - continue CI->>CI: Run deploy steps with production secrets ``` This is the exact mechanism that turns a GitHub Actions pipeline from Continuous Deployment (fully automatic) into Continuous Delivery (automatic up to a human-gated release) as covered conceptually in Part 1 — the only difference between the two, in GitHub terms, is whether the `production` environment has required reviewers configured. --- ## Self-Hosted Runners GitHub-hosted runners (`runs-on: ubuntu-latest`) are fresh, ephemeral VMs GitHub provisions and tears down for every job — zero maintenance, but limited to GitHub's provided hardware/OS images and (on the free tier) limited monthly minutes. A **self-hosted runner** is a machine (physical, VM, container, or Kubernetes pod via the Actions Runner Controller) that a team registers to their own repo, organization, or enterprise, and that GitHub Actions dispatches jobs to instead. ```yaml jobs: build: runs-on: [self-hosted, linux, gpu] # labels: match any runner tagged with ALL of these steps: - uses: actions/checkout@v4 - run: ./build-with-gpu.sh ``` **Why a team reaches for self-hosted runners**, in rough order of frequency: 1. **Hardware GitHub doesn't offer** — GPU-accelerated builds, specialized ARM hardware, more RAM/CPU than the largest GitHub-hosted tier. 2. **Network access to private infrastructure** — the runner needs to reach an internal database or private VPC that a GitHub-hosted (public-cloud, ephemeral) runner cannot reach. 3. **Cost at high volume** — GitHub-hosted minutes are billed per-minute past the included allowance; a team running thousands of build-minutes daily may find dedicated self-hosted capacity cheaper. 4. **Compliance** — some regulatory environments require build infrastructure to stay entirely within the organization's own network boundary. **The security tradeoff, stated plainly (expanded fully in Part 5):** a self-hosted runner attached to a *public* repository is a genuine security risk — anyone who can open a pull request can potentially get arbitrary code executed on that runner via a workflow triggered by `pull_request_target` or similar. Self-hosted runners are broadly considered safe for private/internal repos with a trusted contributor set, and require real hardening (ephemeral, single-job runners; no persistent secrets on the box) for anything public-facing. --- ## Contexts, Expressions, and Conditional Steps Every value inside `${{ }}` in a workflow file is an **expression**, evaluated against one of several built-in **contexts** — structured data GitHub makes available about the run, the event that triggered it, the repo, and the job's own state so far. | Context | Holds | Example | |---|---|---| | `github` | The triggering event, repo, actor, ref | `github.event_name`, `github.actor`, `github.ref` | | `env` | Environment variables defined in the workflow/job/step | `env.NODE_ENV` | | `secrets` | Configured secrets (never printed to logs, auto-masked) | `secrets.DEPLOY_TOKEN` | | `matrix` | The current matrix combination | `matrix.node-version` | | `needs` | Outputs and results from jobs this job depends on | `needs.build.outputs.version`, `needs.build.result` | | `steps` | Outputs from previous steps in the *same* job | `steps.get-version.outputs.value` | | `runner` | Info about the runner executing the job | `runner.os`, `runner.temp` | **`if:` conditions** are how a step or an entire job decides whether to run at all — the GitHub Actions equivalent of a branch in code, and the mechanism behind "only deploy from `main`, never from a feature branch" or "only run this cleanup step even if an earlier step failed": ```yaml jobs: deploy: if: github.ref == 'refs/heads/main' && github.event_name == 'push' runs-on: ubuntu-latest steps: - run: ./deploy.sh - name: Notify on failure if: failure() # only runs if a PRECEDING step in this job failed run: ./notify-slack.sh "Deploy failed" - name: Always clean up if: always() # runs regardless of success/failure/cancellation run: ./cleanup.sh ``` The four status-check functions — `success()` (default, implicit if `if:` is omitted), `failure()`, `cancelled()`, and `always()` — are worth memorizing precisely, because a genuinely common mistake is adding a "notify on failure" step and having it silently never run, because by default GitHub skips every remaining step in a job the instant one step fails, and only `if: failure()` or `if: always()` override that default skip behavior. --- ## Job Outputs — Passing Small Data Between Jobs Artifacts (covered earlier) move *files* between jobs. **Job outputs** move small, individual *values* — a version string, a computed flag, a generated ID — without the overhead of uploading and downloading a file for a single piece of data. ```yaml jobs: build: runs-on: ubuntu-latest outputs: version: ${{ steps.get-version.outputs.value }} steps: - id: get-version # the step needs an `id:` to be referenced later run: echo "value=$(cat VERSION)" >> "$GITHUB_OUTPUT" deploy: needs: build runs-on: ubuntu-latest steps: - run: echo "Deploying version ${{ needs.build.outputs.version }}" ``` **The `>> "$GITHUB_OUTPUT"` pattern is the current, correct way to set an output** — writing to this special environment-file path GitHub provides to the runner, rather than the older `::set-output::` workflow command syntax, which GitHub deprecated for security reasons (it was vulnerable to log injection from untrusted input). Any tutorial or Stack Overflow answer still showing `::set-output::` is describing a removed, no-longer-functional pattern. --- ## Service Containers — Databases for Integration Tests A **service container** runs a real dependency — Postgres, Redis, RabbitMQ — as a sidecar container alongside a job's main container, network-reachable by the job's steps for the job's entire duration. This is how a GitHub Actions job runs genuine integration tests against a real database, rather than mocking it out, directly reusing the "cheap unit tests first, then real integration tests against real dependencies" stage-ordering principle from Part 1. ```yaml jobs: integration-test: runs-on: ubuntu-latest services: postgres: image: postgres:16 env: POSTGRES_PASSWORD: testpass POSTGRES_DB: testdb ports: - 5432:5432 options: >- --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5 steps: - uses: actions/checkout@v4 - run: npm ci - run: npm run test:integration env: DATABASE_URL: postgresql://postgres:testpass@localhost:5432/testdb ``` **The `options:` health-check block is not optional boilerplate — it's what prevents a race condition.** Without it, the job's steps can start running *before* Postgres has finished initializing inside its container, causing intermittent, hard-to-debug "connection refused" failures that look like a flaky test but are actually a startup-ordering bug. GitHub Actions waits for the service container to report healthy (via the configured health check) before starting the job's own steps. --- ## Concurrency Control By default, pushing three commits to the same branch within a minute of each other queues (or runs in parallel) three separate, full workflow runs — wasteful, and for a deploy workflow specifically, actively dangerous (two deploys to the same environment racing each other). **`concurrency:`** groups workflow runs by a key and controls what happens when a new run starts while an older one in the same group is still active. ```yaml concurrency: group: deploy-production # runs sharing this exact group name are serialized cancel-in-progress: false # let the current run finish; queue the new one behind it jobs: deploy: runs-on: ubuntu-latest environment: production steps: [...] ``` A very common pattern uses the branch/PR reference to build a dynamic group name, so **every branch or PR gets its own independent concurrency lane** — stale runs on an old commit within *the same PR* get cancelled, but that never affects a different PR's runs: ```yaml concurrency: group: ci-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }} cancel-in-progress: true # a new push to the SAME pr/branch cancels the old, now-stale run ``` **The two settings genuinely mean opposite things and picking the wrong one has real consequences:** `cancel-in-progress: true` is correct for CI checks on a PR (there's no value in finishing a test run against code that's already been superseded by a newer push) but would be actively dangerous for a production deploy job (`cancel-in-progress: false`, as shown above) — you never want to cancel a deploy that's already partway through applying changes to production; you want the next one to wait its turn instead. --- ## A Full Realistic Multi-Stage Pipeline Tying every mechanism from this chapter together into one realistic pipeline — build → test (matrix) → security scan → deploy to staging → manual approval → deploy to production, directly reusing Part 1's full pipeline-anatomy diagram, now expressed as real GitHub Actions YAML: ```yaml name: Full Pipeline on: push: branches: [main] pull_request: branches: [main] permissions: contents: read jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: ./.github/actions/setup-env - run: npm run build - uses: actions/upload-artifact@v4 with: { name: dist, path: dist/ } test: needs: build strategy: matrix: node-version: ['18', '20', '22'] fail-fast: false runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: { node-version: '${{ matrix.node-version }}', cache: 'npm' } - run: npm ci - run: npm test security-scan: needs: build runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - run: npm audit --audit-level=high deploy-staging: needs: [test, security-scan] runs-on: ubuntu-latest environment: staging steps: - uses: actions/download-artifact@v4 with: { name: dist, path: dist/ } - run: ./deploy.sh --env staging smoke-test: needs: deploy-staging runs-on: ubuntu-latest steps: - run: curl -f https://staging.example.com/healthz deploy-production: needs: smoke-test runs-on: ubuntu-latest environment: production # required reviewers configured here = the manual gate steps: - uses: actions/download-artifact@v4 with: { name: dist, path: dist/ } - run: ./deploy.sh --env production ``` ```mermaid flowchart LR Build["build"] --> Test["test
(matrix ×3)"] Build --> Scan["security-scan"] Test --> DeployStaging["deploy-staging"] Scan --> DeployStaging DeployStaging --> Smoke["smoke-test"] Smoke --> Gate{"production
environment —
required reviewer"} Gate --> DeployProd["deploy-production"] ``` Notice this single file demonstrates every stage-ordering principle from Part 1: cheap/fast checks (`test`, `security-scan`) run before expensive ones (an actual deploy); the production deploy is the last, most gated step; and the `needs:` graph enforces exactly the same left-to-right dependency order as the generic pipeline diagram this series opened with. --- ## Common Mistakes | Mistake | Why it's a problem | Fix | |---|---|---| | Forgetting `permissions:` entirely | `GITHUB_TOKEN` defaults to broad permissions on many repos, violating least privilege | Set `permissions: { contents: read }` at the workflow level, escalate only in the specific job that needs it (full detail in Part 5) | | Using `actions/checkout@v4` but never pinning further | A tag like `@v4` can be moved by the action's author (or, in a compromise, an attacker) | Pin third-party actions to a full commit SHA for anything security-sensitive (full detail in Part 5) | | `fail-fast: true` (the default) on a compatibility matrix | Cancels other combinations before you see their results, hiding the full picture of what's actually broken | Set `fail-fast: false` when the goal is genuinely seeing every combination's outcome | | Treating a cache as guaranteed | Caches are best-effort and can be silently evicted | Never rely on cache for anything correctness-critical — only for speed; use artifacts for anything that must exist | | Copy-pasting the same 5 setup steps into every workflow file | Drift — a fix applied to one copy doesn't reach the others | Extract into a composite action (or reusable workflow if it's a full job) | | No environment protection on `production` | A single bad merge to `main` can auto-deploy straight to real users with no human check | Configure required reviewers on the `production` environment | | Self-hosted runner attached to a public repo with no hardening | A malicious PR can potentially execute code on your infrastructure | Use GitHub-hosted runners for public repos, or heavily hardened ephemeral self-hosted runners (Part 5) | --- ## Worked Practice Problems **Problem 1:** A team's `test` job takes 12 minutes because it reinstalls all npm dependencies from scratch on every run. What's the single highest-leverage fix, and why? *Answer:* Add dependency caching via `actions/cache@v4` (or `setup-node`'s built-in `cache: 'npm'` option), keyed on the lockfile hash. This is the single highest-leverage fix because dependency installation is almost always the largest fixed cost in a JS pipeline, and unlike test parallelization or matrix tuning, caching requires no change to the actual test suite — it's pure infrastructure speedup with no risk to correctness, since the cache key is invalidated automatically the moment the lockfile changes. **Problem 2:** Three product repos in an organization each maintain their own near-identical 40-line "build, scan, and push a Docker image" workflow, and they've already drifted slightly out of sync — one has an extra security scan step the others are missing. Which GitHub Actions mechanism fixes this, and how would you migrate? *Answer:* A reusable workflow (`workflow_call`) in a shared, central repository, since what's being deduplicated is a full self-contained job (build+scan+push), not just a handful of setup steps. Migration: extract the common logic into `shared-workflows/.github/workflows/build-scan-push.yml` with `workflow_call` inputs for anything genuinely repo-specific (image name, Dockerfile path), then replace each product repo's local job with `uses: my-org/shared-workflows/.github/workflows/build-scan-push.yml@v1`. Going forward, a fix to the shared workflow (like adding that missing security scan step) reaches all three repos the next time they pull `@v1`, instead of requiring three separate manual edits. **Problem 3:** A workflow needs to deploy to `staging`, `qa`, and `production` — same deploy logic, different target and different secrets per environment — with production requiring manual approval but the other two not. Sketch the job structure. *Answer:* One reusable workflow (`reusable-deploy.yml`, `workflow_call`, taking `environment` as an input and `DEPLOY_TOKEN` as a required secret) called three times from a caller workflow — either via `strategy.matrix: { target: [staging, qa] }` for the two non-gated environments, plus a separate explicit `production` job with `needs:` on the first two so it only starts after both succeed. The manual-approval behavior needs no extra workflow logic at all — it's entirely a property of the `production` GitHub Environment having required reviewers configured in repo settings, which the reusable workflow's `environment: ${{ inputs.environment }}` line automatically picks up whenever it's called with `environment: production`. --- ## Summary and What's Next GitHub ships its pipeline system directly inside the same product as the code it builds — workflow files live at `.github/workflows/`, triggered by native platform events (push, PR, release, schedule, or a manual button), and structured as jobs (parallel by default, ordered via `needs:`) made of steps. Matrix builds fan a single job definition out across many configurations; composite actions deduplicate steps within a job, while reusable workflows deduplicate entire jobs (and are how larger organizations centralize a golden-path pipeline). Environments are GitHub's concrete implementation of the manual approval gate that separates Continuous Delivery from Continuous Deployment, and self-hosted runners trade GitHub's zero-maintenance hosted infrastructure for custom hardware, network access, or cost control — at the price of real security responsibility. **Part 5** picks up exactly where the security tradeoffs in this chapter left off: `GITHUB_TOKEN` least privilege in depth, pinning actions to a commit SHA against supply-chain attacks, OIDC to eliminate long-lived cloud credentials entirely, branch protection rules and rulesets, CODEOWNERS, Dependabot, and GitHub Advanced Security (CodeQL and secret scanning).