Workspaces, Environments & Real-World Repository Structure

Assumes you're comfortable with Part 2's state splitting and Part 3's module design — this chapter is where those two ideas combine into the actual directory layout and repository shape a real platform team runs day to day, at the scale of dozens of services, several environments, and (by the end of the chapter) more than one cloud.

Table of Contents#

1. Why Repository Structure Deserves Its Own Chapter
2. Terraform Workspaces — What They Actually Are
3. Workspaces vs. Directory-Per-Environment — the Real Tradeoff
4. The Directory-Per-Environment Pattern in Practice
5. The DRY Problem Directory-Per-Environment Creates
6. Terragrunt and the Infrastructure-Live Pattern
7. A Variable Hierarchy: Global, Account, Region, Environment
8. Monorepo vs. Polyrepo for Terraform at Scale
9. A Concrete Repository Layout for a Mid-Size Platform Team
10. Bootstrapping the State Backend Itself
11. Promoting a Change From Dev to Prod
12. Multi-Account AWS — Landing Zones and Organizations
13. Cross-Account Access — the Hub-and-Spoke assume_role Pattern
14. Multi-Cloud Directory Structure — Keeping Clouds Cleanly Separated
15. HCP Terraform Stacks — a Native Alternative to Terragrunt
16. Naming Conventions Across the Tree
17. Choosing Your Structure — a Decision Framework
18. Worked Scenario: Migrating Off Workspaces Into Directory-Per-Environment
19. Worked Scenario: Standing Up a Second Cloud for Disaster Recovery
20. Worked Scenario: the tfvars Override That Silently Hit the Wrong Account
21. Part 4 CLI Cheat Sheet
22. Common Mistakes and Interview Traps
23. Worked Practice Problems
24. Summary and What's Next

Why Repository Structure Deserves Its Own Chapter#

Almost every serious Terraform incident that isn't a state-management problem (Part 2) is a repository- structure problem — a change meant for staging that silently applied to production, a .tfvars file copied from the wrong environment, a shared module change that rippled into every account at once because nothing separated them. None of this is about HCL syntax. It's about how a codebase is physically laid out: which directory maps to which real environment, which account, and — once a company has more than one — which cloud, and how much of that mapping is enforced by structure itself versus trusted to a human reading a folder name correctly under pressure.

This is also the chapter where the throughline system stops being a convenient teaching device and starts looking like what a real platform team actually runs: checkout-service, catalog-service, and inventory-service, each needing dev/staging/prod, all inside one AWS Organization, with a second cloud (GCP) added for disaster recovery by the end of this chapter — and the actual directory tree that holds all of it.

Terraform Workspaces — What They Actually Are#

A CLI workspace is a named, isolated state file within the same backend configuration — terraform workspace new staging creates a second, independent state under the same bucket/key, addressable via terraform.workspace inside your configuration. This is a genuinely different feature from an "HCP Terraform workspace" (a whole managed run environment in the SaaS product) despite the identical name — a recurring source of confusion worth clearing up immediately.

terraform workspace new staging
terraform workspace new prod
terraform workspace select staging
terraform workspace list
#   default
# * staging
#   prod

locals {
  instance_type = terraform.workspace == "prod" ? "m6i.large" : "t3.micro"
}

Workspaces vs. Directory-Per-Environment — the Real Tradeoff#

CLI workspaces switch which state file you're pointed at without changing which .tf files are loaded — which is exactly their strength for genuinely ephemeral, structurally-identical environments, and exactly their weakness for anything long-lived and meaningfully different.

The single sharpest real-world danger with CLI workspaces: switching workspaces is a silent, local CLI state change — terraform workspace select prod followed by an apply intended for staging, because the engineer forgot which workspace they'd last selected, is a genuinely common way to apply a staging-sized change against production. Directory-per-environment structurally prevents this exact mistake: the working directory itself (environments/prod/) is what you'd have to be in, and that's visible in every terminal prompt, every cd, every CI job's working-directory setting — there's no separate, invisible "current workspace" state to forget about.

The Directory-Per-Environment Pattern in Practice#

The straightforward version of this pattern is one directory per environment, each a complete root module in its own right, each with its own backend configuration and its own .tfvars file.

# environments/prod/main.tf
module "checkout_database" {
  source = "../../modules/database"

  instance_class     = "db.r6g.large"
  multi_az           = true
  backup_retention   = 30
}

# environments/dev/main.tf
module "checkout_database" {
  source = "../../modules/database"

  instance_class     = "db.t4g.small"
  multi_az           = false
  backup_retention   = 1
}

Each environment's main.tf is a real, honest description of what that environment actually is — a reader opening environments/prod/main.tf sees multi_az = true right there, with no conditional expression to trace through to figure out what prod actually gets. This directness is the entire value proposition of the pattern, and it's also exactly what creates the next section's problem.

The DRY Problem Directory-Per-Environment Creates#

The moment three environment directories share 90% identical structure and differ only in a handful of values, plain directory-per-environment starts to hurt — a change to checkout_database's module call (a new argument, a renamed variable) now has to be made in three places, by hand, and it's easy for one environment to quietly drift out of sync with the other two.

This is precisely the gap the two mechanisms in the next two sections exist to close — a variable hierarchy (pulling shared defaults up and out) and Terragrunt (a wrapper generating the repetitive parts) both attack the same underlying problem from different angles: keep each environment's actual differences explicit and readable, while eliminating the accidental duplication of everything that's genuinely the same.

Terragrunt and the Infrastructure-Live Pattern#

Terragrunt is a thin wrapper around the terraform CLI, popularized by Gruntwork, that generates boilerplate (backend config, provider config, common variables) from a DRY hierarchy of small .hcl files — the most widely adopted answer to directory-per-environment's duplication problem.

# prod/us-east-1/checkout-database/terragrunt.hcl
include "root" {
  path = find_in_parent_folders()
}

terraform {
  source = "git::https://github.com/meridian-platform/terraform-modules.git//database?ref=v2.3.1"
}

inputs = {
  instance_class   = "db.r6g.large"
  multi_az         = true
  backup_retention = 30
}

The include "root" block pulls in shared configuration (backend generation, provider generation) defined once at the repository root, so every leaf terragrunt.hcl file only ever states what's genuinely specific to that one deployment — the module source/version and the environment-specific input values, nothing else. Running terragrunt apply in this leaf directory generates the full Terraform configuration (backend block, provider block, module call) on the fly from the hierarchy, applies it, and leaves the generated files as disposable artifacts, not something committed to version control.

A Variable Hierarchy: Global, Account, Region, Environment#

Whether implemented via Terragrunt's find_in_parent_folders() mechanism or a simpler hand-rolled .tfvars merge, the underlying idea is the same: values that genuinely vary by scope should be defined at that scope, once, and inherited downward — not copy-pasted into every leaf that needs them.

This chapter's caption: each layer only ever states what's genuinely new at that scope — region.hcl never repeats org_name, and the leaf terragrunt.hcl never repeats the region's AZ list; every layer below inherits everything above it automatically.

A hand-rolled equivalent without Terragrunt is achievable too, using Terraform's own -var-file layering (later files override earlier ones):

terraform apply \
  -var-file=../../global.tfvars \
  -var-file=../account.tfvars \
  -var-file=./region.tfvars \
  -var-file=./prod.tfvars

Monorepo vs. Polyrepo for Terraform at Scale#

Whether all of this infrastructure code lives in one repository or is split across many is a genuinely separate decision from directory structure — and, like CLI-workspaces-vs-directories, the right answer tracks team size and ownership boundaries more than any technical property of Terraform itself.

For the platform team running checkout-service, catalog-service, and inventory-service, a monorepo (infrastructure-live/, one repository, directory-per-service-per-environment inside it) fits comfortably — one small team, tightly coupled ownership, and the cross-cutting refactor benefit (Part 3's module extraction scenario touched all three services in one coordinated change) genuinely matters. A larger organization with dozens of autonomous product teams, each owning its own infrastructure independently, would reasonably split by team or by service instead — Google and Meta's internal monorepo-at-massive-scale approach works because of enormous, purpose-built tooling investment most organizations don't have and shouldn't try to replicate.

A Concrete Repository Layout for a Mid-Size Platform Team#

Pulling every pattern in this chapter together, here's the platform team's actual infrastructure-live repository, at the point in the series where checkout-service, catalog-service, and inventory-service all run across dev/staging/prod in one AWS Organization:

Each service's directory under each environment mirrors Part 2's state-splitting boundaries exactly — one Terragrunt deployment (and therefore one state file) per service per environment, all consuming the shared network deployment's outputs via dependency blocks (Terragrunt's own equivalent of Part 2's terraform_remote_state, resolved automatically from the tree structure rather than a hand-typed backend config).

Bootstrapping the State Backend Itself#

A genuine chicken-and-egg problem every new environment/account hits: Part 2's remote backend (an S3 bucket with locking enabled) is itself infrastructure — so what manages it, if Terraform needs a working backend before it can safely run at all?

The standard resolution is a small, deliberately separate "bootstrap" configuration, using local state (the one legitimate long-term use of local state this series endorses, beyond a throwaway sandbox) purely for the handful of resources — the state bucket, its versioning and locking configuration, and the IAM policy scoping access to it — that every other configuration in that account will depend on:

# bootstrap/prod/main.tf
resource "aws_s3_bucket" "tfstate" {
  bucket = "meridian-platform-tfstate-prod"
}

resource "aws_s3_bucket_versioning" "tfstate" {
  bucket = aws_s3_bucket.tfstate.id
  versioning_configuration { status = "Enabled" }
}

Once applied, every other configuration in that account references this bucket in its own backend "s3" block — but the bootstrap configuration itself has nowhere "further down" to delegate its own state to, which is exactly why it's the one deliberate, permanent exception to "always use a remote backend." Its local state file is small, rarely changes after initial creation, and — critically — should still be backed up (committed to a tightly-access-controlled internal repository, or copied somewhere durable) since it's the literal foundation every other state in that account depends on existing correctly.

Promoting a Change From Dev to Prod#

With environments split into their own directories (or Terragrunt deployments), "promotion" is the actual mechanism by which a change reaches production — and the discipline here is what turns directory isolation from a nice structural property into an enforced deployment process.

This chapter's caption: the same underlying change (a module version bump, or a .tfvars value) moves through three separate, reviewed PRs rather than one PR touching all three environments simultaneously — each promotion step is its own decision point, not an automatic cascade.

The concrete mechanism varies by how modules are sourced: if staging/ and prod/ pin a module by git tag or registry version (Part 3), promotion is literally bumping that pinned version in one directory's terragrunt.hcl (or main.tf) at a time, in order, each its own reviewed PR — never one PR editing all three environments' pins simultaneously, which would defeat the entire point of graduated, observed promotion.

Multi-Account AWS — Landing Zones and Organizations#

At real scale, "environment" and "AWS account" become the same boundary — dev, staging, and prod each get their own AWS account under one AWS Organization, not just separate directories or .tfvars files inside a single shared account. This is the single strongest blast-radius control available: an IAM policy mistake, a leaked credential, or a runaway resource in the dev account has no path to reach prod at all, because they're different accounts with different credentials entirely — not a permissions boundary inside one account that a misconfiguration could erode.

This chapter's caption: dev, staging, and prod are peer accounts under one OU, not nested inside each other — none of them can reach another by default, and every cross-account interaction (state bucket access, CI role assumption) has to be explicitly granted, which is exactly the property a single-account, directory-only isolation strategy can't offer.

For a mid-size organization this typically lands at 10-50 accounts (fewer, coarser OUs); a large enterprise runs 100-500+. AWS Control Tower (or the newer Terraform-native "Account Factory for Terraform," AFT) is the common automation layer for account creation itself — genuinely out of scope for a Terraform-content-only series, but worth knowing exists as the tool that provisions the accounts this chapter's directory structure then targets.

Cross-Account Access — the Hub-and-Spoke assume_role Pattern#

With separate AWS accounts, a Terraform run needs a way to act in the target account without holding that account's own long-lived credentials directly — the standard pattern is a "hub" identity (a CI role, or a human's SSO identity) that assumes a "spoke" role scoped to exactly one target account.

provider "aws" {
  alias  = "prod"
  region = "us-east-1"

  assume_role {
    role_arn     = "arn:aws:iam::444455556666:role/terraform-prod-deployer"
    session_name = "terraform-ci-${terraform.workspace}"
  }
}

The target account's terraform-prod-deployer role's trust policy is the actual security control — it names exactly which hub identity is permitted to assume it, and the role's own IAM policy scopes exactly what that session can do once assumed. No long-lived credential for the prod account ever needs to exist outside that account itself; the CI hub only ever holds credentials for assuming into other accounts, each session expiring automatically.

Multi-Cloud Directory Structure — Keeping Clouds Cleanly Separated#

When a second cloud provider enters the picture — GCP for disaster recovery, per this chapter's closing scenario — the same discipline that separates environments applies again, one level up: never mix two clouds' resources inside the same state file, and keep cloud-specific configuration in its own directory branch rather than interleaved with the primary cloud's.

Splitting at the cloud level, above environment, keeps each cloud's account/project structure, IAM model, and provider configuration fully independent — an AWS Organizational Unit and a GCP Folder are conceptually similar but never actually related, and forcing them into a shared directory branch (interleaving aws-prod/ next to gcp-prod/ as siblings under one prod/ folder, for instance) tends to produce configuration that quietly assumes cross-cloud symmetry that doesn't actually exist.

HCP Terraform Stacks — a Native Alternative to Terragrunt#

Stacks, HashiCorp's own answer to the DRY-multi-deployment problem, reached broader general availability through 2026 (including monorepo support) — a native, first-party alternative to Terragrunt worth knowing about even for a team not currently on HCP Terraform, since it represents where the ecosystem is heading.

A Stack defines reusable "components" (similar in spirit to modules) and "deployments" (similar in spirit to Terragrunt's per-environment leaf configuration), with HCL-native orchestration across them instead of a separate wrapper tool — the goal being Terragrunt's DRY, multi-environment ergonomics without introducing a second tool and syntax dialect on top of Terraform itself.

Naming Conventions Across the Tree#

A consistent naming and tagging scheme, applied identically across every environment/account/cloud directory, is what makes the structure this chapter builds actually navigable and query-able at scale — a different naming habit per environment quietly undermines every other practice in this chapter.

locals {
  name_prefix = "${var.org}-${var.environment}-${var.service}"
  # e.g. "meridian-prod-checkout"

  common_tags = {
    Organization = var.org
    Environment  = var.environment
    Service      = var.service
    ManagedBy    = "terraform"
    Repository   = "infrastructure-live"
  }
}

resource "aws_db_instance" "this" {
  identifier = "${local.name_prefix}-db"
  tags       = local.common_tags
}

Choosing Your Structure — a Decision Framework#

Pulling this entire chapter into one decision sequence:

Worked Scenario: Migrating Off Workspaces Into Directory-Per-Environment#

The platform team's earliest Terraform setup used CLI workspaces for dev/staging/prod, driven by terraform.workspace conditionals throughout. As checkout-service grew genuinely divergent prod-only requirements (a read replica, WAF rules, a different backup schedule), the conditional logic sprawled across a dozen terraform.workspace == "prod" ? ... : ... expressions scattered through the configuration, increasingly hard for a new team member to reconstruct "what does prod actually look like" from.

The migration itself used the state-splitting mechanics from Part 2 — each CLI workspace's state was terraform state pulled independently, and each became the seed state for a new, dedicated environments/<name>/ directory with its own explicit backend key. No resources were destroyed or recreated; only the organizing structure around already-applied infrastructure changed. Six months post- migration, environments/prod/main.tf reads as a complete, honest description of production on its own, with zero conditional expressions — exactly the payoff this chapter's second section promised.

Worked Scenario: Standing Up a Second Cloud for Disaster Recovery#

Following an executive mandate for cross-cloud disaster recovery (not just cross-region), the team stood up a GCP replica of checkout-service's critical path — Cloud SQL as the replicated read target, GKE as the standby compute layer — under a brand-new gcp/ directory branch, exactly per this chapter's multi-cloud structure guidance. The team deliberately did not try to reuse the AWS-side database or eks-node-group modules with cloud-specific conditionals bolted in — a lesson learned directly from Part 3's "god module" anti-pattern — and instead wrote genuinely separate gcp-cloudsql and gke-node-pool modules, accepting real duplication of structure (both provision "a managed database" and "a Kubernetes node pool" conceptually) in exchange for each module staying simple, provider-native, and easy to reason about on its own.

Worked Scenario: the tfvars Override That Silently Hit the Wrong Account#

An engineer running a routine catalog-service change in staging used a shell alias that, on this particular machine, still pointed AWS_PROFILE at the prod account from an unrelated task earlier that day. The staging/ directory's own explicit backend.tf correctly targeted the staging state bucket — Part 2's directory-explicit-backend discipline worked exactly as designed — but the provider credentials used to actually execute the plan came from the shell's ambient AWS profile, which the directory structure has no way to enforce on its own.

The plan showed changes against real prod resources (visible instance IDs, ARNs the engineer immediately recognized as wrong), and the review discipline from Part 1 caught it before apply — but the near-miss prompted a structural fix: every environment directory's own generated backend/provider configuration (via Terragrunt, adopted shortly after this incident) now also asserts the AWS account ID it expects to be operating against, failing plan outright with a clear error if the assumed role's actual account ID doesn't match what that directory declares.

data "aws_caller_identity" "current" {}

resource "terraform_data" "account_guard" {
  lifecycle {
    precondition {
      condition     = data.aws_caller_identity.current.account_id == "222233334444"
      error_message = "This directory targets the staging account (222233334444) — got ${data.aws_caller_identity.current.account_id}."
    }
  }
}

Part 4 CLI Cheat Sheet#

Common Mistakes and Interview Traps#

Worked Practice Problems#

Problem 1: A team runs three near-identical, disposable load-test environments, created and destroyed multiple times per week, alongside a permanent, structurally distinct production environment. What structure fits each, and why shouldn't they use the same mechanism for both?

Answer: The load-test environments fit CLI workspaces well — they're genuinely ephemeral and structurally identical to each other, and creating/destroying a workspace is fast with no new directory scaffolding needed each time. Production should be its own explicit directory (and, following this chapter's stronger guidance, its own AWS account) — it's long-lived, structurally distinct from anything else, and the stakes of an accidental wrong-target apply are high enough to want the structural, not just conventional, protection a dedicated directory (and account) provides. Using CLI workspaces for both would put production one terraform workspace select mistake away from an accidental apply; using a full directory-per-instance structure for the load-test environments would mean constant directory scaffolding for something meant to be quick and disposable.

Problem 2: A platform team has 4 engineers, one shared set of infrastructure modules, and 3 services that change together frequently during releases. Should they use a monorepo or split into per-service repositories, and what would change their answer?

Answer: A monorepo fits this team well — small, tightly-collaborating team, modules and consumers that change together often, and no genuine need yet for per-repository access control. The concrete condition that would change this answer: the team growing into several genuinely autonomous groups each owning one service independently, needing separate release cadences and separate repository-level permissions — at that point, the polyrepo factors from this chapter's comparison table start outweighing the monorepo's current cross-cutting-refactor convenience.

Problem 3: An engineer's terraform plan for the staging directory shows changes against resources with ARNs the engineer doesn't recognize from staging at all. The directory's backend.tf correctly points at the staging state bucket. What's the most likely explanation, and what structural fix prevents this class of mistake going forward?

Answer: The backend (state location) is correct, but the provider credentials actually being used during this run are pointed at a different account than the directory implies — an ambient AWS_PROFILE/session left over from unrelated work, exactly per this chapter's tfvars-override scenario. The state backend being correct doesn't guarantee the provider's actual target account matches; the structural fix is an explicit account-ID (or project-ID, on GCP) assertion via a precondition (or equivalent guard) that fails the plan outright the moment the assumed identity's real account doesn't match what that directory declares, rather than relying on directory naming and human attention alone.

Summary and What's Next#

Repository and directory structure is where Part 2's state-splitting and Part 3's module design actually meet the real world: CLI workspaces for genuinely ephemeral, identical environments; explicit directories for anything permanent and structurally distinct; a DRY tool (Terragrunt today, Stacks increasingly as it matures) once duplication pain across those directories is real; and, at genuine scale, account-per- environment and cloud-level directory separation as the actual enforcement mechanisms behind what a folder name only implies. None of this is Terraform syntax — it's the organizational discipline that keeps a mistake in dev from ever having a structural path to reach prod, which is worth more than almost any single language feature this series covers.

Part 5 returns to configuration-language depth: providers beyond the single-region default this chapter's examples assumed, data sources for reading infrastructure Terraform doesn't manage, and provisioners' narrow, correctly-scoped role — all of it informed by the directory and account structure this chapter just established as the ground everything else in the series stands on.