Drift Detection, Import & Refactoring Existing Infrastructure

Assumes you're comfortable with Part 1's refresh step and Part 2's state manipulation commands — this chapter is the deep, dedicated treatment of everything this series has referenced but not yet fully shown: moved, import, and the newer removed block.

Table of Contents#

1. What Drift Actually Is, and Why It's Inevitable
2. Detecting Drift: -refresh-only
3. Reconciling Drift — Three Legitimate Paths
4. Scheduled Drift Detection in CI
5. Third-Party Drift Tooling
6. Preventing Drift at the Source
7. The moved Block — Refactoring Without Destroying
8. moved Blocks for count-to-for_each Migrations
9. moved Blocks Across Module Boundaries
10. The removed Block — Letting Go Without Destroying
11. terraform import — the Imperative Original
12. The import Block — Declarative, Reviewable Import
13. Auto-Generating Configuration From an Import
14. The Import Workflow, End to End
15. Importing Into a Module
16. Bulk Import for a Large, Previously Unmanaged Estate
17. taint and -replace — Forcing a Clean Recreate
18. Drift Severity Across Environments
19. A Drift Response Runbook
20. Drift With No Human Involved: Provider-Side Default Changes
21. Worked Scenario: a Manual Console Change Nobody Told Terraform About
22. Worked Scenario: Importing Three Years of Hand-Built Infrastructure
23. Worked Scenario: a Module Refactor That Nearly Destroyed Production
24. Part 6 CLI Cheat Sheet
25. Common Mistakes and Interview Traps
26. Worked Practice Problems
27. Summary and What's Next

What Drift Actually Is, and Why It's Inevitable#

Drift is any gap between what Terraform's state believes is true and what's actually true in the real infrastructure — caused by a manual console change, an emergency kubectl/CLI fix during an incident, another tool managing the same resource, or the cloud provider itself changing a default. No team, however disciplined, eliminates drift entirely — incident response routinely requires a fast manual fix under pressure, well before there's time to write and review a proper Terraform change. The goal isn't zero drift; it's detecting drift quickly and reconciling it deliberately, rather than discovering it as a confusing surprise diff weeks later.

Detecting Drift: -refresh-only#

terraform plan -refresh-only (Terraform 0.15.4+, the modern replacement for the old standalone terraform refresh command) updates Terraform's understanding of current reality and shows exactly what changed, without touching real infrastructure and without proposing any remediation — pure detection, fully separated from any remediation decision.

terraform plan -refresh-only

This output is exactly what a "someone widened an ingress rule to the entire internet during an incident and never reverted it" drift incident looks like — a real, security-relevant example of why detection needs to happen faster than "whenever the next unrelated plan happens to run." -detailed-exitcode makes this usable in automation: exit code 0 means no drift, 2 means drift detected (a real, actionable signal), and 1 means an actual error — a CI job or alert can branch on this directly, which the next section builds on.

Reconciling Drift — Three Legitimate Paths#

Once drift is detected, exactly three responses are legitimate — which one is correct depends entirely on whether the drifted value should actually become the new desired state, or whether Terraform's original configuration was right and the manual change needs to be undone.

The middle path deserves care: terraform apply -refresh-only updates state to match reality but doesn't touch .tf files — if the manual change was wrong and the .tf files still declare the original, correct value, the very next ordinary apply will revert it, which is the intended two-step dance (accept the drift into state first, so the next plan's diff is clean and obviously "revert to declared config" rather than a confusing three-way disagreement between old state, new reality, and declared config).

Scheduled Drift Detection in CI#

Building directly on -detailed-exitcode, most mature teams run a scheduled (nightly, or more frequent for security-sensitive resources) drift-check job entirely separate from the PR-driven pipeline Part 8 covers in depth:

# .github/workflows/drift-check.yml (illustrative — Part 8 covers full CI/CD)
on:
  schedule:
    - cron: "0 * * * *"   # hourly

jobs:
  drift-check:
    steps:
      - run: |
          terraform plan -refresh-only -detailed-exitcode
          if [ $? -eq 2 ]; then
            # Post to Slack, page on-call, whatever this org's alerting policy is
            echo "Drift detected"
          fi

Third-Party Drift Tooling#

Beyond the native -refresh-only mechanism, a small ecosystem of dedicated drift tools exists for teams wanting continuous, cross-account drift visibility beyond what a single configuration's own scheduled plan covers — driftctl (created by Snyk, now in maintenance mode but still functional) was the best-known open-source option; newer entrants (Firefly, and drift-detection features built into commercial run-pipeline platforms like Spacelift and env0) have picked up active development since.

Preventing Drift at the Source#

Detecting and reconciling drift well is necessary, but the cheapest drift to handle is the drift that never happens — restricting who and what can make manual changes to Terraform-managed infrastructure at all is a genuine prevention layer, not just a detection-and-cleanup story.

# An SCP (Service Control Policy) at the AWS Organization level (Part 4) —
# denies manual console/CLI changes to resources tagged as Terraform-managed,
# EXCEPT via the specific role Terraform's CI pipeline itself assumes.
{
  "Effect": "Deny",
  "Action": ["ec2:*", "rds:*", "s3:*"],
  "Resource": "*",
  "Condition": {
    "StringEquals": { "aws:ResourceTag/ManagedBy": "terraform" },
    "StringNotEquals": { "aws:PrincipalArn": "arn:aws:iam::444455556666:role/terraform-prod-deployer" }
  }
}

This is a real tradeoff, not a free win — it also blocks the legitimate emergency console fix this chapter's first worked scenario depended on, which is exactly why most teams implement it with a documented break-glass exception process (a separate, audited, time-boxed role that can bypass the restriction, used deliberately and logged) rather than an absolute, no-exceptions block. The goal is raising the bar from "any engineer with console access can silently drift a resource" to "drifting a resource requires a deliberate, audited, logged decision to use the break-glass path" — prevention as friction, not prevention as an impossible wall.

The moved Block — Refactoring Without Destroying#

A moved block declaratively tells Terraform "this resource's address changed, but it's the same real object" — the fix for exactly the "renamed local name → destroy and recreate" trap Part 1 and Part 3 both flagged without yet showing the mechanics.

moved {
  from = aws_instance.web
  to   = aws_instance.app
}

Unlike terraform state mv (Part 2), a moved block lives in version-controlled configuration — it's reviewable in a PR, applies consistently across every environment/workspace that runs this configuration, and runs automatically as part of a normal plan/apply, with no separate manual step anyone could forget to run against one environment. moved blocks are safe to leave in configuration indefinitely (Terraform simply no-ops if the "from" address no longer exists in state), though most teams remove them after confirming every environment has applied the move at least once, to keep the configuration from accumulating stale historical markers forever.

moved Blocks for count-to-for_each Migrations#

Part 1's count-vs-for_each scenario showed the problem a count-indexed resource creates when an item is removed from the middle of the list; moved blocks are the actual migration mechanism for converting an existing count-based resource to for_each without destroying anything already applied.

# Before: count = 3, addressed by index
# resource "aws_db_instance" "replica" {
#   count = 3
#   ...
# }

# After: for_each, addressed by a stable key
resource "aws_db_instance" "replica" {
  for_each = toset(["analytics", "reporting", "backup"])
  # ...
}

moved {
  from = aws_db_instance.replica[0]
  to   = aws_db_instance.replica["analytics"]
}
moved {
  from = aws_db_instance.replica[1]
  to   = aws_db_instance.replica["reporting"]
}
moved {
  from = aws_db_instance.replica[2]
  to   = aws_db_instance.replica["backup"]
}

One moved block per existing index, mapping it explicitly to its new key — this is precisely the mechanism Part 1's own worked scenario ("count vs. for_each, and getting bitten by it") referenced as the eventual fix the team applied, shown here in full.

moved Blocks Across Module Boundaries#

The same mechanism handles Part 3's module-extraction scenario — moving a resource that used to live directly in the root module into a newly-created child module, without destroying it:

moved {
  from = aws_vpc.main
  to   = module.network.aws_vpc.main
}

And, less commonly but just as validly, moving a resource between two different child modules, or up out of a module back into the root — the from/to pair works across any address change, module boundaries included, as long as the underlying resource type is identical on both sides.

The removed Block — Letting Go Without Destroying#

removed (Terraform 1.7+) declaratively stops Terraform from managing a resource — with an explicit, reviewable choice about whether the real infrastructure should be destroyed or left alone.

removed {
  from = aws_s3_bucket.legacy_logs

  lifecycle {
    destroy = false
  }
}

With destroy = false, Terraform drops the resource from state entirely, leaving the real bucket untouched — the declarative, version-controlled, PR-reviewable equivalent of terraform state rm (Part 2), with the same "leaves real infrastructure orphaned but unmanaged" effect, just expressed as code instead of a one-off local command. Omitting lifecycle { destroy = false } (or setting destroy = true) instead makes removed behave like deleting the resource block outright — a genuine destroy, planned and reviewable exactly like any other planned deletion.

terraform import — the Imperative Original#

The original terraform import CLI command brings a real, existing resource under Terraform's state management — but only the state side; you still had to hand-write the matching .tf configuration block yourself, and getting that configuration to exactly match the imported state (with zero drift on the very next plan) was, and still is with this form, a real, tedious exercise in trial and error.

terraform import aws_s3_bucket.legacy_logs meridian-legacy-logs-bucket

# You must ALSO write this yourself, matching the real resource's
# actual configuration closely enough that the next plan shows no changes:
resource "aws_s3_bucket" "legacy_logs" {
  bucket = "meridian-legacy-logs-bucket"
  # ... every other real attribute, guessed/confirmed by hand
}

This imperative form is still fully supported and still the fastest option for a genuine one-off, single- resource import — but it's a local, unreviewed command (the same category of concern Part 2 raised about direct state manipulation), and the config-writing burden it leaves entirely on you is exactly what the next section's declarative form improves on.

The import Block — Declarative, Reviewable Import#

The import block (Terraform 1.5+) moves the same operation into version-controlled configuration, plannable and reviewable exactly like any other change — the modern default over the bare CLI command for anything beyond a quick, disposable one-off.

import {
  to = aws_s3_bucket.legacy_logs
  id = "meridian-legacy-logs-bucket"
}

resource "aws_s3_bucket" "legacy_logs" {
  bucket = "meridian-legacy-logs-bucket"
  # ...
}

terraform plan with an import block present shows the import as an explicit planned action — reviewable in a PR before it ever touches state, exactly the same review discipline Part 1 established for every other kind of change:

Multiple import blocks can exist in one configuration, and — as of newer Terraform releases — import blocks now work inside modules, not just the root configuration, closing an early gap this feature launched with.

Auto-Generating Configuration From an Import#

Pairing an import block with -generate-config-out solves the original CLI command's biggest pain point directly: Terraform inspects the real resource and writes matching HCL for you, rather than leaving you to hand-guess every attribute.

terraform plan -generate-config-out=generated_resources.tf

# generated_resources.tf — written BY Terraform, from the import block(s)
# in your configuration and the real resource's current attributes
resource "aws_s3_bucket" "legacy_logs" {
  bucket = "meridian-legacy-logs-bucket"
  # ... every real attribute Terraform could discover
}

This is a genuinely large improvement over hand-writing import configuration, and still explicitly marked experimental — its output format may change across releases, and it has real gaps: it doesn't work for resources targeted with count/for_each that don't already exist in configuration, and it doesn't reach into module-internal resources. Treat the generated file as a strong first draft, not a final answer — review it line by line, confirm the very next plan shows zero changes, and only then commit it.

The Import Workflow, End to End#

Pulling the last three sections into one sequence — the actual, repeatable process a platform team runs for every real import, not just the individual mechanics in isolation.

This chapter's caption: the "0 to add, 0 to change, 0 to destroy" confirmation step is the actual gate — skipping straight from generated config to apply without confirming a clean plan first is how a subtly wrong generated attribute silently becomes a real, unintended change instead of a faithful import.

Importing Into a Module#

Bringing a pre-existing resource into a module (rather than the root configuration) combines this chapter's import mechanics with Part 3's module addressing:

import {
  to = module.network.aws_vpc.main
  id = "vpc-0a1b2c3d"
}

This is exactly the mechanic Part 2's "recovering from a corrupted state" scenario referenced — a resource that genuinely exists in real infrastructure but has no state record needs import to bring it back under management, and if that resource conceptually belongs inside an already-extracted module (Part 3), the import target is the module-prefixed address, not a bare resource address.

Bulk Import for a Large, Previously Unmanaged Estate#

A single resource's import block is straightforward; bringing an entire pre-existing account under management (this chapter's later "three years of hand-built infrastructure" scenario, at real scale) needs a repeatable, scriptable approach rather than hand-writing dozens or hundreds of import blocks one at a time.

# Enumerate real resources of a given type via the cloud provider's own CLI,
# generate one import block per resource programmatically
aws s3api list-buckets --query 'Buckets[].Name' --output text | tr '\t' '\n' | while read -r bucket; do
  cat >> generated_imports.tf <<EOF
import {
  to = aws_s3_bucket.${bucket//[-.]/_}
  id = "${bucket}"
}
EOF
done

terraform plan -generate-config-out=generated_resources.tf
# Review, resource by resource, exactly as the manual case requires —
# bulk generation does not skip the review step, it only removes the
# tedium of hand-typing each import block's boilerplate

taint and -replace — Forcing a Clean Recreate#

Sometimes the correct reconciliation for drift isn't accepting or reverting a value — it's forcing a clean destroy-and-recreate of a resource that's degraded or misconfigured in a way no attribute-level fix addresses.

# Modern, apply-scoped form — the current recommended approach
terraform apply -replace="aws_instance.worker"

# Older, persistent form — marks the resource tainted in STATE until
# the next apply, which can surprise a later, unrelated apply
terraform taint aws_instance.worker

-replace is scoped to one specific apply invocation — it doesn't persist any marker in state the way the older taint command does, which is exactly why it's the currently recommended approach: a forgotten taint sitting in state can cause a resource to be unexpectedly replaced by a completely unrelated later apply that nobody remembered the taint was even there for.

Drift Severity Across Environments#

Not every environment's drift deserves the same response speed — Part 4's directory-per-environment structure makes it natural to tune drift-check frequency and alerting severity per environment, matching each one's real stakes rather than treating dev and prod identically.

This tuning is exactly why Part 4's account-per-environment and directory-per-environment structure pays off again here — each environment's drift-check job runs against its own state/backend independently, so dev's higher noise tolerance never has to be balanced against prod's low tolerance in one shared configuration or one shared alert channel.

A Drift Response Runbook#

Pulling this chapter's mechanisms into one operational checklist for a platform team's on-call rotation:

• Confirm the drift with terraform plan -refresh-only — read the full diff before deciding anything
• Identify why the drift happened — an incident fix, a different tool, a provider default change (feeds directly into whether this specific drift should recur, or was truly one-off)
• Decide which of the three reconciliation paths applies (config update, revert, or ignore/remove)
• If reverting: apply -refresh-only first, then a normal apply, confirming the plan matches expectations at each step
• If the manual change should become permanent: update .tf files, open a normal reviewed PR — never skip review just because the change already exists live
• If the resource shouldn't be Terraform-managed going forward: use removed { destroy = false }, not an ad hoc state rm
• Document the drift and its resolution somewhere durable — a recurring drift source (the same security group, repeatedly) is itself a signal worth escalating, not just repeatedly reconciling
• If the drift traced back to a genuine emergency console change, confirm the break-glass process (this chapter's prevention section) was actually used, rather than an unrestricted credential
• Confirm the environment's drift-check cadence still matches its actual severity tier — a repeated incident in one environment may justify tightening that environment's own detection frequency
• Close the loop with whoever made the original change, even when it was correct — a quick heads-up keeps the next drift from being an equal surprise to the runbook's own on-call rotation

Drift With No Human Involved: Provider-Side Default Changes#

Not all drift traces back to a person — a provider version bump (Part 1's module-upgrade scenario, applied here to provider versions directly) can change a resource's default value between API versions, producing drift with zero manual changes anywhere.

A concrete, real pattern: a cloud provider's API introduces a new default for a previously-optional field (similar in spirit to the map_public_ip_on_launch default change Part 3's module scenario hit), and the next provider version upgrade that picks up the new API behavior shows every existing resource of that type as drifted — not because anything about the real resource changed, but because the provider's own interpretation of "what does this field's absence mean" changed underneath it.

Worked Scenario: a Manual Console Change Nobody Told Terraform About#

During a genuine production incident, an on-call engineer widened checkout-service's security group to allow traffic from a debugging tool's IP range, directly in the AWS console, to unblock an active outage — the right call under the circumstances, but never followed up afterward. The platform team's scheduled -refresh-only drift check (this chapter's dedicated CI job) caught the change the following night, well before it would have otherwise surfaced in the next unrelated feature PR's plan.

The team's response followed the drift runbook directly: confirmed the change was a genuine incident stopgap (not an intentional permanent widening), ran apply -refresh-only to accept the current state, then a normal apply to revert it back to the originally-declared, narrower rule — closing the gap within a day of the incident rather than leaving an overly-permissive rule live indefinitely until someone happened to notice.

Worked Scenario: Importing Three Years of Hand-Built Infrastructure#

Before the platform team existed, inventory-service's original infrastructure was built entirely by hand in the AWS console over roughly three years — no Terraform, no consistent tagging, real configuration drift even against itself across environments. Bringing it under management used import blocks paired with -generate-config-out, resource by resource, starting with the lowest-risk, easiest-to-verify pieces (S3 buckets, IAM roles) before working up to the database and its security groups.

Each generated configuration file was reviewed by two engineers before being committed — not a rubber stamp, because the generated HCL faithfully captured real inconsistencies (three years of ad hoc console changes had left slightly different tagging on nearly every resource) that needed cleaning up, not blind preservation. The full import took several weeks, deliberately paced rather than rushed, with a terraform plan showing zero changes confirmed after every single resource's import before moving to the next — exactly the discipline this chapter's import-block section recommends, applied at real scale.

Worked Scenario: a Module Refactor That Nearly Destroyed Production#

A well-intentioned refactor renamed several internal resource names inside the database module (Part 3) for clarity — aws_db_instance.this to aws_db_instance.primary, in preparation for adding a future read replica — without adding matching moved blocks, on the mistaken assumption that "it's just an internal module implementation detail, callers won't notice." The very next plan against every environment calling this module showed 1 to destroy, 1 to add for every single database the module managed — the internal rename was, from Terraform's perspective, exactly as consequential as any other address change, module- internal or not.

The plan-review discipline (Part 1) caught it before any apply reached production, and the fix was exactly this chapter's moved block mechanism, applied retroactively:

moved {
  from = aws_db_instance.this
  to   = aws_db_instance.primary
}

Part 6 CLI Cheat Sheet#

Common Mistakes and Interview Traps#

Worked Practice Problems#

Problem 1: A terraform plan -refresh-only shows a security group rule changed from a narrow CIDR block to 0.0.0.0/0. What are the two possible correct responses, and what determines which one is right?

Answer: Either update the .tf configuration to declare 0.0.0.0/0 (if the change was legitimate and should persist) or run apply -refresh-only followed by a normal apply to revert it back to the originally declared narrow CIDR (if the change was a mistake or an incident stopgap that was never meant to be permanent). Which is correct depends entirely on investigating why the change happened — the drift itself carries no information about intent, only about the fact that a disagreement exists.

Problem 2: An engineer renames aws_instance.web to aws_instance.frontend in one PR, with no moved block, reasoning "the plan will just show it as a rename, not a destroy." What actually happens, and why is that reasoning wrong?

Answer: The plan shows 1 to add (aws_instance.frontend), 1 to destroy (aws_instance.web) — Terraform has no concept of "rename" as a first-class plan action; it only ever sees two distinct addresses, one gone and one new, and its default inference is destroy-then-create. The reasoning is wrong because it assumes Terraform reasons about semantic similarity between two resources, when it actually reasons purely about address identity — a moved block is what supplies the "these are actually the same object" information Terraform has no other way to infer.

Problem 3: A team runs terraform import aws_s3_bucket.legacy mybucket successfully, then finds that their very next terraform plan still shows several changes rather than a clean "no changes." What does this indicate, and what's the correct next step — re-running the import?

Answer: This indicates the hand-written (or -generate-config-out-generated but not fully verified) .tf configuration for aws_s3_bucket.legacy doesn't yet exactly match the real bucket's current attributes — the import itself succeeded (state now tracks the resource), but the configuration is still out of sync with it. Re-running the import doesn't fix this — the import already succeeded once and doesn't need repeating. The correct next step is reading the plan diff carefully and adjusting the .tf configuration's arguments to match what the diff shows, repeating until plan shows zero changes, exactly the same iterative process the "importing three years of hand-built infrastructure" scenario describes at real scale.

Problem 4: An organization implements an SCP denying manual changes to Terraform-tagged resources, with no break-glass exception process. Six months later, a production incident requires an emergency security group change, and the on-call engineer's console attempt is denied outright, with no documented alternative. What went wrong in the policy's design, and what should have been included from the start?

Answer: The SCP achieved pure prevention with no accounted-for exception path — exactly the failure mode this chapter's prevention section warns against, treating prevention as "an impossible wall" instead of "a deliberate, audited friction." A production incident occasionally does need a fast manual fix before there's time to write and review a proper Terraform change (this chapter's very first worked scenario depends on this being possible); the fix is a documented, separate, time-boxed break-glass role that can bypass the SCP, used deliberately and logged, paired with the expectation that any break-glass change gets caught and reconciled by the next scheduled -refresh-only drift check — prevention and detection working together, not prevention alone with no escape valve for genuine emergencies.

Summary and What's Next#

Drift is inevitable, not a sign of process failure — the discipline that matters is detecting it on a deliberate schedule (-refresh-only, scripted with -detailed-exitcode) rather than waiting for it to surface as a confusing surprise, and reconciling it through one of exactly three legitimate paths rather than guessing. moved and removed blocks turn what used to require unreviewed, local state commands (Part 2) into version-controlled, PR-reviewable, consistently-applied configuration — and import blocks, especially paired with -generate-config-out, do the same for bringing previously-unmanaged infrastructure under Terraform's control. Every mechanism in this chapter shares one theme: making an operation that used to be imperative, local, and easy to forget into something declarative, reviewable, and consistent across every environment that runs the configuration.

Part 7 shifts from reactive (detecting and fixing problems after they exist) to proactive: testing Terraform configuration and modules before a bad change ever reaches a real environment at all — static analysis with tflint and checkov, the native terraform test framework, and Terratest for genuine integration testing against real, ephemeral infrastructure. Several of that chapter's test cases exist specifically to catch the exact regressions this chapter's worked scenarios showed slipping past a human reviewer, turning a lesson learned the hard way into an automated check nobody has to remember to run by hand.