Companion question bank for the 3-part tutorial series in this folder: 01-dr-fundamentals-and-strategies.md, 02-dr-testing-and-runbooks.md, 03-real-world-disasters-and-case-studies.md.

Answers are short and plain — expand out loud using the diagrams and worked examples in the tutorials.

---

Part 1 Questions: DR Fundamentals & Strategies

1. What kinds of events should a DR plan actually cover, beyond physical disasters?#

Infrastructure/region failure, human error (an accidental deletion, a bad deploy), security incidents, software bugs silently corrupting data, and genuine physical disasters. In practice, human error and process failures happen far more often than fires or earthquakes.

2. Distinguish High Availability, Disaster Recovery, and Business Continuity.#

HA proactively prevents downtime (redundancy, failover). DR reactively recovers once something has already failed badly. Business Continuity is the broadest scope — keeping the entire business functioning (people, process, communication), not just technical systems. DR is a subset of BC.

3. Define RPO and RTO in plain terms.#

RPO looks backward: how much data, created before a disaster, can we afford to lose? RTO looks forward: how long can we afford to be down while recovering? Both are deliberate business decisions, not purely technical ones.

4. Why does tightening RPO/RTO cost non-linearly more as you approach near-zero?#

Going from hours to an hour might mean automating a manual process. Going from an hour to a few seconds might mean running fully duplicate, continuously-synchronized infrastructure in a second region — the cost curve gets dramatically steeper near the top, not linear.

5. Name the four classic DR strategies, from cheapest/slowest to most expensive/fastest.#

Backup & Restore, Pilot Light, Warm Standby, Multi-Site Active-Active.

6. Describe the Pilot Light strategy with an analogy.#

A small, continuously-running core (typically just the database, kept in sync) in the DR region — like a gas stove's pilot light: minimal fuel used continuously, but it lets the stove ignite to full power almost instantly when actually needed, instead of starting a fire from scratch.

7. What's the key difference between Pilot Light and Warm Standby?#

Pilot Light has only the data layer running continuously (app servers are off or minimal). Warm Standby has a full, scaled-down but FUNCTIONAL copy of everything already running, just not receiving real traffic — recovery is mostly scaling up and switching traffic, not provisioning from scratch.

8. Why can Multi-Site Active-Active achieve near-zero RTO?#

There's no real "switch" — both regions are already fully live and serving real traffic simultaneously, so failover is just routing away from the failed region, which the other region was already handling anyway.

9. What genuine complexity does Multi-Site Active-Active reintroduce?#

The exact CAP theorem consistency challenges from the Reliability & Architecture Patterns series — bidirectional replication between two simultaneously-active regions creates real write-conflict scenarios needing a deliberate resolution strategy, not a simple "run it twice" solution.

10. How should a team actually choose a DR strategy for a given system?#

Based on the real, business-driven RPO/RTO requirement for THAT specific system — not defaulting every system to the most expensive strategy "to be safe," exactly like choosing an SLO deliberately rather than always aiming for 100%.

---

Part 2 Questions: DR Testing & Runbooks

11. Why is an untested DR plan described as "a hypothesis, not a verified capability"?#

Real, documented failures happen where a runbook references a decommissioned tool, an unreachable person, or a forgotten dependency — none of which show up just from reading the document. Only actual testing surfaces these gaps, safely, before a real disaster does.

12. Walk through the DR testing maturity ladder.#

Tabletop exercise (talk through it, no systems touched) → walkthrough/simulation (real steps, non-production environment) → partial/component failover test (one real piece, in production, controlled) → full failover test (the entire system, for real) → chaos engineering/unannounced drills (no advance warning).

13. Why is a tabletop exercise so valuable despite touching zero real systems?#

It cheaply and quickly surfaces PROCESS and OWNERSHIP gaps — like unclear declaration authority or a runbook referencing a tool nobody uses anymore — often revealing serious gaps in minutes that would otherwise only surface during a real crisis.

14. Why is testing failback just as important as testing failover?#

Returning to the original primary is a genuinely separate, sometimes more complex operation — especially reconciling data written to the DR environment while it was live — that can fail even when the failover itself worked perfectly.

15. What should a good DR runbook always include?#

Exact, copy-pasteable commands (not vague descriptions), explicit declaration authority, explicit dependency ordering, explicit verification steps at each stage, and a separate failback procedure — kept current through regular testing.

16. Why is hosting a status page on the same infrastructure being tested for failure a real, common gap?#

The exact tool needed to communicate about the disaster becomes a casualty of it — the same "your monitoring shouldn't depend on the thing it's monitoring" principle applied to communication tooling specifically.

17. What is the "dependency mapping problem" in DR planning?#

A DR plan that only considers one headline service, without mapping its full dependency chain (internal services it depends on, external providers), can produce a technically "successful" failover of that one service that still doesn't actually work, because a dependency wasn't also failed over.

18. How does "people are part of the system too" connect to redundancy principles from earlier in the course?#

A DR plan relying on one specific person's tribal knowledge is exactly as fragile as an un-redundant server — the fix is the same principle applied to knowledge instead of infrastructure: cross-train multiple people, validated through real testing.

19. How often should different levels of DR testing happen?#

Tabletop exercises: quarterly (cheap enough to do often). Partial failover tests: quarterly to semi-annually. Full failover tests: at least annually, more often for the most critical systems. Also: re-test after any significant architecture change, since a plan tested against last year's architecture may not reflect this year's reality.

---

Part 3 Questions: Real-World Disasters & Case Studies

20. Summarize GitLab's 2017 incident and its main lesson.#

An engineer accidentally deleted production data while troubleshooting replication lag on the wrong server. Of five believed-redundant backup mechanisms, four were broken or not actually running — recovery relied on a lucky manual snapshot, with real permanent data loss for the gap since. Main lesson: untested backups provide false confidence, not real protection.

21. Summarize the 2017 AWS S3 us-east-1 outage and its main lesson.#

A mistyped command removed far more servers than intended from a core S3 subsystem, taking it down entirely — an enormous number of other services and customer applications across the internet had hidden, unmapped dependencies on that exact subsystem and were affected. Main lesson: hidden dependencies are real and severely underestimated, and a single region CAN experience a genuinely severe, wide-ranging outage.

22. Summarize the 2021 Facebook/Meta BGP outage and its main lesson.#

A maintenance command accidentally withdrew all of the company's BGP route announcements, making their infrastructure unreachable globally — including their own internal DNS and tools, and reportedly even physical badge-reader access, since those also depended on the now-unreachable infrastructure. Main lesson: recovery mechanisms must never depend entirely on the exact systems they're meant to recover — "break glass" independent access matters.

23. Why is Knight Capital relevant to a DR/business-continuity discussion specifically?#

It shows a purely software/deployment-process failure — with zero infrastructure or hardware failure involved — can be just as catastrophic to business continuity as any physical disaster, causing ~$440M in losses in about 45 minutes and threatening the company's survival.

24. What's the common thread across all four case studies?#

None were caused by a fire, flood, or earthquake — every one was human/process error or a hidden dependency. In every case, the severity was made dramatically worse by a genuine preparedness gap (untested backups, unmapped dependencies, no independent recovery path), not by the initial mistake alone.

25. What does a genuine DR culture look like, as opposed to a checkbox exercise?#

DR planning is everyone's concern, not a document written once and filed away. Testing gaps are treated as valuable findings (like a chaos experiment disproving a hypothesis), not embarrassing failures to hide. DR readiness is tracked and reported like an SLO, with real, sustained leadership support for ongoing testing.

26. Name the sections of a complete DR plan template.#

Business impact analysis (with explicit RPO/RTO), chosen DR strategy and why, full dependency map, runbook (with trigger criteria, declaration authority, exact steps, verification, and separate failback), communication plan, testing schedule, and clear ownership.

---

Quick-Fire / Rapid Recall#