# Azure Cloud Architecture — Part 14: Business Continuity: Backup, DR & Migration

> **Series:** Azure Cloud Architecture (14 of 16)
> **Part 1:** `01-fundamentals-and-governance.md` — Fundamentals & Governance
> **Part 2:** `02-identity-and-access.md` — Identity & Access
> **Part 3:** `03-compute-vms-and-scale-sets.md` — Compute: Virtual Machines & Scale Sets
> **Part 4:** `04-networking-foundations-vnets-ip-and-dns.md` — Networking Foundations: VNets, IP & DNS
> **Part 5:** `05-networking-hybrid-connectivity.md` — Networking: Hybrid Connectivity
> **Part 6:** `06-networking-application-delivery.md` — Networking: Application Delivery
> **Part 7:** `07-networking-private-access-and-security.md` — Networking: Private Access & Security
> **Part 8:** `08-storage-blob-files-and-disks.md` — Storage: Blob, Files & Disks
> **Part 9:** `09-databases-and-data-services.md` — Databases & Data Services
> **Part 10:** `10-containers-and-serverless.md` — Containers & Serverless
> **Part 11:** `11-application-architecture-and-messaging.md` — Application Architecture & Messaging
> **Part 12:** `12-security-and-compliance.md` — Security & Compliance
> **Part 13:** `13-monitoring-logging-and-observability.md` — Monitoring, Logging & Observability
> **Part 14:** This file — Business Continuity: Backup, DR & Migration
> **Part 15:** `15-cicd-and-iac.md` — CI/CD & Infrastructure as Code
> **Part 16:** `16-multi-region-cost-optimization-and-cheatsheet.md` — Multi-Region, Cost Optimization & Cheat Sheet
> **Questions:** `questions.md`

## Table of Contents

1. [Business Continuity — RTO, RPO, and the Framework](#business-continuity--rto-rpo-and-the-framework)
2. [Azure Backup — Recovery Services Vault](#azure-backup--recovery-services-vault)
3. [Backup Policies and Retention](#backup-policies-and-retention)
4. [Cross-Region Restore](#cross-region-restore)
5. [Azure Business Continuity Center](#azure-business-continuity-center)
6. [Azure Site Recovery — Architecture](#azure-site-recovery--architecture)
7. [ASR Test Failover and Recovery Plans](#asr-test-failover-and-recovery-plans)
8. [ASR Reprotection and Failback](#asr-reprotection-and-failback)
9. [High Availability vs. Disaster Recovery](#high-availability-vs-disaster-recovery)
10. [Designing for RTO/RPO — Standby Tiers](#designing-for-rtorpo--standby-tiers)
11. [Azure Migrate — Discovery and Assessment](#azure-migrate--discovery-and-assessment)
12. [The Cloud Adoption Framework Migration Phases](#the-cloud-adoption-framework-migration-phases)
13. [Migration Strategies — the Five Rs](#migration-strategies--the-five-rs)
14. [A Full Worked BC/DR Bootstrap for Meridian Freight](#a-full-worked-bcdr-bootstrap-for-meridian-freight)
15. [Part 14 CLI Cheat Sheet](#part-14-cli-cheat-sheet)
16. [Common Mistakes and Interview Traps](#common-mistakes-and-interview-traps)
17. [Worked Practice Problems](#worked-practice-problems)
18. [Summary and What's Next](#summary-and-whats-next)

## Business Continuity — RTO, RPO, and the Framework

Every backup and DR decision in this chapter comes down to two numbers, worth defining precisely before anything else: **Recovery Time Objective (RTO)** — how long can the business tolerate being down — and **Recovery Point Objective (RPO)** — how much data loss (measured in time) is acceptable.

```mermaid
graph TD
    Incident["An incident occurs"] --> RPO["RPO: how much data\nsince the last backup\nis acceptable to lose"]
    Incident --> RTO["RTO: how long until\nservice is restored"]
    RPO --> Design["Backup frequency and\nreplication design"]
    RTO --> Design2["Standby tier and\nfailover automation"]
```

Meridian Freight's `rates-db` (pricing data, changes slowly, tolerable to lose a few hours of updates) has a genuinely different RPO requirement than `shipment-api`'s live order data (near-zero tolerable loss) — this chapter's design decisions follow directly from these two numbers being set deliberately per workload, not applied blanket.

---

## Azure Backup — Recovery Services Vault

```bash
az backup vault create --name rsv-meridian --resource-group rg-shipment-api-prod \
  --location eastus

az backup protection enable-for-vm --vault-name rsv-meridian --resource-group rg-shipment-api-prod \
  --vm vm-driver-portal-01 --policy-name DefaultPolicy
```

The **Recovery Services vault** is the container for backup data and policies — supporting VMs, SQL/SAP HANA on VMs, Azure Files, and (via a separate Backup vault) newer workload types.

---

## Backup Policies and Retention

```bash
az backup policy create --vault-name rsv-meridian --resource-group rg-shipment-api-prod \
  --name policy-daily-retain-90 --backup-management-type AzureIaasVM \
  --policy '{"schedulePolicy": {"scheduleRunFrequency": "Daily"}, "retentionPolicy": {"dailySchedule": {"retentionDuration": {"count": 90, "durationType": "Days"}}}}'
```

**Worth stating precisely the current RPO reality for standard Azure Backup: the Standard policy's primary-region RPO is up to 24 hours, and replication to the secondary region can add up to another 12-36 hours in the worst case** — a genuinely important number to confirm against a workload's actual RPO requirement before assuming Azure Backup alone satisfies it; a near-zero-RPO requirement (like `shipment-api`'s live order data) needs a fundamentally different mechanism (synchronous replication, a failover group — Part 9) layered on top, not standard backup alone.

---

## Cross-Region Restore

```bash
az backup restore restore-disks --resource-group rg-shipment-api-prod \
  --vault-name rsv-meridian --container-name "<container>" --item-name vm-driver-portal-01 \
  --rp-name "<recovery-point>" --target-resource-group rg-driver-portal-dr \
  --use-secondary-region
```

**A genuinely important prerequisite worth stating explicitly: cross-region restore ONLY works for a vault using GRS (or GZRS) replication — a vault configured with LRS has no secondary-region copy to restore from at all**, directly connecting back to Part 8's storage redundancy discussion; the vault's own redundancy setting is a real, easy-to-overlook prerequisite decided at vault creation, not something fixable reactively during an actual regional outage. Enabling cross-region restore after the fact also takes up to 48 hours before it's actually usable — another reason this is a proactive, not reactive, configuration decision.

---

## Azure Business Continuity Center

```bash
az backup vault backup-status show --name rsv-meridian --resource-group rg-shipment-api-prod
```

**Business Continuity Center** provides a single, unified view across BOTH Azure Backup and Azure Site Recovery — genuinely useful for an organization with many workloads spread across both mechanisms, since it surfaces protection status, upcoming test failover schedules, and built-in alerts (unhealthy replication, failover failures, expiring agents) in one place rather than checking each vault and each ASR configuration separately. Alerts surfaced here route to the same Azure Monitor action groups (Part 13) already covering every other operational alert, keeping DR-specific alerting inside the same unified on-call path rather than a separate, easy-to-miss notification channel.

---

## Azure Site Recovery — Architecture

```bash
az site-recovery replication-policy create --resource-group rg-shipment-api-prod \
  --vault-name rsv-meridian --name policy-24h-rpo \
  --recovery-point-retention-in-hours 24

az site-recovery protection-container mapping create --resource-group rg-shipment-api-prod \
  --vault-name rsv-meridian --fabric-name eastus --protection-container-name pc-eastus \
  --target-protection-container pc-westus --policy-name policy-24h-rpo
```

**Azure Site Recovery (ASR)** replicates entire VMs (Azure-to-Azure, or on-premises-to-Azure) continuously, ready for a genuine regional-outage failover — a fundamentally different mechanism from Azure Backup's periodic snapshots, worth stating the distinction precisely: **Backup answers "recover a point-in-time copy after data loss or corruption"; ASR answers "keep a continuously-replicated, ready-to-activate copy for a full site/region failure."** A production-critical workload typically needs both, for different failure modes.

---

## ASR Test Failover and Recovery Plans

```bash
az site-recovery recovery-plan create --resource-group rg-shipment-api-prod --vault-name rsv-meridian \
  --name recovery-plan-shipment-api --primary-fabric-id eastus --recovery-fabric-id westus \
  --failover-deployment-model ResourceManager
```

**A recovery plan** sequences a multi-VM failover in the correct order (database tier before application tier, for instance) rather than failing over every VM simultaneously and hoping dependency order works out. **Test failover** runs this entire sequence into an ISOLATED, non-production network — genuinely essential to state explicitly: a test failover never impacts the actual production environment, making it safe to run as a REGULAR, scheduled DR drill rather than something attempted for the first time during an actual emergency.

> **From the Trenches:** An organization configured Azure Site Recovery for its production fleet, confirmed replication health looked healthy in the portal, and considered DR "done." Eighteen months later, during an actual regional incident, the real failover revealed the recovery plan's VM startup order had never been updated after a database migration changed which VM was now the actual primary — the plan failed over the OLD primary first, in the wrong order, causing a longer outage than the DR investment was supposed to prevent. The corrective practice adopted afterward: a scheduled, quarterly test failover, treated as a real operational requirement rather than a one-time setup checkbox — replication health alone says nothing about whether the RECOVERY PLAN itself still reflects the current architecture.

---

## ASR Reprotection and Failback

```bash
az site-recovery protected-item create --resource-group rg-shipment-api-prod --vault-name rsv-meridian \
  --fabric-name westus --protection-container-name pc-westus --replicated-item-name vm-shipment-api-01 \
  --policy-name policy-24h-rpo
```

**After a genuine failover, the failed-over VMs are now running in the secondary region — "reprotection" starts replicating them BACK toward the original primary region**, so a subsequent "failback" can return to the original region once it's healthy again. This two-step reprotect-then-failback sequence, not a single "undo" operation, is worth knowing precisely rather than assuming failover is trivially reversible.

---

## High Availability vs. Disaster Recovery

A genuinely important conceptual distinction worth stating precisely, since the two are often conflated: **High Availability (HA)** protects against LOCAL failures (a VM crash, a zone outage) with automatic, fast failover WITHIN a region — Availability Zones (Part 3), SQL failover groups (Part 9). **Disaster Recovery (DR)** protects against a FULL REGIONAL failure, typically with a slower, often more manual failover to an entirely different region.

| | High Availability | Disaster Recovery |
|---|---|---|
| Protects against | VM/zone-level failure | Full regional failure |
| Typical RTO | Seconds to minutes | Minutes to hours |
| Mechanism | Availability Zones, failover groups | Site Recovery, cross-region backup restore |
| Automation | Usually automatic | Often requires a deliberate failover decision |

---

## Designing for RTO/RPO — Standby Tiers

```mermaid
graph LR
    Cold["Cold standby —\nrestore from backup,\nlongest RTO, lowest cost"] --> Warm["Warm standby —\nscaled-down secondary\nalready running"]
    Warm --> Hot["Hot standby —\nfull-scale secondary,\nactive-active or\nactive-passive, lowest RTO"]
```

| Tier | RTO | Cost | Best fit |
|---|---|---|---|
| Cold | Hours | Lowest | `rates-db`'s DR — infrequent price changes tolerate a slower recovery |
| Warm | Minutes | Moderate | A secondary kept running at reduced scale, ready to scale up on failover |
| Hot | Seconds | Highest | `shipment-api`'s live order path — customer-facing, minimal tolerable downtime |

**Why matching the standby tier to the ACTUAL RTO/RPO requirement matters concretely, worth stating explicitly: over-provisioning a hot standby for a workload that could tolerate a cold-standby recovery wastes real, ongoing cost — under-provisioning a workload that genuinely needs hot standby risks a business-critical outage lasting far longer than acceptable.** This decision should be made deliberately per workload, exactly the way this chapter's opening RTO/RPO framework recommends, not defaulted uniformly across every service.

---

## Azure Migrate — Discovery and Assessment

```bash
az migrate project create --name migrate-meridian --resource-group rg-shipment-api-prod
```

**Azure Migrate** performs agentless discovery of on-premises VMware/Hyper-V/physical servers, mapping application dependencies and producing right-sizing assessments — directly relevant to Meridian Freight's remaining legacy on-premises freight-routing servers, referenced since Part 1, whose eventual migration this chapter's tooling actually executes.

---

## The Cloud Adoption Framework Migration Phases

```mermaid
graph LR
    Assess["Assess —\ninventory workloads"] --> Plan["Plan —\nsequencing, data paths"]
    Plan --> Mobilize["Mobilize —\nprepare infrastructure"]
    Mobilize --> Migrate["Migrate —\nexecute in waves"]
    Migrate --> Optimize["Optimize —\nmonitor, tune, decommission source"]
```

**A genuinely important current practice worth stating explicitly: treat the assessment as a LIVING document, refreshed quarterly and cross-checked against Azure Advisor recommendations, rather than a one-time snapshot** — an assessment done once at the very start of a multi-year migration goes stale as both the source environment and Azure's own service offerings change.

---

## Migration Strategies — the Five Rs

| Strategy | What it means | Best fit |
|---|---|---|
| Rehost ("lift and shift") | Move as-is, minimal changes | Fast migration, legacy apps not worth re-architecting yet |
| Replatform | Minor optimizations during migration (e.g., moving to a managed database) | Meridian Freight's legacy database moving to PostgreSQL Flexible Server |
| Refactor | Modify code to better use cloud-native services, without full rearchitecture | Moving a monolith's specific components toward managed services incrementally |
| Rearchitect | Substantially redesign for cloud-native patterns (microservices, serverless) | `shipment-api`'s own evolution described throughout this series |
| Rebuild | Discard and rebuild from scratch | When the existing system's technical debt exceeds the cost of rebuilding |

**Why choosing the RIGHT strategy per workload — not defaulting to the same one for everything — matters concretely: Meridian Freight's legacy freight-routing servers are a rehost/replatform candidate (working, low-risk, not worth a full rewrite yet), while `shipment-api` itself has already been rearchitected throughout this series precisely because its growth justified that investment** — the five strategies exist on a real spectrum of effort versus cloud-native benefit, and the right choice depends on each specific workload's value and risk profile, not a blanket organizational policy.

---

## A Full Worked BC/DR Bootstrap for Meridian Freight

```bash
# 1. Recovery Services vault with GRS (prerequisite for cross-region restore)
az backup vault create --name rsv-meridian --resource-group rg-shipment-api-prod

# 2. Backup policy matched to rates-db's actual RPO tolerance
az backup protection enable-for-vm --vault-name rsv-meridian --resource-group rg-shipment-api-prod \
  --vm vm-rates-db --policy-name policy-daily-retain-90

# 3. Site Recovery for shipment-api's hot-standby tier
az site-recovery replication-policy create --resource-group rg-shipment-api-prod \
  --vault-name rsv-meridian --name policy-24h-rpo

# 4. A recovery plan sequencing the multi-tier failover correctly
az site-recovery recovery-plan create --resource-group rg-shipment-api-prod --vault-name rsv-meridian \
  --name recovery-plan-shipment-api

# 5. Azure Migrate project for the remaining legacy on-premises servers
az migrate project create --name migrate-meridian --resource-group rg-shipment-api-prod
```

---

## Part 14 CLI Cheat Sheet

| Area | Command | Purpose |
|---|---|---|
| Vault | `az backup vault create` | Create a Recovery Services vault |
| Policy | `az backup policy create` | Define a backup schedule and retention |
| Protection | `az backup protection enable-for-vm` | Enable backup for a VM |
| Cross-region | `az backup restore restore-disks --use-secondary-region` | Restore from the secondary region |
| ASR policy | `az site-recovery replication-policy create` | Define an ASR replication policy |
| Recovery plan | `az site-recovery recovery-plan create` | Sequence a multi-VM failover |
| Reprotection | `az site-recovery protected-item create` | Start reprotecting a failed-over VM |
| Migrate | `az migrate project create` | Create an Azure Migrate assessment project |

---

## Common Mistakes and Interview Traps

| Mistake | Why It's Wrong | Fix |
|---|---|---|
| Assuming standard Azure Backup satisfies a near-zero-RPO requirement | Standard policy RPO can be up to 24 hours in the primary region alone | Layer synchronous replication (failover groups, Part 9) on top for near-zero-RPO workloads |
| Configuring a Recovery Services vault with LRS and expecting cross-region restore to work | Cross-region restore requires GRS/GZRS — LRS has no secondary copy | Use GRS/GZRS from vault creation if cross-region restore is a requirement |
| Treating a successful test failover once as proof DR remains valid indefinitely | Architecture changes (new VMs, changed dependencies) can silently invalidate a recovery plan | Run test failovers on a regular, scheduled cadence, not just once at initial setup |
| Applying the same standby tier (hot/warm/cold) to every workload uniformly | Over-provisions low-RTO-need workloads and under-provisions critical ones | Match standby tier to each workload's actual RTO/RPO requirement individually |
| Treating a migration assessment as a one-time snapshot | Both the source environment and Azure's offerings change over time | Refresh the assessment on a recurring (e.g. quarterly) cadence |
| Defaulting to the same migration strategy (usually rehost) for every workload | Some workloads genuinely benefit from replatforming or rearchitecting; others don't justify the investment | Choose a migration strategy per workload based on its actual value and risk profile |

---

## Worked Practice Problems

**Problem 1:** Meridian Freight configures Azure Backup on its Recovery Services vault using the default Standard policy, believing this satisfies `shipment-api`'s stated near-zero-data-loss requirement for live order data. During an actual VM failure, the team discovers up to 24 hours of order data since the last backup point is at risk of being lost. What was the design mistake?

*Answer:* The team applied Azure Backup's periodic, point-in-time backup mechanism to a workload with a near-zero RPO requirement, without recognizing that standard backup's primary-region RPO can be up to 24 hours — backup is designed to answer "recover a point-in-time copy after data loss," not to guarantee minimal ongoing data loss for a continuously changing dataset. For `shipment-api`'s live order data, a fundamentally different mechanism is needed: synchronous or near-synchronous replication (a SQL failover group with automatic failover, Part 9) that keeps a continuously up-to-date secondary copy, with Azure Backup layered on top as a SEPARATE protection against a different failure mode (accidental deletion, corruption) rather than the sole DR mechanism for this specific near-zero-RPO requirement.

**Problem 2:** An organization's Azure Site Recovery setup shows healthy replication status in the portal continuously for eighteen months. During an actual regional failover, the recovery plan fails to bring services up correctly, because the VM startup sequence still reflects an architecture from before a significant database migration. What did "healthy replication status" fail to catch, and what practice would have caught it?

*Answer:* Replication health only confirms that DATA is being continuously copied to the secondary region — it says nothing about whether the RECOVERY PLAN'S sequencing logic still reflects the current architecture, since the plan and the replication mechanism are validated independently. A regular, scheduled test failover (run into an isolated network, safe to perform without affecting production) would have caught this specific gap by actually exercising the recovery plan's startup sequence against the CURRENT architecture, revealing the stale VM ordering well before an actual incident — exactly why test failovers need to be a recurring operational practice, not a one-time setup validation.

---

## Summary and What's Next

- **Every backup/DR decision should trace back to an explicit RTO/RPO defined per workload** — not a uniform policy applied blanket across genuinely different criticality levels.
- **Azure Backup (point-in-time recovery) and Azure Site Recovery (continuous replication for regional failover) solve different failure modes** — a production-critical workload typically needs both.
- **Cross-region restore requires GRS/GZRS vault redundancy configured from the start** — not fixable reactively during an actual regional outage, and takes up to 48 hours to become usable after enabling.
- **Test failovers must be a recurring, scheduled practice** — replication health alone says nothing about whether a recovery plan's sequencing still reflects the current architecture.
- **High Availability and Disaster Recovery are genuinely distinct concerns** — local, fast, automatic failover versus regional, slower, often more deliberate failover.
- **Migration strategy (the Five Rs) should be chosen per workload** based on actual value and risk profile, not defaulted uniformly — this series' own `shipment-api` rearchitecture versus the legacy servers' rehost/replatform path is a real, concrete example of that differentiation.

**Continue to Part 15** (`15-cicd-and-iac.md`) for the CI/CD and Infrastructure as Code practices that actually deploy and evolve everything this series has built.