Cloud Network Architecture

Table of Contents#

1. From One VPC to a Cloud Network Estate
2. VPC/VNet Fundamentals — CIDR Planning and Why It's Hard to Change Later
3. Subnets — Public, Private, and the Route Table That Actually Defines the Difference
4. Route Tables — the Real Source of Truth for Where Traffic Goes
5. Internet Gateways and NAT Gateways — Getting In and Out
6. Security Groups vs. NACLs — Stateful and Stateless Defense in Depth
7. VPC Peering — Direct, Point-to-Point Connectivity
8. Transit Gateway — Solving Peering's Mesh Problem
9. VPC Endpoints and PrivateLink — Reaching Cloud Services Without the Public Internet
10. Bastion Hosts and Session Manager — Accessing Private Subnets Without Exposing SSH
11. Site-to-Site VPN — IPsec and WireGuard for Hybrid Connectivity
12. Dedicated Connections — Direct Connect, ExpressRoute, and Cloud Interconnect
13. IP Address Management at Scale — Avoiding CIDR Collisions
14. DNS Inside the VPC — Private Hosted Zones and Split-Horizon Resolution
15. VPC Flow Logs — Network-Layer Observability
16. Multi-Account Landing Zones — Shared VPC and Centralized Egress
17. Multi-Region and Multi-Cloud Network Design Patterns
18. Centralized Egress Filtering — Network Firewall as a Managed Inspection Point
19. Full Worked Scenario: Redesigning the Platform's Network for a Second Region and a Compliance Boundary
20. Common Mistakes and Interview Traps
21. Worked Practice Problems
22. Summary and What's Next

From One VPC to a Cloud Network Estate#

Parts 1 and 2 covered how traffic reaches your network and how it's spread across backends once it arrives — this chapter is about the network it arrives into: how a cloud Virtual Private Cloud (VPC, or VNet on Azure) is actually structured, and how many of them get wired together as an organization grows past "one team, one VPC, one region." checkout-service, catalog-service, and inventory-service — the same throughline platform from the first two chapters — started, like most platforms do, as a single VPC in a single region, with a handful of engineers who could hold the entire network layout in their heads. This chapter follows that platform's network as it grows past the point where that's still true: a second region for latency and compliance, a separate VPC for a data-platform team that needs isolation, a VPN back to an on-premises payment-processing system that can't move to the cloud, and the connectivity glue needed to make all of that work without either a security nightmare or an unmanageable mesh of point-to-point links.

Every mechanism in this chapter answers some version of the same underlying question: as the number of VPCs, accounts, and regions grows, does connectivity and security policy stay something a small platform team can reason about directly, or does it silently become an unmanageable, ad hoc sprawl that nobody fully understands anymore? The patterns favored throughout this chapter — centralized IPAM, hub-and-spoke connectivity, centralized egress — all trade a small amount of upfront design discipline for keeping that question answerable at any scale the platform actually grows to.

VPC/VNet Fundamentals — CIDR Planning and Why It's Hard to Change Later#

A VPC is a logically isolated, private IP address space inside a cloud provider's network — the cloud equivalent of the private network namespaces covered from the kernel side in this site's Linux & Networking Fundamentals series, just operated at the scale of an entire account/subscription rather than one host. Creating one starts with choosing a CIDR block (commonly out of the RFC 1918 private ranges — 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16) — and this single choice is disproportionately consequential, because a VPC's primary CIDR block cannot be freely resized after subnets and resources already depend on it, and an overlapping CIDR between two VPCs makes them fundamentally unable to peer or route to each other later without a costly re-IP.

Subnets — Public, Private, and the Route Table That Actually Defines the Difference#

A VPC's CIDR block is subdivided into subnets, each pinned to exactly one Availability Zone — a subnet itself is never multi-AZ, which is precisely why every genuinely highly-available architecture needs at least one subnet per AZ per tier (public, private, data), not one subnet shared across zones. "Public" and "private" are not a special subnet type or a checkbox — they are purely a description of what that subnet's route table happens to point a 0.0.0.0/0 (default) route at, covered in depth in the next section. A subnet with a default route to an Internet Gateway is, by that fact alone, a "public" subnet; one without such a route (or with its default route pointed at a NAT gateway instead) is "private" — the label is just descriptive shorthand for the routing configuration underneath it.

A minimally viable, genuinely-available two-tier layout — one public and one private subnet per AZ, each AZ independent so a single zone failure never takes down the whole tier.

⚙️ checkout-service's pods run in the private subnets — never directly internet-addressable — reached only through the load balancer sitting in the public subnets, exactly the pattern covered from the load-balancing side in Part 2. The database tier (RDS, in this platform's case) typically gets its own, even-more-restricted subnet group with no route to the internet in either direction, not even via NAT — a third tier beyond the simple two-tier diagram above, common in any architecture handling sensitive data.

Route Tables — the Real Source of Truth for Where Traffic Goes#

Every subnet is associated with exactly one route table, and that table — not any label, tag, or naming convention — is the actual, enforced source of truth for where traffic from that subnet goes. A route table is a simple, ordered list of destination CIDR blocks and the target (a gateway, a peering connection, a NAT gateway, a Transit Gateway attachment) each one is routed through — conceptually identical to the ip route table on a single Linux host covered in this site's Linux & Networking Fundamentals series, just scoped to an entire subnet's worth of resources rather than one machine.

# A representative private-subnet route table
Destination          Target
10.0.0.0/16           local                    # intra-VPC traffic, always implicit
10.1.0.0/16           pcx-0123456789abcdef0     # VPC peering connection to another VPC
0.0.0.0/0              nat-0fedcba9876543210    # everything else goes out via NAT Gateway

The most-specific matching route always wins — the same longest-prefix-match principle covered in Part 1 for BGP applies identically here, just evaluated locally within one route table rather than propagated across ASes. This is what lets a route table have both a broad 0.0.0.0/0 default (catch difficult everything-else traffic) and a narrower, more specific route (like the peering-connection route above) that takes priority for its own destination range without needing any explicit ordering or priority field.

Internet Gateways and NAT Gateways — Getting In and Out#

Two distinct gateway types handle the two distinct directions of internet traffic, and conflating them is a common source of confusion for engineers new to cloud networking:

A NAT Gateway is a real, if unglamorous, single point of failure and cost center worth deliberate design attention. It's zonal (an AWS NAT Gateway lives in one specific AZ), so a genuinely resilient multi-AZ architecture needs one NAT Gateway per AZ — routing every private subnet's outbound traffic through a NAT Gateway in a different AZ than the subnet itself both adds unnecessary cross-AZ data transfer charges and creates an availability dependency between zones that a well-designed architecture specifically avoids.

⚠️ From the trenches: inventory-service's platform team ran a single NAT Gateway (in one AZ) shared across all three AZs' private subnets, reasoning that NAT Gateway "basically never goes down" and the savings from not running three were worth it. During an AZ-level networking event (not a full AZ outage — just degraded connectivity within that one zone), every private subnet across all three AZs lost outbound internet access simultaneously, because they all depended on the single NAT Gateway sitting in the affected zone. Services that needed outbound calls to third-party APIs (payment processing, fraud-detection checks) failed across the entire platform, not just the one genuinely-affected AZ — turning a partial, single-zone degradation into a full platform outage purely due to the NAT topology. The fix — one NAT Gateway per AZ, with each AZ's private subnets routed only to their own zone's NAT Gateway — cost roughly 3x more per month in NAT Gateway hourly charges, a tradeoff the team now considers obviously correct in hindsight, but hadn't consciously evaluated against the actual availability cost of the cheaper design at the time it was built.

NAT Gateway Alternatives, and the IPv6 Case Where NAT Isn't Needed at All#

A managed NAT Gateway isn't the only option, and knowing the alternatives — and when NAT is unnecessary entirely — rounds out the outbound-connectivity picture:

• NAT instances (a plain EC2 instance running NAT software, the pre-managed-NAT-Gateway approach) are now largely legacy — cheaper at very small scale, but self-managed (patching, scaling, HA all become the team's own problem again) in a way that rarely beats the managed NAT Gateway's operational simplicity once the true cost of that ongoing maintenance burden is counted honestly.
• Egress-only Internet Gateways are the IPv6-specific equivalent of a NAT Gateway's purpose (outbound-only internet access for a private subnet) without actually doing address translation at all — a direct consequence of Part 1's IPv6 coverage: since every IPv6 address is globally routable by design, there's no private-address-to-public-address mapping to perform in the first place. An egress-only Internet Gateway simply enforces the directionality restriction (outbound-initiated only, exactly like a NAT Gateway's behavior) without the translation NAT's own name implies, and — notably — without NAT's associated hourly or per-GB processing charge, since there's no translation work being done.

Security Groups vs. NACLs — Stateful and Stateless Defense in Depth#

Two independent, layered filtering mechanisms guard traffic in and out of a VPC, and — echoed from Part 1's netfilter-adjacent material, but at the cloud-managed layer instead of the kernel — the distinction between stateful and stateless filtering is the single most important thing to internalize correctly:

The stateful/stateless distinction has a concrete, easy-to-get-wrong consequence: a Security Group rule allowing inbound HTTPS (443) needs no matching outbound rule for the response to leave — the state table handles it automatically. A NACL doing the exact same job needs an inbound rule allowing 443 and a separate outbound rule allowing the ephemeral port range (typically 1024-65535) the client's OS assigned for that connection's return traffic — a NACL that only opens 443 inbound, with no ephemeral-range outbound rule, silently breaks every connection through it despite looking, at a glance, like it should work.

The Security Group's return path (solid) needs nothing extra configured; the NACL's return path (dashed) needs its own explicit outbound rule — a genuinely common source of "works from one direction, silently fails from the other" tickets.

VPC Peering — Direct, Point-to-Point Connectivity#

VPC peering establishes a direct, private network connection between exactly two VPCs, letting resources in either communicate using private IP addresses as if they were on the same network — no internet gateway, no NAT, no public exposure required. Peering is non-transitive: if VPC A peers with VPC B, and VPC B peers with VPC C, traffic from A cannot reach C through B — each pair that needs to communicate needs its own direct peering connection, a constraint that becomes the central limitation covered in the next section.

# Requesting a peering connection (the accepting side must separately accept it)
aws ec2 create-vpc-peering-connection \
  --vpc-id vpc-checkout-us-east-1 \
  --peer-vpc-id vpc-shared-data-us-east-1

# Both sides' route tables need an explicit route pointing at the peering
# connection for the other VPC's CIDR — peering alone does not add routes

Peering's core scaling problem is combinatorial: connecting N VPCs in a full mesh (every VPC able to reach every other one directly) requires N(N-1)/2 peering connections — the exact same growth curve covered for iBGP full-mesh in Part 1, and it becomes unmanageable at roughly the same scale (a few dozen VPCs). Ten VPCs need 45 connections; twenty need 190. Each connection also needs its own explicit route-table entries on both sides, multiplying the operational overhead further. This is precisely the problem the next section's Transit Gateway pattern exists to solve.

Transit Gateway — Solving Peering's Mesh Problem#

A Transit Gateway (AWS's name; GCP's Network Connectivity Center and Azure's Virtual WAN are the equivalent concepts) is a managed, regional hub that every VPC attaches to once, with the gateway itself handling routing between every attached VPC — replacing peering's combinatorial mesh with a hub-and-spoke model where N VPCs need only N attachments, not N(N-1)/2 connections.

Five attachments instead of ten peering connections for the same full connectivity — and adding a sixth VPC later means one new attachment, not five new peering connections.

The tradeoff, covered from the routing-mechanics side in Part 1's peering-vs-transit discussion, applies here too: Transit Gateway adds a real extra hop (measurably higher latency than direct peering, though usually immaterial for typical application traffic) and a per-attachment, per-GB cost that direct peering doesn't carry. Transit Gateway's real, differentiating power is multiple route tables — a single Transit Gateway can maintain several independent routing domains, so (for example) a compliance-sensitive VPC can be attached with routes that only reach a specific subset of the other attached VPCs, while the rest of the network sees a completely different routing view — enforcing network-level segmentation centrally, at the hub, rather than needing to replicate the same segmentation logic across every individual VPC's own route tables and security groups.

VPC Endpoints and PrivateLink — Reaching Cloud Services Without the Public Internet#

A subtlety easy to miss: calling a cloud provider's own managed service (S3, DynamoDB, a managed Kafka service) from inside a private subnet with no internet route normally fails, because those services' API endpoints are, by default, public internet addresses — even though the traffic never leaves the cloud provider's own network physically, it's still routed, from the VPC's perspective, as internet-bound traffic requiring a route to an Internet Gateway or NAT Gateway to reach at all.

VPC Endpoints solve this by placing a private, in-VPC entry point for a specific AWS service directly inside the VPC's own route table — traffic to that service never needs a route to the internet at all, improving both security posture (no NAT Gateway data-processing charges for that traffic, no possibility of that traffic being intercepted or exfiltrated via a compromised NAT path) and, frequently, latency.

• Gateway endpoints (S3, DynamoDB specifically): free, implemented as a route-table target — a genuinely easy, low-cost win with essentially no downside for any private subnet that talks to either service.
• Interface endpoints (most other services — Secrets Manager, KMS, most managed-service APIs): an ENI with a private IP placed directly in your subnet, billed hourly plus per-GB, functioning as a private DNS target for the service.

PrivateLink extends the same underlying mechanism to expose a custom service — one your own organization runs, in your own VPC — privately to other VPCs (including ones belonging to a different AWS account entirely, a common pattern for a platform team offering a shared internal service to multiple product teams' own VPCs) without peering or Transit Gateway at all, and critically, without exposing that service's IP space or requiring any inbound route from the consumer's side — a one-way, tightly-scoped connectivity primitive specifically suited to a shared internal platform API that many otherwise-unrelated VPCs need to reach.

Bastion Hosts and Session Manager — Accessing Private Subnets Without Exposing SSH#

Engineers still need occasional interactive access to resources sitting in a private subnet — debugging a production issue that genuinely requires shelling into a box, running a one-off database migration script from inside the network boundary. The traditional pattern, and the more modern replacement for it, are worth contrasting directly, because the traditional one carries a real, persistent security liability that's easy to under-appreciate until it's exploited.

• The bastion host (jump box) pattern: a single, hardened EC2 instance sits in a public subnet with SSH (port 22) open to the internet (or, at best, to a specific allowlisted IP range), and engineers SSH into the bastion first, then SSH again from the bastion into the actual private-subnet target. This works, and is still common in older architectures, but it means a real, internet-facing SSH port is permanently exposed — a continuous attack surface (credential-stuffing attempts, unpatched-CVE exploitation risk) that exists every hour of every day, whether or not anyone is actually using it in a given moment.
• AWS Systems Manager Session Manager (and the equivalent identity-aware-proxy-based tooling on GCP and Azure) replaces the bastion entirely: an agent running on the target instance itself establishes an outbound connection to the Systems Manager service (no inbound port needs to be open at all — not even 22), and an authorized engineer's session is brokered through that outbound channel, authenticated via the cloud provider's own IAM rather than a separately-managed SSH key. The target instance needs zero inbound rules from the internet, and often zero inbound rules at all — the entire access path is IAM-authenticated and fully audit-logged (every session, every command, centrally) by default, a real security and compliance upgrade over SSH key management sprawl.

The bastion pattern's inbound-exposed port is a permanent attack surface even when idle; Session Manager's outbound-only connection has no equivalent inbound exposure at all — the private instance never listens on a network port for interactive access.

Site-to-Site VPN — IPsec and WireGuard for Hybrid Connectivity#

Not every workload lives in the cloud — checkout-service's platform still depends on an on-premises payment-settlement system that, for regulatory and legacy-integration reasons, isn't migrating. A site-to-site VPN creates an encrypted tunnel between the cloud VPC and the on-premises network over the public internet, avoiding the cost and lead time of a dedicated physical connection (covered next) at the expense of depending on best-effort public internet transit for the underlying path.

Two protocol families dominate production site-to-site VPN deployments today, and the choice genuinely matters:

Every major cloud's managed site-to-site VPN product (AWS Site-to-Site VPN, Azure VPN Gateway, Google Cloud VPN) is built on IPsec specifically because of its universal enterprise-router interoperability — a practical constraint that keeps IPsec the default choice for this specific use case even as WireGuard gains ground elsewhere (a service mesh's own inter-cluster tunneling, or a modern zero-trust network-access product, being two areas where WireGuard has seen faster adoption than in the classic router-to-router site-to-site case).

Dedicated Connections — Direct Connect, ExpressRoute, and Cloud Interconnect#

For an organization with sustained, high-volume, latency-sensitive traffic between on-premises infrastructure and the cloud — enough to justify the cost and lead time — a dedicated connection (AWS Direct Connect, Azure ExpressRoute, Google Cloud Interconnect) provides a private, physical network link directly into the cloud provider's network, bypassing the public internet entirely. This isn't a VPN over a dedicated line — it's a genuinely different physical/logical connection, typically established at a carrier-neutral colocation facility (frequently the same kind of facility that hosts the IXPs covered in Part 1), with the customer either bringing their own cross-connect or working through a Direct Connect/ExpressRoute Partner who already has presence there.

Production hybrid architectures at real scale frequently run both, with the dedicated connection as primary and a site-to-site VPN as an automatic failover path — the dedicated connection's own physical circuit is a real single point of failure risk (a backhoe cutting the wrong fiber run is the industry's long-running dark joke about this exact failure mode) unless paired with either a second, physically diverse dedicated connection or a VPN fallback that can absorb traffic during an outage.

IP Address Management at Scale — Avoiding CIDR Collisions#

Returning to the CIDR-planning discipline from earlier in this chapter, now at the scale that makes it matter most: once an organization has dozens of VPCs across multiple accounts, regions, and teams, ad hoc CIDR assignment reliably produces overlapping address ranges that block exactly the connectivity patterns this chapter has covered — two VPCs with overlapping CIDRs cannot peer, cannot share a Transit Gateway route table cleanly, and cannot be bridged by a VPN without additional NAT complexity layered on top specifically to work around the collision.

• AWS IPAM (and the equivalent GCP/Azure IP-management tooling) is the managed-service answer: a central pool of address space, carved into nested, non-overlapping allocations per region/account/team, with automatic collision detection at allocation time rather than discovery months later during an attempted peering request.
• The alternative to a managed IPAM tool is the same discipline covered earlier applied as an actual, enforced process: a documented central CIDR registry (even a well-maintained spreadsheet or a Terraform-managed data source, for a smaller organization), with every new VPC's CIDR request checked against it before creation — the specific mechanism matters less than the discipline of never allocating a new VPC's address space without checking it against everything that already exists.

⚙️ Worked example: checkout-service's platform team reserves 10.0.0.0/8 as the organization's entire private address space at the start of their cloud adoption, carving it into /16s: 10.0.0.0/16 for us-east-1 production, 10.1.0.0/16 for eu-central-1 production (used in this chapter's own worked scenario below), 10.2.0.0/16 for the shared-services/network account, and 10.100.0.0/16 reserved, deliberately unused, as headroom for a future third region — planned before it's needed, exactly the discipline this section argues for. Every individual VPC created afterward carves its own smaller CIDR (typically a /20 or /21, sized for real expected instance/ENI count including headroom for auto-scaling) out of its region's reserved /16, checked against the central registry before creation — a five-minute process that has, in this platform's three years of operation since, avoided a single CIDR collision despite the platform growing from one VPC to over a dozen across three accounts.

DNS Inside the VPC — Private Hosted Zones and Split-Horizon Resolution#

A cloud DNS service (Route 53, Cloud DNS, Azure DNS) commonly serves two genuinely distinct roles that are easy to conflate: public DNS (resolving checkout-service's public API hostname for the entire internet, tied directly to the anycast/BGP mechanics from Part 1) and private DNS (resolving internal service names — checkout-service.internal, an RDS endpoint's hostname — only for resolvers inside the VPC, never exposed publicly at all).

A private hosted zone attached to one or more VPCs answers queries for internal-only names, and is a significant operational upgrade over hardcoding private IP addresses directly into application configuration — a backend instance can be replaced, its private IP can change, and every dependent service continues working unmodified because they reference the stable DNS name, not the IP that happens to sit behind it today.

Split-horizon DNS (the same zone name resolving differently depending on whether the query originates inside or outside the VPC) is the pattern that lets checkout-service.example.com resolve to a public, internet-facing IP for external customers while an internal checkout-service.example.com query (from inside the VPC, hitting a private hosted zone with a higher-priority match) resolves to a private, internal load balancer IP for service-to-service traffic that should never leave the VPC or pass through a public-facing load balancer at all — reducing both latency (no unnecessary round trip out to the internet and back) and unnecessary public attack surface for traffic that was always meant to stay internal.

VPC Flow Logs — Network-Layer Observability#

Every mechanism covered so far in this chapter — route tables, Security Groups, NACLs, peering — makes a silent, binary allow/deny decision on every packet, with no default record of what it decided or why. VPC Flow Logs capture metadata about IP traffic flowing through a VPC's network interfaces — source and destination IP/port, protocol, byte/packet counts, and critically, whether the flow was ACCEPTed or REJECTed — without capturing the packet payload itself (a deliberate design choice, keeping flow logs usable for network troubleshooting without also becoming a full packet-capture compliance/privacy liability).

The second row's REJECT — a connection attempt to port 5432 (Postgres) that never completed — is exactly the kind of signal that distinguishes a Security Group block (which appears here, explicitly) from a missing route table entry (which does not — a route table drop happens before flow logging ever sees the packet at all, since there's no interface to attach the log to).

Multi-Account Landing Zones — Shared VPC and Centralized Egress#

A large organization rarely runs every workload inside one flat account with one VPC — the now-standard practice (AWS's Control Tower/Landing Zone, GCP's equivalent Shared VPC and Resource Manager hierarchy, Azure's Management Groups) is one AWS account (or GCP project) per team or workload, with networking centralized and shared from a dedicated network account — isolating a security incident or a cost overrun in one team's account from ever directly touching another's, while still providing centrally managed, consistent connectivity.

Shared VPC / Transit Gateway sharing is the mechanism that makes this practical: a central network account owns the actual VPC (or the Transit Gateway), and shares specific subnets or attachments with other accounts via the cloud provider's cross-account resource-sharing primitive (AWS Resource Access Manager, GCP's Shared VPC host/service project model) — application teams deploy their workloads into subnets they don't own or directly control the routing for, while the central network team retains sole ownership of the actual network architecture, route tables, and Transit Gateway attachments.

Application teams get self-service deployment into a subnet without needing (or being able to accidentally misconfigure) the shared network's own routing, NAT, or Transit Gateway architecture.

Centralized egress is the specific, high-value pattern this model enables: instead of every team's account running and paying for its own NAT Gateway fleet, all outbound internet traffic from every shared account routes through a small number of NAT Gateways owned centrally in the network account — cutting the NAT Gateway cost multiplication that would otherwise scale linearly with team/account count, and giving the security team a single, centrally-controlled, centrally-logged egress point to apply organization-wide egress filtering (an explicit allowlist of permitted outbound destinations, a common requirement in regulated environments) rather than needing to replicate and audit that policy separately in every individual team's account.

Multi-Region and Multi-Cloud Network Design Patterns#

Bringing this chapter's pieces together at the scale most large platforms eventually operate at:

• Multi-region, single cloud: each region gets its own VPC (never a single VPC spanning regions — cloud VPCs are fundamentally regional constructs), connected via the cloud provider's own inter-region backbone (AWS's inter-region VPC peering, or a Transit Gateway peering connection between regions) rather than routing inter-region traffic out over the public internet.
• Multi-cloud: no single managed hub-and-spoke product spans providers — connectivity between an AWS VPC and a GCP VPC genuinely requires either a site-to-site VPN between the two clouds' own gateway products, or a third-party network-as-a-service overlay (Aviatrix and similar products exist specifically to provide the missing "Transit Gateway that spans clouds" abstraction). This is a real, material added complexity most single-cloud platforms never have to plan for, and a driver of genuine platform-team headcount at organizations that adopt multi-cloud primarily for negotiating leverage rather than a workload-specific technical need.
• Compliance-driven regional isolation: checkout-service's EU expansion (first introduced in Part 1's anycast worked scenario) needs its EU customer data to stay within the EU region's network boundary as a hard requirement, not just a preference — implemented at the network layer by deliberately not Transit-Gateway-connecting the EU VPC to the US VPC for data-plane traffic, even though both regions are part of the same overall platform, with only a narrowly-scoped, audited connection (a specific PrivateLink endpoint, not a general routable peering) permitted for the specific cross-region metadata sync the compliance review actually approved.

Centralized Egress Filtering — Network Firewall as a Managed Inspection Point#

The centralized-egress pattern from earlier in this chapter (all outbound traffic routed through a small number of NAT Gateways in a shared network account) creates a natural, single choke point for something beyond plain address translation: deep, policy-driven inspection of what's actually leaving the network, not just permitting or denying based on IP/port the way a Security Group or NACL does.

A managed network firewall (AWS Network Firewall, Azure Firewall, GCP Cloud NGFW) sits inline at exactly that centralized-egress choke point and applies stateful, often domain-name-aware and even payload-aware rules — "outbound HTTPS is only permitted to an explicit allowlist of approved third-party API domains," "block outbound traffic matching known malware command-and-control signatures," "alert on any outbound connection attempting to reach a raw IP address with no matching DNS-based allowlist entry at all" (a real, common signature of a compromised host attempting to exfiltrate data or reach a C2 server directly, bypassing normal DNS-resolved traffic patterns entirely).

A Security Group or NACL alone has no concept of "which domain is this HTTPS connection actually going to" — that's payload/SNI-level inspection, a genuinely different, deeper capability layered on top of the IP/port-level controls covered earlier in this chapter.

Full Worked Scenario: Redesigning the Platform's Network for a Second Region and a Compliance Boundary#

Following directly from Part 1's checkout-service EU-expansion scenario, this chapter's worked example covers the network architecture underneath that expansion in full, rather than just the anycast frontend covered there.

Starting state: a single VPC in us-east-1 (CIDR 10.0.0.0/16), with the standard three-tier subnet layout from earlier in this chapter, one NAT Gateway per AZ, and a Transit Gateway already in place connecting it to a separate shared-data VPC used by the analytics team.

The redesign, applying this chapter's concepts in sequence:

1. CIDR planning first: the EU VPC is allocated 10.1.0.0/16 — deliberately non-overlapping with the existing us-east-1 VPC's 10.0.0.0/16, following the central IPAM discipline covered earlier, so that any future connectivity requirement between the two remains possible without a re-IP.
2. Regional Transit Gateway peering, but deliberately scoped: the EU Transit Gateway is peered to the US Transit Gateway specifically for a narrow, audited set of prefixes (the internal admin tooling subnet, needed for platform-team operational access) — not a blanket "connect everything" peering, directly implementing the compliance-driven isolation pattern from the previous section.
3. No production data-plane route exists between the two regions' application subnets at all — this is the actual enforcement mechanism for "EU customer data stays in the EU," verified concretely by inspecting the Transit Gateway's route table for the EU attachment and confirming the US application subnet's CIDR is genuinely absent from it, not merely restricted by a Security Group that a future change could accidentally loosen.
4. VPC Endpoints for every AWS-managed service the EU workloads depend on (Secrets Manager, KMS), so that a genuinely private, EU-region-only path exists for every dependency — avoiding any scenario where a seemingly-internal API call to a managed AWS service transits back through a US-region endpoint by default.

What the compliance audit caught that the network diagram alone hadn't surfaced: the redesign correctly blocked EU-to-US data-plane routing at the network layer, but the audit found that checkout-service's own application-level configuration still pointed its centralized logging pipeline at a single, US-region log aggregation endpoint — meaning EU request logs, which could contain customer PII depending on what got logged, were being shipped out of the EU region entirely, over the legitimately-permitted admin/tooling Transit Gateway path, silently bypassing the very isolation the network redesign was built to enforce. The lesson that generalizes, and the reason this scenario is worth including in a networking chapter rather than treating it as purely an application-layer concern: network-layer isolation only enforces what traffic the network topology allows — it says nothing about what an application deliberately chooses to send over a connection the network topology does, correctly, permit. The fix required both a network-layer change (a regional logging endpoint, so EU logs never needed a cross-region path in the first place) and an application-configuration change — a reminder that a genuinely audited compliance boundary needs review at every layer, not just the one this chapter's own material covers.

Common Mistakes and Interview Traps#

Worked Practice Problems#

Problem 1: An application in a private subnet can successfully connect to an internal service in a peered VPC, but a curl to an internet-hosted public API from that same private subnet times out. The route table has a route for the peered VPC's CIDR, and a NAT Gateway exists in the VPC. What's the most likely misconfiguration?

Answer: The private subnet's route table is very likely missing (or has an incorrect) default route (0.0.0.0/0) pointing at the NAT Gateway — the presence of a working, specific route to the peered VPC's CIDR doesn't imply the separate, broader default route to the NAT Gateway is also correctly configured; each destination needs its own explicit route table entry, and one working route says nothing about another.

Problem 2: Two VPCs, 10.0.0.0/16 and 10.0.128.0/17, need to be connected. What's wrong with this plan before any peering request is even attempted?

Answer: 10.0.128.0/17 falls entirely within 10.0.0.0/16's address range (10.0.0.0/16 spans 10.0.0.0 through 10.0.255.255, and 10.0.128.0/17 spans 10.0.128.0 through 10.0.255.255 — a strict subset). These two VPCs have overlapping CIDR blocks and cannot be peered at all — the peering request itself would be rejected. This is exactly the outcome the central IPAM/CIDR-planning discipline covered in this chapter exists to prevent, and the only real fix at this point is re-IPing one of the two VPCs entirely, a disruptive, high-effort remediation that upfront planning avoids.

Problem 3: A compliance requirement states that a specific subnet's traffic must never reach the public internet, under any circumstance, even if a future engineer misconfigures a route table. What's the most robust way to enforce this, beyond simply "don't add a NAT Gateway route"?

Answer: Don't rely on the absence of a route as the sole enforcement mechanism, since a future change could add one. A more robust approach layers multiple independent controls: no NAT Gateway or Internet Gateway route configured for that subnet (removes the easy path entirely), a restrictive NACL on the subnet with an explicit DENY for any destination outside the organization's known private CIDR ranges (a control that survives even if a route table change is accidentally made, since the NACL is evaluated independently of the route table), and, where the platform supports it, an automated compliance-as-code check (in the Terraform/IaC pipeline covered in this site's own Terraform series) that fails a deploy outright if it would introduce a route to 0.0.0.0/0 on a subnet tagged as compliance-restricted — turning "don't misconfigure this" from a matter of individual engineer discipline into an automatically enforced guardrail.

Problem 4: A security team wants to guarantee that no workload can exfiltrate data to an arbitrary external IP address, only to a specific, approved list of third-party API domains. A Security Group allowlisting outbound HTTPS to specific IP ranges is proposed as the fix. Why does this fall short of the actual requirement, and what's the right layer to enforce it at?

Answer: A Security Group's outbound rules operate on IP addresses/CIDR ranges, not domain names — and most third-party APIs (and every CDN-fronted one) resolve to IP addresses that change over time and are shared across many unrelated customers of that same CDN/cloud provider, making a stable, accurate IP-based allowlist impractical to maintain and, worse, prone to either blocking legitimate traffic when an IP rotates or under-restricting because the allowlisted range is far broader than the one specific domain actually intended. The requirement described — "only these specific domains" — needs domain-aware (SNI or DNS-based) inspection, which is exactly what a managed network firewall provides, sitting at the centralized egress point covered earlier in this chapter, rather than something a Security Group's IP/port-only model can express at all.

Summary and What's Next#

Every mechanism in this chapter — CIDR planning, subnets and route tables, the peering-vs-Transit-Gateway scaling curve, VPN and dedicated connections, and DNS split-horizon resolution — is the infrastructure that Parts 1 and 2's routing and load-balancing concepts actually run on top of inside a cloud account. Getting this layer right (or wrong) shapes every later networking decision a platform makes, and — as this chapter's worked scenario showed — even a network layer that's genuinely correctly designed only enforces what the network topology allows, not what an application built on top of it actually chooses to do.

The recurring theme worth carrying forward: almost every mechanism in this chapter exists in layered pairs, not single controls — Security Groups paired with NACLs, a NAT Gateway paired with an egress-only Internet Gateway for the address family that doesn't need translation, a dedicated connection paired with a VPN fallback, direct peering paired with a Transit Gateway once the mesh outgrows it. Treating any one of these as sufficient on its own, rather than as one layer of a deliberately redundant design, is the thread running through most of this chapter's From the Trenches callouts.

Part 4 moves from network topology to the security and protocol layer riding on top of it: the TLS handshake in full mechanical detail, how mTLS extends it to mutual authentication between services, and how HTTP itself has evolved from HTTP/1.1 through HTTP/2 to HTTP/3's QUIC-based transport — the protocols that actually carry checkout-service's traffic across every network this series has covered so far, whether that traffic is crossing the public internet from Part 1, being split across a backend fleet by Part 2's load balancer, or routed between the VPCs and subnets this chapter just laid out.