# Linux & Networking Fundamentals — Part 2: TCP/IP & DNS

> **Series:** Linux & Networking Fundamentals (2 of 7)
> **Part 1:** `01-linux-process-and-memory-internals.md` — Process & Memory Internals
> **Part 2:** This file — TCP/IP & DNS
> **Part 3:** `03-linux-troubleshooting-toolkit.md` — The Linux Troubleshooting Toolkit
> **Part 4:** `04-netfilter-iptables-nftables.md` — Netfilter, iptables & nftables
> **Part 5:** `05-namespaces-and-virtual-networking.md` — Network Namespaces & Virtual Networking
> **Part 6:** `06-systemd-deep-dive.md` — systemd Deep Dive
> **Part 7:** `07-ebpf-observability-and-networking.md` — eBPF for Observability & Networking
> **Questions:** `questions.md`

## Table of Contents

1. [Why Networking Fundamentals Matter for an SRE](#why-networking-fundamentals-matter-for-an-sre)
2. [The Layered Model, Simplified](#the-layered-model-simplified)
3. [IP Addresses and Packets](#ip-addresses-and-packets)
4. [Ports — How One Machine Serves Many Things](#ports--how-one-machine-serves-many-things)
5. [TCP vs UDP — The Fundamental Tradeoff](#tcp-vs-udp--the-fundamental-tradeoff)
6. [The TCP Three-Way Handshake](#the-tcp-three-way-handshake)
7. [TCP Connection States](#tcp-connection-states)
8. [The TIME_WAIT Problem](#the-time_wait-problem)
9. [TCP's Reliability Machinery](#tcps-reliability-machinery)
10. [TLS/SSL — Encrypting the Connection](#tlsssl--encrypting-the-connection)
11. [What DNS Actually Is](#what-dns-actually-is)
12. [The Full DNS Resolution Journey](#the-full-dns-resolution-journey)
13. [Common DNS Record Types](#common-dns-record-types)
14. [DNS Caching and TTL](#dns-caching-and-ttl)
15. [DNS Troubleshooting Commands](#dns-troubleshooting-commands)
16. [Common Mistakes](#common-mistakes)
17. [Worked Practice Problems](#worked-practice-problems)
18. [Summary and What's Next](#summary-and-whats-next)

---

## Why Networking Fundamentals Matter for an SRE

Almost every production incident eventually touches the network — a service can't reach its database, a load balancer health check fails, DNS returns a stale IP. If Part 1 was about understanding one machine deeply, this part is about understanding **how machines actually talk to each other** — the layer underneath every "service call" and "API request" mentioned throughout this entire course.

---

## The Layered Model, Simplified

Real networking education uses the formal 7-layer OSI model, but for SRE/DevOps interview purposes, a simplified 4-layer mental model covers what actually gets asked and used day to day.

```mermaid
graph TD
    App["Application Layer<br/>(HTTP, DNS, gRPC — the actual<br/>data your app cares about)"] --> Transport["Transport Layer<br/>(TCP, UDP — how data<br/>gets delivered reliably<br/>or quickly)"]
    Transport --> Internet["Internet Layer<br/>(IP — how data finds its<br/>way across networks to<br/>the right MACHINE)"]
    Internet --> Link["Link Layer<br/>(Ethernet, Wi-Fi — how data<br/>moves across ONE physical<br/>network segment)"]
```

**Simple analogy for the whole stack:** sending an HTTP request is like mailing a letter. The Application layer is the actual letter's content. The Transport layer (TCP) is choosing "registered mail with delivery confirmation" (reliable) vs. a "postcard" (fast, no guarantee). The Internet layer (IP) is the street address that gets it to the right building. The Link layer is the actual truck driving it there.

---

## IP Addresses and Packets

An **IP address** identifies a specific machine (or network interface) on a network. Data travels in small chunks called **packets**, each carrying a source and destination IP address, like a mailing address on every single envelope.

```bash
# Check your own machine's IP address(es)
ip addr show
# or the older, still-common command:
ifconfig

# Trace the actual network path (hop by hop) to a destination
traceroute example.com
```

**IPv4 vs IPv6, briefly:** IPv4 addresses (`192.168.1.1`) are 32-bit, giving about 4.3 billion possible addresses — which the internet has already run out of. IPv6 addresses (`2001:0db8::1`) are 128-bit, providing an astronomically larger address space specifically to solve that exhaustion. Most production systems today still run dual-stack (both) or IPv4-only internally, with IPv6 adoption varying significantly by region and provider.

---

## Ports — How One Machine Serves Many Things

An IP address gets you to the right *machine* — a **port** (a number from 0-65535) gets you to the right *application* running on that machine.

```mermaid
graph TD
    IP["One machine, one IP<br/>address: 10.0.0.5"] --> P1["Port 80: web server<br/>(HTTP)"]
    IP --> P2["Port 443: web server<br/>(HTTPS)"]
    IP --> P3["Port 5432: PostgreSQL<br/>database"]
    IP --> P4["Port 6379: Redis"]
```

**Simple analogy:** the IP address is the apartment building's street address; the port number is the specific apartment/unit number within that building — the mail carrier (network) gets the envelope to the right building using the address, and the building's own internal system (the OS) routes it to the right apartment using the unit number.

| Port | Common Service |
|---|---|
| 22 | SSH |
| 53 | DNS |
| 80 | HTTP |
| 443 | HTTPS |
| 3306 | MySQL |
| 5432 | PostgreSQL |
| 6379 | Redis |
| 9090 | Prometheus (common default) |

---

## TCP vs UDP — The Fundamental Tradeoff

One of the single most common networking interview questions — a genuinely important tradeoff to understand deeply, not just recite.

```mermaid
graph TD
    TCP["TCP:<br/>Connection-oriented,<br/>RELIABLE, ORDERED"] --> TCPNote["Guarantees delivery and<br/>correct order, using<br/>acknowledgments and<br/>retransmission — but this<br/>reliability costs overhead<br/>and latency"]

    UDP["UDP:<br/>Connectionless,<br/>NO guarantees"] --> UDPNote["Fire-and-forget — no<br/>handshake, no<br/>acknowledgment, no<br/>retransmission — much<br/>lower overhead and latency,<br/>but packets can arrive out<br/>of order, duplicated, or<br/>not at all"]
```

**Simple analogy:** TCP is a registered letter — the post office confirms it arrived, and resends it if it doesn't. UDP is shouting a message across a room — fast and simple, but if someone didn't hear it, nobody automatically resends it, and there's no confirmation either way.

| | TCP | UDP |
|---|---|---|
| Delivery guarantee | Yes — retransmits lost packets | No — "best effort" only |
| Ordering guarantee | Yes — reassembles packets in order | No — packets can arrive out of order |
| Overhead | Higher (handshake, acknowledgments, retransmission logic) | Lower — minimal overhead |
| Typical use cases | Web traffic (HTTP), databases, anything where correctness matters more than raw speed | Video/voice calls, DNS, gaming, anything where a dropped packet is better tolerated than added latency |

**Why DNS (mostly) uses UDP, a common specific follow-up question:** DNS queries are small, and if one is lost, the client just retries quickly — the overhead of a full TCP handshake for every tiny lookup would be wasteful. (DNS does fall back to TCP for larger responses, like zone transfers or responses exceeding a certain size — worth knowing this nuance exists.)

**Why video/voice calls use UDP, another common follow-up:** if one small piece of audio is lost, you want the call to keep moving forward smoothly (a tiny, likely-unnoticed glitch) rather than pause and wait for TCP to detect the loss and retransmit the old, now-stale audio — by the time it arrived, the conversation has already moved on.

---

## The TCP Three-Way Handshake

Before any data flows over TCP, both sides perform a specific three-step exchange to establish a reliable connection — one of the most frequently diagrammed sequences in all of networking interviews.

```mermaid
sequenceDiagram
    participant Client
    participant Server

    Client->>Server: SYN (synchronize -<br/>I'd like to connect)
    Server->>Client: SYN-ACK (synchronize<br/>+ acknowledge - OK,<br/>let's connect)
    Client->>Server: ACK (acknowledge -<br/>confirmed, connection established)
    Note over Client,Server: Connection is now OPEN -<br/>actual data can flow
```

**Why three steps, specifically, not two:** both sides need to confirm they can both send AND receive successfully. Step 1 proves the client can send to the server. Step 2 proves the server can both receive from and send to the client. Step 3 proves the client can receive from the server — after all three, both sides have mutually confirmed a working, two-way connection.

### The Four-Way Close

Closing a TCP connection is its own, similarly formal exchange (each side independently signals "I'm done sending," since a connection is technically two independent one-way streams):

```mermaid
sequenceDiagram
    participant Client
    participant Server

    Client->>Server: FIN (I'm done sending)
    Server->>Client: ACK (acknowledged)
    Server->>Client: FIN (I'm done sending too)
    Client->>Server: ACK (acknowledged)
```

---

## TCP Connection States

```mermaid
stateDiagram-v2
    [*] --> LISTEN: server waiting<br/>for connections
    LISTEN --> SYN_RECEIVED: SYN received
    SYN_RECEIVED --> ESTABLISHED: ACK received,<br/>handshake complete
    ESTABLISHED --> ESTABLISHED: data flows
    ESTABLISHED --> FIN_WAIT: this side<br/>initiates close
    FIN_WAIT --> TIME_WAIT: close sequence<br/>completing
    TIME_WAIT --> [*]: connection fully<br/>closed, after a wait
```

```bash
# See all current TCP connections and their states
ss -tan
# or the older equivalent:
netstat -tan

# Count connections by state - genuinely useful for real diagnosis
ss -tan | awk '{print $1}' | sort | uniq -c | sort -rn
```

---

## The TIME_WAIT Problem

A specific, commonly-tested, real production issue worth understanding deeply.

After a connection closes, the side that initiated the close (usually, but not always, the client) holds the connection in **TIME_WAIT** state for a period (commonly 60-120 seconds, OS-dependent) before fully releasing it.

```mermaid
graph TD
    A["Why TIME_WAIT exists:<br/>ensures any DELAYED,<br/>duplicate packets from the<br/>old connection are safely<br/>discarded, rather than being<br/>mistaken for a NEW<br/>connection reusing the<br/>same port"] --> B["Genuinely necessary for<br/>correctness"]

    C["The PROBLEM: a server<br/>handling a very HIGH rate<br/>of short-lived connections<br/>can accumulate THOUSANDS<br/>of TIME_WAIT connections,<br/>eventually exhausting<br/>available local ports"] --> D["🚨 New connections start<br/>failing - looks like the<br/>server is 'down,' but it's<br/>actually just out of<br/>available ports"]
```

**A concrete interview scenario worth having ready:** "A load-testing tool making thousands of short-lived HTTP connections per second can exhaust the client machine's available ephemeral ports, because each closed connection sits in TIME_WAIT for a minute or two before its port is reusable — this can make a perfectly healthy server *look* broken from the client's perspective, when the real bottleneck is the client's own local port exhaustion." Common mitigations: reuse persistent/keep-alive connections instead of opening a new one per request (directly connects to the connection-pooling discussion from the Reliability & Architecture Patterns series), or tune the OS's `net.ipv4.tcp_tw_reuse` setting.

```bash
# Count TIME_WAIT connections specifically
ss -tan state time-wait | wc -l
```

---

## TCP's Reliability Machinery

A few of the specific mechanisms that make TCP's reliability guarantee actually work — worth knowing by name.

```mermaid
graph TD
    Reliability[TCP Reliability Mechanisms] --> Ack["Acknowledgments (ACKs):<br/>the receiver confirms<br/>which data it actually got"]
    Reliability --> Retrans["Retransmission:<br/>if an ACK doesn't arrive<br/>in time, the sender<br/>assumes it was lost and<br/>resends it"]
    Reliability --> Window["Flow control (window<br/>size): the receiver tells<br/>the sender how much data<br/>it can accept right now,<br/>preventing it from being<br/>overwhelmed"]
    Reliability --> Congestion["Congestion control:<br/>the sender deliberately<br/>SLOWS DOWN if it detects<br/>signs of network congestion<br/>(lost packets), then<br/>gradually speeds back up"]
```

**Why congestion control matters for an SRE, specifically:** it's the reason a sudden burst of packet loss on a network path can cause TCP throughput to drop dramatically and then only slowly recover — TCP is deliberately, intentionally cautious about speeding back up after detecting loss, to avoid immediately re-causing the same congestion. A real, sustained network issue (not just a one-off blip) can therefore have an outsized, lingering effect on throughput even after the underlying problem clears.

---

## TLS/SSL — Encrypting the Connection

**TLS** (the modern successor to the older, now-deprecated **SSL**) adds encryption and authentication on top of a TCP connection — this is what turns HTTP into HTTPS.

```mermaid
sequenceDiagram
    participant Client
    participant Server

    Client->>Server: TCP three-way handshake<br/>(as above)
    Client->>Server: ClientHello (supported<br/>TLS versions/ciphers)
    Server->>Client: ServerHello + Certificate<br/>(proves server identity)
    Client->>Client: Verifies certificate against<br/>a trusted Certificate Authority
    Client->>Server: Key exchange - both sides<br/>derive a shared encryption key
    Note over Client,Server: Connection is now<br/>ENCRYPTED - actual<br/>application data flows
```

**Why this adds real, measurable latency, worth knowing as a concrete fact:** the TLS handshake requires its own additional round-trips *on top of* the TCP three-way handshake — this is exactly why techniques like **TLS session resumption** (reusing a previously-negotiated session's parameters to skip a full new handshake) and **connection keep-alive** (avoiding a fresh handshake for every single request) matter for real-world latency, directly connecting to the resilience/connection-pooling patterns from the Reliability & Architecture Patterns series.

```bash
# Inspect a server's TLS certificate details
openssl s_client -connect example.com:443 -servername example.com < /dev/null

# See the full request/response including TLS handshake timing
curl -v https://example.com
```

---

## What DNS Actually Is

**DNS (Domain Name System)** is, at its core, a giant, distributed, hierarchical phone book — it translates human-friendly names (`example.com`) into the IP addresses machines actually need to connect to each other.

```mermaid
graph LR
    Human["Human types:<br/>'example.com'"] --> DNS["DNS resolves this to..."]
    DNS --> IP["...an actual IP address:<br/>'93.184.216.34'"]
```

**Why this exists at all:** IP addresses are hard for humans to remember and can change over time (a server migration, a new load balancer). DNS provides a stable, memorable name that can point to a changing underlying IP address — directly the same idea behind the DNS-based load balancing and failover discussion in the Reliability & Architecture Patterns series.

---

## The Full DNS Resolution Journey

This exact sequence — "what actually happens when you type a URL and hit enter" — is one of the most classic, comprehensive SRE/networking interview questions of all. Being able to walk through it fully, in order, is high-value.

```mermaid
sequenceDiagram
    participant Browser
    participant OS as OS/Local Resolver
    participant Resolver as Recursive DNS Resolver<br/>(e.g. your ISP or 8.8.8.8)
    participant Root as Root DNS Server
    participant TLD as TLD DNS Server<br/>(.com)
    participant Auth as Authoritative DNS Server<br/>(for example.com)

    Browser->>OS: Resolve example.com
    OS->>OS: Check local cache first
    OS->>Resolver: Not cached - ask the<br/>recursive resolver
    Resolver->>Resolver: Not cached either
    Resolver->>Root: Who handles .com?
    Root-->>Resolver: Here's the .com<br/>TLD server
    Resolver->>TLD: Who handles example.com?
    TLD-->>Resolver: Here's example.com's<br/>authoritative server
    Resolver->>Auth: What's the IP for<br/>example.com?
    Auth-->>Resolver: 93.184.216.34
    Resolver-->>OS: 93.184.216.34<br/>(and caches it)
    OS-->>Browser: 93.184.216.34
    Browser->>Browser: NOW opens a TCP<br/>connection to that IP
```

**A strong interview answer names every layer of this hierarchy by name — root servers, TLD servers, authoritative servers — and explicitly notes that caching happens at multiple layers along the way (the OS, the recursive resolver), which is exactly why DNS changes don't propagate instantly (directly connecting to the DNS-based failover latency discussion from the Reliability & Architecture Patterns series).**

---

## Common DNS Record Types

| Record Type | Purpose | Example |
|---|---|---|
| **A** | Maps a name to an IPv4 address | `example.com -> 93.184.216.34` |
| **AAAA** | Maps a name to an IPv6 address | `example.com -> 2606:2800:220:1::` |
| **CNAME** | Maps a name to ANOTHER name (an alias) | `www.example.com -> example.com` |
| **MX** | Specifies mail servers for a domain | `example.com -> mail priority 10` |
| **TXT** | Arbitrary text — often used for domain verification, SPF/email security records | `example.com -> "v=spf1 include:..."` |
| **NS** | Specifies which servers are authoritative for a domain | `example.com -> ns1.example.com` |
| **SOA** | "Start of Authority" — administrative info about a zone (primary server, TTL defaults, etc.) | — |

**A commonly-tested nuance: CNAME records cannot coexist with other records for the same name** (e.g., you can't have both a CNAME and an A record for the same exact hostname) — a real, practical DNS configuration constraint worth knowing exists.

---

## DNS Caching and TTL

Every DNS record has a **TTL (Time To Live)** — how long a resolver is allowed to cache it before checking again.

```mermaid
graph TD
    A["Short TTL (e.g. 60s)"] --> A1["✅ Changes propagate fast"]
    A --> A2["❌ More DNS query load,<br/>slightly more latency on<br/>average (more frequent<br/>lookups needed)"]

    B["Long TTL (e.g. 24h)"] --> B1["✅ Less DNS query load,<br/>better average latency"]
    B --> B2["❌ Changes propagate SLOWLY<br/>- some clients keep using<br/>a stale/old IP for a long time"]
```

**A directly practical, real-world tactic worth knowing: lowering the TTL well in advance of a planned migration or failover event.** If you know you're migrating to a new server next week, dropping the TTL from 24 hours to, say, 5 minutes a day or two ahead of time means that when the actual cutover happens, stale caches expire and pick up the new IP quickly — instead of some users being stuck on the old IP for up to a full day after the change.

---

## DNS Troubleshooting Commands

```bash
# The modern, detailed way to query DNS
dig example.com

# Query a SPECIFIC record type
dig example.com MX

# Query a SPECIFIC DNS server directly (bypassing local cache/resolver)
dig @8.8.8.8 example.com

# See just the answer, nothing else
dig +short example.com

# Trace the FULL resolution path, root servers included
dig +trace example.com

# Older, simpler alternative
nslookup example.com

# Reverse DNS lookup (IP -> hostname)
dig -x 93.184.216.34
```

---

## Common Mistakes

| Mistake | Why It's Wrong | Fix |
|---|---|---|
| Assuming TCP and UDP are interchangeable, "pick whichever" | They have fundamentally different guarantees (reliable/ordered vs. best-effort/fast) — the wrong choice can cause real correctness or latency problems | Choose based on whether correctness/ordering or raw speed/low-overhead matters more for the specific use case |
| Not accounting for TIME_WAIT when load testing or building high-throughput short-connection clients | Can silently exhaust local ports, making a healthy server look broken from the client's perspective | Reuse persistent/keep-alive connections instead of opening a new one per request where possible |
| Assuming a DNS change takes effect immediately everywhere | DNS is cached at multiple layers (OS, recursive resolver) according to each record's TTL | Lower the TTL well in advance of a planned change, and expect propagation delay proportional to the TTL |
| Treating TLS handshake overhead as negligible | It adds real additional round-trips on top of the TCP handshake, especially costly for many short-lived connections | Use connection keep-alive and TLS session resumption to avoid repeating the full handshake unnecessarily |
| Confusing a recursive resolver with an authoritative server | They play very different roles in the DNS hierarchy | Recursive resolvers do the lookup legwork on a client's behalf and cache results; authoritative servers hold the actual, canonical records for a domain |

---

## Worked Practice Problems

**Problem 1:** A load testing tool making 5,000 short-lived HTTPS requests per second against a staging server starts reporting connection failures after a few minutes, even though the server's own dashboards show it healthy and responsive the whole time. What's the likely explanation?

*Answer:* The load testing client is likely exhausting its own local ephemeral port pool due to TIME_WAIT accumulation — each closed short-lived connection holds its local port in TIME_WAIT for roughly 60-120 seconds, and at 5,000 new connections/second, that adds up to potentially hundreds of thousands of ports sitting in TIME_WAIT within just a couple of minutes, exceeding the available ephemeral port range. The fix isn't on the server side at all (which is why its dashboards look fine) — it's reusing persistent, keep-alive connections in the load testing tool instead of opening a brand-new connection per request, or tuning the client OS's port reuse settings.

**Problem 2:** A company plans a database migration next Tuesday, cutting over via a DNS change from the old server's IP to the new one. The DNS record currently has a 24-hour TTL, unchanged. What risk does this create, and what should they do in advance?

*Answer:* With a 24-hour TTL, some clients (and intermediate resolvers) could continue using the old, cached IP address for up to a full day after the DNS record is actually updated — meaning traffic could still be hitting the old, migrated-away-from server well after the cutover, potentially against a database that's no longer being kept in sync. They should lower the TTL (e.g., to 5 minutes) several days before the migration, giving existing longer-TTL caches time to naturally expire and be replaced with the new, short TTL, so that when the actual cutover happens, stale caches clear out quickly across the board.

**Problem 3:** Explain, step by step, what happens between a user typing `https://example.com` into their browser and the first byte of the actual page arriving.

*Answer:* First, DNS resolution: the browser checks its own cache, then the OS resolver's cache, then queries a recursive resolver, which (if not already cached) walks the hierarchy from root servers to the `.com` TLD servers to `example.com`'s authoritative server, ultimately returning an IP address. Next, a TCP three-way handshake (SYN, SYN-ACK, ACK) establishes a reliable connection to that IP on port 443. Then, since this is HTTPS, a TLS handshake occurs on top of that TCP connection — exchanging supported versions/ciphers, verifying the server's certificate against a trusted Certificate Authority, and deriving a shared encryption key. Only after all of that completes does the browser send its actual HTTP GET request over the now-encrypted connection, and the server's first response bytes come back.

---

## Summary and What's Next

- **TCP** is reliable and ordered but has more overhead; **UDP** is fast and simple but offers no delivery or ordering guarantees — choose based on which property (correctness or speed) matters more for the use case.
- The **TCP three-way handshake** (SYN, SYN-ACK, ACK) establishes a connection by mutually confirming both sides can send and receive; closing uses a similar four-step exchange.
- **TIME_WAIT** exists for good reason (discarding delayed duplicate packets safely) but can genuinely exhaust local ports under very high connection churn — a real, specific, commonly-tested production issue.
- **TLS** adds encryption/authentication on top of TCP, at the cost of additional handshake round-trips — connection reuse and session resumption matter for real-world latency.
- **DNS** is a distributed, hierarchical, heavily-cached lookup system (root -> TLD -> authoritative servers) — being able to walk through the full resolution journey step by step is one of the most classic SRE/networking interview questions.
- **DNS TTL** controls the tradeoff between propagation speed and query load — lower it deliberately, well in advance of any planned migration or cutover.

**Continue to Part 3** (`03-linux-troubleshooting-toolkit.md`) to put everything from Parts 1 and 2 into practice with the actual command-line toolkit SREs use to diagnose real production issues.