Companion question bank for the 3-part tutorial series in this folder: 01-scaling-strategies.md, 02-queuing-theory-and-capacity-math.md, 03-autoscaling-and-load-testing.md.

Answers are short and plain — expand out loud using the diagrams and worked examples in the tutorials.

---

Part 1 Questions: Scaling Strategies

1. What's the difference between vertical and horizontal scaling?#

Vertical: make one machine bigger (more CPU/RAM). Horizontal: add more machines and spread the work across them. Vertical is simple but has a real cost/hardware ceiling; horizontal has no practical ceiling but needs real design work.

2. Why does vertical scaling have a "hidden ceiling" beyond just running out of bigger machines to buy?#

Cost grows faster than linearly as machines get bigger — the biggest, most specialized hardware carries a real premium well before you actually hit a hard physical limit.

3. Why is horizontal scaling easy for stateless services but hard for stateful ones?#

A stateless service doesn't remember anything between requests, so any new copy is immediately as good as any existing one. A stateful service (like a database) holds data that matters — a brand-new copy doesn't automatically have the existing data, so scaling it requires a real decision about how state gets shared or partitioned.

4. What problem do read replicas solve, and what do they cost you?#

They scale read capacity by copying data to read-only servers, letting reads bypass the primary entirely — a big win since most applications read far more than they write. The cost is replication lag: a replica can briefly serve slightly stale data, the same eventual-consistency tradeoff from the CAP theorem.

5. How would you handle a user needing to see their own just-written data when reads normally go to replicas?#

Route that specific read back to the primary instead of a replica — the same "read-your-writes consistency" pattern from the CAP theorem tutorial.

6. What's the single biggest risk when scaling one layer of a system without checking the others?#

You just relocate the bottleneck instead of removing it — e.g., scaling app servers 10x just means 10x more traffic now hits the same database, which may never have been a problem before.

7. What's the core tradeoff with caching, and what's the classic joke about it?#

Cache invalidation — a cache that's never invalidated serves stale data; one invalidated too aggressively barely helps at all. "There are only two hard things in computer science: cache invalidation and naming things."

---

Part 2 Questions: Queuing Theory & Capacity Math

8. State Little's Law and explain it in plain English.#

L = λ × W. In plain terms: the number of things "in the system" (waiting + being served) equals the rate things arrive, multiplied by how long each one stays. It works for any queue, and lets you solve for whichever variable you don't know.

9. A service handles 200 requests/sec, each taking 0.25 seconds on average. How many concurrent requests are "in flight" on average?#

L = λ × W = 200 × 0.25 = 50 concurrent requests — a directly actionable number for sizing worker/thread pools.

10. Why does a small slowdown in processing time cause a queue to explode, not just grow a little?#

Because L = λ × W — if arrival rate stays the same but processing time (W) jumps (say, from a slow downstream dependency), queue depth (L) scales proportionally, and even a modest slowdown multiplies into a huge queue buildup.

11. Why does queue length explode as utilization approaches 100%, rather than growing steadily?#

Real traffic is bursty, not perfectly smooth. At low utilization there's spare capacity to absorb bursts invisibly. As utilization nears 100%, that spare capacity shrinks to nothing, so even small bursts have nowhere to go except into a rapidly growing queue.

12. Is running a server at 100% utilization "efficient"? Why or why not?#

No — it's fragile, not efficient. The "unused" capacity at lower utilization is actively doing work: absorbing real-world burstiness so queues don't explode. Running at 100% leaves zero buffer for any random spike.

13. Why is pure trend-based forecasting not enough for capacity planning?#

It only extrapolates smooth historical growth and misses known, discrete events — a marketing campaign, a product launch, a seasonal spike — that don't show up as a smooth trend. Good forecasting combines trend extrapolation with direct business input.

14. Why should capacity be planned against peak traffic, not average traffic?#

Real traffic constantly exceeds its own average — planning for exactly the average guarantees the system gets overwhelmed the moment traffic goes even slightly above it, which happens constantly. Same "averages hide the real story" lesson as latency percentiles.

15. What factors should influence how much headroom (buffer) you plan for?#

How bursty/predictable the traffic is, how fast you can react if the forecast is wrong (autoscaling in seconds vs. ordering hardware over weeks), and how costly running out of capacity would be versus the cost of some idle capacity.

---

Part 3 Questions: Autoscaling & Load Testing

16. What's the actual HPA scaling formula?#

desired replicas = ceil(current replicas × (current metric value / target metric value)).

17. Work through: 6 replicas running, target CPU 50%, current average CPU is 120%. What does HPA calculate?#

ceil(6 × (120/50)) = ceil(6 × 2.4) = ceil(14.4) = 15 replicas.

18. Why is CPU utilization frequently the wrong metric to scale on?#

For I/O-bound services (mostly waiting on a database or network call), CPU can stay low even while the service is completely overwhelmed, because it's stuck waiting, not computing — HPA would never trigger. Scale on whatever actually reflects real load: queue depth, request rate, or a custom metric.

19. What is "flapping" in autoscaling, and how do you prevent it?#

Rapid, wasteful scale-down/scale-up cycles as a metric bounces around its target. Prevented with a stabilization window on scale-down (require the metric to stay low for a sustained period, e.g. 5 minutes, before removing capacity) — scale-up deliberately uses little to no stabilization window since reacting fast to real spikes matters more.

20. What's the relationship between HPA, the Scheduler, and Cluster Autoscaler?#

HPA scales pod replica count. If the cluster's existing nodes don't have room, new pods sit Pending. Cluster Autoscaler notices Pending pods with nowhere to go and provisions new nodes from the cloud provider. It's a layered system — each piece only solves its own part of the puzzle.

21. Why doesn't autoscaling eliminate the need for any static capacity buffer?#

There's real "cold-start" latency between demand rising and new capacity actually serving traffic — detecting the spike, scheduling a pod, pulling the image, app warmup, and possibly provisioning a whole new node. That chain can take anywhere from tens of seconds to a few minutes, during which some static headroom (a sensible minReplicas floor) is what actually absorbs the gap.

22. When would you use predictive/scheduled autoscaling instead of relying purely on reactive HPA?#

For known, repeating traffic patterns (a reliable daily 9am spike, a weekly surge) — scheduled scaling raises the floor ahead of time, removing the cold-start lag for predictable events, while reactive HPA still handles anything above that baseline, including genuinely unpredictable spikes.

23. Distinguish load, stress, spike, and soak tests.#

Load: expected, realistic traffic — can we handle a normal day? Stress: push beyond expected levels until something breaks — where's our actual ceiling? Spike: a sudden sharp jump — can we survive a flash sale or viral moment? Soak: moderate load sustained for hours/days — do we leak memory or slowly degrade?

24. Why can't a 10-minute load test catch a memory leak?#

A leak that adds a small amount of memory per hour is invisible over 10 minutes — it only becomes visible after sustained operation over hours or days, which is exactly what a soak test is specifically designed to catch.

25. Why do tools like k6 support automated "thresholds" (pass/fail criteria)?#

It turns a load test into an automated gate — the test itself fails (non-zero exit code) if p95 latency or error rate breach a defined limit, making it possible to run load tests automatically in CI/CD before every release instead of relying on a human eyeballing a graph.

26. When reading load test results, why check p95/p99 instead of just the average?#

Same reason as always — averages hide tail-latency outliers. A load test's average latency can look perfectly healthy while a meaningful percentage of requests are experiencing real, painful slowness.

27. What's a safe way to load test using real production traffic patterns without risking actual users?#

Shadow traffic — mirror a copy of real production traffic to a new system without it affecting real users, or canary load testing (send synthetic load only to a small canary deployment, never the full fleet). Always paired with circuit breakers, rate limits, and a clear kill switch.

---

Quick-Fire / Rapid Recall#