Autoscaling: HPA, VPA, KEDA & Cluster Autoscaling

Assumes you're comfortable with resource requests/limits and QoS classes from Part 2, and Karpenter/Cluster Autoscaler's node-level scaling from Part 7 — this chapter fills the pod-level autoscaling gap those parts referenced but never fully covered.

Table of Contents#

1. Why This Part Exists
2. Three Kinds of Autoscaling, and How They Compose
3. HPA Deep Dive — the Control Loop Mechanics
4. HPA Metric Types — Resource, Pods, Object, External
5. HPA behavior — Stabilization Windows and Scaling Policies
6. Scaling on Custom Metrics — a Prometheus Adapter Worked Example
7. VPA Deep Dive — Recommender, Updater, Admission Controller
8. VPA updateMode — Off, Initial, Recreate, InPlaceOrRecreate
9. Why HPA and VPA Conflict on the Same Metric
10. KEDA Deep Dive — Architecture
11. A Worked KEDA Example: Scaling on Queue Depth, Including Scale-to-Zero
12. KEDA ScaledJob — Scaling Jobs, Not Deployments
13. KEDA vs. Plain HPA — Decision Framework
14. Autoscaling StatefulSets — What's Different
15. Tying Pod Scaling to Node Scaling
16. A Full Worked Scenario: The Whole Stack Under Load
17. Monitoring Autoscaler Health
18. Cost Implications of Autoscaling Choices
19. Common Pitfalls: Flapping, Mismatched Requests, Autoscaler Races
20. Scheduled and Predictive Scaling — Handling Known Traffic Patterns
21. Part 12 CLI Cheat Sheet
22. Quick Reference: Every Autoscaler, Side by Side
23. Common Mistakes and Interview Traps
24. Sane Starting Defaults by Workload Tier
25. Worked Practice Problems
26. Summary and What's Next

Why This Part Exists#

Autoscaling is one of the highest-leverage levers a platform team has, and also one of the easiest to misconfigure quietly — a wrongly-tuned autoscaler rarely fails loudly, it just costs more or serves worse than it should for months before anyone notices. Autoscaling references have appeared throughout this series without a dedicated treatment — Part 2 mentioned it as the trigger condition for scheduling pressure, Part 7 referenced "VPA (Part 2)" for a right-sizing decision that Part 2 never actually covered — and that gap matters, because autoscaling is where most of a cluster's real cost and reliability tradeoffs actually get made.** This chapter covers all three pod/cluster autoscaling mechanisms Kubernetes offers in the depth CKA and CKAD both expect, and — per the industry research behind this series' chapter list — KEDA specifically, now a CNCF-graduated project running in production at thousands of organizations.

The throughline system continues: checkout-service handling synchronous HTTP traffic (a natural fit for HPA), and a new order-processor component consuming from a queue (the natural fit for KEDA's event-driven model), both living in the checkout namespace.

Three Kinds of Autoscaling, and How They Compose#

These three mechanisms answer three different questions, and a mature production setup typically runs some combination of all three simultaneously, not one instead of the others.

The three layers are complementary, not competing — HPA/KEDA decide how many pods to run, VPA decides how big each one should be, and cluster-level autoscaling makes sure there's somewhere to actually put them once HPA has decided to add more. A cluster running only HPA with permanently wrong, hand-guessed resource requests wastes money on oversized requests or gets OOMKilled on undersized ones (Part 10) no matter how well HPA itself is tuned — VPA is what keeps the inputs to that scaling decision honest. Losing sight of this layering is a common source of confused troubleshooting: a symptom that looks like "HPA isn't working" is often actually a VPA sizing problem, or a node-capacity problem one layer below HPA entirely.

HPA Deep Dive — the Control Loop Mechanics#

The HorizontalPodAutoscaler is a controller running its own reconciliation loop (Part 1's general controller pattern, specialized for this one job) — by default every 15 seconds, it compares a target metric against the desired value and computes a new replica count.

The formula itself is worth memorizing exactly, because it explains behavior that otherwise looks surprising: desiredReplicas = ceil(currentReplicas × currentMetricValue / desiredMetricValue). If checkout-service runs 4 replicas averaging 80% CPU against a 50% target, HPA computes ceil(4 × 80 / 50) = 7 — a jump of 3 replicas in one reconciliation, not a gradual +1. This is why HPA without behavior tuning (next section) can look "twitchy" under bursty load: the math itself is proportional, not incremental.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: checkout-service
  namespace: checkout
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: checkout-service
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 50

HPA Metric Types — Resource, Pods, Object, External#

autoscaling/v2 supports four distinct metric source types, and picking the right one for a given signal is what separates HPA configs that actually track real load from ones that technically work but scale on the wrong thing.

metrics:
  - type: Resource
    resource:
      name: cpu
      target: { type: Utilization, averageUtilization: 50 }
  - type: Pods
    pods:
      metric: { name: http_requests_per_second }
      target: { type: AverageValue, averageValue: "200" }

When an HPA lists more than one metric, Kubernetes computes a desired replica count independently for each one and takes the largest — the autoscaler is deliberately biased toward over-provisioning rather than under-provisioning when signals disagree, on the reasoning that the cost of a few extra replicas is almost always smaller than the cost of an outage from under-scaling on the metric that mattered.

HPA behavior — Stabilization Windows and Scaling Policies#

Without behavior tuning, HPA's default scale-down stabilization window is 300 seconds and scale-up is 0 seconds — meaning it reacts to a load spike almost instantly but waits five minutes of sustained lower load before scaling back down, a deliberately asymmetric default to avoid flapping.

spec:
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
        - type: Percent
          value: 100          # can double replica count in one step
          periodSeconds: 60
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Pods
          value: 2            # remove at most 2 pods per step
          periodSeconds: 60

The stabilization window doesn't delay the decision — it changes which value the decision is based on. During the scale-down stabilization window, HPA doesn't wait silently and then check once; it continuously tracks the highest recommended replica count seen across the entire window and only scales down to that value, which is precisely what prevents a brief dip in load from triggering a scale-down that a load spike 30 seconds later would immediately have to reverse.

Scaling on Custom Metrics — a Prometheus Adapter Worked Example#

Scaling checkout-service on requests-per-second instead of CPU requires a metrics adapter translating Prometheus queries into the Kubernetes Custom Metrics API — CPU/memory work out of the box via metrics-server, but any other signal needs this extra piece.

# prometheus-adapter config: maps a PromQL query to a Kubernetes metric name
rules:
  - seriesQuery: 'http_requests_total{namespace="checkout",pod!=""}'
    resources:
      overrides:
        namespace: { resource: "namespace" }
        pod: { resource: "pod" }
    name:
      matches: "http_requests_total"
      as: "http_requests_per_second"
    metricsQuery: 'sum(rate(<<.Series>>{<<.LabelMatchers>>}[2m])) by (<<.GroupBy>>)'

kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1/namespaces/checkout/pods/*/http_requests_per_second" | jq .

Verifying the adapter is actually exposing the metric before troubleshooting HPA itself is the fast path here — if the custom.metrics.k8s.io query above returns data, HPA will work; if it returns an empty list or an error, the problem is entirely in the Prometheus/adapter pipeline, and no amount of editing the HPA object will fix it.

VPA Deep Dive — Recommender, Updater, Admission Controller#

The Vertical Pod Autoscaler is three separate components working together, not one monolithic controller — understanding the split explains exactly when and how it actually changes a running pod's resources.

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: checkout-service
  namespace: checkout
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: checkout-service
  updatePolicy:
    updateMode: "Off"    # recommend only — see next section
  resourcePolicy:
    containerPolicies:
      - containerName: checkout-service
        minAllowed: { cpu: 100m, memory: 128Mi }
        maxAllowed: { cpu: 2, memory: 2Gi }

The Recommender never touches a running pod by itself — it only ever writes a recommendation to the VPA object's status. Whether that recommendation actually changes anything running depends entirely on updateMode, which is why Off mode is both completely safe and genuinely useful on its own: it gives you real, workload-specific sizing data (kubectl describe vpa checkout-service) without any risk of an unexpected pod eviction.

VPA updateMode — Off, Initial, Recreate, InPlaceOrRecreate#

Why HPA and VPA Conflict on the Same Metric#

Running HPA and VPA on the same Deployment, both targeting CPU, creates a feedback loop neither one is designed to resolve: VPA raises CPU requests to fit observed usage, which changes the utilization percentage HPA is computing against, which changes HPA's replica count, which changes per-pod load and therefore VPA's next observed usage.

Working the numbers through one full cycle makes the loop concrete rather than abstract. Start with 4 replicas, each requesting 500m CPU, each actually using 400m (80% utilization against a 50% target — HPA wants to scale up). VPA's recommender, watching the same usage data, recommends raising the request to 800m per pod to reduce the ratio of usage-to-request it considers healthy. Once applied, that same 400m of real usage against an 800m request is now only 50% utilization — exactly at HPA's target, so HPA now computes zero need to scale. If usage later climbs to 480m per pod under the new 800m request (60%), HPA scales up again, redistributing load and dropping per-pod usage — which VPA's next observation window sees as evidence its 800m recommendation was too generous, nudging it back down, restarting the cycle. Neither controller is malfunctioning; each is behaving exactly as designed on an input the other one keeps moving.

The supported combination is HPA on one metric and VPA on a different one it doesn't influence — the most common working pattern is HPA scaling on a custom/external metric (requests-per-second, queue depth) while VPA right-sizes CPU/memory requests, since neither one's decision then feeds back into the other's input. Scaling both mechanisms on CPU or memory simultaneously for the same workload is explicitly unsupported and will produce exactly the oscillation described above — the fix is always changing which input each controller reacts to, never tuning either one's thresholds more finely, since finer tuning only changes how fast the cycle oscillates, not whether it does at all.

KEDA Deep Dive — Architecture#

KEDA doesn't replace HPA — it extends it, by generating and managing an HPA object behind the scenes while adding two things vanilla HPA doesn't have: a huge catalog of 60+ built-in event-source scalers, and native scale-to-zero.

Scale-to-zero is the genuinely new capability HPA structurally cannot offer — a standard HPA's minReplicas can't go below 1, because HPA has no way to know a workload should wake back up once there are zero pods left to observe any metric from. KEDA solves this by running its own lightweight polling loop outside the HPA/pod lifecycle entirely: at zero replicas, KEDA itself watches the event source directly, and the moment a new message/event appears, it scales the Deployment back up to minReplicaCount (which can genuinely be 0), at which point the normal HPA mechanism takes back over for everything above 1.

A Worked KEDA Example: Scaling on Queue Depth, Including Scale-to-Zero#

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: order-processor
  namespace: checkout
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: order-processor
  minReplicaCount: 0
  maxReplicaCount: 30
  cooldownPeriod: 120
  pollingInterval: 15
  triggers:
    - type: aws-sqs-queue
      metadata:
        queueURL: https://sqs.us-east-1.amazonaws.com/123456789/order-events
        queueLength: "5"          # target: 5 messages per replica
      authenticationRef:
        name: keda-trigger-auth-aws-credentials

kubectl get scaledobject order-processor -n checkout
kubectl get hpa -n checkout    # confirm KEDA's generated HPA: keda-hpa-order-processor

The cooldownPeriod/pollingInterval pair matters for cost vs. responsiveness tradeoffs the same way HPA's behavior stabilization windows do — a short pollingInterval reacts to a new message quickly (low latency to first response) but polls the event source more often (a real, if usually small, cost for a managed queue service); a long cooldownPeriod avoids flapping to zero and back but means paying for idle replicas slightly longer after traffic genuinely stops.

KEDA ScaledJob — Scaling Jobs, Not Deployments#

Everything above (ScaledObject) targets a scalable resource like a Deployment — ScaledJob is a distinct KEDA CRD for a different shape of problem entirely: creating one Kubernetes Job per unit of work, rather than scaling replica count on a long-running Deployment.

apiVersion: keda.sh/v1alpha1
kind: ScaledJob
metadata:
  name: invoice-batch-processor
  namespace: checkout
spec:
  jobTargetRef:
    template:
      spec:
        containers:
          - name: invoice-processor
            image: registry.internal/invoice-processor:2.1.0
        restartPolicy: Never
    backoffLimit: 2
  minReplicaCount: 0
  maxReplicaCount: 50
  pollingInterval: 10
  triggers:
    - type: aws-sqs-queue
      metadata:
        queueURL: https://sqs.us-east-1.amazonaws.com/123456789/invoice-batches
        queueLength: "1"

ScaledJob is the better fit whenever "one message, one isolated unit of work, then exit" describes the processing model more accurately than "a pool of workers continuously pulling" — invoice generation, one-off video transcoding, or any batch task where a single item's failure genuinely shouldn't risk the state of whatever else a shared long-running pod happened to be doing at the same time.

KEDA vs. Plain HPA — Decision Framework#

Autoscaling StatefulSets — What's Different#

HPA and VPA both technically support targeting a StatefulSet instead of a Deployment, but the ordinal, identity-bound nature of StatefulSet pods (Part 2) changes what "scaling" actually costs and how safely it can happen.

Tying Pod Scaling to Node Scaling#

Part 7 covered Karpenter and Cluster Autoscaler in full depth — the connection worth making explicit here is that pod-level and node-level autoscaling are two independent control loops linked only by one signal: an unschedulable pod.

HPA has no awareness of node capacity at all, and Karpenter/Cluster Autoscaler have no awareness of HPA's scaling decisions — the only coupling between the two systems is the Scheduler leaving a pod Pending when it can't place it, which is precisely why a burst of new replicas from HPA can show up briefly stuck Pending (Part 10's scheduling diagnostic tree) even in a perfectly healthy cluster: it's waiting on a new node, not failing.

A Full Worked Scenario: The Whole Stack Under Load#

Walking a single traffic event through every layer covered in this chapter and Part 7, in the order things actually happen:

The end-to-end latency from "traffic spikes" to "fully absorbed" is dominated by node provisioning (t=15s to t=90s above), not by HPA's own decision speed — HPA reacted within one sync period, but new compute capacity takes real wall-clock time to boot regardless of how fast the scaling decision was made. This is exactly why over-provisioning a small buffer of headroom (or using Karpenter's consolidation settings conservatively, Part 7) matters for genuinely latency-sensitive services: the autoscaling math can be perfect and a service can still serve degraded traffic for 60-90 seconds purely waiting on physical node boot time.

Monitoring Autoscaler Health#

An autoscaler that's silently misbehaving is worse than one that's off, because the system looks automated and handled right up until the moment it isn't — a handful of specific metrics turn "is autoscaling actually working" from a guess into a real, alertable signal.

kubectl get hpa -n checkout -w
kubectl describe hpa checkout-service -n checkout | tail -15

Cost Implications of Autoscaling Choices#

Every decision in this chapter has a direct FinOps consequence, and treating autoscaling as a pure reliability lever while ignoring its cost side produces predictably expensive surprises.

Common Pitfalls: Flapping, Mismatched Requests, Autoscaler Races#

Scheduled and Predictive Scaling — Handling Known Traffic Patterns#

Every mechanism covered so far is reactive — it scales after a metric crosses a threshold. For load that's genuinely predictable (a checkout system's traffic reliably climbs at 9am on a weekday, or spikes hard on a known sale date), waiting for a reactive signal wastes the exact node-boot latency window this chapter's worked scenario walked through.

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: checkout-service-scheduled
  namespace: checkout
spec:
  scaleTargetRef: { name: checkout-service }
  minReplicaCount: 3
  maxReplicaCount: 20
  triggers:
    - type: cron
      metadata:
        timezone: America/New_York
        start: 0 8 * * 1-5     # 8am weekdays: pre-scale ahead of the 9am traffic ramp
        end: 0 19 * * 1-5      # 7pm weekdays: relax back to reactive-only scaling
        desiredReplicas: "10"

KEDA's cron trigger composes with any other trigger on the same ScaledObject — the effective replica count is the maximum across all active triggers, so a cron trigger here acts as a scheduled floor (guaranteeing at least 10 replicas during known business hours) while a CPU or queue-depth trigger can still scale above that floor reactively if real load exceeds what was predicted.

Part 12 CLI Cheat Sheet#

Quick Reference: Every Autoscaler, Side by Side#

A single comparison table worth keeping close, since these four mechanisms are easy to mix up under exam or incident pressure.

Common Mistakes and Interview Traps#

Sane Starting Defaults by Workload Tier#

Teams new to tuning these controllers often either leave every default untouched or over-tune from day one — a reasonable starting point by workload criticality avoids both extremes.

These are starting points to tune from with real traffic data, not permanent settings — revisit them after the workload has run long enough to show its actual pattern, the same way VPA's own recommendations improve as it accumulates more observed history. A quarterly review of these settings against actual observed traffic and cost data is a reasonable cadence for most teams — frequent enough to catch a workload that's outgrown its original sizing assumptions, infrequent enough not to become its own maintenance burden.

Worked Practice Problems#

Problem 1: An HPA targeting 50% CPU utilization shows <unknown> under TARGETS in kubectl get hpa, and the Deployment never scales regardless of load. What's the single most likely cause, and how do you confirm it in one command?

Answer: The target container almost certainly has no CPU resources.requests set — Utilization targets are computed as a percentage of the request, and with no request there's no denominator for that percentage, so the metrics pipeline has nothing valid to report. Confirm with kubectl get deployment checkout-service -o jsonpath='{.spec.template.spec.containers[*].resources}'; an empty or missing requests field confirms the cause immediately.

Problem 2: A team configures both an HPA and a VPA on the same Deployment, both targeting CPU, hoping for "the best of both." A week later they notice replica count oscillating unpredictably with no corresponding change in real traffic. Explain what's happening.

Answer: VPA raising the container's CPU request lowers the observed utilization percentage HPA computes (same usage divided by a now-larger request), which HPA interprets as reduced load and scales down for; fewer replicas then absorb the same total traffic, raising per-pod usage back up, which VPA's next recommendation cycle reacts to by raising the request further — the two controllers are each reacting to a symptom the other one caused, with no traffic change required to keep the cycle going. The fix is scaling HPA and VPA on different metrics for this workload, or using VPA only in Off (recommendation-only) mode alongside HPA.

Problem 3: A ScaledObject sets minReplicaCount: 0 for a queue-consuming Deployment, but the team notices new messages sit in the queue for up to 30 seconds before a replica appears to process them, even though the queue was previously empty. Is this a bug, and if not, what governs that delay?

Answer: Not a bug — it's KEDA's pollingInterval (how often it checks the event source while scaled to zero) plus the real time needed to schedule and start a new pod from zero. A pollingInterval of 15-30 seconds means, in the worst case, a message can sit for nearly that entire interval before KEDA even notices it exists, on top of normal pod startup time; lowering pollingInterval trades a small increase in polling cost against lower worst-case latency for the first message after an idle period.

Problem 4: A ScaledJob processing invoices has maxReplicaCount: 50 and a queue that briefly backs up to 400 messages during a batch import, then drains. The team wants to know whether this is safe to leave as configured. What should they check before deciding?

Answer: maxReplicaCount on a ScaledJob directly caps concurrently running Jobs, so 50 concurrent invoice-processor pods will each need scheduling capacity (Part 7's Karpenter/Cluster Autoscaler) and each likely holds a connection to whatever backing service (a database, a third-party invoicing API) the processor calls — the real question isn't whether KEDA can create 50 Jobs, it's whether that downstream dependency can absorb 50x its normal concurrent load without being the actual bottleneck or getting rate limited. Checking the downstream service's own concurrency/rate limits before treating maxReplicaCount as a safe ceiling is the necessary follow-up, not just confirming the KEDA config is syntactically fine.

Summary and What's Next#

HPA, VPA, and KEDA solve three different sizing questions and are designed to compose — the one hard rule is never letting two of them react to the same metric on the same workload. KEDA extends rather than replaces HPA, adding scale-to-zero and a wide event-source catalog while generating a standard HPA underneath for everything above one replica. None of this operates in isolation from node-level scaling — an HPA scale-up decision and a Karpenter/Cluster Autoscaler node-provisioning decision are two separate control loops linked only through the Scheduler's Pending state, which is why real-world scale-up latency is often dominated by node boot time rather than any autoscaler's own decision speed.

Part 13 moves to a structural rather than reactive concern: multi-tenancy. Running many teams or customers safely on one shared cluster raises questions ResourceQuota and LimitRange (Part 2) only partially answer — Part 13 covers the harder isolation boundaries (namespace-based multi-tenancy limits, virtual clusters, and policy-based tenant isolation) that a growing platform team eventually has to solve. The two chapters connect directly: a shared cluster running autoscaling per the patterns in this chapter needs multi-tenant guardrails to stop one team's maxReplicas ceiling (or a KEDA scaler gone haywire on a bad event source) from starving every other tenant's workload of the very node capacity Karpenter or Cluster Autoscaler just provisioned in response. Recognizing which of HPA, VPA, or KEDA a given production symptom actually calls for — rather than reaching for whichever one a team happens to already know — is most of the practical skill this chapter aimed to build.