Table of Contents#
- Why This Part Exists — And Why Right Now
- The Problem With Ingress — Why It Needed Replacing
- Gateway API's Persona-Based Resource Model
- GatewayClass — Selecting an Implementation
- Gateway — The Cluster Operator's Listener Configuration
- HTTPRoute — Application-Team-Owned Routing
- A Minimal Gateway + HTTPRoute, Built Up Step by Step
- GRPCRoute — Native gRPC Routing
- TCPRoute, UDPRoute, and TLSRoute — Beyond HTTP
- Advanced HTTPRoute — Header Matching, Filters, and Traffic Splitting
- Cross-Namespace Routing and ReferenceGrant
- Migrating From Ingress — Ingress2Gateway and the ingress-nginx Retirement
- Envoy Gateway — Architecture Overview
- Envoy Gateway's xDS Control Plane
- Installing Envoy Gateway
- The EnvoyProxy CRD — Customizing the Data Plane
- EnvoyPatchPolicy — the Escape Hatch for Advanced Envoy Config
- A Full Worked Example: Envoy Gateway End to End
- Security Policies in Envoy Gateway
- Gateway API Conformance Levels — Core, Extended, and Implementation-Specific
- Observability With Envoy Gateway
- Istio as a Gateway API Implementation — Ambient Mode, ztunnel, Waypoints
- Cilium Gateway — the eBPF-Native Implementation
- NGINX Gateway Fabric, Traefik, and Kong — the Rest of the Field
- Implementation Feature Comparison — a Reference Table
- Choosing an Implementation — a Decision Framework
- Gateway API and Service Mesh — Where the Line Blurs
- Debugging a Misconfigured Gateway — Status Conditions
- Common Mistakes
- Worked Practice Problems
- Key Terms Glossary — This Chapter's Vocabulary in One Place
- Summary and What's Next
Why This Part Exists — And Why Right Now#
Part 3's networking chapter mentioned Gateway API in a single paragraph — "worth knowing exists, even briefly" — and moved on, because that chapter's job was establishing the Service/Ingress model everyone actually ran for the past decade. That deferral has an expiration date: ingress-nginx, the single most widely deployed Ingress controller in the ecosystem, is officially retired as of March 2026 — no further security patches, no further updates, from the Kubernetes project itself. Every cluster still running it is now running unmaintained, security-exposed ingress infrastructure. This Part exists because "Gateway API is the future" quietly became "Gateway API is the thing you migrate to this year" for a very large share of the ecosystem, and a course claiming production depth on Kubernetes networking can no longer treat it as an optional footnote.
This Part covers the Gateway API specification itself and Envoy Gateway as its most architecturally instructive, purpose-built implementation — deliberately not a cloud-managed one, so the mechanics are visible rather than hidden behind a provider's own abstraction. Part 9 immediately following covers what changes when a team runs Gateway API on a specific managed platform instead.
The Problem With Ingress — Why It Needed Replacing#
Ingress was never actually a complete specification — it was a minimal common denominator that every controller vendor (NGINX, Traefik, HAProxy, cloud load balancers) then extended through annotations, because the base Ingress object simply didn't have fields for the routing behavior real production traffic needed (canary weighting, header-based routing, TLS passthrough policy, rate limiting).
This annotation sprawl is the concrete, structural problem Gateway API was designed to fix, not a vague "Ingress is old" complaint. A Kubernetes-wide, YAML-only definition for "canary weight" or "header match" simply never existed, so every implementation independently invented its own string-keyed annotation dialect. Migrating a real production Ingress manifest between controllers routinely meant translating dozens of vendor-specific annotations by hand, with no compiler or validator to catch a missed one. Gateway API's routing resources (HTTPRoute, GRPCRoute) express traffic splitting, header matching, and request/response filtering as first-class, strongly-typed API fields — checked by the Kubernetes API server's own schema validation, not string-matched by whatever controller happens to be watching.
A second, structural gap: Ingress has one object type for everyone — the same Ingress resource is edited by the platform team configuring TLS certs and the application team configuring path routing, with no separation of concern or RBAC boundary between those two very different jobs. Gateway API's persona-based split (covered next) fixes this directly.
Gateway API's Persona-Based Resource Model#
Gateway API's single biggest architectural decision, worth understanding before any individual resource: it deliberately splits configuration across three separate object types, each owned by a different team/persona, with Kubernetes RBAC enforcing the boundary between them.
This is the single largest practical difference from Ingress worth internalizing: an application team can now own and edit their own routing rules (an HTTPRoute in their own namespace) without ever needing write access to the shared Gateway object that defines the actual listener, port, and TLS certificate — a real RBAC boundary Ingress's single flat object never offered. A platform team retains sole control of the shared entry point's TLS/listener configuration while delegating routing-rule ownership outward to the teams who actually know their own service's routing needs — the same "narrow the blast radius of who can touch what" discipline this series has argued for throughout, from RBAC in Part 1 to NetworkPolicy in Part 3.
GatewayClass — Selecting an Implementation#
GatewayClass is the cluster-scoped resource an infrastructure provider (or, for a self-managed cluster, the platform team standing up Envoy Gateway) installs once — it names an implementation (via a controllerName the implementation's own controller watches for) and optionally references implementation-specific configuration through parametersRef.
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: envoy-gateway
spec:
controllerName: gateway.envoyproxy.io/gatewayclass-controller
parametersRef:
group: gateway.envoyproxy.io
kind: EnvoyProxy
name: custom-proxy-config
namespace: envoy-gateway-systemA cluster can run multiple GatewayClass objects side by side, each pointing to a different implementation's controller — a genuinely useful pattern this chapter returns to in its decision-framework section: an internal-only GatewayClass backed by Envoy Gateway for lightweight, purpose-built routing, alongside an istio GatewayClass for workloads that also need full mesh mTLS, both coexisting in the same cluster, each application team simply choosing which GatewayClass their own Gateway references.
Gateway — The Cluster Operator's Listener Configuration#
A Gateway is namespace-scoped and defines the actual listeners — ports, protocols, and TLS certificate references — that the referenced GatewayClass's implementation should provision.
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: production-gw
namespace: gateway-infra
spec:
gatewayClassName: envoy-gateway
listeners:
- name: https
protocol: HTTPS
port: 443
tls:
mode: Terminate
certificateRefs:
- name: prod-tls-cert
allowedRoutes:
namespaces:
from: AllallowedRoutes.namespaces.from: All is the specific field worth calling out, since it's the exact mechanism that makes the persona split practical rather than theoretical — it tells the implementation "accept HTTPRoute attachments from any namespace in the cluster," which is what lets application teams attach their own routes from their own namespaces without the platform team pre-authorizing each one individually. The alternative values (Same, restricting attachment to the Gateway's own namespace, or Selector, restricting to namespaces matching a label selector) exist for platform teams who want tighter control over which application namespaces may attach routes at all — worth choosing deliberately rather than defaulting to All on a genuinely multi-tenant cluster where namespace isolation matters, per Part 7's own multi-tenancy discussion.
HTTPRoute — Application-Team-Owned Routing#
HTTPRoute is the resource an application team actually edits day to day — it lives in the application's own namespace and attaches to a Gateway via parentRefs.
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: checkout-route
namespace: checkout
spec:
parentRefs:
- name: production-gw
namespace: gateway-infra
hostnames:
- "checkout.example.com"
rules:
- matches:
- path:
type: PathPrefix
value: /api
backendRefs:
- name: checkout-svc
port: 8080This is the direct functional replacement for the path/host rules that used to live inside an Ingress object — the meaningful difference is that this HTTPRoute lives in the checkout namespace, editable by the checkout team, referencing a shared Gateway in a completely separate gateway-infra namespace they don't otherwise have write access to. A single HTTPRoute can also define multiple rules, each with its own matches and backendRefs, letting one route object express several distinct paths for the same hostname — the same consolidation an Ingress object's own multi-path rule list used to provide, just with typed, per-rule filter and weight fields available where Ingress had none.
A Minimal Gateway + HTTPRoute, Built Up Step by Step#
Worth walking the full request path end to end, tying GatewayClass, Gateway, and HTTPRoute together into one coherent picture before moving into any implementation-specific detail.
Step 1 — the platform team installs an implementation and creates one GatewayClass (done once, cluster-wide). Step 2 — the platform team creates a Gateway in a shared namespace, defining the actual listener (port 443, TLS cert) that the implementation's controller watches and provisions real infrastructure for (an Envoy proxy Deployment and Service, in Envoy Gateway's case — covered in depth later in this chapter). Step 3 — each application team creates its own HTTPRoute, in its own namespace, referencing that shared Gateway by name via parentRefs, and defining only the routing rules relevant to its own service. The implementation's controller watches all three resource types together and continuously reconciles the actual data-plane configuration (Envoy's xDS config, in this chapter's reference implementation) to match — the exact same declarative, continuously-reconciled model this entire series has built up since Part 1's control-plane discussion, now applied specifically to L7 routing.
GRPCRoute — Native gRPC Routing#
GRPCRoute, standardized in Gateway API v1.1 and now widely implemented, gives gRPC traffic its own first-class routing resource rather than forcing it through HTTP path-matching semantics that don't map cleanly onto gRPC's own service/method addressing model.
apiVersion: gateway.networking.k8s.io/v1
kind: GRPCRoute
metadata:
name: inventory-grpc
namespace: inventory
spec:
parentRefs:
- name: production-gw
rules:
- matches:
- method:
service: inventory.v1.InventoryService
method: CheckStock
backendRefs:
- name: inventory-grpc-svc
port: 9090Matching by service/method directly, rather than by URL path, is the concrete improvement worth naming — a gRPC service's actual routing unit is a service-and-method pair, not a URL path segment, and forcing gRPC through Ingress's path-based matching historically meant either brittle path-convention workarounds or bypassing Ingress-level routing for gRPC traffic entirely. GRPCRoute closes that gap as a genuinely native primitive, not a bolted-on extension.
TCPRoute, UDPRoute, and TLSRoute — Beyond HTTP#
Three further route types extend Gateway API below Layer 7, worth knowing exist even for a team whose immediate need is purely HTTP:
| Route type | Layer | Use case |
|---|---|---|
| TLSRoute | L4, TLS-aware | Routes based on SNI hostname without terminating TLS — the gateway passes the encrypted connection through, useful when the backend itself must terminate TLS (mTLS to the pod, or a non-HTTP TLS protocol) |
| TCPRoute | L4 | Pure TCP forwarding by listener port, with no protocol awareness at all — a database proxy, a custom binary protocol |
| UDPRoute | L4 | Pure UDP forwarding — DNS, some gaming/streaming protocols, syslog |
These three route types are worth flagging explicitly as Experimental-channel resources in the Gateway API project itself, and — critically for this chapter's later provider-specific coverage — not every implementation supports all of them. Envoy Gateway supports all three; several of the cloud-managed implementations covered in Part 9 (most notably AWS's own EKS Gateway API Controller) do not yet support TCPRoute/UDPRoute at all, meaning a team needing raw TCP/UDP load balancing on EKS specifically still has to fall back to a plain LoadBalancer Service or a different implementation — a concrete gap this chapter's Part 9 companion covers by provider.
A minimal TCPRoute example is worth seeing concretely, since its shape differs from HTTPRoute in one important way — it has no hostnames or matches field at all, since TCP has no concept of hostname or path; routing is purely by which listener port the traffic arrived on:
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: TCPRoute
metadata:
name: postgres-proxy-route
namespace: data
spec:
parentRefs:
- name: internal-tcp-gw
rules:
- backendRefs:
- name: postgres-proxy-svc
port: 5432Advanced HTTPRoute — Header Matching, Filters, and Traffic Splitting#
HTTPRoute's rules array supports genuinely rich matching and modification, expressed as typed fields rather than annotations — worth seeing the three most commonly used capabilities concretely.
Header-based matching, useful for beta-feature routing or internal-only debug traffic:
rules:
- matches:
- headers:
- name: x-beta-user
value: "true"
backendRefs:
- name: checkout-svc-beta
port: 8080Weighted traffic splitting, the exact mechanism a progressive-delivery tool (Argo Rollouts, Flagger — covered in the CI/CD & GitOps series' own progressive delivery chapter) drives programmatically during a canary rollout:
rules:
- backendRefs:
- name: checkout-svc-stable
port: 8080
weight: 90
- name: checkout-svc-canary
port: 8080
weight: 10Request header modification filters, replacing what used to require a controller-specific annotation:
rules:
- filters:
- type: RequestHeaderModifier
requestHeaderModifier:
add:
- name: x-request-source
value: gateway-api
backendRefs:
- name: checkout-svc
port: 8080The weight field on backendRefs deserves the strongest emphasis of the three, since it's the field a progressive-delivery controller like Argo Rollouts or Flagger actually programmatically edits during a canary rollout — Gateway API's native traffic-splitting primitive is precisely what lets those tools drive a canary without depending on a service mesh's own separate traffic-splitting CRD, a meaningful simplification for a team that wants progressive delivery without adopting a full mesh purely for that one capability.
Cross-Namespace Routing and ReferenceGrant#
A subtle but important security primitive: by default, an HTTPRoute cannot reference a backend Service in a different namespace from itself, even if a Gateway permits routes from many namespaces to attach. This default exists specifically to prevent one namespace's route from silently exfiltrating traffic to a Service in a namespace it was never granted access to.
ReferenceGrant is the resource that grants this permission — created in the target namespace, by the team that owns it, explicitly consenting to being referenced from a named source namespace and kind. This is the same "the resource being accessed must consent to being accessed, not just the resource doing the accessing" pattern already familiar from cross-namespace NetworkPolicy design in Part 3 — worth recognizing as a repeated Kubernetes security idiom rather than a Gateway-API-specific novelty.
Migrating From Ingress — Ingress2Gateway and the ingress-nginx Retirement#
The ingress-nginx retirement (March 2026, per this chapter's opening) turned Gateway API migration from a "someday" project into an active, time-bound one for a large share of the ecosystem — worth covering the actual migration path concretely rather than leaving it abstract.
ingress2gateway (a kubernetes-sigs project, reaching its 1.0 release in March 2026 — timed directly with the ingress-nginx retirement it was built to ease) reads a cluster's existing Ingress objects and generates the equivalent Gateway/HTTPRoute YAML automatically, understanding several vendor-specific annotation dialects (including ingress-nginx's own) well enough to translate common cases correctly. The tool is explicitly not a zero-touch migration — any annotation without a direct Gateway API field equivalent (a genuinely vendor-specific extension with no standardized concept) gets flagged for manual review rather than silently dropped or guessed at, consistent with this course's own repeated caution against blind, unverified automation for anything touching production traffic routing.
A team beginning this migration for real should run ingress2gateway against a non-production cluster's Ingress objects first, diff the generated HTTPRoute/Gateway YAML against the original Ingress's actual observed behavior, and only then plan a production cutover — the exact "verify before trusting an automated tool's output" discipline this course applies everywhere else automation touches a production system.
| Migration step | What to check |
|---|---|
Run ingress2gateway against a non-prod cluster | Tool completes without crashing; review its own flagged-annotation warnings first |
| Diff generated resources against original behavior | Every route, header rule, and TLS cert mapping matches the old Ingress's actual observed traffic handling |
| Test in a non-production environment | Real request traffic against the new Gateway/HTTPRoute stack, not just kubectl apply succeeding |
| Cut production traffic over | Only after the above two steps pass, ideally with a rollback path (keeping the old Ingress controller running in parallel briefly) |
| Decommission the old Ingress controller | Only once the new Gateway API stack has run cleanly under real production load for a genuine observation window |
Envoy Gateway — Architecture Overview#
Envoy Gateway is a dedicated, Gateway-API-native controller with no service-mesh ambitions of its own — worth stating precisely, since it's the single biggest architectural distinction from Istio (covered later in this chapter): Envoy Gateway exists purely to implement Gateway API's ingress/north-south traffic model well, not to also provide east-west mesh mTLS between every pod in a cluster.
The control plane / data plane split is worth internalizing precisely, since it explains where a real production incident's root cause tends to live: the envoy-gateway controller itself is a single control-plane Deployment — it never touches actual application traffic. The managed Envoy Proxy Deployment, provisioned and continuously reconciled by that controller per Gateway object, is the actual data plane handling every real request — meaning a production traffic issue (5xx errors, latency) is almost always a data-plane (Envoy Proxy Pod) problem, while a routing rule failing to apply at all (an HTTPRoute change with no observed effect) points back toward the control plane, worth distinguishing quickly during an actual incident rather than searching both simultaneously.
Envoy Gateway's xDS Control Plane#
Envoy's own configuration model — shared with Istio, Contour, and Gloo, all of which likewise translate Kubernetes-native resources into it — is xDS (Discovery Service), a family of gRPC streaming APIs (Listener Discovery Service, Route Discovery Service, Cluster Discovery Service, Endpoint Discovery Service) that push configuration changes to a running Envoy process dynamically, in memory, with no process restart or reload required.
This "no restart, in-memory, streamed" property is the concrete mechanism behind Gateway API's own promise of near-instant reconciliation — a routing change applied via kubectl apply is genuinely live against real traffic within milliseconds of the envoy-gateway controller translating it, not after some poll interval or reload window. This is the same class of property that made NGINX Ingress Controller's own reload-based model (a config change historically required an Nginx worker reload, briefly dropping in-flight connections on some versions) a real, if usually minor, operational rough edge that xDS-based implementations avoid structurally.
Installing Envoy Gateway#
The standard installation path is a single Helm release, followed by creating the GatewayClass this chapter's earlier section already showed:
helm install eg oci://docker.io/envoyproxy/gateway-helm \
--version v1.8.3 \
-n envoy-gateway-system \
--create-namespace
kubectl wait --timeout=5m -n envoy-gateway-system \
deployment/envoy-gateway --for=condition=AvailableAfter the controller is running and a GatewayClass referencing gateway.envoyproxy.io/gatewayclass-controller exists, creating any Gateway object that references that class automatically provisions a dedicated Envoy Proxy Deployment and Service for it — no separate, manual "install the data plane" step, since the controller handles that provisioning itself as part of its normal reconciliation loop.
The EnvoyProxy CRD — Customizing the Data Plane#
The EnvoyProxy custom resource, referenced from a GatewayClass's parametersRef (shown earlier in this chapter), is how a platform team customizes the actual managed Envoy Deployment Envoy Gateway provisions — resource requests/limits, replica count, Service type (LoadBalancer vs. ClusterIP vs. NodePort), and pod-level scheduling constraints.
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
name: custom-proxy-config
namespace: envoy-gateway-system
spec:
provider:
type: Kubernetes
kubernetes:
envoyDeployment:
replicas: 3
container:
resources:
requests:
cpu: 500m
memory: 512Mi
envoyService:
type: LoadBalancerWithout an EnvoyProxy CRD referenced, Envoy Gateway applies sensible defaults — but any real production deployment should set this explicitly, for exactly the same right-sizing and availability reasoning the CI/CD & GitOps series' self-hosted runner chapter argued for compute generally: a single-replica data-plane Deployment is a real availability risk for something sitting directly in every request's critical path, and default resource requests are rarely correctly sized for a specific cluster's actual traffic volume.
A PodDisruptionBudget for the managed Envoy Proxy Deployment deserves the same explicit attention — without one, a node drain or cluster upgrade (per Part 7's own upgrade-strategy discussion) can legally evict every Envoy Proxy replica simultaneously if the scheduler's own timing allows it, briefly taking down the entire ingress path for every route attached to that Gateway. The EnvoyProxy CRD's envoyDeployment spec accepts a standard Kubernetes PodDisruptionBudget-shaped configuration for exactly this reason — worth setting on any Gateway carrying real production traffic, not an optional hardening step reserved for later.
EnvoyPatchPolicy — the Escape Hatch for Advanced Envoy Config#
Gateway API's typed resources deliberately don't expose every one of Envoy's own extensive configuration surface — doing so would recreate the annotation-sprawl problem this chapter opened by criticizing, just with typed fields instead of annotation strings. EnvoyPatchPolicy is Envoy Gateway's own escape hatch for the genuinely rare case where a team needs to reach configuration Gateway API's standard resources don't yet expose.
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyPatchPolicy
metadata:
name: custom-envoy-tweak
namespace: gateway-infra
spec:
targetRef:
group: gateway.networking.k8s.io
kind: Gateway
name: production-gw
type: JSONPatch
jsonPatches:
- type: type.googleapis.com/envoy.config.listener.v3.Listener
name: production-gw/https
operation:
op: add
path: "/per_connection_buffer_limit_bytes"
value: 65536This is worth flagging explicitly as a last-resort mechanism, not a routine configuration path — a team reaching for EnvoyPatchPolicy regularly is a signal worth noticing, since it means genuinely standard Gateway API resources aren't covering that team's real needs, and heavy reliance on raw Envoy JSONPatches re-creates exactly the vendor-lock-in and non-portability problem Gateway API's typed resource model exists to avoid in the first place. This chapter's own worked practice problems return to this exact tension directly.
A Full Realistic Multi-Stage Pipeline: Envoy Gateway End to End#
Tying every mechanism from this chapter together into one coherent, deployable example — a platform team standing up production HTTPS ingress for two application teams sharing one Envoy Gateway installation:
# 1. GatewayClass — installed once by the platform team
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: envoy-gateway
spec:
controllerName: gateway.envoyproxy.io/gatewayclass-controller
---
# 2. Gateway — shared listener, owned by the platform team
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: production-gw
namespace: gateway-infra
spec:
gatewayClassName: envoy-gateway
listeners:
- name: https
protocol: HTTPS
port: 443
tls:
mode: Terminate
certificateRefs:
- name: prod-tls-cert
allowedRoutes:
namespaces:
from: All
---
# 3. HTTPRoute — owned by the checkout team, in their own namespace
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: checkout-route
namespace: checkout
spec:
parentRefs:
- name: production-gw
namespace: gateway-infra
hostnames:
- "checkout.example.com"
rules:
- backendRefs:
- name: checkout-svc
port: 8080
---
# 4. HTTPRoute — owned by the catalog team, independently, in THEIR namespace
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: catalog-route
namespace: catalog
spec:
parentRefs:
- name: production-gw
namespace: gateway-infra
hostnames:
- "catalog.example.com"
rules:
- backendRefs:
- name: catalog-svc
port: 8080Worth reading this as the concrete payoff of the whole persona model this chapter opened with: the platform team applies steps 1-2 exactly once and never touches them again for routine application changes; the checkout and catalog teams each independently manage steps 3-4 in their own namespaces, with neither able to see or modify the other's routing rules, both sharing the exact same underlying Envoy Proxy Deployment and its one TLS certificate — one shared, efficiently-utilized data plane, cleanly partitioned ownership.
Security Policies in Envoy Gateway#
Envoy Gateway extends Gateway API with its own SecurityPolicy and BackendTrafficPolicy CRDs for concerns Gateway API's core spec doesn't standardize — rate limiting, JWT authentication, and CORS, among others.
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: SecurityPolicy
metadata:
name: jwt-auth
namespace: checkout
spec:
targetRefs:
- group: gateway.networking.k8s.io
kind: HTTPRoute
name: checkout-route
jwt:
providers:
- name: auth0
remoteJWKS:
uri: https://example.auth0.com/.well-known/jwks.jsonThe targetRefs pattern used here — attaching a policy object to an existing Gateway API resource by reference, rather than embedding the policy inline inside that resource — deserves its own explicit callback, since it's a deliberate, repeated Gateway API extension pattern, not unique to SecurityPolicy. This "attach an extension policy CRD to a core Gateway API object" idiom is exactly how the wider Gateway API ecosystem is designed to be extended without bloating the core spec itself — the same pattern other implementations (including several covered later in this chapter) use for their own respective policy extensions, worth recognizing as a design convention rather than an Envoy-Gateway-specific quirk.
BackendTrafficPolicy covers a genuinely separate concern from SecurityPolicy — traffic shaping and resilience rather than identity — and is worth seeing concretely, since rate limiting is one of the most commonly requested capabilities that Gateway API's own core spec deliberately leaves to implementation-specific extension:
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: BackendTrafficPolicy
metadata:
name: checkout-rate-limit
namespace: checkout
spec:
targetRefs:
- group: gateway.networking.k8s.io
kind: HTTPRoute
name: checkout-route
rateLimit:
type: Global
global:
rules:
- clientSelectors:
- headers:
- name: x-api-key
type: Distinct
limit:
requests: 100
unit: MinuteThe clientSelectors field is worth calling out specifically, since it's what makes this a genuinely per-client rate limit rather than one shared bucket for the entire route — type: Distinct on the x-api-key header means Envoy Gateway maintains a separate counter per distinct header value, so one noisy API consumer hitting their own 100-requests-per-minute ceiling doesn't consume budget that belongs to a different, well-behaved consumer sharing the same route. This same policy object also supports circuit-breaking, retry budgets, and connection-pool tuning — worth knowing exists as a single, coherent home for resilience configuration, rather than scattered across several narrower mechanisms.
Gateway API Conformance Levels — Core, Extended, and Implementation-Specific#
Worth naming precisely, since this chapter has repeatedly flagged that "conformant" doesn't mean "identical feature set" across implementations: the Gateway API project itself defines three explicit support tiers, and understanding them is what makes sense of why six different, genuinely conformant implementations can still behave so differently in practice.
This tiering is the precise, structural reason this chapter's own comparison table works the way it does: every implementation covered in this chapter passes Core conformance identically — that's the actual, portable guarantee Gateway API provides. Extended fields are where real, if usually minor, behavioral differences start appearing between implementations. Implementation-Specific extensions — SecurityPolicy, BackendTrafficPolicy, and their equivalents in other implementations — are where the genuine non-portability lives, by design, since standardizing every possible policy concern into the core spec would recreate exactly the annotation-sprawl problem this chapter opened by criticizing. A team designing for genuine portability between implementations should keep routing logic itself on Core/Extended fields, and treat any Implementation-Specific policy CRD as a conscious, documented point of vendor coupling.
Observability With Envoy Gateway#
Envoy itself exposes rich, native metrics (request counts, latency histograms, upstream connection pool stats) in Prometheus format by default, directly reusable by this course's own Observability series without any additional instrumentation work — Envoy has been a first-class Prometheus-metrics emitter for years, predating its adoption as a Gateway API data plane.
The per-route granularity is worth emphasizing specifically, since it's a genuine step up from what a typical Ingress controller exposed by default: because Gateway API's HTTPRoute objects are individually named, distinct Kubernetes resources (rather than rules buried inside one large annotated Ingress object), Envoy's own per-route metrics naturally carry that same granularity — a platform team can build a Grafana dashboard breaking down latency and error rate by individual HTTPRoute name directly, without needing custom label-extraction logic to recover that boundary from a flatter Ingress-based metrics stream.
This also composes directly with the golden-signals framing from this course's own Observability series: envoy_cluster_upstream_rq_time (latency), envoy_cluster_upstream_rq_total and its 5xx-suffixed counterpart (traffic and errors), and envoy_cluster_circuit_breakers_default_cx_open (a genuine saturation signal) map cleanly onto latency/traffic/errors/saturation without any custom metric authored specifically for this chapter's purposes — the same four-signal model, populated automatically by Envoy's own default instrumentation.
Istio as a Gateway API Implementation — Ambient Mode, ztunnel, Waypoints#
Istio, already introduced in Part 4 as a service mesh, is also a fully conformant Gateway API implementation — worth understanding the architectural contrast with Envoy Gateway precisely, since it's the single most consequential decision point covered later in this chapter's decision framework.
The concrete architectural distinction from Envoy Gateway, stated precisely: Istio is a full service mesh (east-west mTLS between every pod, plus north-south Gateway API ingress) that happens to also implement Gateway API, while Envoy Gateway is Gateway API implemented with zero mesh ambitions at all. Istio's newer Ambient mode (the now-standard alternative to the older sidecar-per-pod model) splits this further: a lightweight, shared ztunnel proxy runs once per node handling L4 mTLS and basic policy for every pod on that node with minimal overhead, while genuinely L7-aware processing (the kind HTTPRoute header matching and traffic splitting require) is offloaded only to waypoint proxies — full Envoy instances deployed per-namespace or per-workload, only where L7 features are actually needed. The practical decision this distinction drives: a team that only needs Gateway API's ingress routing, with no mesh requirement, takes on real unnecessary operational complexity adopting Istio purely for that purpose — Envoy Gateway's narrower scope is the better-fitted tool. A team that already needs (or will soon need) full mesh mTLS between services gets Gateway API "for free" as part of adopting Istio anyway, making the calculus a different one entirely.
Worth one further, concrete detail tying this back to Part 4's own sidecar-based coverage of Istio: Ambient mode is not a separate product from the sidecar-based Istio Part 4 already introduced — it's an alternative data-plane mode of the same Istio control plane, selectable per-cluster or even per-namespace during a migration. A team already running sidecar-based Istio from Part 4 does not need to re-adopt anything new to also get Gateway API support; the same Istio installation already implements it.
Cilium Gateway — the eBPF-Native Implementation#
Cilium — already familiar to readers of this course's networking material as a CNI plugin — also ships its own conformant Gateway API implementation, Cilium Gateway, distinguished from every other implementation covered so far by running its data plane in the Linux kernel via eBPF rather than as a userspace proxy process.
The practical performance argument, stated precisely rather than as marketing language: any traffic path that can be handled entirely in-kernel via eBPF skips the userspace proxy hop (the context switch and memory copy into a process like Envoy) that every other implementation in this chapter incurs for every single request. This genuinely matters at very high request-rate, latency-sensitive workloads — the kind of workload where shaving a userspace hop off the hot path is worth the real additional operational complexity of adopting eBPF-based networking. The honest caveat worth stating alongside that benefit: not every Gateway API feature maps cleanly onto pure in-kernel processing — genuinely complex L7 logic still needs a userspace fallback path even in Cilium's own architecture, and a team already running Cilium as its CNI (making Cilium Gateway a natural, low-additional-complexity extension of infrastructure already in place) is in a meaningfully different adoption position than a team that would need to newly introduce Cilium purely to get its Gateway implementation.
NGINX Gateway Fabric, Traefik, and Kong — the Rest of the Field#
Three further implementations worth knowing by name and rough positioning, completing this chapter's implementation survey before its decision framework:
| Implementation | Positioning |
|---|---|
| NGINX Gateway Fabric (NGF) | The official, Gateway-API-native successor to NGINX Ingress Controller OSS — explicitly positioned as NGINX's long-term Kubernetes ingress strategy following that controller's own retirement, the natural migration target for a team already standardized on NGINX's own configuration idioms and comfortable staying within that vendor's ecosystem |
| Traefik v3 | A simple, low-operational-overhead controller with direct, native Gateway API support — a strong default fit for smaller, on-prem, or developer-centric platforms that don't need enterprise-scale policy management, prized for ease of operation over deep feature breadth |
| Kong Kubernetes Gateway | An enterprise-oriented implementation built on Kong's own proxy, offering deep built-in authentication (JWT, OAuth2), OPA/WASM-based extensibility, and advanced rate limiting out of the box — the strongest fit for a team that specifically wants a full API-gateway feature set (not just routing) bundled with its Gateway API conformance |
Worth reading this table alongside the earlier Envoy Gateway/Istio/Cilium sections as one unified picture, not a separate, lesser tier: all six implementations covered across this chapter pass the same official Gateway API conformance test suite for their respective supported feature sets — conformance itself is not the differentiator. What actually differs, and what this chapter's decision framework next section addresses directly, is each implementation's additional scope beyond the shared conformant core: a service mesh (Istio), an eBPF dataplane (Cilium), a purpose-built lightweight gateway (Envoy Gateway, Traefik), or a full API-gateway product (Kong).
Implementation Feature Comparison — a Reference Table#
Worth consolidating every implementation covered across this chapter into a single, scannable reference — a direct extension of this course's own established pattern of closing a multi-option survey with one lookup table rather than leaving the comparison scattered across several sections' worth of prose.
| Implementation | Dataplane model | Mesh capability | TCPRoute/UDPRoute | Built-in rate limiting | Best-fit team profile |
|---|---|---|---|---|---|
| Envoy Gateway | Userspace Envoy, xDS-driven | None (Gateway API only) | Yes | Yes (BackendTrafficPolicy) | Wants Gateway API done well with no additional infrastructure scope |
| Istio (Ambient) | ztunnel (L4) + waypoint Envoy (L7) | Full mesh (mTLS, east-west) | Yes | Yes, mesh-wide policy | Already needs, or will soon need, full service-mesh mTLS |
| Cilium Gateway | eBPF, in-kernel where possible | Partial (CNI-level network policy, not full mTLS mesh) | Yes | Yes | Already running Cilium as CNI |
| NGINX Gateway Fabric | Userspace NGINX | None | Varies by release | Yes | Standardized on NGINX's own ecosystem already |
| Traefik v3 | Userspace Traefik | None | Yes | Yes | Small/on-prem clusters, developer-centric platforms, simplicity-first |
| Kong Kubernetes Gateway | Userspace Kong (Nginx/Envoy-based, version-dependent) | None | Yes | Yes, enterprise-grade | Needs a full API-gateway feature set (JWT/OAuth2, OPA/WASM) beyond routing |
The "Mesh capability" column is worth reading as the single most decision-relevant column in this table, directly feeding the decision framework's first gate — it's the one dimension that maps to genuinely different classes of infrastructure commitment (full mesh vs. none vs. partial), rather than a routine feature-checklist difference between otherwise-similar options.
Choosing an Implementation — a Decision Framework#
Worth closing the implementation survey with a single, walkable decision flow, directly mapping this chapter's own comparisons onto a concrete choice.
The two gates worth the strongest emphasis, since they're the ones a team most commonly evaluates in the wrong order: mesh-or-not comes first, CNI-already-in-place comes second — both are "does adopting this implementation mean adopting a large, separate piece of infrastructure I don't otherwise need" questions, and getting that order backwards (e.g. evaluating feature checklists before asking whether a mesh is even wanted) is how teams end up running Istio's full operational surface purely for its Gateway API routing, an outcome nobody actually wanted. Everything after those two gates is a genuine, narrower feature-fit decision between purpose-built options, none of which carries hidden, large-scope infrastructure the team didn't explicitly choose to adopt.
Gateway API and Service Mesh — Where the Line Blurs#
Worth a closing clarification, since the Istio section above genuinely blurs a line this chapter has otherwise kept clean: Gateway API itself is not exclusively an "ingress" specification — its GAMMA (Gateway API for Mesh Management and Administration) initiative extends the same HTTPRoute resource to also express east-west, mesh-internal traffic routing (service-to-service, not just external-to-cluster), meaning a mesh like Istio can use the identical HTTPRoute object type both for its north-south Gateway and for internal mesh traffic-splitting policy.
This is worth knowing exists rather than treated as this chapter's own focus — GAMMA's mesh-traffic use of HTTPRoute is a genuinely separate application of the same resource type, not an extension of anything covered so far in this chapter's own ingress-focused treatment. A team encountering an HTTPRoute with no Gateway parentRef at all, attached instead to a Service directly, has run into GAMMA's mesh-mode usage — worth recognizing rather than assuming a misconfiguration, but genuinely out of scope for this chapter's own north-south focus.
Debugging a Misconfigured Gateway — Status Conditions#
Worth a dedicated troubleshooting section, since this is where a team new to Gateway API most commonly gets stuck: unlike a traditional Ingress controller's often-opaque failure modes (a route silently not working, with little indication why), Gateway API standardizes status conditions directly on Gateway and HTTPRoute objects, giving a genuinely structured starting point for diagnosis.
kubectl get gateway production-gw -n gateway-infra -o yamlstatus:
conditions:
- type: Accepted
status: "True"
reason: Accepted
- type: Programmed
status: "True"
reason: Programmed
listeners:
- name: https
conditions:
- type: ResolvedRefs
status: "False"
reason: InvalidCertificateRef
message: "referenced Secret prod-tls-cert not found"Three condition types are worth knowing by name, since they isolate the failure to a specific layer rather than leaving a team guessing: Accepted (True means the implementation's controller has accepted this Gateway object as valid and intends to act on it — False usually means a spec-level problem, like an unsupported field combination); Programmed (True means the actual data plane, the managed Envoy Proxy Deployment in this chapter's case, has been successfully configured to match — False with Accepted: True means the object was valid but the implementation couldn't actually provision infrastructure for it, worth checking cloud quota or RBAC next); and ResolvedRefs (per-listener, False means a referenced object — most commonly a TLS Secret, as in the example above — couldn't be found or resolved). The example status above is a genuinely common real failure: the Gateway itself is valid and accepted, but its HTTPS listener can't actually come up because the referenced TLS certificate Secret doesn't exist in the expected namespace — a fast, structured diagnosis that would otherwise require digging through Envoy's own logs directly.
HTTPRoute objects carry an analogous status.parents field, one entry per Gateway the route is attached to, each with its own Accepted and ResolvedRefs conditions — meaning a route silently not receiving traffic is diagnosable the same structured way: check whether the specific Gateway it's attached to actually accepted the attachment, and whether its own backendRefs resolved to a real, existing Service.
Common Mistakes#
| Mistake | Why it's a problem | Fix |
|---|---|---|
| Treating Gateway API as "just a new Ingress" with a 1:1 field mapping | Missing the deliberate persona/RBAC split that's the actual point of the redesign | Design Gateway vs. HTTPRoute ownership around real team boundaries, not old Ingress habits |
Setting allowedRoutes.namespaces.from: All by default on a multi-tenant cluster | Any namespace can attach routes to the shared Gateway with no platform-team review | Use Selector or Same deliberately where namespace isolation actually matters |
| Assuming every implementation supports every route type | AWS's EKS Gateway API Controller, for example, lacks TCPRoute/UDPRoute support | Check the specific implementation's conformance/route-type support before committing to a design |
| Adopting Istio purely to get Gateway API, with no actual mesh need | Takes on full service-mesh operational complexity for a routing-only requirement | Default to Envoy Gateway (or Traefik) unless a genuine mesh requirement already exists |
Reaching for EnvoyPatchPolicy as a routine configuration path | Recreates the vendor-lock-in/non-portability problem Gateway API's typed resources exist to avoid | Treat it as a rare escape hatch; push standard needs back into typed Gateway API/policy CRD fields |
Migrating off ingress-nginx via blind ingress2gateway auto-conversion with no manual review | Vendor-specific annotations with no Gateway API equivalent get silently flagged, not auto-fixed | Diff generated resources against real observed Ingress behavior in a non-production cluster first |
Running a single-replica Envoy Proxy Deployment in production with no EnvoyProxy customization | A real availability risk for infrastructure sitting directly in every request's critical path | Set replica count, resource requests, and PodDisruptionBudget explicitly via the EnvoyProxy CRD |
Worked Practice Problems#
Problem 1: Two application teams, checkout and catalog, each want to manage their own HTTP routing rules without needing write access to the shared TLS certificate or listener configuration. Which two Gateway API resources solve this, and who owns each?
Answer: The platform team owns a single Gateway object (in a shared namespace, e.g. gateway-infra) defining the listener, port, and TLS certificate — this is what requires elevated, centrally-controlled access. Each application team independently owns its own HTTPRoute object, living in its own namespace (checkout, catalog respectively), referencing the shared Gateway via parentRefs without needing any write access to it. This persona split — infra/operator-owned Gateway, app-team-owned HTTPRoute — is the core architectural answer Gateway API was designed around, replacing Ingress's single flat object with no such boundary.
Problem 2: A team migrating off ingress-nginx runs ingress2gateway against their cluster and the tool completes successfully, generating Gateway and HTTPRoute YAML with no errors. Is it now safe to delete the old Ingress controller and cut over to the generated resources in production? Why or why not?
Answer: Not automatically safe — ingress2gateway explicitly does not guarantee a complete, zero-touch conversion; any ingress-nginx-specific annotation without a direct Gateway API field equivalent gets flagged for manual review rather than silently and correctly translated. The tool completing "successfully" means it ran without crashing, not that every piece of the old Ingress's actual routing behavior was faithfully preserved. The correct next step is running the generated resources against a non-production cluster, diffing actual observed traffic behavior against the original Ingress's behavior, and only then planning a production cutover.
Problem 3: A platform team needs raw TCP load balancing (not HTTP) for a custom binary protocol running on EKS, and has chosen AWS's own EKS Gateway API Controller for their Gateway API implementation. What should they check before committing to this plan, given this chapter's own coverage?
Answer: Whether the specific implementation actually supports TCPRoute at all — this chapter flagged explicitly that AWS's EKS Gateway API Controller does not support TCPRoute/UDPRoute (a gap covered in more depth in Part 9's provider-specific treatment). Not every implementation supports every Gateway API route type, and this is exactly the kind of implementation-specific gap that needs checking against the target platform's own actual conformance before a design commits to it — a plain LoadBalancer Service, or a different implementation with TCPRoute support, would be the practical alternative here.
Problem 4: A team is deciding between Envoy Gateway and Istio purely to get Gateway API's HTTP routing capability — they have no current or planned need for service-mesh mTLS between their own internal services. Which is the better-fitted choice, and why, per this chapter's decision framework?
Answer: Envoy Gateway — it's a dedicated, Gateway-API-native controller with no mesh ambitions, matching the team's actual stated need exactly. Adopting Istio purely for Gateway API routing means taking on the full operational complexity of a service mesh (ztunnel, waypoint proxies, mesh-wide mTLS machinery) for a capability that requires none of it — precisely the "mesh-or-not comes first" ordering mistake this chapter's decision framework flags as the most common way teams end up over-provisioning infrastructure complexity relative to their actual need.
Problem 5: A security-conscious platform team notices an HTTPRoute in the checkout namespace referencing a backend Service in a completely different payments namespace, and the request is failing. What Gateway API resource is most likely missing, and why does this failure mode exist by design?
Answer: A ReferenceGrant, created in the payments namespace (the target namespace), explicitly consenting to being referenced by an HTTPRoute from the checkout namespace. This restriction exists by design specifically to prevent one namespace's route from silently forwarding traffic into a Service in another namespace that never explicitly agreed to be a routing target — the same "the resource being accessed must consent, not just the resource doing the accessing" security idiom already established for cross-namespace NetworkPolicy design.
Problem 6: A platform team notices a developer has created an EnvoyPatchPolicy to work around a missing feature, and it's now the third one added this quarter, each by a different team. What does this pattern suggest, and what should the platform team investigate?
Answer: Per this chapter's own framing of EnvoyPatchPolicy as a last-resort escape hatch, three separate teams reaching for raw Envoy JSONPatches in one quarter is a signal that genuinely common needs aren't being met by standard Gateway API resources or Envoy Gateway's own policy CRDs (SecurityPolicy, BackendTrafficPolicy) — not that three unrelated, one-off edge cases happened to occur. The platform team should investigate whether these three patches share a common underlying need (a missing header-manipulation capability, a missing timeout/retry field) that could instead be exposed as a proper, typed, reusable policy — the same instinct that motivated Gateway API's design away from Ingress's own annotation sprawl in the first place, now applied to the platform team's own extension surface.
Problem 7: A team wants Gateway API's native traffic-splitting (weight on backendRefs) to drive an automated canary rollout, referencing this course's CI/CD & GitOps series. What's the connection between HTTPRoute weights and a tool like Argo Rollouts or Flagger?
Answer: Argo Rollouts and Flagger programmatically edit an HTTPRoute's backendRefs weight values as a canary progresses through its analysis steps — Gateway API's native weighted traffic-splitting is precisely the mechanism those progressive-delivery controllers drive under the hood when configured against a Gateway-API-based ingress, rather than requiring a separate service-mesh-specific traffic-splitting CRD purely for that purpose. This is a genuine simplification for a team that wants progressive delivery without adopting a full mesh solely to get traffic-splitting capability.
Key Terms Glossary — This Chapter's Vocabulary in One Place#
| Term | Meaning in this chapter's context |
|---|---|
GatewayClass | Cluster-scoped resource naming a Gateway API implementation, owned by the infrastructure provider persona |
Gateway | Namespace-scoped listener/TLS configuration, owned by the cluster operator persona |
HTTPRoute | Application-team-owned routing rules, attached to a Gateway via parentRefs |
ReferenceGrant | Target-namespace-issued consent allowing a route in another namespace to reference its backend |
| xDS | Envoy's native, streaming gRPC configuration protocol — Discovery Service family (Listener/Route/Cluster/Endpoint) |
EnvoyProxy CRD | Envoy Gateway's customization surface for the managed Envoy Proxy Deployment |
EnvoyPatchPolicy | Last-resort raw Envoy JSONPatch escape hatch, not a routine configuration path |
| Ambient mode | Istio's sidecar-free mesh architecture — ztunnel (L4) plus per-namespace waypoint proxies (L7) |
| GAMMA | Gateway API's mesh-traffic (east-west) extension, reusing HTTPRoute for service-to-service routing |
| Core / Extended / Implementation-Specific | Gateway API's three conformance tiers, from universally guaranteed to fully implementation-owned |
BackendTrafficPolicy | Envoy Gateway's resilience/traffic-shaping policy CRD — rate limiting, circuit breaking, retries |
SecurityPolicy | Envoy Gateway's identity/auth policy CRD — JWT, CORS, and related concerns |
| GRPCRoute | Native gRPC routing by service/method, rather than URL path matching |
This table deliberately mirrors the same closing-glossary pattern already used in the CI/CD & GitOps series' own self-hosted runner chapter — a quick lookup aid for a chapter this dense in newly introduced, closely related terminology.
Summary and What's Next#
Gateway API replaces Ingress's single, annotation-sprawling object with a deliberately split, persona-based model — GatewayClass (infrastructure provider), Gateway (cluster operator, listener/TLS ownership), and HTTPRoute/GRPCRoute/TCPRoute/UDPRoute/TLSRoute (application team, routing-rule ownership) — turning what used to require vendor-specific annotation strings into strongly-typed, schema-validated API fields, with real RBAC boundaries between the teams that own each layer. This split isn't cosmetic: it's the concrete mechanism that lets an application team manage its own routing rules without ever touching the shared listener/certificate configuration a platform team owns, while ReferenceGrant closes the corresponding cross-namespace security gap by requiring explicit, target-side consent. Envoy Gateway, this chapter's reference implementation, translates these Kubernetes-native resources into Envoy's own xDS configuration through a clean control-plane/data-plane split — a single controller Deployment watching Gateway API resources, and a separately provisioned, per-Gateway Envoy Proxy Deployment actually handling traffic, customizable through the EnvoyProxy CRD and (as a genuine last resort) EnvoyPatchPolicy. Istio, Cilium, NGINX Gateway Fabric, Traefik, and Kong each implement the identical Gateway API conformant core while adding meaningfully different additional scope — a full service mesh, an eBPF-native dataplane, vendor-ecosystem continuity, operational simplicity, or a full API-gateway feature set respectively — making the real choice between them a question of which additional infrastructure a team is actually willing to adopt, not a question of Gateway API conformance itself. The ingress-nginx retirement (March 2026) and the ingress2gateway migration tooling built specifically around it make this a live, time-bound migration for much of the ecosystem right now, not a someday concern.
Part 9, immediately following, takes this exact same specification and asks a different question: what actually changes when a team runs it on GKE, EKS, AKS, or bare-metal on-prem infrastructure — each cloud provider implements Gateway API as a facade over its own existing, proprietary load-balancing product (Google Cloud Load Balancing, AWS VPC Lattice, Azure Application Gateway for Containers respectively), with real, provider-specific capability gaps worth knowing before committing a production design to any one of them, while on-prem infrastructure has no such managed facade to lean on at all.
Everything covered in this chapter — the core resource model, Envoy Gateway's architecture, and every implementation compared — remains the shared foundation Part 9 builds on; nothing there replaces it.