How Jetstack Set Up a Global Load Balancer for Multiple Kubernetes Clusters

Originally reported by InfoQ. This article is an independent technical explanation based on Jetstack’s publicly described approach.

Background and Business Requirements

Jetstack’s engineering team documented how they set up a single Global Load Balancer (GLB) across multiple Google Kubernetes Engine (GKE) clusters for one of their customers. The goals were to add Google Cloud Armor DDoS protection, adopt container-native load balancing for improved traffic visibility and network performance, and consolidate routing behind a single IP address rather than relying on DNS-based geographic routing.

The customer’s existing architecture used multiple Kubernetes clusters with geolocation-based DNS routing via AWS Route 53. Route 53 can return different IP addresses based on where DNS queries originate, effectively directing users to their nearest cluster. While functional, this approach lacked DDoS protection and added unnecessary network hops that increased latency.

Kubernetes Load Balancing Fundamentals

Understanding the migration requires a clear picture of how Kubernetes exposes services. At the Service level, Kubernetes defines three types — excluding Ingress:

• ClusterIP — exposes the service on an internal cluster IP only; not reachable from outside the cluster.
• NodePort — exposes the service on each node’s IP at a static port, accessible externally but unrefined.
• LoadBalancer — provisions a cloud-provider load balancer in front of the cluster. In GKE, this creates a Network Load Balancer (NLB), which operates at Layer 4.

The customer had been using the LoadBalancer service type, which in GKE maps to a Network Load Balancer (NLB). For internet-facing traffic, a Kubernetes Ingress resource is the more typical entry point. In GKE, creating an Ingress resource automatically provisions a Global Load Balancer (GLB) operating at Layer 7.

Why a Network Load Balancer Was Insufficient

Google’s NLBs do not support Cloud Armor, which is Google’s DDoS and WAF service. Cloud Armor supports Layer 3 through Layer 7 network rules, but it can only be attached to a backend service fronted by an L7 GLB — not an NLB. This architectural constraint made switching to an L7 Global Load Balancer a prerequisite for Cloud Armor integration.

An L7 GLB also brings additional capabilities:

• URL-based routing and path rules.
• TLS termination at the load balancer level.
• A globally anycast IP address, replacing the need for geographic DNS routing.

One trade-off: the L7 GLB restricts traffic-serving ports to 80, 8080, and 443. The customer’s application had previously used additional non-standard ports, requiring application changes before migration could proceed.

Migration Phase 1: Introducing the L7 Global Load Balancer

The first phase focused on replacing the NLBs with L7 GLBs by creating Ingress resources in each GKE cluster. In GKE, provisioning an Ingress resource automatically creates a corresponding GLB. At the end of this phase, each cluster still had its own GLB with its own IP address, and DNS continued to point to these separate addresses. The primary gains were Cloud Armor integration and URL-based routing capability. Application port changes were completed during this phase.

Migration Phase 2: Enabling Container-Native Load Balancing

GKE offers a feature called container-native load balancing that allows a load balancer to route traffic directly to individual pod IP addresses, bypassing the kube-proxy and iptables routing layer that normally sits between an LB and a pod.

Without container-native load balancing, the traffic path from an external GLB to a pod is:

• GLB → Node IP (via NodePort) → kube-proxy → Pod IP

This introduces extra network hops and obscures true pod-level health from the load balancer. Container-native load balancing shortens this to:

• GLB → Pod IP (directly)

This is achieved using Network Endpoint Groups (NEGs) — a GCP-specific construct that holds the IP addresses and ports of individual pods registered as load balancer backends. NEGs are provisioned automatically when an Ingress is annotated appropriately in GKE.

It is important to note that container-native load balancing via NEGs is a GKE-specific feature and is not part of the upstream Kubernetes specification. It cannot be replicated on other managed Kubernetes platforms without equivalent vendor-specific tooling.

Migration Phase 3: Consolidating to a Single Global Load Balancer

The third and most architecturally complex phase replaced the per-cluster GLBs with a single GLB IP address that routes traffic to backends across all clusters. This eliminated the need for geographic DNS routing entirely.

The Core Challenge: Kubernetes Has No Multi-Cluster Ingress Primitive

Kubernetes does not have a native mechanism for defining an Ingress that spans multiple clusters. A single Ingress resource is scoped to one cluster. At the time this work was carried out, Google had a beta-stage tool that aimed to address this gap, but it was considered too early for production use.

It is also worth clarifying the distinction between two related but different concepts:

• Single GLB in front of multiple independent clusters: One anycast IP routes incoming internet traffic to independent Kubernetes clusters acting as separate backends. Each cluster manages its own workloads. This is what Jetstack implemented.
• Single control plane across multiple clusters (multi-cluster federation): A unified control plane coordinates workloads across clusters. This is a different problem solved by tools like KubeFed.

The Terraform-Based Approach

Jetstack’s team used Terraform with the Google Cloud provider to automate the solution. Rather than relying on GKE’s automatic Ingress-to-GLB provisioning (which would create a separate GLB per cluster), they:

1. Defined the NEG resources for each cluster’s pods explicitly via Terraform, rather than letting GKE auto-create them.
2. Created a single GLB in Terraform, separate from any specific Kubernetes cluster.
3. Registered the NEGs from all clusters as backends on the single GLB, tying them together using GCP annotations.

This approach treats the GLB as infrastructure-as-code, fully decoupled from the Kubernetes Ingress resource lifecycle, giving the team precise control over backend configuration and backend weights.

Key Takeaways for Platform Engineers

• Cloud Armor requires an L7 GLB. If your GKE workloads need DDoS protection, NLBs are not sufficient — you must migrate to an L7 Ingress-backed load balancer.
• Container-native load balancing reduces latency but is GCP-specific. NEGs enable pod-direct routing, improving health checking accuracy and reducing hops, but they are not portable across clouds.
• Multi-cluster GLB requires stepping outside the Kubernetes Ingress abstraction. To front multiple GKE clusters with one GLB, you need to manage NEGs and the GLB at the infrastructure layer (e.g., Terraform), not via standard Ingress resources.
• Port restrictions matter. Migrating to a GKE L7 GLB means restricting to ports 80, 8080, and 443. Applications using non-standard ports need modification before migration.
• This is not trivial on other clouds. The pattern of a single external load balancer fronting multiple independent Kubernetes clusters is inherently cloud-provider-specific and complex regardless of platform.

Source: Originally reported by InfoQ, based on Jetstack’s engineering documentation.