This article is based on a case study originally reported by InfoQ, covering details shared by Monzo’s engineering team about their Kubernetes network policy implementation.
Background: Monzo’s Infrastructure at Scale
Monzo is a mobile-only digital bank that runs its core infrastructure on AWS. Its backend is built on Kubernetes, with Apache Cassandra as the primary database, Apache Kafka for messaging, and Go for most application code. At the time this case study was shared, Monzo operated approximately 1,500 microservices with more than 9,300 inter-service interactions.
Managing network access at that scale is non-trivial. The Monzo security engineering team set zero-trust networking as a core goal: no service—inside or outside the cluster—should be trusted by default. Instead, each service should be permitted to communicate only with a pre-approved list of other services.
What Zero-Trust Networking Means in a Kubernetes Context
Kubernetes, by default, allows unrestricted communication between all pods in a cluster. Zero-trust networking inverts this: all traffic is denied unless explicitly permitted. Achieving this in a cluster with hundreds or thousands of services requires a disciplined, automated approach to policy management—manual configuration does not scale.
Kubernetes provides a native NetworkPolicy resource for this purpose, but its feature set has limits. Monzo used the Calico CNI (Container Network Interface) plugin, which extends standard Kubernetes network policies with additional capabilities—such as logging traffic that would have been dropped, without actually dropping it. This logging-only mode was critical to the team’s rollout strategy.
Discovering Inter-Service Dependencies with Static Analysis
Before enforcing any network policy, the team needed to understand which services actually talk to each other. They built a custom tool called rpcmap, which discovers inter-service dependencies through static code analysis of the Go codebase.
The team explicitly chose static analysis over two alternatives:
- Integration test observation: Monzo’s codebase is large and not every code path has an integration test. Testing-time observations would produce an incomplete dependency map.
- Runtime observation: Just because a service call is rare does not mean it never happens. As Backend Engineer Jack Kleeman noted, a bank can have yearly processes—a service dependency exercised once a year would be missed by short-window runtime sampling.
Static analysis produces a deterministic, complete picture of intended call paths, making it the most reliable foundation for policy generation at this scale.
Initial Approach: Testing on a Single Service
Rather than rolling out policies cluster-wide immediately, the team isolated a single service first. This pilot revealed several early limitations:
- The initial design placed the burden of maintaining the allowed-caller list on the destination service team, creating an ownership and coordination problem.
- Dev teams had to edit Kubernetes config files by hand—error-prone and difficult to review.
- Policy testability was weak; it was hard to verify that policies behaved correctly before enforcing them.
Using Calico’s Extended Policy Features for Safe Rollout
To address testability, the team engaged with the Calico community. They discovered that Calico offers features beyond what standard Kubernetes NetworkPolicy exposes—specifically, the ability to log traffic that would be denied without actually dropping it.
The rollout strategy was:
- Configure the enforcement policy to run at the lowest priority (last in the evaluation chain).
- Enable audit/log mode so that traffic which would be blocked is logged instead of dropped.
- Monitor network traffic to identify which services would have had packets dropped under full enforcement.
- Adjust the policy rules based on findings before switching to hard enforcement.
This approach allowed the team to validate their generated policies against real cluster traffic with zero disruption to production services.
Shifting Policy Ownership to the Invoking Service
One of the most important design decisions was flipping who owns the allowed-access list. Rather than making the destination service responsible for declaring who may call it, the team made the calling (invoking) service responsible for declaring which services it needs access to.
This was expressed through Kubernetes labels. Each service used labels in the network policy’s ingress spec to declare its required downstream dependencies. This has a key advantage: the team that owns a service also owns its dependency declarations, which aligns responsibility with knowledge.
Handling High-Fan-Out Services with Service-Type Grouping
Some services—such as monitoring or observability tooling—call a large number of other services. Listing every individual dependency for such services would be unmanageable. To handle this, services were grouped by a service-type label, and access policies were applied to the group rather than individual service names. This kept policies readable and maintainable even for high-fan-out scenarios.
Automating Policy Generation in the Deployment Pipeline
With the design validated, the team integrated rpcmap into the CI/CD pipeline. It runs on every commit, statically analysing the code to produce dependency rule files. The deployment pipeline then converts these rule files into the Kubernetes/Calico service labels and network policy objects automatically.
This eliminates manual Kubernetes config editing and ensures that network policies stay in sync with the code as it evolves.
Future Direction: Moving to a Service Mesh
The team acknowledged that enforcing access control at the CNI layer has limitations—primarily that it operates at the IP/port level rather than at the application (L7) level. Monzo indicated plans to move this enforcement to a service mesh layer, and noted they had already begun migrating to a custom mesh based on Envoy. A service mesh allows richer, identity-aware policy enforcement, mutual TLS between services, and more granular observability than CNI-layer policies alone.
Key Takeaways for Platform and Security Engineers
- Default-deny is achievable at scale, but requires automated policy generation—manual approaches break down past a few dozen services.
- Static code analysis is a more reliable source of dependency truth than runtime observation when code paths include infrequent but critical processes.
- Use audit/log mode before enforcement to validate policies against real traffic without risking production outages.
- Assign policy ownership to the calling service, not the destination, to align responsibility with the team that has the most context.
- Group services by type to manage high-fan-out dependencies without bloating individual policy definitions.
- CNI-layer network policies are a starting point; a service mesh provides deeper, L7-aware enforcement as a natural next step.
Monzo’s approach demonstrates that zero-trust networking inside a Kubernetes cluster is operationally feasible at significant scale, provided it is backed by automation, static analysis, and a phased rollout strategy.