This article is based on a report originally published by InfoQ, which interviewed Max Williams, Principal System Engineer at Delivery Hero AG, about the company’s Kubernetes tooling strategy.
Overview
Delivery Hero (DH) operates Kubernetes clusters at significant scale — reportedly hundreds of nodes and thousands of pods across multiple environments. To keep this infrastructure manageable, their engineering teams rely on a coordinated set of tools built around Helm, the Kubernetes package manager. The full stack they use includes Helm itself, helmfile, helm-diff, and helm-secrets.
Why Helm?
Helm is a CNCF-hosted project that functions as a package manager for Kubernetes. Rather than maintaining a growing collection of individual YAML manifests for every resource in a cluster, Helm groups all the Kubernetes objects needed to deploy an application into a single, versioned unit called a chart.
Most teams start by writing raw YAML files to declare cluster state. This approach works at small scale but becomes increasingly difficult to maintain as the number of services and environments grows. Helm addresses this by:
- Packaging all related Kubernetes resources (Deployments, Services, ConfigMaps, etc.) into a single chart
- Enabling parameterisation through values files, so the same chart can be deployed differently across environments
- Providing versioned releases, making rollbacks straightforward
At Delivery Hero, multiple engineering teams use Helm charts to package both individual applications and cluster-level tooling.
helmfile: Declarative Management Across Clusters
helmfile is a declarative spec for deploying Helm charts. It maintains information about which charts are installed on which clusters, together with the relevant chart versions, values files, and Kubernetes cluster contexts. Williams describes it as analogous to Ansible playbooks — a single source of truth for the desired state of Helm releases across environments.
Because helmfile configurations are stored in Git, the workflow naturally supports progressive rollouts: changes can be reviewed in pull requests and applied to non-production clusters first before being promoted to production. This gives teams a clear audit trail and a safe promotion process.
helm-diff: Previewing Changes Before Deployment
helm-diff is a Helm plugin that renders a colour-coded diff of the changes that will be applied when a chart is upgraded. This allows engineers to review exactly what will change in the cluster before committing to a deployment, reducing the risk of unexpected modifications reaching production.
Williams acknowledges that a clean diff is not a guarantee of a clean deployment — issues can still arise after apply. However, he notes that Delivery Hero had not encountered problems attributable to helm-diff itself beyond issues already present in Helm’s own behaviour.
helm-secrets: Encrypting Sensitive Configuration
Managing secrets in Kubernetes is one of the most common pain points teams face when moving to GitOps workflows. Storing sensitive values in plaintext in a Git repository is not acceptable, but keeping secrets entirely out of version control creates its own operational problems.
helm-secrets solves this by wrapping Helm values files with encryption. Under the hood it uses Mozilla sops (Secrets OPerationS), which acts as a transparent encryption/decryption layer around a standard text editor or CI pipeline. Engineers edit secrets in plaintext locally; sops handles encryption before the file is committed and decryption when it is consumed.
sops integrates with major cloud Key Management Systems (KMS), including:
- AWS KMS
- GCP KMS
This is a natural fit for Delivery Hero’s infrastructure, which Williams describes as hosted “mostly on EKS and GKE, with some teams still using kops clusters on AWS.”
The Broader Toolchain
Helm and its companion tools are not the only pieces of Delivery Hero’s infrastructure puzzle. The team also uses:
- Terraform — for provisioning and managing underlying cloud infrastructure
- Sysdig — for container security and monitoring
- Prometheus — for metrics collection and alerting
Key Takeaways
Delivery Hero’s approach offers a practical template for engineering teams managing Kubernetes at scale:
- Use Helm to package applications and cluster tooling into versioned, reusable charts
- Use helmfile to maintain a declarative, Git-tracked record of which charts and versions are deployed to which clusters
- Use helm-diff to review changes before they are applied, catching unintended modifications early
- Use helm-secrets with Mozilla sops to safely version-control encrypted secrets alongside the rest of your Helm configuration
- Combine with cloud KMS (AWS or GCP) to centralise key management without adding operational overhead
This layered toolchain addresses the three main challenges of operating Kubernetes at scale: configuration management, change visibility, and secrets handling — without requiring teams to build custom tooling from scratch.