Why Containers and Kubernetes Matter for HPC

Software containers and Kubernetes have become foundational tools for building, deploying, and managing enterprise applications at scale. They allow organisations to deliver software faster and more reliably while using compute resources more efficiently and reducing operational costs.

High-performance computing (HPC) is increasingly moving closer to the enterprise world. As a result, the HPC community stands to benefit from the container and Kubernetes ecosystem — particularly the ability to quickly allocate and deallocate large computational resources on demand. Unlike the traditional HPC model, where capacity was planned years in advance, modern compute infrastructure must be elastic and responsive to workload spikes.

Cloud providers and software vendors are continuously improving on-demand resource provisioning. In enterprise computing, packaging software in container images is now standard practice. Kubernetes has become the dominant container orchestrator in large enterprises. The HPC community — led largely by AI and machine learning workloads — is now adopting these patterns and applying them to batch jobs and interactive scientific applications.

The Challenge for HPC Teams

HPC leaders face a rapidly shifting technology landscape. Containerisation has become ubiquitous, Kubernetes has emerged as the standard for container orchestration, deep learning workloads continue to grow, and Site Reliability Engineering (SRE) practices have spread across data centres. Evaluating which of these technologies genuinely applies to HPC-style workloads — tightly coupled parallel jobs, massive batch throughput, specialised hardware — remains a real challenge.

A Brief Introduction to Kubernetes

When running a single container on a developer laptop, Docker alone is sufficient. But when deploying hundreds of containers across dozens or hundreds of machines, manual management quickly becomes impractical. Kubernetes addresses this by providing scheduling, container lifecycle management, networking, and self-healing capabilities in a scalable, extensible platform.

The core Kubernetes architecture consists of:

• Control plane (master): Contains the API server, scheduler, and controller manager. Controllers continuously compare the current state of resources against the desired state and take corrective action when they diverge.
• Worker nodes: Run the kubelet agent, which communicates with the control plane and manages containers via the Container Runtime Interface (CRI). Supported runtimes include containerd, CRI-O, and Docker (via shim).
• kube-proxy: Handles network proxying and load balancing rules on each node.

Kubernetes vs. Traditional HPC Workload Managers

Kubernetes and HPC workload managers such as SLURM, PBS Pro, or Univa Grid Engine both perform workload and resource management — but they were designed for fundamentally different workload profiles.

Strengths of HPC Workload Managers

Traditional HPC schedulers have been refined over decades for demanding scientific computing scenarios. Key capabilities include:

• Support for shared-memory and distributed-memory (MPI-based) parallel jobs
• Advance reservations — blocking resources ahead of time for scheduled workloads
• Fair-share scheduling to manage resource allocation across users, projects, and departments
• Resource reservation to accumulate capacity for large multi-node jobs
• Preemption — suspending lower-priority jobs to free resources for high-priority ones
• NUMA-aware scheduling for precise allocation of CPU cores and sockets
• Standards compliance for job submission and management (DRMAA, DRMAAv2)
• Proven scalability for millions of batch jobs per day on the world’s largest supercomputers

What traditional HPC managers lack are native support for microservice architectures, deeply integrated container lifecycle management, overlay networking, and rolling application updates.

Strengths of Kubernetes

Kubernetes was built from the ground up for containerised microservice applications. Its notable capabilities include:

• Management of Pods — groups of one or more co-located containers sharing network and storage namespaces
• Pluggable overlay networking via CNI plugins
• Self-healing through the controller loop (desired state vs. actual state)
• Declarative, version-controlled configuration via YAML manifests
• Built-in load balancing and service discovery
• Rolling updates and canary deployments
• Native integrations with monitoring and logging stacks (Prometheus, Fluentd, etc.)
• Persistent storage hooks through Persistent Volumes and CSI drivers

What Kubernetes currently lacks for HPC use cases is a high-throughput batch job queuing system with the sophisticated fair-share, reservation, and preemption rules that scientific computing requires. Additionally, many legacy HPC engineering applications are not designed to interface with Kubernetes APIs — though this is changing as new AI-centric frameworks such as Kubeflow are built natively for Kubernetes.

Combining Kubernetes and HPC Workload Managers

The most practical near-term approach is not to replace one system with the other, but to combine them. The recommended pattern works as follows:

1. Kubernetes manages HPC containers and supporting services — handling deployment, scaling, networking, and lifecycle across the cluster.
2. Inside those containers, an HPC workload manager (SLURM, Univa Grid Engine, etc.) runs — either connected to an existing HPC cluster or deployed as a self-contained instance within the containerised environment.

This layered approach preserves compatibility with existing HPC engineering applications and retains advanced batch scheduling capabilities, while the overall deployment gains the portability and operational consistency of Kubernetes.

Operational Benefits of This Architecture

Simplified Administration

HPC environments can involve large numbers of containers serving different purposes — compute nodes, head nodes, license servers, pre- and post-processing services. Kubernetes abstracts away the complexity of managing individual container runtimes, making scaling operations — adding or removing compute capacity — significantly simpler and more repeatable.

Hybrid and Multi-Cloud Portability

Portability is one of the core value propositions of containers, and Kubernetes extends that portability to entire fleets of containers. An HPC stack packaged for Kubernetes can run on-premises, on a public cloud provider, or across both simultaneously. Engineers interact with the same interfaces regardless of the underlying infrastructure, allowing organisations to make infrastructure decisions based on cost, performance, and available capabilities — not on application lock-in.

On-premises deployments can offer a true hybrid-cloud experience: consistent HPC application behaviour with the ability to burst to cloud resources on demand.

What’s Next for Kubernetes and HPC?

Adapting Kubernetes to the specific demands of HPC and engineering workloads is not trivial. MPI job patterns, tight inter-node communication latency requirements, and high-throughput batch scheduling are all areas where additional tooling — such as the Kubernetes Job API extensions, MPI Operator, or volcano batch scheduler — continues to mature.

The primary driver of this progress is the AI and machine learning community, whose workloads share many characteristics with traditional HPC (GPU clusters, parallel training jobs, large data movement) but have adopted Kubernetes natively. Projects like Kubeflow, volcano, and the Kubernetes SIG-Scheduling working group are expanding what is possible for batch and HPC-style workloads on Kubernetes.

For HPC practitioners, the practical path forward is to treat Kubernetes as the infrastructure and operations layer, while preserving proven HPC workload managers for the scheduling logic that scientific computing depends on — at least until native Kubernetes batch capabilities reach the required maturity.

About the Original Authors

Daniel Gruber, Burak Yenier, and Wolfgang Gentzsch are with UberCloud, a company that began developing HPC container technology and containerised engineering applications in 2013 to facilitate access to HPC workloads in shared on-premises and cloud environments. This article draws on their published white paper describing their experience with UberCloud HPC containers and Kubernetes.