This article is based on a theCUBE interview conducted at KubeCon + CloudNativeCon in San Diego, California, featuring Renaud Gaubert, lead container and OSS engineer at NVIDIA Corp., and Diane Mueller, director of community development at Red Hat Inc. The segment was sponsored by Red Hat Inc. Neither Red Hat nor other sponsors had editorial control over theCUBE or SiliconANGLE content.
The Infrastructure Complexity Problem for Data Scientists
Artificial intelligence and machine learning have become essential disciplines for data scientists, but they bring significant operational overhead. Setting up GPU clusters, managing drivers, configuring runtimes, and keeping infrastructure healthy can consume time that would be better spent extracting insights from data.
“We’re working towards reducing that complexity and making sure that people that are data scientists, AI, or machine-learning engineers are able to focus on their core business values,” said Renaud Gaubert, lead container and OSS engineer at NVIDIA Corp., speaking at KubeCon + CloudNativeCon.
The ideal experience, as Diane Mueller, director of community development at Red Hat, described it, is straightforward: data scientists want to open a JupyterHub notebook, find everything they need already in place, train their models, and move on — without worrying about what is running underneath.
“[Data scientists] would like to just open up a JupyterHub notebook, have everything there they need, train their models, have them run, and then after they’re done, they’re done.”
Diane Mueller, Director of Community Development, Red Hat Inc.
Kubernetes Operators as the New Deployment Pattern
Kubernetes Operators encode operational knowledge — installation, upgrading, failure recovery, and monitoring — into software that runs inside the cluster itself. Rather than requiring a human operator to manage complex stateful workloads by hand, an Operator continuously reconciles the actual state of the cluster with the desired state defined in a custom resource.
At KubeCon, Red Hat announced an alpha release of OpenShift 4 built entirely on Operators. Mueller described this as a significant shift in how applications and services are deployed and managed on Kubernetes platforms.
“It’s an all Operators-based deployment of OpenShift. I think the Operator framework really has been the big thing that we’ve been really getting a lot of uptake on. It’s been the new pattern for deploying applications, or services, and getting things beyond just a basic install of a service on OpenShift or any Kubernetes.”
Diane Mueller, Red Hat Inc.
OperatorHub.io: A Shared Marketplace for Kubernetes Operators
Alongside the OpenShift 4 alpha announcement, Red Hat has been building out OperatorHub.io as a community hub for upstream Kubernetes Operators. Operators listed there — including the NVIDIA GPU Operator — are designed to run on any conformant Kubernetes distribution, not just OpenShift.
“OperatorHub.io, everything in there runs on any Kubernetes. And the goal is to be able to take stuff in a hybrid-cloud model [and] be able to run it anywhere you want.”
Diane Mueller, Red Hat Inc.
Free access to these Operators means platform teams can deploy them on any Kubernetes environment, supporting a hybrid-cloud model where workloads are not locked to a single provider or distribution.
The NVIDIA GPU Operator: Four Phases of Automation
The NVIDIA GPU Operator is built on Red Hat’s Operator SDK and demonstrates how the Operator pattern can dramatically simplify AI/ML infrastructure management. Gaubert outlined four distinct phases of automation the GPU Operator handles:
Phase 1: Component Installation
The Operator automates the installation of every component a GPU cluster requires: the NVIDIA driver, the container runtime, the Kubernetes device plugin, and the necessary mount points. This replaces a manual, error-prone bootstrap process with a single declarative resource definition.
Phase 2: Infrastructure Automation and Updates
Once the cluster is running, the Operator automates infrastructure builds and manages rolling updates to individual components. This removes the need for manual intervention each time a driver or plugin version changes.
Phase 3: Lifecycle Management and Self-Healing
In a fleet of GPU nodes, individual machines will inevitably enter degraded or failed states. Phase three addresses lifecycle management: the Operator detects nodes that have drifted from the desired state and automatically remediates them, recovering from bad states to good ones without manual intervention.
Phase 4: Monitoring and Observability
The final phase surfaces operational insights to users. By integrating monitoring into the Operator itself, GPU cluster health and utilization data becomes visible without requiring separate, manually configured tooling.
“The Operator SDK has helped us a lot here, just laying out these different steps, and, in a way, it’s done the same thing as what we’re trying to do for our customers, the different data scientists. Which is, basically, get out of our way and allow us to focus on core business values.”
Renaud Gaubert, NVIDIA Corp.
Why This Matters for AI/ML Teams
The Operator pattern addresses a genuine pain point: the gap between Kubernetes as a general-purpose platform and the specific, complex requirements of GPU-accelerated AI/ML workloads. By encoding operational expertise into software, Operators allow platform teams to offer a managed-service-like experience on self-hosted or hybrid infrastructure.
- Reduced toil: Routine tasks such as driver installation, updates, and node recovery are automated rather than handled manually.
- Consistency: The same Operator runs identically on OpenShift, upstream Kubernetes, or any cloud provider’s managed Kubernetes service.
- Self-service for data scientists: With infrastructure managed by Operators, data scientists can provision GPU-backed environments without engaging the platform team for each request.
- Hybrid-cloud portability: Operators listed on OperatorHub.io are not tied to a single distribution, supporting workload portability across environments.
Further Resources
To explore Operators available for your Kubernetes environment, visit OperatorHub.io. For documentation on building your own Operators, the Operator SDK is the recommended starting point for Go, Ansible, and Helm-based Operators.
Source: SiliconANGLE / theCUBE interview at KubeCon + CloudNativeCon, San Diego, November 2019.