This article is based on reporting originally published by Environmental Leader.
HPE and NREL Join Forces on AI-Driven Data Center Operations
Hewlett Packard Enterprise (HPE) has announced an AI Ops research and development collaboration with the US Department of Energy’s National Renewable Energy Laboratory (NREL). The goal of the partnership is to develop artificial intelligence and machine learning technologies that automate and improve operational efficiency — including resiliency and energy usage — in data centers designed for the exascale era of computing.
The project is structured as a three-year collaboration and focuses on introducing monitoring and predictive analytics to the power and cooling systems at NREL’s Energy Systems Integration Facility (ESIF) High Performance Computing (HPC) Data Center.
Training ML Models on Years of Historical Sensor Data
A key element of the project is the use of more than five years of historical operational data — totalling over 16 terabytes — gathered from sensors embedded in NREL’s supercomputers, Peregrine and Eagle, as well as the facility infrastructure itself. This data is being used to train machine learning models for anomaly detection, with the objective of predicting and preventing problems before they impact operations.
Early results have been encouraging. Models trained on the historical dataset successfully identified or predicted events that had previously occurred in NREL’s data center, validating the practical potential of applying predictive analytics to real-world HPC environments.
Addressing Energy and Water Consumption at Scale
The collaboration also has a clear sustainability mandate. Data centers in the US collectively consume significant quantities of energy and water, and the HPE–NREL effort aims to provide a model for reducing that footprint. The project will monitor energy and water usage and optimize efficiency as measured by three industry-standard metrics:
- Power Usage Effectiveness (PUE): The ratio of total facility energy consumption to the energy delivered to IT equipment.
- Water Usage Effectiveness (WUE): The amount of water used by a data center relative to the energy consumed by IT equipment.
- Carbon Usage Effectiveness (CUE): A measure of total carbon dioxide emissions caused by data center energy consumption.
Rooted in PathForward and Exascale Computing Requirements
The AI Ops initiative grew directly out of HPE’s involvement in PathForward, a US Department of Energy program designed to accelerate the nation’s technology roadmap toward exascale computing. Exascale systems — capable of performing a quintillion (1018) calculations per second — represent a roughly thousandfold increase in speed over contemporary supercomputers.
Managing infrastructure of that scale and complexity requires a new approach to operations. AI-driven automation is seen as essential to maintaining energy efficiency and ensuring reliability at exascale, where the volume of telemetry data and the speed of infrastructure changes will exceed human operators’ ability to respond manually.
Open Source Stack: TensorFlow, NumPy, and Scikit-learn
The project’s ML pipeline is built on widely adopted open source libraries. HPE and NREL are using TensorFlow, NumPy, and Scikit-learn to develop the machine learning algorithms underpinning the anomaly detection and predictive analytics capabilities.
Four Functional Pillars of the AI Ops Framework
The collaboration is organized around four core operational areas:
- Monitoring: Collecting, processing, and analyzing large volumes of IT and facility telemetry from disparate sources in real time, then applying ML algorithms to that stream of data.
- Analytics: Using big data analytics and machine learning to analyze data from diverse tools and devices across the data center facility, generating actionable insights.
- Control: Applying algorithms that enable systems to resolve issues autonomously, automate repetitive operational tasks, and perform predictive maintenance on both IT hardware and facility infrastructure.
- Data Center Operations: Evolving AI Ops into a validation tool that supports continuous integration (CI) and continuous deployment (CD) workflows for core IT functions across the modern data center.
HPE’s Roadmap: HPCM and InfoSight Integration
Beyond the immediate NREL collaboration, HPE has outlined plans to expand these capabilities more broadly. The company intends to enhance its HPE High Performance Cluster Management (HPCM) system to support complete provisioning, management, and monitoring for clusters scaling to 100,000 nodes.
HPE also plans to explore integration with HPE InfoSight, a cloud-based, AI-driven management platform that continuously monitors, collects, and analyzes data on IT infrastructure. InfoSight is designed to predict and prevent potential system events, maintaining server health and performance — capabilities directly aligned with the goals of the NREL AI Ops project.
Why This Matters for DevOps and Infrastructure Teams
While this collaboration is focused on HPC-scale environments, the underlying principles — telemetry-driven anomaly detection, ML-based predictive maintenance, and AI-assisted autonomous remediation — are increasingly relevant to enterprise data center and cloud operations teams. As infrastructure grows in scale and complexity, automated intelligence is transitioning from a competitive advantage to an operational necessity. The NREL project offers a high-visibility proof point for these approaches at extreme scale, and its use of open source tooling means the lessons are broadly applicable.