Bringing AI and HPC expertise to cloud computing

Source:-cio.com

You don’t have to look far to see that artificial intelligence is becoming an integral part of business operations. Today, businesses everywhere are reinventing their processes to take advantage of AI-driven capabilities, from chatbots in customer call centers to robotic vehicles in distribution facilities.

This rapid adoption of AI-driven services has brought about a corresponding demand for high performance computing. That’s because it takes a whole lot of computational power to train deep learning applications, do AI inferencing in real time and run analytics on massive amounts of data.

Hyperion Research reports, “Machine learning, deep learning, inferencing, training, graph analytics, and other AI methods, as well as high performance data analytics (HPDA) have all seen robust growth in the past few years, both in budget allocations as well as organizational focus. Growing at more than 30 percent, HPC-enabled AI is projected to be a $3.5 billion market in 2024.”

“HPC is crucial to AI developments, and the growth of HPC in the cloud, as well as the attention paid to HPC by Cloud Service Providers (CSPs), has resulted in many AI workflows being created and run in cloud environments. For some AI workflows, access to enough data for training can be challenging for on-premise execution or expensive to transfer from a cloud environment. In these cases, data residing or collected in a cloud environment, from sensors or the combination of multiple different data sets, can be better run where the data exists.”

The firm signals a major shift in the HPC market, with the percentage of HPC workloads running in the cloud jumping up to 20% in 2019. Here are a few possible reasons:

It’s now more viable to do HPC in the cloud, thanks to the increased availability of hybrid and multi-cloud options. These offerings are designed to provide HPC users with a near seamless computing environment between on-premises hardware and cloud resources.
As HPC applications grow in complexity, as is the case with the addition of new AI techniques to a wide range of applications, the ability to easily burst into the cloud provides a more flexible, responsive computing environment.
Multi-cloud capabilities allow HPC users to manage multiple third-party cloud resources from a single pane of glass. This is a feature that is especially attractive to global corporations that use various CSPs in different geographic locations.

Many CSPs now provide support that is specifically targeted for HPC workloads, in-house HPC experts and services to help HPC users better understand the process of running their applications in the cloud. With this expertise, HPC users can get the guidance they need to run the right instances on the right platform for each of their jobs.
“Many end users are beginning to understand how AI, and its many subsegments, can aid their current workloads, whether through running AI in conjunction with traditional modeling and simulation workloads, or in entirely new application spaces with AI,” according to Hyperion. “With their HPC & AI Innovation Lab, Dell brings together a robust hardware and software infrastructure, combined with a staff of experts, including engineers, computer scientists, and subject matter experts. These experts are all deeply engaged in the HPC community and understand the best practices for solving HPC and AI problems.”