AWS regularly ships new services and features, but its AI and machine learning portfolio has seen particularly rapid growth. At re:Invent 2018 alone, AWS announced 13 machine learning products. That pace of change is welcome for users who need capable tools, but it makes it genuinely difficult for DevOps teams and data scientists to track what is available and when to use it.
This article breaks down the most significant recent additions to the AWS machine learning platform, organized by audience: infrastructure-focused developers and DevOps engineers who build and deploy custom models, and data scientists or non-specialists who need managed services to analyze large datasets without hand-rolling ML pipelines.
Who AWS Is Building For
AWS machine learning investments fall into two broad categories:
- Developer and DevOps tooling — Deep Learning AMIs and Deep Learning Containers target engineers building sophisticated, custom models and the teams responsible for deploying them on cloud infrastructure.
- Managed AI services — Services like Amazon Forecast and Amazon Personalize are designed for data scientists and non-specialists who want to apply advanced ML techniques to large datasets without managing the underlying infrastructure or writing training code from scratch.
AWS Deep Learning Containers
Setting up a reproducible environment for deep learning development has historically been a time-consuming task. Data scientists had to manually resolve dependencies across multiple source repositories, manage framework versions, and tune configuration for their target hardware — a largely DIY process prone to environment drift and hard-to-debug failures.
NVIDIA addressed this problem early with NVIDIA GPU Cloud (NGC), a registry of pre-built containers that bundle all the libraries and drivers needed to run deep learning workloads on NVIDIA hardware. AWS adopted a similar approach and introduced AWS Deep Learning Containers: a set of Docker images pre-configured with the most popular deep learning frameworks.
Supported Environments
AWS Deep Learning Containers can run across a range of AWS compute services and orchestration platforms:
- Amazon Elastic Kubernetes Service (EKS)
- Amazon Elastic Container Service (ECS)
- Amazon SageMaker
- Individual EC2 instances
- Self-managed Kubernetes clusters backed by EC2 instances
Frameworks and Runtime Support
At the time of this writing, the containers shipped with support for TensorFlow 1.14.0, MXNet 1.4.1, and the NVIDIA CUDA toolkit 10.0. Both frameworks expose Python APIs, which is the dominant scripting language in the data science community.
Key capabilities include:
- Model training and inference with both TensorFlow and MXNet.
- Distributed training with TensorFlow, supported via Horovod — an open-source distributed training framework originally developed by Uber.
- CPU and GPU instance compatibility on Ubuntu 16.04.
- SageMaker integration — useful for teams that want to run custom algorithms and containers without managing the training infrastructure directly.
Availability and Deployment
Deep Learning Containers are available at no additional charge from the AWS Marketplace or Amazon Elastic Container Registry (ECR). You pay only for the underlying compute resources you consume.
For Kubernetes-based deployments, the workflow is straightforward: create a pod specification file that references one of the Deep Learning Container images, then schedule it onto your cluster using kubectl. The same container images can be used consistently across local development, EKS, and SageMaker, reducing environment parity issues between development and production.
Deep Learning AMIs: Updates for Smaller-Scale Projects
For teams working on smaller projects — those involving a handful of VMs rather than a full container orchestration cluster — AWS maintains Deep Learning Amazon Machine Images (AMIs). These provide a similar pre-configured environment directly on EC2 instances, without the need to manage container infrastructure.
In 2019, AWS expanded framework support for Deep Learning AMIs with the following additions:
- PyTorch 1.1, Chainer 5.4, and CUDA 10 support for MXNet
- Amazon Linux 2 support, along with TensorFlow 1.13.1, MXNet 1.4.0, and Chainer 5.3.0
AWS also extended the Elastic Inference service to support TensorFlow 1.12, adding a new EIPredictor Python API. Elastic Inference allows you to attach a fractional GPU accelerator to any SageMaker or EC2 instance — rather than provisioning a full GPU instance — which can significantly reduce the cost of deep learning inference workloads while still improving throughput over CPU-only instances.
AWS AI Managed Services: Now Generally Available
Several AI services previewed at re:Invent 2018 reached general availability in the months that followed. These managed services abstract away model training, infrastructure provisioning, and hyperparameter tuning — users supply data, and AWS handles the ML pipeline.
Amazon Forecast
Amazon Forecast is a fully managed time series forecasting service built on technology Amazon developed for its own internal demand-planning use cases. Users supply historical time series data and any relevant metadata, and Forecast automatically selects and trains a predictive model. The service is designed to handle the kind of complex, multi-variable forecasting problems — such as product demand, resource utilization, or financial metrics — that are difficult to address with simple statistical methods.
Amazon Personalize
Amazon Personalize is a managed recommendation and personalization engine that draws on the same underlying models used in Amazon’s retail platform. Teams provide an interaction dataset — such as page views, purchases, or content engagement events — along with an item catalog. Personalize handles model training, tuning, and hosting, and returns ranked recommendations via an API. This makes it practical to add real-time personalization to applications without dedicated ML engineering resources.
Amazon ML Insights for QuickSight
ML Insights is an add-on module for Amazon QuickSight, AWS’s interactive business intelligence service. It introduces machine learning and natural language processing features directly into the BI workflow, including:
- Anomaly detection — automatically surfaces unexpected changes in metrics across large datasets.
- Forecasting — projects future values of time series data within QuickSight dashboards.
- Auto-narratives — generates plain-language textual descriptions of chart data, making it easier to communicate findings to non-technical stakeholders.
ML Insights is aimed at analysts who need more than standard statistical aggregation but do not want to build and maintain separate ML pipelines outside of their BI tooling.
Key Takeaways for DevOps and ML Teams
- If your team builds custom deep learning models, Deep Learning Containers remove the environment setup burden and work across EKS, ECS, SageMaker, and raw EC2.
- For smaller VM-based workloads, Deep Learning AMIs offer a faster path to a pre-configured training environment without container orchestration overhead.
- Elastic Inference is worth evaluating if inference cost on GPU instances is a concern — attaching fractional accelerators can reduce spend compared to full GPU instances.
- Managed services like Forecast and Personalize are production-ready options for common ML use cases where building and maintaining custom models is not justified by the available ML engineering capacity.
Source: Originally based on reporting from searchaws.techtarget.com (2019).