Run an Amazon SageMaker Notebook Locally with Docker

Originally sourced from towardsdatascience.com. This article targets data scientists and ML engineers who are using or planning to use Amazon SageMaker. Basic familiarity with SageMaker Notebook instances and Docker is assumed.

What Is Amazon SageMaker?

Amazon SageMaker is AWS’s managed cloud machine learning platform. It is built around four core offerings that support the end-to-end data science workflow:

1. Ground Truth — A managed service for large-scale, on-demand data labeling, used to generate training data for supervised learning algorithms.
2. Training — A managed service to train and tune models at any scale.
3. Inference — A managed service to host pre-built models and run predictions against new data.
4. Notebook — An AWS-hosted Jupyter server that gives data scientists a single interface to manage the above three services alongside general data science tasks.

The hosted Notebook instance is a convenient, batteries-included environment. However, there are real-world scenarios where running an equivalent environment locally — inside a Docker container — is preferable.

TL;DR

• The SageMaker Notebook instance comes pre-loaded with ML frameworks, multiple kernels, GPU support, Git integration, and AWS example notebooks.
• It costs money to run, requires an internet connection, demands AWS Console sign-in, and can be difficult to customize deeply.
• A Docker container that mirrors the Notebook instance environment lets you work locally on any laptop or desktop.
• Replicated features include Jupyter Notebook and JupyterLab, multiple Conda-based kernels, AWS and SageMaker SDKs, the AWS CLI, Docker CLI passthrough, Git integration, a Conda tab, and the SageMaker Examples tab.
• The cloud instance and the local container are complementary, not mutually exclusive. With proper Git integration, you can switch between them seamlessly.

Why Use the Hosted SageMaker Notebook Instance?

Before considering a local alternative, it is worth understanding what the hosted instance provides:

• Complete system on demand. With a few clicks in the AWS Console, you get a fully configured system with the most commonly needed ML libraries and tools — no infrastructure management required.
• Multiple ML kernels. Built-in kernels cover standard Python 2 and 3, popular frameworks (TensorFlow, MXNet, PyTorch, Chainer), and other runtimes (R, PySpark).
• Multiple instance sizes. You can choose from general-purpose instance types (t2, t3, m4, m5), compute-optimized (r4, r5), or GPU-backed instances (p2, p3) depending on your workload.
• Elastic Inference. For GPU acceleration without the full cost of a p2/p3 instance, you can attach an Elastic Inference accelerator (eia1 type) to your notebook instance.
• Built-in security. Access is gated by AWS Console authentication by default.
• Git integration. You can link the instance to a Git repository hosted on AWS CodeCommit, GitHub, or any other provider, enabling version control for all notebook work.
• Curated examples. Every instance ships with a rich set of SageMaker example notebooks drawn from AWS’s public GitHub repository. You can import any example into your workspace with minimal effort.

Why Run a Local SageMaker Notebook Container?

Despite the advantages above, the hosted instance has limitations that a local Docker container can address:

• Cost. The hosted instance incurs EC2 charges whenever it is running. Spot instances are not available for Notebook instances, so there is no built-in way to reduce that cost. A local container runs free.
• Local data access. Because the hosted instance runs in the cloud, local datasets must be uploaded to S3 or another online store before use. A local container can mount any host directory directly using a Docker volume.
• No daily Console sign-in. The hosted instance requires AWS Console authentication and sessions time out after 12 hours. With the local container, you configure AWS credentials once on the host machine and do not need to sign in again for routine notebook work.
• Offline access. The local container is accessible without an internet connection — useful for data cleaning, feature engineering, and exploratory analysis that does not require SageMaker training or inference endpoints.
• Customization. The hosted instance supports lifecycle configurations and terminal access for customization, but this requires shell scripting knowledge and can feel constrained. A Dockerfile gives teams full control over their environment, which can then be shared and version-controlled like any other code artifact.
• Flexibility. The Docker image can be hosted centrally on a high-capacity on-premises server, distributed to individual developer machines, or used to spin up clusters of notebook servers. The deployment model is entirely up to the team.

Note: The AWS-hosted instance and the local container are not mutually exclusive. Used together — with a shared Git repository — they complement each other well, letting you alternate between cloud and local environments for the same project.

Features Replicated in the Local Container

The goal of the local container is to reproduce the most important features of the AWS-hosted Notebook instance while adding the benefits of local execution. Here is what has been replicated:

Jupyter Notebook and JupyterLab

The container is based on Jupyter’s official Docker images with modifications to align with SageMaker’s defaults:

• The default user is named ec2-user with passwordless sudo access, matching the hosted instance.
• Custom Miniconda and Conda versions can be specified at build time.
• JupyterHub is skipped (not needed for single-user local use).
• The default notebook directory is set to /home/ec2-user/SageMaker.

Multiple Kernels

Multiple kernels are created using Conda environments (conda env create), mirroring the kernel names used by the hosted instance. Examples include:

• conda_python2
• conda_python3
• conda_tensorflow_p36
• conda_mxnet_p36

The set of kernels is configurable at image build time via the CONDA_ENVS build argument, so a given image can include one kernel or many. Additional kernels can be added by placing Conda environment YAML files in the base/utils/envs directory and rebuilding the image.

AWS and SageMaker SDKs

Every kernel includes both the AWS Boto3 SDK and the SageMaker Python SDK. These are essential for any Python code that interacts with AWS services, including SageMaker training jobs and model deployment.

AWS CLI

The AWS CLI is installed inside the container for shell scripts and terminal workflows that interact with AWS services such as S3.

Note: Both the AWS SDK and CLI require AWS credentials to be configured on the host machine (for example, via ~/.aws/credentials or environment variables), which are then passed into the container.

Docker CLI (Host Engine Passthrough)

Many SageMaker examples require Docker to build custom training images. Rather than running a full Docker-in-Docker setup (which is complex and carries security implications), the container installs only the Docker CLI and connects it to the host machine’s Docker Engine via a socket mount.

Include the following flag when starting the container:

-v /var/run/docker.sock:/var/run/docker.sock:roCode language: JavaScript (javascript)

Windows note: Update the mount path to //var/run/docker.sock:/var/run/docker.sock:ro when running on Windows.

Git Integration in JupyterLab

Git is installed in the container, and the jupyterlab-git extension is included in JupyterLab, providing a graphical interface for common Git operations without leaving the browser.

Conda Tab

Mirroring the hosted instance, the local container includes a Conda tab for browsing and managing Conda environments.

Recommendation: Rather than making ad-hoc changes through the Conda tab, update the corresponding YAML files in base/utils/envs and rebuild the Docker image. This keeps environment changes reproducible and shareable with the rest of your team.

SageMaker Examples Tab

AWS-provided SageMaker example notebooks are bundled in the Docker image, along with the same two-click copy workflow found in the hosted instance. This lets you pick any example and start experimenting immediately. (Third-party examples such as those from fast.ai and PyTorch were not included in the initial version but can be added.)

Known Gaps: Features Not Yet Replicated

The local container does not yet replicate every feature of the hosted instance. The following are known gaps:

GPU Support

The container currently targets CPU workloads only. If you need local GPU-accelerated training or inference, you must use the AWS-hosted instance or a separately configured GPU-enabled Docker setup. Note that if you submit SageMaker training jobs to GPU-backed cloud instances from within the local container, you do not need local GPU support for that workflow.

Non-Python Kernels

Additional Python-based kernels can be added with a new Conda environment YAML file. However, R and PySpark runtimes require more substantial work and were not supported in the initial version.

SageMaker Local Mode

The AWS-hosted Notebook instance supports SageMaker’s local mode, which runs training and inference containers directly on the notebook server. This feature was not configured in the initial local container release; network configuration work is needed to support it.

Automated Kernel Updates

When AWS updates the SageMaker Notebook instance (for example, bumping framework versions), the kernel Conda environment files in the local container must be updated manually. A potential future improvement would be to automate this using CloudWatch Events and AWS Lambda to detect SageMaker version changes and trigger a rebuild.

Pre-installed Python Libraries

To keep image sizes manageable, many Python libraries available in the hosted instance are not pre-installed in the local container kernels. Teams with specific library requirements should add them to a custom Conda environment YAML file and rebuild the image.

Additional Tooling

Tools such as nbdime (Jupyter notebook diff and merge) and plotlywidget (interactive graphing for Python) were identified as candidates for future inclusion.

Getting Started

To run the local SageMaker Notebook container, you need Docker installed on your machine. The general approach is:

1. Clone or build the Docker image that replicates the SageMaker Notebook environment, specifying your desired kernels via the CONDA_ENVS build argument.
2. Configure AWS credentials on your host machine (for example, via aws configure or by placing credentials in ~/.aws/).
3. Run the container with volume mounts for your working directory, the Docker socket, and your AWS credentials directory. For example:

   docker run -p 8888:8888 \
     -v $(pwd)/notebooks:/home/ec2-user/SageMaker \
     -v ~/.aws:/home/ec2-user/.aws:ro \
     -v /var/run/docker.sock:/var/run/docker.sock:ro \
     your-sagemaker-notebook-image

4. Open JupyterLab at http://localhost:8888 in your browser.

Check the source repository for the full Dockerfile, environment YAML files, and any additional run scripts specific to the implementation described in this article.

Summary

The Amazon SageMaker Notebook instance is a powerful, convenient starting point for working with the AWS ML platform. A local Docker container that replicates its core environment extends that convenience to offline work, reduces costs for development and exploration tasks, and gives teams the customization and flexibility that a managed cloud service cannot easily provide.

The two approaches work best together: use the local container for iterative development, data exploration, and cost-sensitive workflows, and switch to the hosted instance when you need GPU-backed compute, Elastic Inference, or the full library of pre-installed kernels. With Git integration keeping both environments in sync, the transition between them can be nearly seamless.

AWS also provides separate Docker containers for SageMaker training and deployment, which complement this notebook container as part of a fully containerized ML development workflow.