Shipping a machine learning model to production is fundamentally different from shipping a conventional application. The code may be straightforward, but the data, the model artifact, and the predictions it generates all introduce challenges that standard DevOps pipelines were never designed to handle. That gap is exactly what MLOps is trying to close.
Why MLOps Exists
Machine learning models learn patterns from training data and then apply those patterns to new, unseen inputs. Managing that data safely—ensuring it is clean, unbiased, private-compliant, and representative of real-world conditions—is genuinely hard. It is widely cited (including by Gartner) that the majority of data science projects never reach production. The operational complexity of managing data, model versions, and live predictions is a significant contributing factor.
Real-world data changes continuously. A model trained on historical house prices in one economic climate may perform poorly after a market shift. Inputs and predictions must therefore be monitored over time for distribution drift—a challenge that simply does not exist in the same form for a traditional web service.
How MLOps Differs from DevOps
Conventional DevOps is built around the build-and-release cycle. Source code is compiled into executable artifacts, tested, and deployed. The non-code assets involved—configuration files, environment variables—are relatively small and static. The system behaves deterministically: given the same input, it produces the same output.
Machine learning inverts this model. Instead of encoding rules explicitly, a model infers weighted patterns from large datasets. A classic example is predicting house prices: the model is trained on known sale prices alongside features such as location, square footage, and number of bedrooms, and then applied to new properties. The “build” in this context is a multi-step pipeline that:
- Ingests and preprocesses potentially large volumes of training data
- Runs one or more training jobs, often with many hyperparameter combinations
- Produces a model artifact (e.g., a serialised set of learned weights)
- Evaluates the model against held-out test data
- Packages and deploys the artifact to a serving environment
This workflow is experimental by nature. Data scientists iterate on feature engineering, model architecture, and training parameters in ways that do not map neatly onto a conventional CI/CD pipeline.
Versioning and Reproducibility
DevOps has well-established practices for versioning code and rolling back releases. MLOps does not yet have equivalent standardisation for versioning the data used to train a model. If a deployed model behaves unexpectedly, being able to reproduce the exact training run—same data snapshot, same parameters, same environment—is essential for diagnosis. Without it, debugging a production model is extremely difficult.
Monitoring in Production
Standard application monitoring tracks error rates, latency, and resource utilisation. These metrics apply to ML services too, but they do not tell you whether the model’s predictions are good. Evaluating prediction quality in production is harder because:
- Ground truth (the correct answer) may only become available long after the prediction is made, if at all.
- Proxy signals—conversion rates, user feedback, downstream outcomes—must often be used instead.
- Data drift can degrade model quality gradually rather than causing a sudden, obvious failure.
- Training data may represent the general population well but miss important edge cases that appear in production.
Mitigating these risks requires careful monitoring design and a cautious, staged rollout strategy for new model versions—analogous to canary deployments in conventional DevOps, but with model-specific quality metrics rather than error codes.
The MLOps Tooling Landscape
A growing ecosystem of tools addresses different parts of the MLOps problem. Organisations broadly face two choices: adopt a managed platform or assemble an in-house stack from open-source components.
Managed Platforms
Cloud providers offer integrated ML platforms that handle infrastructure, experiment tracking, model registry, and deployment. Examples include AWS SageMaker and Azure Machine Learning. These reduce operational overhead but tie an organisation to a specific cloud provider. Vendor-neutral alternatives such as Databricks MLflow offer a managed or self-hosted option that is not bound to a single cloud.
Self-Assembled Open-Source Stacks
Organisations with niche requirements—specific data residency rules, integration with bespoke internal systems, or a need to avoid cloud vendor lock-in—may prefer to build their own platform from open-source components. The landscape is broad and can be difficult to navigate. The Linux Foundation’s LF AI & Data project maintains a landscape diagram and categorised project lists that can help map available tools to specific needs.
For teams already running Kubernetes, the Kubeflow project is worth evaluating. It curates a set of open-source ML tools—covering pipelines, experiment tracking, hyperparameter tuning, and model serving—and makes them work together on Kubernetes. The project is led by Google and has broad industry participation.
Governance, Explainability, and Regulatory Compliance
Reproducibility and monitoring are not just engineering concerns—they are governance concerns. A production ML system may need to satisfy auditors, regulators, or end users who ask why a particular prediction was made. In the European Union, the General Data Protection Regulation (GDPR) gives individuals a right to “meaningful information about the logic involved” in automated decisions that affect them. Similar requirements are emerging in other jurisdictions and sectors.
This creates a direct engineering requirement: the system must be able to explain its predictions. Modelling approaches are often categorised as:
- White-box models (e.g., linear regression, decision trees): the relationship between inputs and outputs is inherently inspectable and human-readable.
- Black-box models (e.g., deep neural networks): the internal mechanics are opaque, and additional interpretation techniques must be layered on top.
In highly regulated domains—medical diagnosis, credit scoring, criminal justice—the inability to explain a prediction may be a hard blocker. A medical diagnosis model, for example, may need to surface the specific features that drove a conclusion so that a clinician can evaluate it critically. This can constrain model choice entirely, ruling out approaches that would otherwise offer better predictive performance.
Tooling for post-hoc explainability of black-box models (approaches that explain individual predictions after the fact rather than by design) is an active research and engineering area. Capabilities in this space are expanding, but they should be evaluated carefully for the specific regulatory context rather than assumed to satisfy compliance requirements automatically.
Where MLOps Is Headed
MLOps is still maturing. Standardisation of practices—especially around data versioning, model registries, and production monitoring—lags behind what DevOps achieved for conventional software delivery. Different organisations have different ML use cases, and a single prescriptive workflow is unlikely to fit all of them.
What is clear is that machine learning systems cannot be treated as black boxes dropped into production and forgotten. They require the same operational discipline as any other production system—and additional discipline specific to the data and model lifecycle. As adoption grows and best practices solidify, expect to see the tooling converge and the gap between a data science experiment and a production-grade system narrow significantly.
This article is based on original reporting from The Register (theregister.co.uk, March 2020).