Michael Berthold on End-to-End Data Science Using KNIME Analytics Platform

This article summarises Michael Berthold’s keynote at the KNIME Fall Summit 2019, as reported by InfoQ.

Overview

Michael Berthold, CEO and co-founder of the open-source data analytics platform KNIME, delivered the keynote at the KNIME Fall Summit 2019. His talk focused on the complete end-to-end data science cycle, which he divided into two top-level categories—Create and Productionize—each containing two distinct phases.

Category	Phases
Create	Gather & Wrangle → Model & Visualize
Productionize	Deploy & Consume → Optimize

Phase 1 – Gather & Wrangle

The first phase covers data acquisition and preparation. KNIME addresses this through a rich library of connector and transformation nodes, as well as big data extensions that integrate third-party technologies directly into a visual workflow. Supported integrations mentioned by Berthold include:

• Amazon Redshift
• H2 (in-memory database)
• Apache Hive and Impala
• Apache Spark

At the time of the keynote, KNIME had also extended its cloud file system connectivity, making it easier to read from and write to cloud storage buckets alongside on-premises data sources.

Phase 2 – Model & Visualize

This phase covers model building, library integration, and result visualisation. KNIME provides domain-specific nodes for several verticals, including:

• Life science – specialised cheminformatics and bioinformatics nodes
• Text processing – NLP preprocessing and classification
• Image analysis – pixel-level processing workflows
• Time-series analysis – lag features, smoothing, and forecasting nodes

For practitioners who need more control, KNIME integrates popular ML frameworks such as H2O and XGBoost, and allows custom code to be embedded via R and Python script nodes. This hybrid approach—low-code visual pipelines plus escape hatches for custom code—means teams with mixed skill levels can collaborate on the same workflow.

Phase 3 – Deploy & Manage

Once a model is ready for production, KNIME provides mechanisms to expose it to end users and downstream systems:

• Analytics applications – interactive web-based frontends for non-technical consumers
• Web services – REST endpoints that expose workflows as APIs
• KNIME Model Process Factory – enables on-demand model retraining, automated initialisation, and monitoring and alerting

KNIME also addresses production governance concerns—versioning, backwards compatibility, compliance controls, and documented best practices—so data science teams can maintain audit trails as models evolve.

Phase 4 – Consume & Optimize

The final phase closes the loop by making ML workflows accessible to business consumers. KNIME supports two primary consumption modes:

• Workflow as a Service – direct deployment so downstream systems can invoke a model programmatically
• Analytics Application – a guided UI that lets non-data-scientists run and interact with a model without writing code

Three Generations of Data Science Automation

Berthold framed the maturity of ML automation in three generations:

First Generation – Pre-canned Pipelines

Clean data enters a fixed pipeline, a standard set of transformations is applied, and a model is produced. There is little to no customisation or feedback loop. This covers basic batch scoring scenarios.

Second Generation – Interactive and Continuous

Customised interactions are introduced in the Model & Visualize and Deploy & Manage phases. Continuous deployment practices are applied, and users can provide feedback that influences subsequent model versions.

Third Generation – Mix-and-Match Automation

The Gather & Wrangle and Consume & Optimize phases become fully automated. Model building still involves customised human interaction, while deployment introduces guided interaction—where the platform makes suggestions and the practitioner confirms them rather than configuring everything manually.

Data Science Abstraction Through Components

Berthold concluded by discussing how KNIME uses Components—encapsulated, reusable workflow segments—to raise the level of abstraction across the pipeline. Components can encapsulate complex logic for steps such as:

• Feature selection and engineering
• Model interpretation and explainability
• Model management (versioning, comparison)
• Model deployment to production

By packaging these steps as components, organisations can standardise how ML problems are solved internally, reducing duplication and lowering the barrier for less experienced team members.

Four Adoption Profiles for Data Science

Berthold suggested that organisations should align their data science investment with their actual business context. He described four adoption profiles:

1. Standard problems with limited business impact – Simple, commodity data science tools are sufficient; the cost of a custom solution outweighs the benefit.
2. Applications requiring no specialist in-house expertise – Pre-built ML models or no-code tools can solve the problem without a dedicated data science team.
3. Productionising data science for value creation – Continuous delivery of models, integration of user feedback, and adoption of new technologies become critical differentiators.
4. Deep innovation embedded in the business – The organisation explores cutting-edge ML/AI research, integrates proprietary R&D, and treats data science as a core strategic capability.

KNIME Product Ecosystem

For teams looking to implement the patterns Berthold described, KNIME offers several products that map to different points in the end-to-end cycle:

• KNIME Analytics Platform – The open-source desktop environment for building and testing data science workflows.
• KNIME Server – A commercial server product that adds scheduling, team collaboration, and centralised deployment. Includes the KNIME Web Portal for browser-based workflow execution.
• Data Science as a Service – A managed offering for organisations that do not want to host their own KNIME Server infrastructure.
• KNIME Hub – A community and enterprise repository for sharing and discovering workflow components.
• KNIME Cloud Cognitive Services – Integrations with cloud-based AI APIs (announced at the time of the summit).

For the most up-to-date product information and documentation, visit the official KNIME website.