Overview

Michael Berthold, CEO and co-founder of the open-source data analytics platform KNIME, delivered the keynote at the KNIME Fall Summit 2019. His talk focused on the complete end-to-end data science cycle, which he divided into two top-level categories—Create and Productionize—each containing two distinct phases.

Phase 1 – Gather & Wrangle

The first phase covers data acquisition and preparation. KNIME addresses this through a rich library of connector and transformation nodes, as well as big data extensions that integrate third-party technologies directly into a visual workflow. Supported integrations mentioned by Berthold include:

• Amazon Redshift
• H2 (in-memory database)
• Apache Hive and Impala
• Apache Spark

At the time of the keynote, KNIME had also extended its cloud file system connectivity, making it easier to read from and write to cloud storage buckets alongside on-premises data sources.

Phase 2 – Model & Visualize

This phase covers model building, library integration, and result visualisation. KNIME provides domain-specific nodes for several verticals, including:

• Life science – specialised cheminformatics and bioinformatics nodes
• Text processing – NLP preprocessing and classification
• Image analysis – pixel-level processing workflows
• Time-series analysis – lag features, smoothing, and forecasting nodes

For practitioners who need more control, KNIME integrates popular ML frameworks such as H2O and XGBoost, and allows custom code to be embedded via R and Python script nodes. This hybrid approach—low-code visual pipelines plus escape hatches for custom code—means teams with mixed skill levels can collaborate on the same workflow.

Phase 3 – Deploy & Manage

Once a model is ready for production, KNIME provides mechanisms to expose it to end users and downstream systems:

• Analytics applications – interactive web-based frontends for non-technical consumers
• Web services – REST endpoints that expose workflows as APIs
• KNIME Model Process Factory – enables on-demand model retraining, automated initialisation, and monitoring and alerting

KNIME also addresses production governance concerns—versioning, backwards compatibility, compliance controls, and documented best practices—so data science teams can maintain audit trails as models evolve.

Phase 4 – Consume & Optimize

The final phase closes the loop by making ML workflows accessible to business consumers. KNIME supports two primary consumption modes:

• Workflow as a Service – direct deployment so downstream systems can invoke a model programmatically
• Analytics Application – a guided UI that lets non-data-scientists run and interact with a model without writing code

Three Generations of Data Science Automation

Berthold framed the maturity of ML automation in three generations:

First Generation – Pre-canned Pipelines

Clean data enters a fixed pipeline, a standard set of transformations is applied, and a model is produced. There is little to no customisation or feedback loop. This covers basic batch scoring scenarios.

Second Generation – Interactive and Continuous

Customised interactions are introduced in the Model & Visualize and Deploy & Manage phases. Continuous deployment practices are applied, and users can provide feedback that influences subsequent model versions.

Third Generation – Mix-and-Match Automation

The Gather & Wrangle and Consume & Optimize phases become fully automated. Model building still involves customised human interaction, while deployment introduces guided interaction—where the platform makes suggestions and the practitioner confirms them rather than configuring everything manually.

Data Science Abstraction Through Components

Berthold concluded by discussing how KNIME uses Components—encapsulated, reusable workflow segments—to raise the level of abstraction across the pipeline. Components can encapsulate complex logic for steps such as:

• Feature selection and engineering
• Model interpretation and explainability
• Model management (versioning, comparison)
• Model deployment to production

By packaging these steps as components, organisations can standardise how ML problems are solved internally, reducing duplication and lowering the barrier for less experienced team members.

Four Adoption Profiles for Data Science

Berthold suggested that organisations should align their data science investment with their actual business context. He described four adoption profiles:

1. Standard problems with limited business impact – Simple, commodity data science tools are sufficient; the cost of a custom solution outweighs the benefit.
2. Applications requiring no specialist in-house expertise – Pre-built ML models or no-code tools can solve the problem without a dedicated data science team.
3. Productionising data science for value creation – Continuous delivery of models, integration of user feedback, and adoption of new technologies become critical differentiators.
4. Deep innovation embedded in the business – The organisation explores cutting-edge ML/AI research, integrates proprietary R&D, and treats data science as a core strategic capability.

KNIME Product Ecosystem

For teams looking to implement the patterns Berthold described, KNIME offers several products that map to different points in the end-to-end cycle:

• KNIME Analytics Platform – The open-source desktop environment for building and testing data science workflows.
• KNIME Server – A commercial server product that adds scheduling, team collaboration, and centralised deployment. Includes the KNIME Web Portal for browser-based workflow execution.
• Data Science as a Service – A managed offering for organisations that do not want to host their own KNIME Server infrastructure.
• KNIME Hub – A community and enterprise repository for sharing and discovering workflow components.
• KNIME Cloud Cognitive Services – Integrations with cloud-based AI APIs (announced at the time of the summit).

For the most up-to-date product information and documentation, visit the official KNIME website.

Cloud Is Reshaping Enterprise Business Intelligence

As once-common objections to cloud adoption — particularly around security — began to ease in the mid-2010s, organizations started identifying which IT and business initiatives would benefit most from cloud deployments. Business intelligence (BI) and analytics emerged as one of the clearest candidates. Research from that era consistently pointed toward cloud becoming the dominant delivery mechanism for analytics workloads. According to a joint Deloitte, EMA, and Informatica State of Cloud Analytics Report, 70.1% of respondents already considered cloud an essential part of their analytics strategy.

The 17 statistics below — drawn from leading research firms and industry surveys circa 2016–2017 — paint a picture of the momentum that was building behind cloud BI, and provide useful historical context for understanding how the market evolved.

Market Growth and Spending Trends

• Enterprise cloud spending was forecast to grow at a 16% compound annual growth rate (CAGR) between 2016 and 2026. — Wikibon 2017 True Private Cloud Forecast and Market Shares
• Adoption of cloud BI solutions grew 50% over three years, from 29% to 43%. — BARC Research and Eckerson Group Study, BI and Data Management in the Cloud: Issues and Trends 2017
• By 2020, more than $1 trillion in IT spending was projected to be directly or indirectly affected by the shift to cloud. — Gartner Market Insight: Cloud Shift — The Transition of IT Spending from Traditional Systems to Cloud

Strategic Adoption and Executive Priorities

• By 2020, any non-cloud strategy for new IT initiatives was expected to require formal justification at more than 30% of large enterprises. — Gartner Predicts 2017: Cloud Computing Enters Its Second Decade
• By 2021, more than half of global enterprises already using cloud were forecast to adopt an all-in cloud strategy. — Gartner Predicts 2017: Cloud Computing Enters Its Second Decade
• 74% of technology CFOs said cloud computing would have the most measurable business impact in 2017. — 2017 BDO Technology Outlook Survey
• CIOs ranked (1) security, (2) cloud computing, and (3) big data analytics and mobile as their top 2017 IT budget priorities. — Nomura Holdings CIO Survey Research

Why Organizations Are Moving BI to the Cloud

Understanding the motivations behind cloud BI adoption helps explain why momentum built so quickly. Research from KPMG and Deloitte identified both technical and business drivers.

Business Drivers

• The three top reasons CIOs adopted cloud were: (1) improve agility and responsiveness, (2) accelerate product development and innovation, and (3) reduce costs. — KPMG, Journey to the Cloud: The Creative CIO Agenda
• 29% of CIOs cited data center modernization as their primary reason for cloud adoption. — KPMG, Journey to the Cloud: The Creative CIO Agenda
• The top three business drivers of cloud analytics adoption were: enhancing business processes (23.5%), improving customer experience (22.2%), and better collaboration (19.1%). — Deloitte, EMA and Informatica State of Cloud Analytics Report

Technical Drivers

• The top three technical factors driving cloud analytics adoption were: reducing costs (19.5%), data security (15.1%), and collaboration (14.4%). — Deloitte, EMA and Informatica State of Cloud Analytics Report

Cloud BI Adoption and Organizational Readiness

• Cloud was an essential part of the analytics strategy for 70.1% of organizations; a further 21.6% said it was necessary as part of their analytics adoption path. — Deloitte, EMA and Informatica State of Cloud Analytics Report
• Nearly 50% of executive management teams viewed cloud BI as critical or very important to their information needs. — 2017 Cloud Computing and Business Intelligence Market Study, Dresner Advisory Services
• Financial services, higher education, business services, and retail were the four industries placing the greatest importance on cloud BI. — 2017 Cloud Computing and Business Intelligence Market Study, Dresner Advisory Services

How Organizations Use Cloud BI

• The top three cloud BI use cases were: dashboard-based reporting (76%), ad-hoc analysis and exploration (57%), and dashboard authoring (55%). — BARC Research and Eckerson Group Study, BI and Data Management in the Cloud: Issues and Trends 2017
• Top planned investments in cloud BI for the coming year included advanced and predictive analytics (53%), operational planning and forecasting (44%), and strategic planning and simulation (44%). — BARC Research and Eckerson Group Study, BI and Data Management in the Cloud: Issues and Trends 2017
• 69% of companies planned to increase their use of cloud BI, and more than one quarter of companies that had not yet implemented cloud BI said they would do so in the coming year. — BARC Research and Eckerson Group Study, BI and Data Management in the Cloud: Issues and Trends 2017

Key Takeaways for DevOps and Data Engineering Teams

Even viewed as historical data, these statistics underscore a structural shift that has continued well beyond 2017. Several patterns remain relevant for teams architecting modern data and analytics platforms:

• Agility and speed-to-insight outweigh pure cost savings as the primary justification for cloud BI — cost is a secondary benefit, not the lead argument.
• Security perception shifted from barrier to driver: by 2017, data security was already listed as a factor motivating cloud adoption, not blocking it.
• Dashboard reporting dominates near-term use cases, but predictive analytics investment was already the top planned priority — a trend that has accelerated significantly with modern ML platforms.
• Collaboration and cross-team access consistently appeared in both business and technical driver lists, pointing to the importance of governed, shared data platforms over siloed on-premise warehouses.

For teams evaluating or evolving their current cloud analytics stack, these foundational drivers — agility, collaboration, cost efficiency, and advanced analytics readiness — remain valid criteria for architectural decisions today.