This article is based on a 2017 prediction piece originally sourced from insidebigdata.com and attributed to SOASTA, a performance analytics company. It is preserved here as a historical reference point for how the industry viewed data analytics trends at the time.
Data Analytics and E-Commerce: The 2017 Outlook
By early 2017, it was increasingly clear that data analysis was becoming a core driver of e-commerce revenue. The growing importance of data-driven decision-making was pushing e-commerce firms to hire more data scientists, with the goal of understanding how customer engagement patterns translated into sales outcomes.
Performance analytics firm SOASTA published predictions arguing that data analytics would become central not just to marketing teams, but to every part of the organization — from digital transformation initiatives to day-to-day digital performance management. Their five predictions offer a useful snapshot of where the industry was heading.
The Five Predictions
1. Engineering and DevOps Teams Would Manage Performance Like a Product
SOASTA predicted that DevOps and engineering teams would move away from reactive firefighting — fixing the most recent outage or manually tuning the slowest pages — toward a more proactive, data-driven approach. By combining monitoring solutions with advanced analytics, teams would be able to identify the areas of highest business impact before problems escalated. In practice, this meant treating application performance as a product with measurable KPIs rather than as an infrastructure concern handled only when things broke.
2. Data Scientists Would Become the Social Glue Between Business and Technology
The prediction here was that data scientists would evolve into cross-functional collaborators rather than siloed specialists. As data teams continued to surface strong correlations between business metrics (revenue, conversion rates, customer retention) and technical metrics (page load times, error rates, API latency), they would create a shared language that compelled business stakeholders and engineering teams to work more closely together. This foreshadowed the broader trend toward embedded analytics and shared dashboards across disciplines.
3. Machine Learning Would Move from Theory into Business Reality
In 2017, machine learning was still widely viewed as an emerging or experimental technology in most enterprise contexts. SOASTA predicted that ML would begin delivering practical business value, particularly in the area of anomaly detection and alerting. Specifically, ML-powered alerts would carry contextual information derived from Big Data insights, enabling engineering teams to diagnose and remediate issues faster rather than spending time manually correlating data from multiple sources.
4. Stream Processing and In-Stream Data Integration Would Emerge as Key Technologies
The fourth prediction focused on the infrastructure layer. New technologies enabling stream processing — the ability to analyze data in motion rather than in batch — were expected to gain significant traction. In-stream data integration, which allows data from multiple sources to be joined and enriched as it flows through a pipeline, was seen as a necessary evolution to support real-time analytics at scale. Technologies in this space (such as Apache Kafka and Apache Flink) were already maturing at this point, and this prediction broadly aligned with the direction the market was moving.
Looking Back: How Accurate Were These Predictions?
Viewed from the present, several of these predictions held up well. DevOps observability and performance engineering have become established disciplines. Data scientists and analytics engineers now routinely sit at the intersection of business and technology teams. Machine learning is embedded in production alerting systems at organizations of all sizes. And stream processing frameworks are now foundational components of modern data architectures.
What the predictions could not fully anticipate was the speed and scale of the MLOps movement, the rise of the modern data stack, or the degree to which cloud-native tooling would democratize access to these capabilities for smaller engineering teams.
Key Takeaways for DevOps Teams
- Treating application performance as a measurable product — with defined metrics and ownership — remains a best practice for engineering teams.
- Cross-functional data collaboration between engineering, product, and business stakeholders is now an expectation, not an exception.
- ML-assisted alerting and anomaly detection have matured significantly and are available through most major observability platforms.
- Real-time stream processing is no longer a niche capability; it is a standard component of production data infrastructure.
Original predictions attributed to SOASTA via insidebigdata.com, February 2017.