‘At scale’ or ‘large-scale’ Agile and DevOps development – sometimes separately but often in the same breath – was a topic that has been cropping up more frequently in the enterprise-IT world. The ‘scale’ message is topical because it chimes with how software development increasingly involves the entire enterprise. For instance, IoT products cover hardware and software engineering, while video game development encompasses coders and artists with various specialities. This broader application of Agile is also affected by having to work for distributed teams, stretching what ‘classic’ Agile practices such as SCRUM covers.

Many organisations have already experienced success with Agile and DevOps. However, extrapolating the foundations on which those successes are built on a much bigger scale can be a challenge, though there is an increasing wealth of large-scale Agile and DevOps stories out there. Many of the IT industry’s brightest brains are also focusing their attention on how to address the ‘at scale’ issue to deal with the complexity, sheer volume of content and contributors, often across dispersed environments and locations.

Within the Agile community, fast-growing and emerging approaches include Large-Scale Scrum (LeSS) and large-scale Kanban. This is in addition to the more established Scaled Agile Framework (SAFe), which gives some organisations – particularly more traditional ones – a structure to move to Agile, while others may find it too inflexible. There are also several hybrid variants, for instance combining Gantt scheduling with SCRUM. The latter is indicative of a growing realisation by many organisations it is often better to pick and choose elements that best fit each situation, that rather than sticking rigidly to a prescribed Agile or DevOps approach. There is no ‘one size fits all’, even less so at scale.

What is perhaps more important to understand is how to address some of the stumbling blocks that will occur in large-scale Agile or DevOps implementations, regardless of framework. A classic roadblock is a realisation that getting – and keeping – everyone on board is hard to sustain, yet change will be the only on-going constant. Management who initially approved of an Agile or DevOps implementation may find enough reason to abandon it at the first problem, viewing it as a failure rather than acknowledging that change is never easy, it requires work and dedication to reap the long-term rewards. This is why it is essential to look at the entire lifecycle of an application and the concrete business results to understand if sustainable change is happening.

Toolchains need to be reassessed for their scalability: not just performance and speed, but also how usable they are for different contributors (which in large-scale Agile or DevOps can include less technical individuals), whether they fit around people’s existing working processes and tools (such as finding ways to enable them to carry on using Git for version control, without sacrificing project-wide control). Also, the file types that need to be included go far beyond just code. This is especially true in IoT projects where there are both hardware and software designers, or in games and entertainment where coders and artists need to collaborate, combining the code with large assets and other types of files.

Adding in new types of tools or re-evaluating the role of existing ones can support ‘at scale’ Agile and DevOps.  The right tools help to automate processes and create consistent efficiency. Questions to ask include, for example, what does it mean to have a true product backlog? How is that different from just having a flat list of issues or spreadsheets to make decisions? These are crucial questions to answer as no Agile organisation can be go beyond the limits of their tooling.

Continuous code inspection and automated static code analysis should always happen in the background with minimal manual involvement. Building enterprise-wide development standards help with consistent reproducibility and code quality across large teams, while also contributing to any compliance requirements.

Automation clearly contributes towards large-scale projects, speeding up productivity and reducing manual error. While not everything can be automated, many routine tasks can, such as automation of test processes, which is becoming increasingly important with the trend towards continuous-testing and ‘Shift Left’, where developers are taking on more responsibility for testing earlier in the development process.

However, tools should always be the supporting act and both Agile and DevOps will always rely on the right mindset. While tools can help provide speed and quality, a culture that is truly collaborative, flexible, constantly open to change and seeking improvement, is essential to large-scale adoption of any software development methodology.

By early 2019, cloud computing had moved well beyond its origins as a low-cost alternative to on-premises servers. It had become the default platform for enterprise digital transformation. The combination of low-cost infrastructure and high-value managed services was accelerating cloud adoption faster than most analysts had predicted even a few years earlier. Here are the ten trends that were expected to define cloud computing in 2019.

1. Enterprise Cloud Spending Would Surge

The expectation was that 2019 would see a significant jump in enterprise cloud budgets, driven by two converging needs: modernizing legacy applications and extracting value from years of accumulated data. Organizations were expected to invest heavily in SaaS platforms while simultaneously adopting cloud-native technologies — particularly Kubernetes, containers, and serverless functions — to support new architectures and faster release cycles.

The underlying logic was straightforward: once leadership recognized cloud as a competitive differentiator rather than just a cost-saving measure, budget constraints eased and adoption accelerated across the board.

2. Multi-Cloud and Hybrid-Cloud Orchestration Would Gain Momentum

By 2019, many enterprises had already committed to both on-premises infrastructure and at least one public cloud provider. The resulting hybrid and multi-cloud environments introduced serious operational complexity — managing costs, enforcing policies, and maintaining visibility across disparate platforms became full-time engineering challenges.

The prediction was that dedicated orchestration platforms would emerge to address this complexity. Rather than teams juggling separate consoles for AWS, Azure, GCP, and private data centers, unified orchestration layers would provide a single pane of glass for cost tracking, workload placement, and governance. This trend proved accurate and has since matured into a category of its own, with tools like Terraform, Crossplane, and vendor-native control planes filling this role.

3. Open-Source Would Drive Technology Democratisation

The democratisation of cloud infrastructure tools was already underway by 2019, led largely by the open-source ecosystem. Technologies that had previously required specialized vendors — hypervisors, cloud management platforms, container runtimes, service meshes — were becoming available as community-maintained open-source projects.

Developer communities were identified as the primary driver of this shift. As developers gained more influence over infrastructure decisions (a trend closely related to the rise of DevOps culture), open-source tools became the default starting point for new projects, reducing lock-in and lowering the cost of experimentation.

4. Specialized Cloud Service Models Would Proliferate

The cloud service model had already expanded well beyond IaaS, PaaS, and SaaS by 2019. The prediction was that increasingly specialized service categories would gain traction, including:

• iPaaS (Integration Platform as a Service) — for connecting disparate applications and data sources
• dPaaS (Data Platform as a Service) — managed data pipeline and analytics environments
• BaaS (Blockchain as a Service) — hosted distributed ledger capabilities
• FaaS (Function as a Service) — serverless compute, exemplified by AWS Lambda and Azure Functions
• mBaaS (Mobile Backend as a Service) — managed backends for mobile application developers

This fragmentation reflected a broader shift: cloud providers were no longer competing solely on raw infrastructure price, but on the depth and breadth of managed services they could offer to reduce developer toil.

5. Full-Stack Developers Would Need Data Engineering Skills

As organizations moved more of their workloads to the cloud and adopted data-driven decision making, the boundaries between application development and data engineering began to blur. The prediction was that full-stack engineers who understood distributed data processing frameworks — particularly Apache Spark — would become significantly more valuable.

This reflected a real skills gap: many development teams knew how to build APIs and front-end interfaces, but lacked the background to design efficient data pipelines or work with large-scale streaming and batch processing systems. Cloud platforms lowered the barrier to entry for these capabilities, but developers still needed a conceptual foundation to use them effectively.

6. Cloud-Based AI and Machine Learning Would Accelerate

The major public cloud providers had begun embedding AI and machine learning services into their platforms around 2017. By 2019, these offerings had matured into comprehensive ecosystems covering managed model training, inference APIs, AutoML tooling, and pre-built cognitive services for vision, speech, and natural language processing.

The expectation was that 2019 would see accelerated enterprise adoption, as organizations that had been evaluating or piloting AI capabilities would move toward production deployments. This prediction aligned with the broader digital transformation narrative: companies that delayed adoption risked falling behind competitors who were using AI to optimize operations, personalize customer experiences, and reduce costs.

7. Organizations Would Create Dedicated Digital Skills Roles

A recurring challenge for enterprises adopting cloud technology was the gap between available training programs and the practical skills their engineering teams actually needed. Generic certification courses often failed to align with a team’s specific technology stack or business context, leaving a disconnect between training investment and measurable outcomes.

The prediction was that organizations would respond by creating dedicated Digital Skills Officer roles — similar in concept to a Chief Learning Officer — positioned to work alongside CTOs and CIOs. These roles would be responsible for translating technical roadmaps into targeted learning programs, ensuring that training efforts produced engineers capable of executing on the organization’s specific cloud strategy rather than passing generic exams.

8. Apache Spark and Data Processing Expertise Would Become a Core Engineering Skill

Related to prediction five, this point emphasized the specific importance of Apache Spark as a framework that full-stack and generalist engineers would increasingly encounter. As cloud-based data platforms made it easier to spin up Spark clusters without managing underlying infrastructure, the expectation was that familiarity with Spark’s programming model — DataFrames, RDDs, streaming APIs — would become a standard part of the cloud engineer’s toolkit rather than a specialized data engineering niche.

This prediction reflected the growing importance of real-time and batch analytics as first-class features of cloud-native applications, rather than separate analytical workloads handled by a separate data team.

9. Container Adoption Would Accelerate, Especially in Microservices

Containers package an application with all of its dependencies — libraries, binaries, configuration — into a single portable unit that runs consistently across environments. By 2019, containers had moved from early-adopter experimentation to mainstream enterprise consideration, driven in large part by Docker’s popularity and the rapid maturation of Kubernetes as an orchestration standard.

The prediction was that 2019 would see continued acceleration, particularly in microservices architectures where containers provide an ideal deployment unit: lightweight, independently deployable, and portable across cloud providers and on-premises infrastructure. Managed Kubernetes services from AWS (EKS), Google (GKE), and Azure (AKS) were all either recently launched or expanding, lowering the operational barrier to container adoption significantly.

10. Cloud Would Extend and Enhance Legacy Applications

Early cloud narratives often assumed that organizations would wholesale migrate their legacy systems to the cloud. In practice, this rarely happened on the expected timeline. Most enterprises found that core legacy systems — billing platforms, ERP systems, mainframe workloads — were too deeply embedded in business processes to replace quickly, and the risk of disruption outweighed the benefits of a full migration.

The more realistic prediction for 2019 and beyond was a hybrid approach: legacy systems would remain in place, but cloud services would be used to augment them. Specific augmentation patterns included:

• Connecting legacy data stores to cloud-based analytics and machine learning pipelines
• Using edge computing for IoT workloads that legacy systems were not designed to handle
• Wrapping legacy functionality in modern APIs to enable new front-end experiences
• Offloading compute-intensive workloads to cloud functions while keeping core logic on-premises

This pattern — often called “cloud augmentation” or the “strangler fig” migration pattern — became one of the dominant enterprise cloud strategies precisely because it reduced risk while delivering incremental value.

Looking Back: How Accurate Were These Predictions?

In retrospect, most of these 2019 predictions proved directionally correct. Multi-cloud complexity became one of the defining operational challenges of the following years. Container and Kubernetes adoption continued at pace. AI and ML services expanded dramatically across all major cloud platforms. The skills gap in cloud and data engineering remains a persistent issue for engineering leaders today.

The prediction that aged least well — or at least evolved significantly — was the focus on Apache Spark as a specific skill for full-stack developers. While Spark remains important in data engineering, the broader ecosystem around managed data services (Databricks, BigQuery, Snowflake, dbt) shifted how practitioners interact with distributed data processing, often abstracting away the framework-level details that the 2019 prediction emphasized.

The core insight from this 2019 analysis remains valid: cloud is not simply an infrastructure cost play. It is the platform on which enterprise competitiveness is built, and the organizations that invest in skills, orchestration, and architecture — not just compute budgets — are the ones best positioned to benefit from it.