This article is based on remarks by Rob Vanstone, technical director at continuous delivery vendor XebiaLabs, delivered at the Computing DevOps Summit. The original report was published by computing.co.uk.
The Core Challenge: Scaling What Works in Pockets
High-performing software organisations share a common trait: they release code frequently. The business case for faster iteration is well established—shorter feedback loops, reduced risk per release, and tighter alignment between engineering output and business value. The hard part is not proving the principle; it is industrialising it across a large enterprise.
According to Rob Vanstone, a recurring pattern plays out in organisations that attempt to scale DevOps. Individual teams achieve genuine breakthroughs—usually through scripting and automation—and become internal poster children. Leadership notices and asks for the same results everywhere. That is precisely where progress stalls.
“Management sees those poster children and say ‘we want more of that’, and that’s where the problem starts.”
Rob Vanstone, XebiaLabs
What works cleanly for one autonomous team does not automatically transfer to dozens of teams sharing toolchains, governance processes, and release gates. Four organisational and technical barriers consistently explain why the scale-up fails.
Barrier 1: Waterfall Thinking Persists Beneath Agile Ceremonies
Adopting Scrum or Kanban at the team level does not automatically change the mindset of the wider organisation. Many large enterprises retain a fundamentally Waterfall structure—clear divisions of skills and labour, handoffs between separate departments, and no shared ownership of outcomes—even while individual teams run sprints.
The consequence is a focus on the technology itself rather than on the business outcomes the technology should enable. Velocity metrics, sprint completion rates, and deployment counts become ends in themselves rather than proxies for faster, better software delivery.
What to address:
- Align team-level Agile practices with an organisation-wide delivery philosophy, not just a project management framework.
- Establish shared ownership of outcomes across development, operations, and QA—not just shared tools.
- Measure success in terms of business value delivered, not process milestones reached.
Barrier 2: Leadership Is Watching Instead of Driving
DevOps transformations in large organisations frequently stall because leadership treats the initiative as something to monitor rather than something to actively lead. Silos persist. Isolated pockets of development iterate independently but remain disconnected from each other and from the broader delivery pipeline.
Incremental improvements within those silos are necessary but insufficient. Without executive involvement in the actual decision-making process—not just dashboard reviews—the organisational changes required to join those pockets together cannot be driven through.
“We’re not just talking dashboards, but the whole decision making process.”
Rob Vanstone, XebiaLabs
What to address:
- Ensure senior leaders are active participants in pipeline governance, not passive consumers of status reports.
- Give leadership teams visibility into delivery bottlenecks at the value-stream level, so they can make resourcing and process decisions based on data.
- Treat DevOps transformation as a change management programme, not an IT project.
Barrier 3: A False Sense of Maturity
Teams that have successfully automated a previously manual process—reducing a three-week task to three hours, for example—often conclude they have reached a high level of DevOps maturity. The automation is real and valuable, but it is not the same thing as organisational maturity if it does not translate into measurable business outcomes.
“They think they’re mature if they have a level of automation, irrespective of their ability to deliver business outcomes.”
Rob Vanstone, XebiaLabs
IT maturity is only meaningful if it produces business value or a meaningfully better user experience. Without that connection, the business case for further automation investment becomes difficult to justify—and funding dries up before the transformation reaches the scale required to make a real difference.
What to address:
- Define maturity models that include business outcome metrics, not just technical delivery metrics.
- Map automation improvements to their downstream effect on release frequency, mean time to recovery, or customer-facing quality.
- Communicate automation wins in business language to maintain executive sponsorship.
Barrier 4: QA and Testing Are Treated as an Afterthought
The fourth barrier is the systematic undervaluation of quality assurance and testing as a first-class component of the delivery pipeline. At small scale, manual or late-stage testing is inconvenient but manageable. As DevOps scales, it becomes a hard ceiling on delivery throughput—and a growing security risk if security testing is similarly bolted on at the end.
A common pattern is a central testing practice that is heavy on user acceptance testing (UAT) and integration testing but light on automated unit, component, and contract tests earlier in the pipeline. UAT and integration testing are necessary, but they are slow and cannot absorb the volume of changes that a mature CI/CD pipeline produces.
“The focus is more on user acceptance testing and integration testing which while necessary do nothing to speed the delivery cycle.”
Rob Vanstone, XebiaLabs
What to address:
- Shift testing left: invest in automated unit and integration tests that run on every commit, not just at the end of a release cycle.
- Set explicit KPIs for test automation coverage and pipeline gate pass rates.
- Align QA goals with DevOps goals—speed and safety are complementary, not competing, objectives.
The Underlying Principle: Joined-Up Toolchains and Data-Driven Improvement
All four barriers share a common root: the absence of coherence. Tools, teams, practices, and metrics operate in isolation rather than as parts of a single, instrumented delivery system. The path forward requires connecting those components deliberately.
Vanstone framed the goal clearly: tools need to be “joined together in a coherent way so you can get your software in front of your customer as quickly and as safely as you can.” Equally important is collecting data about the delivery process itself—pipeline lead times, failure rates, deployment frequency, and recovery times—and using that data to guide investment decisions and identify bottlenecks before they become crises.
Scaling DevOps is fundamentally an organisational challenge with a technical dimension, not the other way around. The teams that succeed are those that treat culture, leadership, measurement, and toolchain coherence as equally important workstreams—not sequential phases.
Summary
| Barrier | Root Cause | Key Action |
|---|---|---|
| Waterfall mindset | Agile ceremonies without cultural change | Align ownership and metrics to business outcomes |
| Passive leadership | Executives monitoring rather than deciding | Embed leaders in delivery governance |
| False maturity | Automation divorced from business value | Define maturity by outcomes, not activities |
| Undertested pipelines | Late-stage, manual QA at scale | Shift testing left with clear automation KPIs |