This article draws on findings originally published on itproportal.com, based on survey data from a “How the World Tests” report. Specific survey figures are preserved from the source; see review notes for caveats.
Why Scaling Agile Is Harder Than Adopting It
The past decade has fundamentally changed what it means to ship software quickly. Agile development methodologies have moved from fringe practice to mainstream expectation, yet organisations consistently report that broad-scale adoption—across multiple teams, departments, or enterprise business units—surfaces a new set of problems that a single-team pilot never exposed.
Survey data from the “How the World Tests” report (itproportal.com) identified the most common obstacles companies face when scaling Agile testing practices. The findings are grouped below into four recurring themes, each with practical guidance for engineering leaders.
1. Lack of Test Automation: The Biggest Single Barrier
Agile’s defining characteristic is its short iteration cycle. Two- to three-week sprints leave testing teams with roughly three business days to configure and execute test cases, followed by a one- to two-day window to fix defects, retest, and deploy. That leaves almost no margin for the kind of slow, manual regression cycles that were acceptable under waterfall models.
According to the survey, 35% of respondents identified lack of test automation as the single biggest obstacle to Agile adoption. Without automation, teams face a hard choice: miss sprint deadlines, or reduce the number of features shipped per sprint. Neither outcome is acceptable at enterprise scale.
What to do about it
- Audit your current test suite. Identify which regression tests are executed every sprint and are therefore the highest-value automation candidates.
- Start narrow. Automate the highest-frequency, lowest-complexity tests first to build team confidence and demonstrate ROI before tackling complex integration scenarios.
- Treat test code as first-class code. Apply code review, version control, and CI pipeline integration to automated test scripts from day one.
- Measure coverage over time. Track the ratio of automated to manual tests per sprint and set quarterly improvement targets.
2. The Automation Skills Gap
Even when organisations recognise that test automation is necessary, they often lack the in-house skills to implement it effectively. The survey found that 45% of respondents cited a skills gap as the primary reason they had not yet automated their testing—making it the leading root cause behind the barrier described above.
Writing reliable, maintainable automated test scripts requires a mix of software engineering discipline and domain knowledge of the system under test. This skill set is not evenly distributed across QA teams, particularly in organisations that built their QA function around manual testing roles.
Options for closing the gap
- Invest in upskilling existing staff. Structured training in a chosen automation framework is usually faster than recruiting. However, training takes engineers off active projects, so plan capacity accordingly.
- Hire specialists with the target skill set. Once an automation tool is selected, the recruiting pool narrows to candidates experienced with that specific toolchain. Factor this into platform decisions.
- Embed automation engineers in delivery teams. Rather than siloing automation expertise in a centralised QA team, place automation-capable engineers inside each Agile squad so skills transfer organically.
- Avoid adopting a tool without a training plan. Teams handed an automation tool without adequate training will fall back on manual processes or misuse the tool, generating brittle scripts that erode confidence in automation overall.
3. Organisation Size Changes the Problem
The challenges of scaling Agile are not uniform across company sizes. Larger enterprises and smaller organisations face structurally different problems.
Smaller companies and startups
Survey data showed that while 70% of smaller companies follow an Agile methodology, only 30% use automated testing frequently in deployments. Smaller teams often operate under a “shipped is better than perfect” philosophy driven by real constraints: limited headcount, tight budgets, and pressure to get a product to market before the opportunity closes.
The risk here is that technical debt in the testing process compounds as the product grows. Automation frameworks that are not established early become progressively harder and more expensive to retrofit onto a maturing codebase.
Enterprise organisations
Larger organisations have greater resources to invest in automation tooling and training, and survey data reflected a higher rate of automated testing adoption at enterprise scale. However, size introduces coordination complexity: multiple teams working in parallel can produce inconsistent automation standards, duplicated test coverage, and integration conflicts when test suites are merged.
Enterprises scaling Agile across dozens of teams often benefit from a dedicated Platform Engineering or DevOps enablement team that owns shared test infrastructure, CI/CD pipelines, and automation standards—freeing delivery squads to focus on business logic rather than tooling.
4. Choosing the Right Automation Tooling
Platform choice directly constrains automation tool choice. A team building a web application, a mobile application, and a backend API will find that no single tool covers all three equally well. The survey reported that 22% of respondents identified missing toolsets or support deficits as a barrier to automated testing.
This is a compound problem: once a platform is chosen, the available automation tools narrow; once a tool is chosen, the available talent pool narrows further. Poor tooling decisions made early can lock teams into frameworks that are difficult to migrate away from.
Evaluation criteria for automation tools
- Platform coverage. Does the tool support the target technology stack (web, mobile, API, desktop)?
- Community and vendor support. Is the tool actively maintained? Is documentation current? Is there an accessible community for troubleshooting?
- Talent availability. Can you realistically hire or train engineers on this tool in your region and at your budget?
- CI/CD integration. Does the tool integrate cleanly with your existing pipeline (Jenkins, GitHub Actions, GitLab CI, etc.)?
- Reporting and observability. Does the tool produce machine-readable test results that can be surfaced in dashboards and tracked over time?
5. Deciding What to Automate vs. What to Test Manually
Even teams that have the tools and the skills often stall on a deceptively simple question: what should we automate? Without clear decision criteria, teams default to what they know—manual testing—even when automation would save significant time. The survey found that 13% of respondents struggled with defining the boundary between automated and manual testing, creating roadblocks to adoption.
Manual testing is not inherently bad; it is the right approach for exploratory testing, usability evaluation, and genuinely novel functionality where test cases cannot yet be defined with precision. The problem arises when manual testing is applied to repetitive regression scenarios where automation would be faster, cheaper, and more consistent.
A practical framework for the automation decision
- Automate if: the test is executed every sprint, the inputs and expected outputs are deterministic, and the test is time-consuming to run manually.
- Manual-test if: the feature is new and requirements are still evolving, the test requires human judgment (e.g., visual design, UX flows), or the test is executed infrequently enough that automation ROI is negative.
- Document the decision. Record why a given test is manual or automated. This prevents recurring debate and surfaces patterns over time—many “manual for now” decisions can be revisited once the feature stabilises.
The Case for Executive Sponsorship
Scaling Agile across multiple teams is not solely an engineering challenge—it is an organisational one. Each of the barriers described above (automation investment, training budgets, tooling evaluation, headcount) requires decisions and budget authority that sits at or above the engineering director level.
C-level stakeholders who mandate Agile adoption without funding the automation infrastructure and skills development needed to support it will see teams struggling to meet sprint cadences, quietly reverting to waterfall-style delivery cycles with Agile terminology layered on top. This is a well-documented anti-pattern sometimes called “Agile in name only” (AINO).
Successful enterprise Agile transformations typically share a common characteristic: explicit executive commitment to the investment required in tooling, training, and process change—not just to the delivery speed improvements that Agile is expected to produce.
Key Takeaways
- Test automation is the most critical technical enabler for scaling Agile; without it, sprint timelines cannot be consistently met.
- The skills gap is the root cause behind most automation deficits—tool adoption without training investment produces poor outcomes.
- Organisation size shapes the nature of the problem: startups need lightweight, fast-to-implement approaches; enterprises need shared infrastructure and standards.
- Tooling decisions have long-term talent and maintenance implications—evaluate carefully before committing.
- Define explicit criteria for what to automate versus manually test to prevent decision paralysis.
- Scaling Agile requires top-down commitment and budget, not just bottom-up engineering effort.