This article is based on commentary by Antony Edwards, CTO at TestPlant, originally published on devopsonline.co.uk. The analysis below expands on the key arguments for a DevOps engineering audience.
What DevOps Actually Means — and Why It Matters for Testing
“DevOps” is one of the most overloaded terms in the software industry. At its core, DevOps is the organizational and technical practice of aligning development and operations teams so that software can be built, validated, and deployed reliably and quickly. It became essential with the rise of Software-as-a-Service (SaaS), where development teams deliver software to their own operations teams rather than shipping a boxed product to a customer.
Before DevOps, a common deadlock emerged: development teams said the product was ready; operations teams said the environment wasn’t fit for production. Neither side fully owned the outcome, and the customer paid the price. DevOps was conceived to break that deadlock through communication, shared ownership, and — critically — automation.
Today, DevOps has become synonymous with two concrete practices:
- Continuous delivery: Delivering new value to users faster, with development and operations working in lockstep to shrink the gap between a code commit and a production deployment.
- Automation: When releases happened every few months, a manual build-test-package-deploy pipeline was painful but possible. When new versions ship multiple times per day, manual processes are simply not viable. Automation is not a best practice — it is a prerequisite.
Both of these changes place significant pressure on testing teams. The rest of this article examines the specific challenges and how organizations can respond.
Speed Over Service: When Automation Crowds Out User-Centric Testing
When organizations talk about DevOps, speed dominates the conversation. Automation enables hundreds of tests to run in a pipeline that would be impossible to execute manually. But this inward focus on pipeline efficiency can quietly erode attention to user experience.
DevOps teams naturally concentrate on their own processes — build stability, deployment frequency, mean time to recovery — rather than on the problems real users encounter day to day. The result is testing that emphasizes compliance with technical specifications rather than user-centric validation.
A common organizational response is to create a dedicated team immediately downstream of the DevOps pipeline, focused specifically on user testing and customer experience. In digital enterprises such as retail and banking, this team often reports to a Chief Digital Officer rather than a Chief Information Officer, reflecting its outward-facing remit.
Finding the right balance between inward-facing automation and outward-facing user validation typically takes one to two years to establish — but the long-term quality gains justify the investment.
Too Many Cooks: The Risk of Collapsing Distinct Roles
Some organizations interpret DevOps as a mandate to eliminate role boundaries entirely, creating an “everyone does everything” structure. In theory, this sounds appealing — generalists who understand the full software lifecycle. In practice, it creates serious productivity problems.
When team members with deep specialization in one area are assigned responsibility for functions they are not suited to, quality suffers. More subtly, the team as a whole tends to default to the function most members are skilled at. Because DevOps initiatives frequently emerge from Agile development projects, development skills predominate — meaning testing and operations concerns get underserved even when they are nominally everyone’s responsibility.
The antidote is not to pretend specialization doesn’t exist, but to deliberately build genuinely cross-functional teams. A high-performing DevOps team, like a strong sports team, is composed of people with distinct, complementary skills — not a group of developers who have been handed additional responsibilities without the training or aptitude to fulfill them.
Practically, this means:
- Keeping dedicated testers (or QA engineers) as recognized members of the DevOps team rather than treating testing as a shared afterthought.
- Defining clear ownership of quality gates, not diffuse collective responsibility.
- Hiring or training for the skills the team actually lacks, rather than assuming existing members will fill every gap.
Putting DevOps to the Test: Structural Gaps in Testing Coverage
Even when teams have the right people, the testing process itself can be challenged by the pace of DevOps. Organizations often need to test processes or integrations for which no automated tests yet exist. Building those tests takes time — time that competes directly with the delivery cadence DevOps is designed to accelerate.
The tests that do exist in a fast-moving pipeline are frequently rapid smoke tests: quick checks that confirm a build hasn’t catastrophically broken core functionality. Smoke tests are valuable, but they are not a substitute for deeper testing. Twenty minutes of regression checks cannot tell you whether a UI change has confused users, whether an edge-case workflow is broken, or whether a performance regression will affect a subset of customers under real load.
What many organizations are missing is a structured layer of testing that sits alongside — not instead of — their automated pipeline tests. This layer should include:
- Exploratory testing by skilled testers who can identify unexpected failures that scripted tests miss.
- User experience testing that evaluates the product from a real customer’s perspective.
- Performance and load testing that validates behavior under realistic conditions, not just synthetic benchmarks.
Without this broader testing coverage, automated pipelines can deliver broken or confusing software very efficiently.
A Better Future: Letting Testing Flourish Inside DevOps
DevOps is not the enemy of quality — but it does demand that organizations think carefully about how quality is maintained at speed. The core insight is that automation and human judgment are complementary, not interchangeable.
Organizations that get this right tend to share a few characteristics:
- They automate the build, test, and deployment pipeline thoroughly — but they also fund and staff dedicated quality roles rather than assuming automation replaces human testers.
- They treat customer experience as a first-class concern in the DevOps workflow, not a secondary concern left to a separate team that sees the product only after release.
- They maintain clear role definitions within cross-functional teams, so that testing expertise is preserved and applied, not absorbed and diluted.
- They invest in the test infrastructure needed to validate new processes, accepting that the upfront cost of building tests is less expensive than the downstream cost of undiscovered defects in production.
DevOps is still maturing as a discipline, and the terminology around it remains contested. But the underlying challenge for testing teams is concrete: deliver quality at a pace that matches continuous delivery, without sacrificing the depth of validation that users and the business depend on. Getting that balance right is one of the defining engineering challenges of modern software organizations.