This article summarizes findings from the OverOps State of Software Quality report, as reported by HelpNetSecurity. Survey data is attributed to OverOps and is based on responses from over 600 software development and delivery professionals.
The Problem: DevOps Adoption Isn’t Eliminating Production Failures
Organizations are investing heavily in DevOps tooling, CI/CD pipelines, and delivery automation — yet customer-impacting software errors remain stubbornly common. According to the OverOps State of Software Quality report, based on a survey of more than 600 software development and delivery professionals, the current level of DevOps investment is not sufficient to guarantee software reliability at the pace modern teams are expected to ship.
The core tension is clear: teams are moving faster than ever, but speed and quality are proving difficult to achieve simultaneously — even with substantial tooling investment.
“The move to DevOps and the increasing velocity at which teams are delivering software is unprecedented. At the same time, they are dealing with much higher levels of risk and vulnerability.”
Herb Krasner, Advisory Board Member, Consortium for IT Software Quality (CISQ)
Key Survey Findings
1. Quality Is the Priority — But Errors Still Happen
The survey found that 70% of respondents say quality is more important than speed, stating they would rather delay the product roadmap than risk a critical error reaching users. However, prioritizing quality in principle does not translate directly into fewer incidents in practice.
53% of respondents reported encountering critical or customer-impacting issues in production at least once a month. This figure cuts across teams regardless of whether they self-identify as speed-focused or quality-focused — suggesting that the problem is systemic, not a matter of individual team philosophy.
2. Customers Are Finding Bugs Before Internal Teams Are
One of the more concerning findings: a quarter of respondents said that over 40% of critical production issues are first reported by end users or customers, rather than being caught by internal monitoring, alerting, or QA mechanisms. This points to a significant gap in observability and production monitoring coverage — even in teams with mature CI/CD pipelines.
3. Delivery Velocity Is Accelerating
Teams are shipping faster than ever. Among the top areas of DevOps investment reported by survey respondents:
- Continuous Integration (CI): 54% of respondents cited it as a primary investment area
- Continuous Delivery (CD): 42% of respondents
Meanwhile, 59% of respondents said they release new code or features anywhere from bi-weekly to multiple times per day. The speed of delivery is increasing, but the tools and practices being used to maintain quality at that speed are not yet keeping pace.
4. Pressure to Move Fast Undermines Quality
45% of respondents identified “pressure to move fast” as one of their top software quality challenges. This is a structural problem: when release cadence is driven by business or competitive pressure rather than readiness, quality controls can be bypassed or compressed — regardless of the tools in place.
5. Developer Productivity Is Being Eroded by Debugging
Frequent production errors are not just a customer experience issue — they have a direct cost in engineering time. The survey found that two out of three respondents spend at least one full day per week troubleshooting code-related issues. More strikingly, 30% of respondents spend between two days and a full work week on troubleshooting each week.
This is significant lost capacity. Time spent debugging production incidents is time not spent building new features, paying down technical debt, or improving reliability proactively.
What Teams Are Planning to Do About It
The survey data suggests that engineering organizations are beginning to look beyond CI/CD pipeline automation as the primary lever for quality improvement. The top planned investments in software quality tooling at the time of the survey included:
- DevOps tooling broadly: 58%
- Microservices and containers: 45%
- Static code analysis: 37%
- Dynamic code analysis: 27%
The planned adoption of static and dynamic code analysis is particularly notable. These tools operate at different stages of the development lifecycle:
- Static analysis examines source code without executing it, catching potential bugs, security vulnerabilities, and code quality issues before deployment.
- Dynamic analysis monitors application behavior during runtime, enabling teams to catch errors and anomalies in staging or production environments that static analysis cannot detect.
Together, these approaches address different blind spots in the software delivery lifecycle and can help close the gap that currently allows so many issues to reach end users undetected.
What This Means for DevOps Teams
The survey results illustrate a broader industry challenge: DevOps adoption has successfully accelerated delivery, but quality engineering practices have not scaled proportionally. Several structural issues are worth addressing:
- Observability gaps: If customers are reporting more than 40% of critical issues before internal systems detect them, production monitoring and alerting coverage needs significant improvement.
- Shift-left testing: Investing in static analysis and other pre-deployment quality gates reduces the number of defects that reach production in the first place.
- Incident cost awareness: When developers are spending one to five days per week on debugging, teams should quantify this overhead and treat it as a business case for quality tooling investment.
- Release pressure governance: Organizational pressure to ship fast without corresponding investment in reliability engineering is a root cause, not just a symptom.
As Herb Krasner of CISQ noted, the industry has reached a turning point. Faster delivery without proportionally better quality controls creates compounding risk — and the data suggests that many teams are already feeling that impact in production.
Source: HelpNetSecurity, reporting on the OverOps State of Software Quality report.