Originally announced May 2020. Source: finance.yahoo.com / OverOps press release.
Overview
In May 2020, OverOps announced two related milestones: joining the CloudBees Technical Alliance Partner Program (TAPP) as a Premier member, and releasing significant enhancements to its Jenkins plugin. Together, these developments were aimed at helping CloudBees and Jenkins users enforce code quality gates inside their CI/CD pipelines and prevent unreliable builds from reaching production.
Why This Partnership Matters for CI/CD Teams
As release velocity increases, so does the risk of shipping defects. CI/CD platforms like CloudBees CI (formerly CloudBees Core) accelerate the path from commit to production, but speed alone does not guarantee reliability. Static analysis, unit tests, and log grepping have historically been the primary quality signals inside a pipeline — each with meaningful blind spots when it comes to runtime behavior.
OverOps positioned its Continuous Reliability (CR) approach as a complement to Continuous Integration and Continuous Delivery. The core idea is that quality gates informed by actual runtime error data — not just pre-production test results — provide a more accurate signal for whether a build is safe to promote.
“Continuous Reliability complements CloudBees offerings around continuous integration and continuous delivery. OverOps provides seamless integration with code quality gates for identifying critical runtime errors in new releases.”
— Francois Dechery, Chief Strategy Officer and Co-founder, CloudBees
How the OverOps Jenkins Plugin Works
The OverOps plugin integrates directly into a Jenkins or CloudBees CI pipeline. At a high level, it works as follows:
- Runtime code analysis: OverOps analyzes application code at runtime — during staging or testing phases — to surface errors that static analysis and unit tests may not catch.
- Error classification: The plugin categorizes errors as new, increasing in frequency, resurfaced after a previous fix, or flagged as critical. This classification helps teams prioritize what actually needs attention before a release ships.
- Quality gate evaluation: Based on the error report, configurable quality gates determine whether the build passes or fails. If the gates fail, the pipeline is blocked automatically, preventing promotion to production.
- Developer routing: Issues are surfaced with enough contextual information — including the exact line of code and the variable state at the time of the error — to route the defect directly to the developer responsible, reducing back-and-forth between Dev and QA.
What Was New in the Enhanced Plugin
The 2020 update to the OverOps Jenkins plugin focused on improving the usability of its quality reporting interface:
- High-level summary view: A new at-a-glance snapshot shows which quality gates passed or failed without requiring users to dig through detailed logs first.
- Drill-down from summary: Engineers can click directly from the summary into specific error details and event analysis, reducing the number of context switches needed to diagnose a failing gate.
- Streamlined UX: The updated interface was designed to reduce the time between a failed gate and a developer understanding which error caused it.
The Case for Runtime Quality Gates in CI Pipelines
Traditional CI quality signals have well-understood limitations:
- Static analysis catches code-pattern issues but cannot observe behavior under real data and load conditions.
- Unit and integration tests validate known scenarios; they miss edge cases and runtime interactions that only appear under realistic usage.
- Log analysis is reactive and depends on developers having logged the right information in advance.
Runtime-informed quality gates add a layer of observability that is grounded in what actually happens when code runs, rather than what developers anticipated when writing tests. Blocking a promotion based on a spike in NullPointerExceptions or an uncaught exception in a critical code path is more actionable than a coverage percentage.
“Without feedback loops and insight into critical runtime errors, it’s impossible for engineering teams to validate the quality of their build and prevent poor quality code from reaching production.”
— Eric Mizell, VP of Solution Engineering, OverOps
CloudBees Technical Alliance Partner Program
The CloudBees Technical Alliance Partner Program (TAPP) is designed to certify and promote third-party integrations that work alongside CloudBees CI and the broader CloudBees Software Delivery Management platform. Premier membership — the tier OverOps joined — indicates a deeper level of technical integration and joint go-to-market commitment. For Jenkins users, this kind of alliance typically means the plugin is actively maintained and tested against current CloudBees CI releases, reducing the risk of compatibility issues as the platform evolves.
About OverOps
At the time of this announcement, OverOps was a continuous reliability platform backed by Lightspeed Venture Partners and Menlo Ventures, with offices in San Francisco, Orlando, and Tel Aviv. The platform was designed to give engineering teams runtime visibility into when, where, and why code breaks — both in pre-production CI pipelines and in live production environments — through deep code-level instrumentation rather than log-based or metric-based monitoring alone.
Note: OverOps rebranded to Digma in later years. Readers researching this product should verify current product availability and branding.