High-Profile Incidents That Put DevOps Security in the Spotlight

In late September 2019, Avast disclosed that it had detected an intrusion into its internal network via a temporary VPN profile that lacked two-factor authentication. Security researchers believed the attackers’ goal was to plant malware inside CCleaner—a widely used system-cleanup utility—repeating a technique that had succeeded in 2017, when cybercriminals successfully trojanized CCleaner and distributed it to millions of users. The 2019 attempt was detected and disrupted before any official build was compromised.

Earlier in April 2019, Docker disclosed a breach of a single Docker Hub database that exposed sensitive information from approximately 190,000 accounts. Investigators suspected the attackers may have attempted to implant malware in official Docker images used to instantiate containers. Docker confirmed that no official images were ultimately compromised, but the incident underscored how a single point of failure in a container registry can threaten countless downstream environments.

These are examples of software supply chain attacks—a category in which adversaries compromise widely used software or infrastructure upstream so that the malware propagates automatically to every organization that installs or pulls from that source. The technique is attractive precisely because a single successful intrusion can reach millions of targets simultaneously.

Why DevOps Environments Are Prime Targets

1. Rapid Cloud Adoption Creates Sprawl

DevOps pipelines are increasingly running on public cloud platforms. Keeping pace with the dynamic nature of cloud environments—constantly changing components, configurations, permissions, and services—places an enormous operational burden on teams that are already moving fast. The resulting configuration drift can leave critical gaps, including:

• Accounts without multi-factor authentication (MFA/2FA) enabled
• Object storage buckets misconfigured with public read or write access
• Container API endpoints exposed without access controls
• Serverless function invocation URLs left unprotected
• Over-permissioned IAM roles and service accounts

2. Modern Software Is a Web of Third-Party Components

Software development today rarely means writing every line from scratch. Production applications routinely depend on dozens or hundreds of open-source packages, commercial SDKs, base container images, and CI/CD plugins. Each dependency is a potential entry point. Compromising any single component in this ecosystem—a popular npm package, a PyPI library, a Docker base image—can give an attacker an immediate foothold across every project that consumes it.

What DevOps Teams Can Do to Defend Against These Attacks

Preventive controls such as MFA, firewalls, web application firewalls (WAF), anti-malware tools, and secure email gateways are necessary but no longer sufficient on their own. Sophisticated supply chain attackers routinely bypass these layers. A robust strategy must combine prevention with continuous monitoring and rapid incident response.

Monitor DevOps Pipelines and Cloud Infrastructure

Early detection is the single most important factor in limiting blast radius. Every meaningful activity in the DevOps workflow—code pushes, build triggers, artifact publishes, and deployments—should be logged and profiled against a baseline of normal behavior. Anomalies to investigate include:

• Access from unexpected geographic regions or IP ranges
• Logins outside normal working hours
• Unusual changes to build configurations or pipeline definitions
• Artifact publishes that do not correspond to a recognized commit or build job

Platform-level event streams—for example, Azure DevOps audit logs, GitHub audit logs, or GitLab’s audit events—can feed a SIEM or monitoring platform to provide this visibility without requiring agents on individual build hosts. Underlying cloud infrastructure (virtual machines, storage accounts, management consoles, PaaS services) should be monitored in the same way, since infrastructure-layer compromises are frequently used as a stepping stone to reach the pipeline itself.

Monitor Production Container Environments

Container security requires visibility at multiple layers:

• Intra-container events: Process execution, file-system writes, and outbound network connections from within a running container can reveal compromised processes or malware payloads. Achieving this visibility typically requires embedding lightweight monitoring agents or eBPF-based probes as part of the golden image build.
• Container-to-container communication: Profiling expected network flows between containers—using data from the orchestration layer (Kubernetes network policies, service mesh telemetry)—makes it possible to detect rogue containers establishing unexpected connections.
• Image provenance: Enforce image signing and verify digests at deploy time so that only images from trusted registries and with verified signatures can run in production.

Continuously Assess Cloud Configuration

A point-in-time audit is not enough in a cloud-native environment where configuration changes happen constantly. Automated cloud security posture management (CSPM) tools can continuously benchmark running configurations across virtual machines, storage, databases, management consoles, container clusters, and identity policies. The CIS Benchmarks are a widely adopted, vendor-neutral baseline for this kind of assessment and are available for major cloud providers, operating systems, and Kubernetes.

Scan Every Software Component for Vulnerabilities and Malware

Dependency scanning and software composition analysis (SCA) should be mandatory gates in the CI/CD pipeline, not optional afterthoughts. Effective component security includes:

• Scanning open-source dependencies for known CVEs before they are merged or promoted
• Scanning container images for vulnerable OS packages and application libraries
• Analyzing first-party and third-party code for suspicious patterns or embedded malware
• Generating and maintaining a Software Bill of Materials (SBOM) so you can respond quickly when a new vulnerability is disclosed in a component you use

The Bigger Picture

Cybercrime syndicates and state-sponsored threat actors have recognized that compromising software at its source—inside the DevOps pipeline—delivers a much higher return than attacking individual organizations one by one. A single implant in a widely distributed package or container image can propagate to thousands of organizations before anyone raises an alarm.

Defending against this threat model requires treating security as a first-class concern throughout the software delivery lifecycle: enforcing least-privilege access, enabling MFA everywhere, scanning every artifact, monitoring every pipeline event, and being prepared to respond the moment something anomalous appears. The organizations that invest in this discipline now will be far better positioned to detect and contain the supply chain attacks that are certain to continue.

TIMWOOD maps seven categories of waste onto team and individual activities. Some are strategic in nature and require cross-team collaboration to resolve; others are tactical and can be addressed within a single feature team. Used alongside Kanban boards or value stream maps, the framework gives DevOps teams a concrete starting point for reducing friction and accelerating delivery.

The Seven Types of Waste in DevOps

The seven TIMWOOD waste types, as applied to DevOps, are:

• Transfer — handoffs between teams or individuals (strategic)
• Inventory — backlog of partially completed or shelved work (strategic)
• Motion — task switching, interruptions, and context changes within a process (tactical)
• Waiting — delays caused by approvals, decisions, or environment readiness (strategic)
• Over-Processing — unnecessary effort, including re-learning undocumented code (tactical)
• Over-Production — building features nobody will use (strategic)
• Defects/Rework — quality issues that require code to be revisited and fixed (tactical)

Tactical wastes should be resolved within the feature team. Strategic wastes are more systemic and demand cross-functional collaboration. Here is a detailed look at each waste type and how to address it in practice.

1. Transfer — Bad Handoffs Between Teams

Transfer waste occurs when too many dependencies exist between individuals or teams — for example, requiring sign-off from a separate team before work can proceed. Each handoff introduces a potential delay and a point of communication failure.

How to address it:

• Cross-functional pairing: Have developers pair with testers to sign off work in real time rather than as a sequential gate. This removes the waiting period between handoffs.
• Continuous Integration (CI): Build a unified CI pipeline that covers multiple gating concerns — code quality, security scanning, test coverage — so software moves between team members through automation rather than manual hand-to-hand transfers.
• Break down silos: Where possible, create teams that are end-to-end capable of delivering a feature without requiring external approval at every step.

2. Inventory — Unfinished Work Piling Up

Inventory waste includes items that were added to the backlog and never touched (stale documentation is a classic example), features that were partially started and then abandoned, or work that was shelved due to shifting priorities. Large backlogs create cognitive overhead and make it harder to see what actually matters.

How to address it:

• Team leaders should agree on a clearly prioritised, short list of features to deliver next.
• Visualise that list on a Kanban board and enforce a work-in-progress (WIP) limit: only add new items once the current ones are shipped.
• Regularly prune the backlog to remove items that no longer align with product goals, rather than letting them accumulate indefinitely.

3. Motion — Context Switching and Interruptions

Motion waste is the cost of switching between tasks, responsibilities, or tools mid-stream. It also includes time lost navigating remote or inaccessible resources. Every context switch carries a cognitive cost that compounds across a team.

How to address it:

• Reducing transfer and inventory waste naturally reduces motion waste, since clearer priorities and fewer handoffs mean fewer reasons to switch tasks.
• Invest in a well-integrated toolchain so that developers, testers, and operations engineers share a common environment rather than toggling between disconnected systems.
• Protect focus time: team agreements such as no-meeting blocks or async-first communication norms can significantly reduce interruptions.

4. Waiting — Delays in Approvals and Environments

Waiting waste is perhaps the most universally recognised: sitting idle while waiting for a product decision, an environment to be provisioned, or an answer from another team. It is often the largest source of lead time bloat in software delivery.

How to address it:

• Build in slack: Explicitly schedule buffer time for known delays rather than treating them as exceptions. This keeps the team moving rather than blocked.
• Start complex tasks earlier: Tasks that are known sources of waiting — environment provisioning, third-party integrations, security reviews — should be initiated as early as possible in the iteration.
• Adopt Infrastructure as Code (IaC): Tools that codify environment configuration allow teams to spin up consistent, on-demand environments automatically, eliminating the manual work of configuring networks, virtual machines, and dependencies from scratch each time.

5. Over-Processing — Excess Effort That Adds No Value

Over-processing waste means investing time and effort in work that is not actually required to deliver value. Common examples include building elaborate solutions for simple problems, writing speculative code, or — perhaps most painfully — asking new team members to reverse-engineer undocumented legacy code before they can contribute.

How to address it:

• Use well-written user stories with clear acceptance criteria to keep developers focused on features that deliver concrete value.
• Timebox feature implementation to prevent gold-plating and scope creep.
• Write code with future maintainability in mind: good naming, modular design, and up-to-date documentation reduce the ramp-up cost for the next person who works on the codebase.

6. Over-Production — Building What Nobody Needs

Over-production occurs when teams build features or entire products that no one will use. This is a strategic waste because it misallocates developer time and creates long-term maintenance overhead for code that delivers no business value.

How to address it:

• Ensure every story in the sprint backlog traces back to a validated epic, initiative, or business outcome — if it doesn’t, question whether it should be built at all.
• Conduct frequent scope reviews within the feature team to catch deviation early, before it becomes a delivered-but-unused feature.
• Engage product owners and stakeholders regularly so that the team builds based on current needs, not assumptions from three months ago.

7. Defects and Rework — The Cost of Poor Quality

Defect waste is the time and effort consumed fixing bugs that escaped earlier in the pipeline. The further a defect travels from its origin before it is caught, the more expensive it becomes to fix. Rework disrupts flow, erodes team morale, and delays delivery.

How to address it:

• Implement fast feedback loops through CI pipelines that run automated tests on every commit.
• Use a layered testing strategy that combines unit tests, functional tests, integration tests, and end-to-end tests to catch different categories of defects at the right level.
• Shift testing left: involve testers during story refinement and development, not just at the end of the sprint, so quality is built in rather than inspected in.

Identifying Your Own DevOps Slowdowns

Not every team will encounter all seven waste types with equal severity. Some organisations will relate strongly to transfer and waiting waste; others will find over-processing or defects to be their primary drag. Some teams will also identify organisation-specific sub-categories of waste that don’t map neatly onto the standard seven.

What is universal is this: scaling and maturing a DevOps practice requires a clear understanding of where waste lives in your delivery pipeline and what it costs — in engineering time, operational overhead, and customer experience. Visualising your workflow through a Kanban board or a value stream map is the most direct route to making that waste visible.

Once the waste is visible, it can be prioritised, assigned, and systematically eliminated. As with any engineering problem, accurately diagnosing what you are dealing with is always at least half the solution.

Key Takeaways

• The lean TIMWOOD framework — Transfer, Inventory, Motion, Waiting, Over-Processing, Over-Production, Defects — applies directly to DevOps workflows.
• Strategic wastes (Transfer, Inventory, Waiting, Over-Production) require cross-team solutions; tactical wastes (Motion, Over-Processing, Defects) can often be resolved within the feature team.
• Kanban boards and value stream maps are the primary tools for making waste visible.
• CI pipelines, IaC, cross-functional pairing, WIP limits, and clear user stories are the most practical levers for reducing waste across the seven categories.
• There is no universal playbook: identify which waste types affect your team most and prioritise accordingly.

The following five-step framework, originally discussed in a webinar hosted by DBTA and featuring Russ Savage (product manager, InfluxData) and Chris Churilo (director of product marketing, InfluxData), provides a practical path to getting control of a DevOps monitoring environment.

Why DevOps Monitoring Is Harder Than It Looks

Even teams that have invested in monitoring tools commonly face the same recurring problems:

• Tooling limitations that prevent collecting all relevant signals
• Manual capacity planning that cannot keep pace with dynamic workloads
• Long lead times for instrumenting new services or metrics
• An overwhelming number of competing monitoring solutions with overlapping scope

The solution is not to buy yet another tool—it is to establish a repeatable process that starts with what you are collecting today and builds toward proactive, data-driven decision-making.

Step 1: Collect the Data

Comprehensive data collection is the foundation. Before you can monitor anything meaningfully, you need to know what data you have, why you are collecting it, and what gaps exist.

• Inventory what you are collecting. List every data source: server metrics, application logs, container stats, network telemetry, and business-level signals. Gaps at this stage propagate into every subsequent step.
• Understand the goal behind each metric. Collecting CPU utilization is only useful if you know which decisions it informs. Align every collected metric to a business or operational question.
• Decide how to collect. Options include reading from known file locations, deploying specialized agents alongside workloads, or using APIs. You also need to choose between push-based collection (the monitored system sends data) and pull-based collection (a central collector scrapes endpoints). Each approach has trade-offs in scalability, security, and operational overhead.
• Identify what is missing. After the inventory, audit for blind spots—new microservices not yet instrumented, third-party dependencies, or ephemeral container workloads that existing agents never capture.

Step 2: Correlate and Triage

Raw data from many sources rarely tells a coherent story on its own. Correlation turns isolated signals into actionable information.

• Handle data dimensionality. Metrics arrive at different frequencies (every second vs. every five minutes), cover different time frames, and come from heterogeneous sources. Your storage and query layer must handle this without forcing you to normalise everything to a common resolution before storage.
• Establish baselines for availability. Before you can detect an anomaly, you need to know what normal looks like. Start by capturing steady-state behaviour for critical services so that deviations are obvious and measurable.
• Identify remaining gaps. As you begin correlating data, new blind spots typically emerge—for example, you may have application metrics but no corresponding infrastructure metrics for the hosts running that application.

Step 3: Identify Trends

Once baselines exist, you can look forward rather than only backward.

• Forecast from historical data. Use time-series data to project future resource consumption—disk fill rates, memory growth, request volume growth. Forecasting gives capacity planning a quantitative basis instead of relying on guesswork or annual reviews.
• Establish alerting thresholds. Thresholds derived from observed trends are far more accurate than static, hand-tuned values. A threshold set at 80% CPU is arbitrary; a threshold set at two standard deviations above the rolling weekly average is grounded in reality.
• Manage inventory and safety margins. Know your safety levels—how much headroom do you have before a resource constraint causes a service degradation? Define these margins explicitly so that on-call engineers have clear action triggers rather than ambiguous dashboards.

Steps 4 and 5: Act and Iterate

The original webinar summarised three core steps, but a complete monitoring maturity model requires two further stages that naturally follow trend identification.

• Step 4 — Act on the data. Insights are only valuable if they drive decisions. This means closing the loop between monitoring data and operational actions: automated remediation for known failure patterns, escalation workflows for anomalies that require human intervention, and capacity orders triggered by forecast thresholds.
• Step 5 — Continuously refine. DevOps environments change constantly. New services are deployed, traffic patterns shift, and infrastructure evolves. Treat your monitoring strategy as a living system: revisit your metric inventory, update baselines after major changes, and retire metrics that no longer serve a purpose.

Tooling Considerations: Time-Series Databases

Several of these steps—particularly correlation, trend analysis, and forecasting—are significantly easier when metrics are stored in a purpose-built time-series database. Time-series databases are optimised for high-throughput writes of timestamped data, efficient range queries, and automatic data retention policies that age out high-resolution data after a configurable period.

When evaluating a time-series platform for a DevOps monitoring stack, look for:

• High write throughput to handle burst instrumentation during incidents
• Flexible retention policies so that raw, high-resolution data can be downsampled and archived rather than deleted outright
• Time-aware query semantics (relative time ranges, aggregations over windows) that map naturally to monitoring questions
• Horizontal scalability to grow with the number of monitored services
• Integration with common collection agents and visualisation tools already in your stack

Summary

Gaining control of a DevOps environment is a process, not a product purchase. Start by understanding what you are collecting and why. Build baselines through correlation, then use historical trends to forecast and set meaningful thresholds. Finally, close the loop by acting on insights and continuously improving your instrumentation as the environment evolves. Teams that follow a structured approach move from reactive fire-fighting to proactive, evidence-based operations.

Enterprises are embracing the need to transform digitally, and they are facing challenges in delivering high-quality solutions in shorter cycle times. OpsHub enables enterprises to create a unified DevOps environment within their existing infrastructure by integrating HPE Application Lifecycle Management (ALM), Application Lifecycle Management Octane (ALM Octane) and Quality Center (QC) solutions to other ALM, DevOps, CRM, and ITSM systems. As a result, organizations can increase delivery speed and quality by enhancing collaboration between sales, service and development teams – while ensuring availability of information with context.

In a DevOps ecosystem created by OpsHub Integration Manager (OIM), all cross-functional teams work with unified sets of facts, towards delivering a holistic experience to the customer. This ecosystem also supports end-to-end automation during all stages of development and operations. Additional features such as support for automatic recovery and reconciliation, proactive conflict management, implicit traceability, and historical tracking introduced in the ecosystem by OIM make the ecosystem future-ready, robust, and reliable for cross-functional collaboration.

“A unified DevOps ecosystem created by integrating HPE ALM, ALM Octane and QCE with other third party systems not only accelerates productivity, but also boosts business agility. This enables enterprises to deliver quality solutions and phenomenal customer experiences leading to delighted customers in the digital age,” said Sandeep Jain, CEO of OpsHub Inc.

“An expanding set of tools and technologies creates complex organizational challenges for enterprises shifting to DevOps,” said Vandan Nayak, Senior Product Manager for ALM Octane, Hewlett Packard Enterprise. “Leveraging tools like HPE ALM, ALM Octane and Quality Center, in conjunction with OpsHub Integration Manager, to streamline enterprise systems can improve cross-team collaboration – and ultimately help organizations rapidly deliver high-quality, high-performing applications.”

About OpsHub

OpsHub is the leading provider of integration and migration solutions for the ALM, CRM, and DevOps tool chains. OpsHub improves the efficiency and effectiveness of agile teams in ALM and DevOps environments by making necessary and current data available to each user, in that user’s preferred system, with full context, in real-time. OpsHub’s solutions for integration and migration speed up development processes, reduce errors, improve decision-making, and result in delivering innovative products and services faster, with higher quality at lower cost.

What Is Accessibility (AX) Testing?

Accessibility testing — commonly abbreviated as AX testing — is a specialised subset of usability testing. While general usability testing focuses on the ease of use for the average user, AX testing explicitly includes people with disabilities: visual impairments, hearing loss, motor difficulties, and cognitive challenges. The goal is to verify that an application is genuinely usable by the broadest possible audience.

Two major standards frameworks govern what “accessible” means in practice:

• WCAG (Web Content Accessibility Guidelines) — Published by the W3C, WCAG provides a tiered set of success criteria (Levels A, AA, and AAA) for making web content perceivable, operable, understandable, and robust.
• Section 508 of the Rehabilitation Act (US) — Requires US federal agencies and contractors to make their Information and Communication Technology (ICT) accessible. The 2017 refresh of Section 508 aligned its technical standards closely with WCAG 2.0 Level AA.

Other relevant regulations include the Americans with Disabilities Act (ADA) and, for organisations operating in the EU, the European Accessibility Act.

Why Accessibility Testing Belongs in DevOps

DevOps pipelines are built around speed and continuous delivery. Accessibility cannot be an afterthought bolted on before a release — by that point, remediating deeply embedded issues is expensive and disruptive. The same “shift-left” principle that applies to security and performance testing applies equally to AX: find defects as early as possible, when the cost to fix them is lowest.

Practically, this means:

• Defining the accessibility scope and acceptance criteria during the design phase, not during QA.
• Selecting AX tools that integrate with your existing CI/CD toolchain early in the project.
• Including automated AX checks as a dedicated stage in the delivery pipeline so violations are surfaced before code reaches the test team.

The Limits of AX Test Automation

Automated accessibility testing is powerful, but it cannot replace human judgement entirely. Not every WCAG success criterion can be evaluated programmatically. For example, determining whether an image’s alternative text is meaningful, or whether a page’s reading order actually makes sense, requires human review.

A practical rule of thumb: automated tools can reliably flag a subset of WCAG checkpoints — estimates in the field commonly cite figures in the range of 25–30% of total checkpoints — and even those flagged results benefit from manual verification to reduce false positives. For CI pipeline integration, a more conservative approach is to automate only checks that produce fully deterministic, objective results (for example: missing alt attributes, absent form labels, insufficient colour-contrast ratios). Covering around 10% of WCAG checkpoints reliably and objectively through automation is considered a solid baseline for a continuous integration suite.

Manual testing with assistive technologies — screen readers, switch access devices, and keyboard-only navigation — must complement automated checks to achieve meaningful accessibility coverage.

Integrating AX Test Automation into the CI/CD Pipeline

Adding automated AX checks to a DevOps pipeline follows the same pattern as any other automated test type:

1. Define pass/fail criteria. Agree upfront on which WCAG level (typically AA) the application must meet, and decide which violation categories will block a build versus generate a warning.
2. Instrument your existing UI tests. Many AX tools integrate with Selenium or Playwright, so accessibility audits can run alongside your existing functional test suite without requiring a separate browser session.
3. Add a dedicated accessibility stage. Run AX scans as a pipeline stage after the functional test stage. Capture screenshots and structured reports as build artefacts so developers can inspect violations in context.
4. Track violations over time. Integrate scan results with your issue tracker or dashboard so regressions are visible and ownership is clear.
5. Complement with manual review sprints. Schedule periodic manual accessibility reviews (using screen readers and real assistive devices) to catch what automation misses.

Common AX Testing Tools for Developers

A range of tools is available to support different stages of accessibility testing, from early developer checks to full pipeline integration:

Browser-Based Developer Tools

• Web developer toolbars — Browser extensions that let developers disable CSS to inspect the logical reading order of a page. The unstyled view approximates what a screen reader will encounter.
• WAVE (Web Accessibility Evaluation Tool) — A browser extension (available for Chrome and Firefox) that overlays accessibility information directly on the rendered page, highlighting errors, alerts, and structural elements. Useful for quick developer feedback during development.

Colour and Visual Simulation Tools

• Color Oracle — A desktop tool that simulates how the screen appears to users with various forms of colour-blindness, helping designers verify that colour is not the only means of conveying information.

Screen Readers

• JAWS (Windows) and VoiceOver (macOS/iOS) are the most widely used desktop screen readers. Testing with actual screen readers is the most reliable way to validate the assistive technology experience.
• Mobile accessibility options (TalkBack on Android, VoiceOver on iOS) should also be considered for applications with a mobile audience.

Industry-Standard AX Automation Tools

ATF-WAVE

An automated testing framework integration that combines WebDriver-based UI automation with the WAVE accessibility engine. The typical workflow is:

1. Navigate the application under test using WebDriver.
2. Capture the rendered HTML of each unique page.
3. Pass that HTML through the WAVE engine to analyse the DOM for accessibility errors.
4. Take a screenshot and fail the test if WAVE reports violations above the defined threshold.

This approach runs as a dedicated accessibility stage in the CI pipeline, keeping AX results separate from — but integrated with — functional test results.

IBM Digital Content Checker

A tool designed for verifying the accessibility of HTML content and EPUB documents. Key capabilities include:

• Upload and scan HTML or EPUB content for accessibility violations.
• Produces detailed reports that list violations and recommend specific remediation steps.
• Can be embedded in internal content development and approval workflows so accessibility is reviewed before publication.

IBM Automated Accessibility Tester

Integrates accessibility auditing directly into the Selenium testing framework, adding AX checkpoints that execute during the same test run as functional tests. Violations are reported alongside functional failures, enabling developers to remediate accessibility issues during the development sprint rather than after deployment.

Conclusion

Accessibility is no longer optional. Regulatory requirements, expanding user expectations, and an increased focus on inclusive design mean that AX compliance must be built into the software delivery process — not treated as a final checkpoint before release.

Integrating automated AX testing into a DevOps pipeline delivers tangible benefits: violations are caught earlier, remediation costs are lower, and releases carry less compliance risk. Automation handles the objective, deterministic checks at speed; human testers with assistive technologies cover the qualitative judgements that machines cannot reliably make. Used together, these two approaches give engineering teams the best practical coverage of accessibility standards throughout continuous delivery.

From “Do More With Less” to “Hire 1,000 Developers”

For years, the dominant philosophy in enterprise IT was austerity: squeeze more output from fewer resources. Express Scripts, a large pharmaceutical benefits management company, flipped that model on its head. After adopting DevOps practices across its engineering organisation, leadership approved the hiring of 1,000 new developers—a striking vote of confidence in the business value that a well-executed DevOps transformation can generate.

Brian Gregory, Director of IT at Express Scripts, shared the company’s journey at a technology event, offering candid lessons on tooling decisions, cultural change, and how DevOps shifted IT from a cost centre to a recognised value driver.

Context: A Company Built on M&A

Express Scripts grew significantly through mergers and acquisitions beginning in the 1980s. That history left the company with a fragmented landscape of disparate processes, legacy systems, and technology stacks acquired from different organisations over decades. DevOps became the mechanism for unifying those systems—and for accelerating the delivery of value across all of them.

Building the DevOps Toolchain

Gregory’s team didn’t start from scratch. They already had the ability to spin up and provision virtual machines quickly, which gave them a foundation to build on. From there, they moved up the stack, assembling a toolchain around several well-known open-source and commercial products:

• Pivotal Cloud Foundry — chosen as the core platform for application hosting and runtime management.
• Jenkins — used as the primary automation server to orchestrate CI/CD pipelines.
• Concourse — an additional pipeline automation tool (its maintainers describe it as a “continuous thing-doer”) used alongside Jenkins for orchestration.
• Artifactory — used to automatically download, cache, and compile code dependencies, bringing consistency and speed to build processes.
• Git — used as the source control repository, providing a single source of truth for all codebases.

Together, these tools enabled the team to implement continuous integration and continuous deployment (CI/CD) pipelines that dramatically reduced the number of manual steps between a developer committing code and that code running in production.

Buy vs. Build: A Critical Decision

One of Gregory’s most direct pieces of advice for teams embarking on a DevOps journey concerns the temptation to build internal tooling from scratch. His position is pragmatic:

“If I told you that you’ve got to figure out a way to get to work, what would you do? Would you build a car? You could, but if you could buy a car, then you could get to work faster and you could start delivering value.”

— Brian Gregory, Director of IT, Express Scripts

The principle isn’t that custom tooling is always wrong. Rather, teams should ask a pointed question before investing engineering time in building internal tools: Does this tool create differentiated business value that justifies the investment? If an off-the-shelf or open-source solution covers the use case adequately, the better ROI usually comes from using it and redirecting engineering effort toward the organisation’s actual product.

The Business Outcome: 1,000 Developer Hires

Once the toolchain was in place and CI/CD pipelines were running, the results became visible to leadership. Deployment steps from development to production were reduced significantly, and both developers and operations staff reported increased productivity. IT began to demonstrate that it could drive business outcomes—not just keep the lights on.

That shift in perception had a concrete consequence: company leadership approved the hiring of 1,000 new developers. To appreciate the scale, Express Scripts employs roughly 30,000 people in total, spanning healthcare professionals, pharmacists, and administrative staff. Adding 1,000 engineers is a substantial commitment. Gregory noted the magnitude himself:

“I started two and a half years ago, and I would’ve told you that would never have happened then—I would have said we would have been lucky to get 100. But it’s really taken off.”

— Brian Gregory, Director of IT, Express Scripts

DevOps Extends Beyond the Dev-Ops Boundary

Gregory was clear that the benefits of DevOps at Express Scripts weren’t limited to bridging the traditional gap between development and operations teams. The feedback loops and collaboration practices extended outward to business partners as well. That broader reach created new obligations for the IT organisation:

• IT teams must develop a genuine understanding of business goals and partner expectations—not just technical requirements.
• All technical teams must remain open to feedback and willing to adjust course when something isn’t working.
• Failure should be treated as a learning mechanism, not a reason to retreat to safer, slower practices.

“You’re not going to hit the mark on day one, nobody does that. It’s what you learn when you fail or you learn in that process that’s so important. That’s where the real value is.”

— Brian Gregory, Director of IT, Express Scripts

People and Culture: The Hard Part of DevOps

Technology choices matter, but Gregory was emphatic that the harder challenge in any DevOps transformation is cultural. The shift requires a fundamentally different mindset—one that not everyone will adopt immediately or enthusiastically. His recommendations for managing this:

• Find champions who embrace risk. DevOps adoption needs people who aren’t paralysed by the possibility of failure. Calculated risk-taking and experimentation are core to the practice.
• Develop leaders who empower, not gatekeep. Command-and-control management styles conflict with the autonomy that high-performing DevOps teams require.
• Bring people along at their own pace. Not everyone will understand the mission immediately. The goal is to keep the overall organisation moving in the same direction, even if individuals are at different stages of the journey.

Key Takeaways for DevOps Practitioners

Express Scripts’ story offers several actionable lessons for engineering teams and IT leaders considering or mid-way through a DevOps transformation:

1. Start with what you have. The team leveraged existing VM provisioning capability before layering on more sophisticated tooling. Build on existing strengths rather than waiting for a perfect starting point.
2. Choose a coherent, integrated toolchain. Pivotal Cloud Foundry, Jenkins, Concourse, Artifactory, and Git work together as a system. Evaluate tools for how well they integrate, not just their individual feature lists.
3. Default to buying before building. Reserve custom engineering effort for problems that genuinely differentiate your product or organisation.
4. Measure outcomes that matter to leadership. Reduced deployment steps and increased developer velocity are meaningful, but translating those into business impact is what earns investment.
5. Treat culture as a long-term project. The mindset shift required by DevOps is real and takes time. Plan for it explicitly rather than assuming the tooling will carry the transformation on its own.

The headline outcome—1,000 developer hires—is a consequence of demonstrating value, not a starting condition. Express Scripts earned that investment by showing, incrementally, what a well-structured DevOps practice could accomplish.

The Most Common Misconception About DevOps

When organisations first explore moving to a DevOps model, a near-universal fear surfaces: “If a developer checks in a single line of code, is it going straight to production?” That fear is understandable, but it rests on a false premise. DevOps does not mandate that every commit triggers a live deployment.

The terms continuous integration (CI) and continuous deployment (CD) are often treated as binary switches — either you have them or you don’t — but in practice they are dials. Your team decides how far to turn each one based on risk appetite, tooling maturity, and the pace of change in your applications.

Defining “Continuous”: It Is Not the Same for Every Team

“Continuous deployment” means moving completed developer changes into a production environment where end users can work with them. The word deploy has a fixed meaning; the word continuous does not. Your organisation owns the definition of continuous that fits your context.

Automated deployment pipelines consume real resources — build agents, test infrastructure, environment capacity, and engineering attention. As a purely practical matter, teams have to decide how often it makes sense to marshal those resources. Common deployment cadences include:

• On every commit — maximum feedback speed, lowest increment size per release
• Nightly — balances feedback with infrastructure cost
• Weekly — common in organisations still building automation confidence
• Monthly — acceptable for applications with a very low rate of change

The Increment of Functionality Problem

The longer the interval between deployments, the more changes accumulate in any single release. This accumulated set of changes — sometimes called the increment of functionality — is directly correlated with deployment risk. A release containing three small changes is far easier to validate, roll back, and diagnose than one containing three months of accumulated work. This is one of the strongest structural arguments for shorter deployment cycles, even when that feels counterintuitive.

“Potentially Deployable” Does Not Mean “Must Deploy”

A nuance often lost in CI/CD discussions: the goal of a continuous deployment pipeline is to produce a potentially deployable artifact — not necessarily to deploy it immediately. Your pipeline can build, test, and validate a package against a production-like environment on every cycle without ever pushing it live until a human or a release gate approves it.

This approach — sometimes distinguished as continuous delivery versus continuous deployment — gives organisations the safety net of frequent validation without the perceived disruption of frequent live releases. The pipeline proves the code could go to production; a separate decision determines when it does.

The Hidden Cost of Infrequent Deployments

Choosing a longer deployment interval to reduce disruption trades one risk for another. The most significant hidden costs are:

• Delayed user feedback. Real production feedback only arrives after an actual deployment. The longer the gap, the longer your team is building without knowing whether the previous release met user needs.
• Expensive rework. When a developer receives feedback on a feature shipped weeks or months ago, context has been lost. Fixing it takes longer and risks destabilising newer work built on top of the original change.
• Compounding integration risk. As features pile up between releases, isolating the cause of a production issue becomes progressively harder.

A common pattern in organisations adopting DevOps is to start with monthly or weekly deployments to build confidence in their automation, and then gradually shift toward shorter cycles as the team realises the risks of large increments outweigh the perceived safety of infrequent releases.

How Integrated Should Your Integration Be?

Continuous integration means merging developers’ changes into a shared mainline frequently, rather than leaving branches in isolation for days or weeks. The fundamental goal is that all developers are working from the same codebase at all times, preventing the “integration hell” that occurs when long-lived branches must eventually converge.

Historically, the minimum bar for CI was simply that the application could still compile after a merge (the “don’t break the build” rule popularised by the Windows NT team, among others). Modern CI pipelines go much further, typically including:

• Automated unit tests run by the developer before or immediately after commit
• Static code analysis and linting
• Integration tests that exercise the merged application as a whole
• Contract tests between services in a microservices architecture
• Performance and stress tests (often deferred to a separate stage due to resource cost)

Developers can — and should — write automated unit tests that verify their own code in isolation. But bugs that only surface when components interact can only be caught during integration testing, after code is merged. This distinction matters when deciding where to place which tests in your pipeline.

Calibrating Your Integration Testing Strategy

The ideal is to run all integration tests against a full duplicate of the production environment on every commit. In practice, cost and environment complexity force most teams to make trade-offs. Two variables determine where tests end up in your pipeline:

• How easy it is to keep the integration environment in sync with production. Environment drift is a hidden tax on test reliability. If your integration environment frequently diverges from production, test results become less meaningful.
• How resource-intensive the tests are. Functional correctness tests are usually fast and cheap. Stress tests, user-acceptance tests, and full end-to-end tests can be slow and expensive to run on every commit.

A practical approach many teams use is a tiered test strategy:

1. On every commit: unit tests and fast integration smoke tests
2. On every build (nightly or per-branch merge): full functional integration tests
3. On release candidates: performance, stress, and user-acceptance tests

The important principle is that deferring a test increases the cost of fixing any defect it catches, because the developer must re-establish context from work done further in the past. As your pipeline becomes more efficient, the economics shift: it becomes cheaper to run more tests earlier, and teams naturally pull tests further left in the process.

The Natural Evolution of a CI/CD Programme

Organisations rarely reach an optimal CI/CD cadence on day one. A realistic progression typically looks like this:

1. Establish the pipeline. Get automated builds and basic tests running. Deploy infrequently (weekly or monthly) to build confidence in automation.
2. Increase deployment frequency. As confidence grows, the team notices that large increments of functionality are themselves a source of risk, and begins deploying more often.
3. Move tests left. As the pipeline matures, expensive tests that were deferred to the deployment stage are moved into the integration stage, or even into per-commit validation.
4. Reach a sustainable balance. The team finds a cadence that minimises overall risk — covering both deployment disruption and the risk of delayed feedback — and continues to adjust as the application and organisation evolve.

Key Takeaways

• CI and CD are spectrums, not binary states. Your team decides where on each spectrum to operate.
• Longer deployment intervals reduce disruption frequency but increase per-release risk and delay user feedback.
• “Potentially deployable” is a valid outcome of a CD pipeline — actual deployment can be gated by humans or automated release policies.
• The right integration testing strategy is determined by environment parity costs and test resource costs, not by ideology.
• As pipeline maturity increases, the economics of shifting tests earlier improve. Treat your CI/CD posture as something that evolves continuously alongside your team’s capabilities.

DevOps does not impose a one-size-fits-all deployment rhythm on your organisation. It gives you the tooling and practices to make deployment safe enough to do more frequently — and then leaves the decision about how frequently firmly in your hands.

DevOps Has Moved Past the Hype Cycle

By the mid-2010s, DevOps had firmly established itself as a mainstream enterprise discipline rather than an emerging trend. Survey data from that period illustrated rapid adoption: the 2016 RightScale State of the Cloud survey reported that 74% of 1,060 IT professionals had adopted DevOps in their organizations, up from 66% in 2015. A separate Vanson Bourne report commissioned by Appvance found that 73% of 200 IT decision-makers had adopted some DevOps processes.

The compensation story was equally striking. Puppet’s 2016 DevOps Salary Report found that 58% of practitioners earned more than $100,000 annually—up from 47% the prior year—while 43% of managers exceeded $150,000, compared with just 26% in 2015.

This growth created intense demand for DevOps engineers and raised an important strategic question: how should an organization actually staff a DevOps transformation?

The Core Insight: It’s Not Build vs. Buy—It’s Build and Buy

A common instinct among CIOs is to treat DevOps adoption as a recruiting problem: find experienced DevOps engineers, hire them, and the transformation follows. IT staffing experts and analysts push back on this framing. You can’t build a DevOps environment simply by purchasing talent. Equally, you can’t retitle existing staff as “DevOps engineers” and expect the culture and practices to appear on their own.

James Stanger, senior director of products at CompTIA, framed it directly: “People who just want that trained DevOps person are kind of barking up the wrong tree. What they need are people who are in operations or development who have a willingness to learn about each other.”

The practical answer is a deliberate combination: hire externally for specific hard-to-build technical skills, then use those new hires to accelerate the upskilling of existing staff who bring irreplaceable organizational knowledge.

What DevOps Actually Requires

DevOps is a methodology—roughly a decade old at the time of the original reporting—in which software development and operations teams collaborate closely throughout the entire software lifecycle: building, testing, and releasing software frequently and in small increments. The goals are faster delivery, higher quality, and better security. DevOps depends on both cultural change and specific enabling technologies.

An organization adopting DevOps therefore needs two distinct types of capability:

• Collaborative mindset and cross-functional fluency: Developers who understand operations constraints, and operations engineers who understand the development lifecycle.
• Specific technical skills: Proficiency with the tools and platforms that make continuous delivery and infrastructure automation possible.

Where to Buy: Technical Skills That Are Hard to Build Quickly

Connor Leech, a senior technical recruiter at tech-staffing agency Mondo, identified the technical areas where external hiring typically makes the most sense. Organizations moving to DevOps commonly need people who can:

• Automate infrastructure provisioning and application deployment pipelines
• Design systems that automatically scale in response to demand fluctuations
• Work fluently with major cloud providers such as AWS

The specific tools and technologies that map to these needs include:

• Configuration management and automation: Puppet, Chef
• Application performance monitoring: New Relic
• Scripting and general-purpose programming languages: Python, Ruby, PHP
• Containerization: Docker (container runtime for packaging and deploying applications consistently across environments)

The reason external hiring makes sense here is pragmatic. These tools are complex, and hiring managers consistently want candidates who have used them in large production environments—not just experimented with them in sandboxes. Training existing staff to production-level proficiency in multiple platforms simultaneously is slow and risky when a transformation is underway.

“It’s definitely possible to retrain existing engineers in these areas, but because the tools are kind of complex, [CIOs] want people who have used them rather than have to teach people on them. It’s a common theme with hiring managers: They want someone who has not only played around with these, but have used them in a large production environment.”

— Connor Leech, Senior Technical Recruiter, Mondo

Using Contract Engineers as a Force Multiplier

One effective model is bringing in experienced DevOps engineers on short-term contracts—typically three to six months. These contractors come in, establish automation pipelines, define standards, and actively upskill the existing team. They are expensive, but they function as a force multiplier: when they leave, the permanent team is meaningfully more capable than it was before.

“What’s really common is bringing in DevOps engineers as contractors. You bring in people on contract for six months, and they come in, help automate and ramp everyone up and then go and do another contract. They’re generally pretty expensive, but they’re incredibly valuable. They can turn a company’s existing engineers into DevOps engineers.”

— Connor Leech, Mondo

Where to Build: Mindset and Organizational Knowledge

Technical skills are necessary but not sufficient. DevOps depends on a fundamentally collaborative way of working—the deliberate dismantling of the wall between development and operations. This requires a different kind of capability that is much harder to hire for and much easier to cultivate in existing staff.

Stanger identified the key characteristics of good internal DevOps candidates:

• Curiosity about the other side of the dev/ops divide, even if their primary expertise leans one way
• Familiarity with project management disciplines to keep iterative delivery cycles on track
• Comfort working within agile and Scrum frameworks, which are common structures in DevOps environments
• Willingness to share responsibilities that were previously siloed

Critically, Stanger argued that many existing IT professionals already possess the raw material for this. Developers have often had some exposure to operations concerns, and operations engineers frequently have development experience. The gap is not as wide as it might appear—what’s often missing is not knowledge but permission and encouragement to apply it.

“Many of the people who have been working in programming for years, if they have the right mentality, are already DevOps people. Anyone could easily be turned into that DevOps person. Most people have the seeds of DevOps in them, but the seeds haven’t been nurtured.”

— James Stanger, CompTIA

Why Existing Staff Are Not Optional

Jay Lyman, a principal analyst covering DevOps at 451 Research, reinforced this dual approach from an analyst’s perspective. Organizations need an infusion of new skills from external hires, but they also depend on the organizational context and institutional knowledge that existing staff carry. No external hire will understand the business’s specific systems, constraints, and history as well as a long-tenured internal engineer.

There is also a practical alignment benefit. IT professionals who understand that DevOps skills increase their market value are generally motivated to participate in training and upskilling programs. This makes change management easier—staff become advocates for the transformation rather than passive recipients of it.

The Leadership Imperative: Understand and Sell DevOps

Staffing strategy is only one part of the equation. CIOs and engineering leaders must themselves develop a genuine understanding of DevOps—not just at the conceptual level, but operationally. They need to be able to articulate what DevOps delivers for the business (speed, quality, reduced risk) and why it requires changes to how teams are structured and how work is measured.

Not all staff will embrace the transition. Some IT professionals are more comfortable in traditional, more sequential delivery models. This is not necessarily a problem—some projects and contexts do suit more structured, phased approaches. But organizations moving fully toward continuous delivery may eventually face a mismatch between those delivery expectations and staff who resist the required way of working. In those cases, external hiring may eventually be necessary not just to add skills but to replace resistance.

Summary: A Practical Staffing Framework

The organizations that successfully form a DevOps environment are those that resist the false choice between hiring and training. They hire deliberately and specifically for the hardest-to-build technical skills, use those new hires to accelerate team capability, and invest simultaneously in developing the collaborative mindset and cross-functional fluency that no job posting can deliver.