Methodology builds cyber considerations into DOD programs from the beginning.

The Department of Defense is rethinking how it approaches software and systems development in its technology programs by using more flexible methods to streamline the process and to improve cybersecurity from the start.

Because traditional DOD program development processes don’t have the speed and flexibility to keep up with rapid technological changes or fast-paced modern adversaries, new methodologies are being considered. One approach gaining traction in many parts of the DOD is Development, Security and Operations, or DevSecOps.

At its simplest, the goal of DevSecOps is to make everyone in a program accountable for security with the goal of implementing security decisions and actions at the same scale and speed as development and operations decisions and actions. DevSecOps differs from both the traditional Waterfall project delivery method approach to DOD programs, which focuses on product delivery, time frames, and coordinating those activities, and the modern Agile development methodologies.

DevSecOps Versus Agile Development

Agile development methodologies that emphasize iterative development cycles and feedback to find and correct errors throughout the software building process have become more common in federal government technology programs. “DevSecOps often gets confused as either another name for Agile development or an offshoot of it in the DOD community,” says Derek Strausbaugh, chief digital officer for Microsoft’s National Security Business.

“While DevSecOps does build on some Agile development principles, such as the continuous integration and delivery of software systems in cycles, its key emphasis from the beginning of the process is to integrate security features: Agile only focuses on delivering software,” Strausbaugh explains.

“You’re really taking security and putting it on the same level as continuous integration and delivery. So it’s not just Agile development, it’s actually creating another leg of the stool for quality from the get go, and integrating your developers, security and your operations functions into the software development lifecycle,” he says.

In a standard Waterfall or Agile program scenario, one team of developers builds an application and then hands it off to another group to install security or other features. DevSecOps shifts left security accountabilities, allowing programs to operate more efficiently and to create more organizational awareness when problems do show up.

“DevSecOps also helps with software and technology development pipelines. This is important because it is possible for programmers to build code and deliver products without understanding the infrastructure underpinning it,” says Zach Kramer, Microsoft’s Azure Government engineering lead. He adds, “This can make it hard to understand a project’s full security footprint, which is where Microsoft Azure and other cloud providers can help provide developers with a full view of the situation.”

“This approach not only helps with code and the underlying software infrastructure, but provides programmers with a standard set of telemetry and a top-to-bottom understanding of the platform they’re working on,” Kramer says. Another advantage of DevSecOps is that it shifts the program manager’s perspective from making sure that software is in compliance or meets a specification or audit to ensuring that the code is written correctly and securely and that it’s deployed in a repeatable manner. This is important because it ensures that a project can provide continuous software delivery at scale.

“If I’m not sure my application is still secure between build one and build two, then I have no chance of doing continuous delivery,“ Strausbaugh explains. “With the right DevSecOps practices in place, I ensure that continuous integration checks take place for the right security areas in order to achieve continuous delivery.”

DOD Applications and Changing Culture

DevSecOps fits into the DOD’s modernization strategy to upgrade legacy systems and incorporate new capabilities such as machine learning or artificial intelligence into its mission. “From a military perspective, it is about increasing velocity—speeding up decision making and operational effectiveness rather than simply delivering software,” says Strausbaugh.

The core aspects of DevSecOps such as providing predictable software delivery and ensuring that that software is secure from cyber attack fit into the military’s view of future multidomain missions. “This is important because if a modern command and control system can be compromised by a cybersecurity vulnerability, that liability can undermine an entire military operation,” Strausbaugh says.

“At the bottom line, DevSecOps is about predictability and speed of delivery while ensuring security. While Agile development quickly produces code that can be deployed to warfighter applications with a repeatable frequency, it runs up against security and operational considerations such as round-the-clock operations and cyber defense,” Kramer says.

“As a methodology, there are a variety of tools and development container framework products organizations can use to help with DevSecOps applications. But it is more about the methodology and approach to building a trusted execution environment that is secure and scalable, without having to be locked into any kind of single source provider,” says Strausbaugh.

Organizations previously mitigated risks through complicated layers of approvals and checks, which slowed delivery in an attempt to reduce risks. These processes still did not eliminate risks but gave a level of comfort to those who approve applications for production. Strausbaugh maintains that while DevSecOps may seem uncomfortable, putting the right practice in place enables a better security posture and faster innovation and ability to respond quicker when the inevitable problem arises.

Organizations also need to evolve certain aspects of their internal cultures to get a DevSecOps program off the ground because the methodology creates a new way to look at a system’s stakeholders.

Strausbaugh notes that Agile development involves the owners of the application or system being built into the process. DevSecOps continues this evolution by bringing “the type of folks you absolutely have to have at the table, and including all the skills you need to have.”

DevSecOps requires teamwork because the goal is creating something to support warfighters and that requires both operational and security responsibilities. The methodology also requires a change in some of the developer skills sets because they need to do much more than write good code.

“It requires developers who are well-rounded and have an eye on writing secure code and code that doesn’t put an operational burden on the organization,” Strausbaugh explains.

People working in this environment also need to accept change as a constant for software development, delivery and sustainment. This requires a cultural shift to create a greater sense of individual accountability, which makes the entire end-to-end process more critical to the overall outcome, Kramer says.

Automation is Key

Automation is another important aspect of the DevSecOps process, especially when deploying software at scale across large DOD agencies or entire services. But these large-scale, fast-paced operations often have problems caused by human error that usually occurs in repetitive, maintenance and security assessment functions. The best way to avoid such common mistakes is to automate those processes.

“If I can get a set of functions to be repeatable and known, I reduce error,” Kramer says. “That means my deployments should not be a human following a script. They should be automated with infrastructure as code. If I’m updating it, it should be done with the same infrastructure-as-code that did my deployment.”

“This same approach applies to security. Instead of requiring a person to go through checklists of hundreds of controls, this is done automatically as part of the software development/supply process pipeline. That transition to automating these basic processes is at the heart of DevSecOps,” Kramer says.

“Automation also frees up personnel who would be doing these manual tasks for other, more challenging and creative things. This also helps prevent employee burnout because the automated processed built into DevSecOps helps manage rote maintenance and security operations and it allows for operational rollbacks if a problem is found so that instead of having developers work overnight to solve a problem, it can be identified and dealt with the following day,” Kramer says.

Growing Use Across DOD

DevSecOps is beginning to make inroads into the DOD. Strausbaugh sees two areas where this is happening. The first is in large scale software “factory” type development programs now underway in technology hubs across the military for developing systems to support warfighters.

The second area embracing DevSecOps is in programs where a major challenge for sustaining the project’s software systems turned out to be monolithic design processes that don’t deliver enough software iterations or updates with the frequency to catch major security vulnerabilities. These systems are also too slow to keep ahead of new capabilities developed by potential U.S. adversaries.

As result of global competitive pressure, some of the DOD’s largest platforms and intelligence, surveillance and reconnaissance (ISR) systems are considering DevSecOps as a way to redesign or move away from software systems developed decades ago.

Another plus is a new generation of younger military and civilian personnel in the DOD who are comfortable with technology and using it in new ways. Strausbaugh notes that there has been a sea change in the DOD in terms of moving to and using cloud-based services and also the adoption of modern software development techniques at many of the military’s tech development centers.

“They’re building microservices architecture and consuming cloud services and looking at commercial as a goal post for the way they should be doing things rather than boxing themselves into the misnomer that the DOD has to do things a certain way just because of history or security,” Strausbaugh says.

Strausbaugh sees the move toward development strategies like DevSecOps accelerating throughout the military. This is reflected by the fact that the DOD is actively working with its industry partners to provide support and to make commercial tools and techniques available because the services are seeing the benefits of their use, he says.

The following five best practices will help you build a dependable Docker container backup strategy that protects both infrastructure state and application data.

1. Track Version Numbers Across All Components

Docker requires that UCP, DTR, and the Docker Engine all run matching versions. This version alignment is critical at restore time: if you restore a DTR backup that was created against an older cluster version, the restored component will be out of sync with the rest of the cluster and the restoration will fail or cause unpredictable behavior.

Best practice: record the exact version of every component at the time each backup is taken. Store this metadata alongside the backup artifact — in a database record, a sidecar file, or your backup management tool’s notes field. Before initiating any restore, verify that the backup’s recorded versions match the current cluster state, or that you are prepared to restore all components together to achieve a consistent version set.

2. Back Up a Single Manager Node — But Track Which One

In a Docker Swarm cluster, every manager node holds an identical copy of Swarm and UCP state. You do not need to back up all manager nodes — backing up one is sufficient to capture the full control-plane state.

However, you must record which specific node you backed up, including its IP address. When restoring, the backup must be applied to a node with the same IP address as the source node. Restoring to a node with a different IP will break the Swarm membership records embedded in the backup, causing the restore to fail.

• Choose a consistent manager node as your designated backup source.
• Log the node’s hostname, IP address, and Swarm node ID alongside every backup.
• Automate this logging in your backup scripts to eliminate human error.

3. Plan for Manager Unavailability During Backups

Running a Swarm or UCP backup pauses the UCP manager node. While the backup is in progress, the UCP user interface will display a warning, and the paused manager is temporarily unavailable for scheduling decisions. In a cluster with only one or two managers, this pause can disrupt cluster operations.

Docker recommends provisioning clusters with five manager nodes. With five managers, a Raft quorum (three nodes) is maintained even if one manager is paused for backup purposes, keeping the cluster fully operational. Schedule backups during low-traffic windows and alert your team before initiating a backup so they are not alarmed by the UCP warning banner.

4. Back Up Each Component Separately

Docker does not provide a single unified backup that covers all components simultaneously. You must run separate backups for:

• Docker Swarm — stores orchestration state, service definitions, secrets, and configs.
• Universal Control Plane (UCP) — stores user accounts, role-based access control (RBAC) policies, and cluster configuration.
• Docker Trusted Registry (DTR) — stores image metadata, access policies, and vulnerability scan results. Image layers stored in an external object store (e.g., S3) must be backed up separately through that store.

Critically, none of these component backups capture application-level data stored in container volumes. Stateful application data — databases, file uploads, configuration files written at runtime — must be backed up independently using volume-aware tools or application-consistent snapshot methods appropriate to the workload.

5. Treat Node-Level Restore as a Last Resort

When a node fails in a highly available cluster, the instinct may be to restore from a node backup. Docker’s guidance is the opposite: restoring a node backup should be an absolute last resort. The preferred recovery path for a failed node is to remove the unhealthy node from the cluster and replace it with a fresh node that re-joins the Swarm.

Replacing rather than restoring a failed node is faster, avoids potential version or state inconsistencies introduced by a node backup, and leverages Swarm’s built-in ability to reschedule workloads onto healthy nodes. A node-level backup restore is only appropriate in catastrophic scenarios where the entire cluster state has been lost and no other recovery option exists.

Summary: Docker Container Backup Checklist

• ✅ Record UCP, DTR, and Engine version numbers with every backup.
• ✅ Back up one manager node and log its IP address and node ID.
• ✅ Provision at least five managers to maintain quorum during backup pauses.
• ✅ Run separate backup jobs for Swarm, UCP, and DTR.
• ✅ Back up application volume data independently of component backups.
• ✅ Prefer node replacement over node restore in HA clusters.

Implementing these practices reduces your risk of failed restores, version conflicts, and unnecessary cluster downtime. Regularly test your restore procedures in a staging environment — a backup that has never been tested is not a reliable backup.

Key Takeaways

• Continuous Delivery (CD) is now a standard practice in modern software development, but systematically measuring it remains uncommon.
• Continuous Delivery Indicators, introduced by Steve Smith in Measuring Continuous Delivery, quantify CD in terms of stability and speed.
• Adopting CD Indicators encourages teams to view their deployment pipeline as a value stream to be actively optimized.
• CD Indicators provide a process optimization framework with clear stability and speed goals and visible progress tracking.
• Effective rollout requires a near-real-time tooling layer that collects raw data and computes indicators automatically and transparently.
• Treating indicator data as process signals—not people KPIs—is essential to maintaining data integrity.

Introduction: The Case for Data-Driven Development

Software delivery organizations increasingly ship complex systems at high frequency. The three core activities involved are Product Management, Development, and Operations—understood here as activities rather than siloed departments. Each activity demands rapid, well-informed decisions: Product Management decides which features to prioritize; Development focuses on process efficiency; Operations focuses on reliability.

Decisions can be made based on team experience alone, or they can be grounded in data. A data-driven approach produces more objective, transparent outcomes—particularly important as delivery velocity increases and the number of teams grows. Transparency through data reduces the need for time-consuming synchronization meetings and helps align everyone continuously.

This article focuses on how Development activities can be supported by Continuous Delivery Indicators, and how that data enables rapid, defensible prioritization decisions. It draws on the real-world implementation at Siemens Healthineers’ teamplay platform, a globally distributed organization comprising 16 software delivery teams across three countries.

Process Indicators, Not People KPIs

Before introducing any measurement framework, organizations must establish a clear principle: the data collected represents process signals, not evaluations of individual contributors. When metrics are used to assess people, those people are incentivized to game the numbers, destroying data quality and trust.

Leadership must explicitly set this expectation. When teams understand that CD Indicators reveal where the process is struggling—not who is failing—they engage honestly with the data and use it to improve rather than to defend themselves.

What Are Continuous Delivery Indicators?

Once a team knows what to build, the central development question becomes: how do we build it efficiently? The answer lies in analysing the team’s value stream: Code → Build → Deploy. This value stream is visible on the team’s deployment pipeline.

Continuous Delivery Indicators, as defined by Steve Smith in Measuring Continuous Delivery, quantify two dimensions of value-stream health:

• Stability – how reliably value flows through the pipeline without failures.
• Speed – how quickly value flows from commit to deployment.

The Speed definition here extends beyond simple lead time. Steve Smith defines speed as a combination of lead time for a change and the interval between changes—an extension of the lead-time-only framing found in Accelerate by Nicole Forsgren, Jez Humble, and Gene Kim.

All indicators are designed so that lower values signal better performance, making graphs immediately readable: values trending close to the X-axis indicate healthy flow; high values indicate bottlenecks.

Stability Indicators

Speed (Throughput) Indicators

Visualizing these indicators over time gives teams an at-a-glance view of their value stream. Aligned with the Theory of Constraints, teams can identify their largest bottleneck and systematically invest effort to eliminate it before moving to the next constraint.

Reading the Graphs: Bottlenecks vs. Healthy Flow

When CD Indicator graphs are plotted over time:

• Values asymptotic to the X-axis → healthy behaviour, low friction in the value stream.
• Persistently high values → a bottleneck requiring attention.
• Upward trend over time → a worsening problem that should trigger investigation before it reaches crisis level.

This visual model means developers and product owners can engage with pipeline health without needing deep expertise in statistical analysis.

How CD Indicators Change Prioritization Behaviour

Without CD Indicators, teams tend to invest in development process improvements only after major outages or based on the loudest voice in the room. With CD Indicators, investment decisions become evidence-based:

• Test Automation – targeted where build failure rates are high.
• Deployment Automation – applied where deployment failure rates or lead times are elevated.
• TDD / BDD Process Improvement – directed at stages where commit lead times are long.

The three-way prioritization trade-off that product owners must navigate becomes explicit:

• Invest in new features to improve product effectiveness.
• Invest in development efficiency based on CD Indicator data.
• Invest in service reliability to reduce operational risk.

CD Indicators make the development efficiency dimension visible and quantifiable, so it can take its rightful place alongside feature and reliability priorities.

Tooling: Making Indicators Actionable

At Siemens Healthineers, the team developed an in-house proprietary tool to compute and visualize CD Indicators for all 16 development teams. Key design decisions included:

• Data source: Azure DevOps services—Azure Repos for code data, Azure Pipelines for build and deployment data.
• History retention: Default Azure DevOps data retention (30 days) was extended to 365 days to enable meaningful trend analysis.
• Layout: Value Stream Stability on the left; Value Stream Speed on the right, mirroring the indicator framework.
• Automated bottleneck detection: The tool identifies the largest stability and speed bottlenecks and highlights them in red, allowing teams to act without needing to interpret raw data tables.
• Automated suggestions: Contextual guidance for resolving detected bottlenecks is surfaced directly in the tool.
• Slack integration (planned): Monthly value stream snapshots delivered to team Slack channels, with deep links back to the tool for further analysis.

The near-real-time feedback loop the tool provides is essential: a team working heads-down on features has no natural opportunity to reflect on whether their deployment pipeline is slowing them down. The tool surfaces that awareness automatically.

Adoption Experience at Siemens Healthineers

The CD Indicators were rolled out across 16 teams working on teamplay, a global digital healthcare service. Introducing value stream thinking required deliberate change management. Most development teams do not habitually think of their deployment pipeline as an assembly line with measurable throughput and stability—the concept needed explanation and workshops to land effectively.

Teams provided substantial feedback during rollout that shaped the tool’s data collection logic:

• Count only Master Branch builds, not Pull Request builds, for throughput metrics.
• Track Pull Request lead times separately to identify review wait-time bottlenecks.
• Report Deployment Lead Time and Deployment Interval both in aggregate across all environments and broken out per environment.

A common early reaction was positive surprise: teams had never before seen their own value stream visualized in terms of stability and speed. The data revealed dynamics that were invisible during day-to-day development.

Real-World Example: Deployment Stability Improvement

One team’s Deployment Stability Indicator revealed a high Deployment Failure Rate alongside fast Deployment Failure Recovery Times. The combination indicated frequent, easily fixed failures—likely from a fixable configuration or scripting issue rather than fundamental instability. Once the team saw the data, they resolved the root cause within a few days, dramatically improving deployment reliability. The key enabler was simply awareness: the fix itself was straightforward once the problem was visible.

Real-World Example: Interpreting Trend Data

In another team’s drilldown, the Deployment Failure Rate trend showed a steady increase from September onward, reaching nearly 100% recently. Simultaneously, Deployment Failure Recovery Time spiked. This meant the pipeline was staying red almost continuously—a critical bottleneck blocking value from reaching production environments.

In contrast, the same team’s Master Branch Commit Lead Time and Build Interval trends showed recent improvement. The appropriate response: prioritize the deployment stability problem (the biggest constraint), while maintaining the process discipline that was already improving code and build throughput.

Future Directions

Two extensions to the CD Indicator framework are under exploration:

• Predictive analytics: Using machine learning to predict stability and speed in downstream pipeline environments based on upstream indicator data—potentially enabling earlier intervention before failures propagate.
• Automated deployment gates: Automatically blocking deployments into certain pipeline environments when CD Indicator thresholds are breached, enforcing quality standards without manual intervention.

Summary

Continuous Delivery Indicators give development teams a structured, data-driven lens for continuously improving how they build software. By quantifying value stream stability and speed, teams can move away from gut-feel and post-incident reactions toward deliberate, evidence-based investment in process improvement.

Critically, the approach depoliticizes development prioritization. When bottlenecks are visible in data rather than surfaced through anecdote or influence, the conversation shifts from opinion to evidence. This supports healthier team dynamics, clearer trade-off decisions, and ultimately better business outcomes—higher user engagement and more reliable revenue-generating software.

Acknowledgements

The CD Indicators Tool described in this article was implemented with instrumental contributions from Kiran Kumar Gollapelly, Krishna Chaithanya Pomar, and Bhadri Narayanan ARR, with initial funding from Frances Paulisch. Thanks to the entire teamplay team at Siemens Healthineers for introducing and adopting these methods.

This article is part of the Data-Driven Decision Making for Software Product Delivery Organizations Series, which covers data-driven approaches across Product Management, Development, and Operations.