This article is based on reporting originally published at searchitoperations.techtarget.com. All quotes are attributed to Chris Judson, VP of Engineering at Choice Hotels, as reported in that source.
The Problem: Homegrown Scripts and Brittle Pipelines
Choice Hotels began its DevOps transition more than four years before this article was written and adopted Jenkins for continuous integration (CI) testing early in that journey. However, the company relied on internally developed scripts to handle the continuous delivery (CD) side of its pipeline — and over time, those scripts became a significant pain point.
“We connected Jenkins CI to custom code in JSON files that executed commands on the infrastructure side via the [AWS] API,” said Chris Judson, VP of Engineering at Choice Hotels, headquartered in Rockville, Md. “This was a brittle process, especially with complex apps, and it led to development hours spent hunting down typos [in scripts].”
This is a common pattern in organizations that grow CI maturity faster than CD maturity. Jenkins handles builds and tests well, but wiring together AWS API calls through hand-crafted JSON is difficult to maintain, audit, and scale across hundreds of applications.
Evaluating the CD Tool Landscape
When Choice Hotels decided to formalize its continuous delivery pipeline, it evaluated several platforms:
- XebiaLabs — an enterprise release orchestration platform
- AWS native tooling — including CodePipeline and CodeDeploy
- Spinnaker — Netflix’s open-source multi-cloud CD platform
- Harness — a SaaS-first CD platform that launched from stealth in late 2017
Choice Hotels was already satisfied with its CI workflow. The existing Jenkins setup integrated well with SonarQube for code quality analysis and Checkmarx for static application security testing (SAST). The priority was therefore to find a CD platform that complemented those tools rather than replacing them.
The team ultimately chose Harness. “Harness appealed to us because it’s offered via SaaS, so we don’t have to babysit the tool and do its care and feeding,” Judson said. “But it’s also focused on the deployment piece, which meant we could keep the Jenkins build jobs we already had in place.”
Why CI and CD Are Different Disciplines
One of the key lessons from Choice Hotels’ journey is that continuous integration and continuous delivery, while often mentioned together, affect different teams and require different skill sets.
- CI is primarily a developer concern: compiling code, running unit and integration tests, and surfacing quality issues early.
- CD is primarily an IT operations concern: automating the infrastructure required to safely deploy, verify, and if necessary roll back application releases.
Treating them as a single monolithic toolchain can obscure this distinction and create friction between development and ops teams. Choice Hotels found that adopting a dedicated CD platform actually sharpened team roles rather than blurring them.
Results: Faster Deployments and Clearer Ownership
The most immediate win came in data center environments. “We’ve seen the biggest improvement in continuous delivery in data centers, where we used scripts on load balancers to segment traffic for canary deployments,” Judson said. “We’re able to deploy much faster there and we’re much more confident in our deployments.”
According to the reporting, for applications where Harness was in use, Choice Hotels was able to deploy application updates approximately 20% more frequently in the year of initial rollout, with that figure rising to more than 30% in the following year.
The platform rollout was still in progress at the time of reporting, spanning more than 1,200 applications running in both AWS and on-premises data centers in Phoenix.
Operational Impact: Reduced Cross-Team Friction
A structured CD pipeline reduced the need for manual coordination between developers and ops engineers — particularly around routine tasks like security patching.
In the previous model, patching Java applications required manual effort from both sides. With Harness, the IT ops team could automate patching independently, freeing developers from infrastructure concerns they shouldn’t need to manage directly.
“The developer and ops side have actually become more focused on their separate roles, and ops has a dashboard to see what’s coming and how deployments are going for themselves.”
Chris Judson, VP of Engineering, Choice Hotels
Automated rollbacks were also a benefit. The Harness platform allowed the IT ops team to revert failed deployments without manual intervention — an important capability when maintaining a large, mixed-environment estate.
Remaining Challenges and Areas for Improvement
The rollout was not without complexity. At the time of reporting, Choice Hotels was still working through integrations between Harness and two key parts of its infrastructure:
- Kafka data pipelines — ensuring that deployment automation does not trigger unnecessary infrastructure duplication or leave orphaned resources.
- Terraform — coordinating infrastructure-as-code provisioning with CD-driven deployments so that rollbacks cleanly reverse infrastructure changes as well as application changes.
These are non-trivial integration problems. When a CD platform orchestrates deployments that provision cloud infrastructure via Terraform, rollback logic must account for stateful infrastructure — not just application artifacts. Getting this right requires careful design of Terraform state management and rollback strategies.
Feature Gap: Predictive vs. Retrospective Cost Analysis
Harness includes cloud cost analysis features that let teams review the efficiency of their deployments after the fact. Judson noted that retrospective data is useful but expressed interest in predictive cost analysis — the ability to forecast infrastructure spend based on historical deployment patterns before a deployment is triggered.
A Harness spokesperson, as reported in the original article, declined to share roadmap specifics but indicated the feature was under consideration at the time.
Key Takeaways for DevOps Teams
- CI and CD solve different problems. Don’t assume a single tool or team should own both. Evaluate them separately.
- Homegrown scripts don’t scale. JSON-driven API orchestration works in early stages but becomes brittle and expensive to maintain as application counts grow.
- A SaaS CD platform reduces operational overhead. Teams avoid the burden of maintaining the tooling itself, which is a real cost in engineering time.
- Structured CD clarifies team responsibilities. Rather than blending dev and ops work together, a well-designed pipeline can give each team clearer ownership and better visibility.
- Terraform and CD rollbacks require careful integration. Stateful infrastructure changes add complexity to automated rollback — plan this integration explicitly.