DevOps: Common mistakes and how to avoid them
Across organisations of all sizes, software is driving real business value as it has transformed processes, broken down legacy systems and unleashed the wave of app-driven business models. The software is changing the business landscape, levelling the playing field for businesses of all sizes to compete and innovate through software development.
To stay competitive, market-leading businesses are committed to producing software in the right way, adopting the principles of continuous integration, continuous delivery and DevOps. While continuous integration involves automation, builds and quick changes, continuous delivery focuses on moving code through the pipeline. DevOps ties the two principles together and is focused on changing business culture to support and promote collaboration between departments.
Indeed, all organisations want to succeed and reach their development goals across projects. In reality, however, this journey is rarely direct and it often takes time and a collective effort. The role of DevOps is to embrace failure as part of this journey and ensure that if you fail, you fail fast and learn from your mistakes. From this, you can leverage the newly gained insights to improve your process going forward.
However, if there are ways to avoid some of these mistakes, then perhaps many businesses would be able to improve and learn faster. Considering this DevOps philosophy, there are common failures teams are likely to come across. Here are some common mistakes businesses might face and how best learn from them.
Typical DevOps failures and how to avoid them:
1) Unforseen lock-outs
It is easy for developers to lock themselves out when automating deployments using configuration as code, forcing them to manually log in to each machine to fix it when an incorrect change is pushed and deployed across all machines.
How to avoid this: Developers must ensure they have the right protections setup, validating the configuration, and all the changes put in place frequently and from the early stages of development. By implementing an official process for both application code and infrastructure code changes, companies will reduce the risk of affecting other environments, saving developers time and improving the quality of the project.
2) Deploying out of line
Often developers will run both continuous integration (CI) and continuous delivery (CD) at the same time to speed up automated testing and feedback cycles. A common mistake to avoid when doing this, is to run deployment as well. This will misconfigure the code, implementing new changes to the final product before it has been tested – defeating the purpose of automated testing before deployment.
How to avoid this: It is crucial to put in place certain gates and checks before any code can reach the end user. For example, constructing the CD pipelines so that parallel testing is managed as a gate to production will ensure that tested and validated changes are not automatically deployed.
While DevOps principles state that teams should be able to deploy at any time, deployment must always be managed, occurring at the end of the pipeline after all the required tests, validation checks and approvals before it reaches the customer.
3) Software Configuration Management conflicts
One of the biggest challenges that a DevOps team will face is Software Configuration Management (SCM). It isn’t uncommon for solution architects early on in their career to be given access to Git (source code management and version control tracking system) repositories – which is a sure-fire way to cause problems in the first place. When trying to master the basics of Git, teams will probably encounter merge conflicts, involving one or more conflicts in the program, and detached HEAD’s (when a specific commit is checked out instead of a branch) which are not only difficult to get out of, but any changes made in this mode will be lost.
How to avoid this: It is best practice to put a branching and merging model in place to ensure the appropriate inputs are being fed into the DevOps pipeline to allows continuous integration of any test and deploy changes.
4) Poorly defined objectives
Though developers work together to push code through the pipeline, different teams can be prone to set their own KPIs. For example, a quality engineering team may use the number of tests executed per sprint as a success metric. However, as employees are almost always driven by incentives, the natural tendency would be to add more and more tests, potentially without considering outdated ones. This could affect other teams and increases test cycle time and therefore time to production.
How to avoid this: Understanding the ‘why’ of DevOps is often the most underemphasised, yet critical, a lesson in DevOps. Businesses need to develop goals and objectives for that particular initiative and then align to those goals. Once these goals are defined, the KPIs and success metrics can be aligned to the objectives. Businesses must develop a set of mutual goals, success metrics and KPIs, as this will dictate what gets prioritised when building pipelines.
5) Failure to recognise organisational barriers
A difficulty businesses frequently face is often one of company culture. Failing to fully integrate teams creates a new set of siloes, segregating departments. This causes issues when looking to implement internal development as departments often shift the blame over, rather than evaluate the business as a whole.
How to avoid this: The best way to organise teams is based on subjects such as product or functionality. Additionally, all stakeholders should be included from the start of the software development process to understand the roles of the teams. Together, this will better align stakeholders and development teams to cooperate. This in turn, will enable a business to truly practice DevOps. The culture of DevOps is just as important as the tools and technology when developing software, though it is a difficult element to change.
When taking into account the common mistakes businesses face when implementing DevOps, it is key to recognise that all businesses are different. Consequently, each will experience an individual DevOps journey and its own mishaps. There is no easy way to pull off a DevOps transformation and it certainly cannot be achieved quickly. DevOps will always present a fair share of challenges and lead to some mistakes along the way, the recipe for success is to learn and adapt. Only when doing this, will businesses be able to create their individual DevOps culture.