Why domain experts are vital for DevOps test automation
Source – techtarget.com
At a recent DevOps test automation conference, a participant in a panel discussion suddenly interrupted himself and looked out at the audience. “How many of your companies have a shadow IT?” A few hands tentatively went up, and he explained further: “How many of you have people outside of IT, in the functional groups, manufacturing, marketing, accounting, who are constantly doing unexpected things with your applications? Are they using them for different purposes, entering nontraditional data, even writing their own add-ons to do more analysis with the results?”
More hands went up, as people understood the phrase. “You want to engage these people,” he continued. “They are the ones who can make your applications better. And you may even want to hire the best of them for your team.”
Whence automation?
And, in fact, what is old is new again. Amidst a wave of DevOps test automation among traditional and Agile teams that removes the human tester from the process, forward-thinking groups are rediscovering the value of the domain expert who is interested in computing.
As testing struggled to become faster and more efficient in recent years, testers have increasingly turned to test automation — the ability to record tests and play them over and over again during the development and delivery process, with little or no tester intervention. Even today, DevOps test automation continues to hold the promise of faster, more complete and more consistent testing.
But despite being at perhaps the height of its popularity and capability, DevOps test automation has plenty of problems with which to contend. First, it requires skills in scripting and script maintenance that are not widely available in the testing community, even today. This is especially true for specialized automation tools used for automated testing on SaaS applications. Companies are scouring the internet looking for proven automation skills in testers, and they’re not being very successful at it. Second, DevOps test automation typically uses ways to identify buttons and other user interface controls that can change during the course of development, requiring either rerecording or manual changes to scripts.
And automation doesn’t address the fact that tests have to be designed and recorded to begin with. Further, while automated tests are an essential part of the smoke testing of new builds, no matter how often you build, you will rarely run your entire test suite as a part of smoke testing. Many tests don’t need to be run more than once or twice.
More important, DevOps test automation just looks at the user interface controls and responses. In the vast majority of applications, testing is much more than that. Testers typically also look at programming interfaces, unit tests and integrations as a part of deployment. DevOps test automation typically has difficulty getting under the hood of an application.
And last, automation can lull teams into a false sense of security. Looking at the results from automated testing runs may not be an accurate reflection of the overall health of a project. The test data and statistics may not provide a complete picture of the quality or readiness of the software product under development. That has to be supplemented by testers and domain experts working with the product itself.
Automation and domain expertise coexist
None of this means DevOps test automation doesn’t have an important role in testing practice. Automation is critical to improving speed and creating reliability in testing. But it’s not the all-consuming role that many envisioned when we started down this path several years ago. It can accelerate many aspects of testing, but not without cost. The upfront and maintenance effort for automated test scripts can be significant, and the results are less than ideal. And it doesn’t look at every aspect that human testers do.
That’s why there is an important role for domain expertise, even if those people aren’t professional testers. They stress an application in ways that no one on the DevOps team could think of doing. And it turns out that DevOps test automation teams are recognizing this capability and are starting to rely on it for their own work.
There was a time when domain expert testers looked like they were going the way of the dinosaur. If this sounds like you, you may be hearing from one or more of your DevOps teams in the near future. In addition to wanting your feedback, they may have a job offer for you.