Why new feature testing should be at the heart of DevOps
Source – techtarget.com
The idea of continuous testing generally entails an always-on, massive build server that does more than just build. It should create a test environment and run end-to-end comparison checks against the system to simulate user behavior. Continuous testing is supposed to check all these things, but when it comes to software, that is naive at best. Complete testing is impossible.
If you think about it, mass inspection masks a problem. When you commit to mass inspection, you’re giving up on good software development practices. You end up developing bad software and, hopefully, catch the biggest problems before they hit production. Mass inspection treats the symptoms of low software quality and is often a very expensive way to do it. Once the automated tools start to run, mass inspection stops being test automation and instead becomes change detection. Meanwhile, the entire job of the programming staff is to create change! That’s why we need to make the case for new feature testing.
Balanced DevOps testing
I am sure you see the problem. If you want to reduce the need for mass inspection, you need to improve first-time quality. This often comes up when we talk through the “Manifesto for Software Craftsmanship,” and that is a good and important thing. But it starts with new feature testing.
Most of automated or continuous testing is regression testing, which checks to make sure a change didn’t break some other bit of code. But between code resilience and craftsmanship, we can reduce the chances of such an issue happening. We still need to test changes, and it’s likely that’s done by a human and likely with a wide range of scenarios. A small percentage of those tests, though, need to be institutionalized and simulated with tools and then put into the deploy process.
New feature testing for DevOps
Even in a full-fledged DevOps process, your team will still need to run end-to-end user simulation tests before rollout. But you don’t need to test every minute detail. Instead, work on how to find the problems before things become an issue.
If you want to claim DevOps status, build well-crafted code so defects do not cascade into other areas of the system. Have real humans perform new feature testing early in the process. A new feature tester should be “technical” and able to look in the developer toolstab in the browser, but the important work is primarily exploration. Some of our advanced customers do this during development, with a mostly-testing person pairing with a mostly-programming person.
In the rush to automation and continuous testing, we can miss the entire point of testing, which is to release better software. Break testing down to its component parts — like new feature testing — and you see why it’s important to pay less attention to testing all the thingsand more attention to testing the right things the right way.