Applying DevOps to Improve the Effectiveness and ROI of IoT Testing

Source – einfochips.com

With the power of the connected world (that unifies multiple objects, sensors, and devices known as the Internet of Things) comes the complexity of realizing real-life conditions and ensuring the delivery of IoT services and functionality over the heterogeneous environment. To make sure the connected devices and objects perform as per their specifications and interoperate with the physical world to make data and services available on time, IoT testing is a must for every enterprise.

According to a research from Markets and Markets, IoT testing market size is estimated to reach USD 1,378.5 Million by 2021, at a Compound Annual Growth Rate (CAGR) of 35.4%. Considering these stats, there cannot be any other way to support the testing of this kind of infrastructure which comprises of millions of devices, sensors, gateways, routers, cloud applications, servers etc., except for adopting open source tools and technologies.

Open-source tools and technologies have become the norm in IoT testing as they fuel rapid integration and testing of IoT platforms with other vendors. It is interesting to note that, open-source adoption lies at the heart of DevOps given that the majority of DevOps tools and technologies in the market are open-source, which reduce the total cost of ownership and testing infrastructure.

How DevOps’ Open-source Ecosystem Helps in IoT Testing

While shift left testing, risk-based testing, a software developer in test (SDET) remain some of the most sought-after IoT testing strategies, DevOps is considered the most effective strategy when it comes to generating the best coverage and test efficiency metrics along with tangible cost savings owing to its support for open-source tools and technologies. Moreover, DevOps or intelligent automation is being adopted by organizations to manage IoT scale and achieve rapid innovation. DevOps also addresses the needs of IoT test automation, covering build, deployment, testing, network, and infrastructure automation.

DevOps can be applied to test multiple IoT product pipelines and application endpoints of an IoT platform. While testing device to cloud use-case, the individual functionality of each component in the IoT platform, including sensors, network protocols, cloud, web, mobile or APIs also needs to be tested. This is where different tools and technologies of DevOps can help.

Role of DevOps Tools and Technologies in IoT Testing & Automation Scenarios

Let us understand how common tools and technologies of DevOps aid in IoT testing:

UI, web and mobile functional test automation can be done with DevOps tools like Robotium, Selenium, Appium etc., with multiple variants and operating systems.
JMeter can be used for functional API automation.
Python is the used widely in the industry today for network automation. Interoperability with multiple protocols like Zigbee, MQTT, CoAP can be automated using Python or Ruby scripts. Sensor, service and API virtualization with Python can be used to simulate end to end scenarios for load and performance testing of IoT applications.
Test environment automation using container technology like Docker or virtualization tools such as Vagrant helps in four to six times more server application instances than traditional virtual machines, saving huge infrastructure costs.
Infrastructure as Code as famously said in DevOps, aids in test environment automation through configuration and deployment automation. DevOps configuration and deployment tools like Ansible, Chef, Puppet etc. can be used for this purpose.
Continuous integration using DevOps tools like Jenkins and Bamboo ensures that the latest piece of code is being tested and in turn promote detecting defects early in the testing cycle. The CI/CD (continuous integration and continuous delivery) pipeline built with these DevOps tools also aid in building the base for continuous testing of microservices through contract and integration testing. This enables faster provisioning and deployment of an IoT test environment, which means faster recovery from test failures, directly improving test cycle time.

DevOps for IoT Regression Testing

Regression testing is one of the most cost accumulating activity as testing needs to be performed iteratively with every newly added feature. Setting up an automated workflow to execute end-to-end sensor to cloud regression tests for IoT using the above DevOps tools saves testing efforts and cost of rework. Hence incorporating DevOps tools in IoT testing strategy provides better coverage in terms of functional testing, compatibility testing, load testing, performance testing and regression testing. Once a DevOps led IoT test automation engine is built, it can also be easily scaled by applying the same test strategy across multiple verticals. DevOps pipeline can be integrated with continuous monitoring tools like NewRelic, AppDynamics, or Pagerduty for incident management. This leads to monitoring the live application performance, directly improving the uptime and user experience.