What is the Next of Virtualization
Source- virtualization.cioreview.com
It has been more than half a century in the history of virtualization since IBM’s first effort to virtualize mainframe in the mid-1960s. Multiple applications could run at once, and hardware utilization and productivity were increased. The evolution of virtualization has been accelerated in the past 20 years. The resources such as a server, desktop, storage, file system, operating system and, networking are virtualized, and workloads are managed and run over virtual resources to achieve scalability and elasticity. Virtualization is the key to the benefits of cloud computing, i.e. the economy of scale. It makes it possible for the enterprise, IT and service providers to reduce the CapEx and OpEx, increase the elasticity of service provisioning, and improve the quality and reliability of services.
Today, virtualization has been used extensively beyond the traditional IT (i.e. server, storage and networks). DevOps paradigm combines software development (Dev) with IT operations (Ops) in order to shorten product development lifecycle and closely align with business objectives. Modern techniques such as Agile methodologies, test-driven methodologies, continuous integration and continuous delivery (CI/CD) etc. have been widely adopted in the industry to enable and accelerate the DevOps paradigm.
DevOps paradigm involves many aspects in the development and operations process, such as code development and review, build and CI, continuous testing, packaging and staging, change management and release, and live monitoring. Each stage of the process needs a toolchain to achieve its goal, such as source code management tool (e.g. git and git review), build and CI tool (e.g. Maven and Jenkins), artifact repository (e.g. Nexus) etc. The fundamental aspect of a successful DevOps is the infrastructure configuration and management of the toolchain, and toolchain automation.
With the adoption of virtualization almost everywhere today, it becomes more cost-effective, more scalable and more elastic to build DevOps process upon virtualized infrastructure and/or cloud-native infrastructure. This trend gives rise to the need of an automation platform to build and deploy a DevOps process on top of virtualized infrastructure and cloud-native infrastructure. This automation platform is expected to have the following basic features:
– The ability to create a virtual infrastructure cluster
– The ability to integrate the toolchain used in the DevOps process, e.g. Maven, Jenkins, Nexus, Sonar etc.
– The ability to deploy the toolchain to the virtual infrastructure cluster
– The ability to perform the configuration and management of the virtual infrastructure and toolchain
– The ability to automate the entire steps above and provides the users with turn-key experience
There are some commercial platforms for DevOps automation to some extent. However, on the other hand, an open source solution is still missing. This is a gap and could be one of the next moves in near future.
Let us look at the other pillar. Several IT vendors simultaneously proposed the concept of service virtualization, e.g. refer to https://en.wikipedia.org/wiki/Service_ virtualization. Currently, the context and scope of service virtualization are quite limited to DevOps in IoT devices.
IoT devices are heterogeneous in nature and bring the challenge of physical devices that are either hard to access or unavailable at scale. Thus, DevOps in IoT runs into problems due to this challenge of hardware availability, especially for testing and test automation.
Service virtualization intends to bridge the gap by emulating the behavior of IoT devices and their components for those component-based IoT applications, e.g. API-driven applications, cloud-based applications and service-oriented architectures. This emulation (or virtualization) of the behaviour of IoT devices and their components enables the workflow of DevOps to proceed without accessing the actual physical device.
There are some service “mocking” tools, mostly are commercial and some are open source. The key difference of “mocking” tools and service virtualization is that “mocking” tools in general skip unavailable physical device while service virtualization emulates the unavailable physical device.
Certainly, emulating the behaviour of IoT devices itself has many challenges, too, e.g. handling different types of hardware, a wide range of protocol stack, many proprietary layers and protocols in communication, lack of standards in messaging and data model, heterogeneity and asynchrony etc, to name a few. With machine learning technique, service virtualization can be built upon:
– First, the communication flow in IoT field is observed and logged, and big data set is generated
– Secondly, the big data set is used as a training set of machine learning to train the model
– Thirdly the model is applied to emulation of the physical device
In conclusion, while virtualization is being adopted ubiquitously, and the next move would be to enable and accelerate the adoption of a DevOps paradigm with an automation platform that provides users with turn-key experience. In the specific area of DevOps in IoT field, service virtualization will play a more and more important role in its enablement and accelerated adoption. Machine learning will be a key technique to build a reliable model of service virtualization for DevOps in IoT field. And more importantly, the gaps are the missing open source solutions for both DevOps automation platform and machine-learning-based service virtualization platform. Those open source solutions are really needed by industry, now.