How to avoid a Kubernetes food fight!

National Lampoon’s Animal House is a 1978 American, comedy film with an iconic scene. Bluto (played by John Bulushi) yells “Food fiiiiiiiight!”, and a crazy chaos ensues as the students wildly fling food all over the cafeteria.

As I listen to CIOs describe how their employees are installing the free, open source Kubernetes software, that food fight scene plays out in my mind. So why is there so much chaos concerning Kubernetes installations? Well, because it’s free–meaning anyone can install it anywhere and at any time.

Remember shadow IT and public cloud?

Let’s back up and give some context to the current Kubernetes dilemma. I find the current Kubernetes food fight reminiscent of what happened with shadow IT several years ago. Developers were frustrated with the extended length of time it took their central IT organizations to provide them with compute resources, so they headed to the public cloud to obtain the resources they needed – quickly and efficiently. Although the developers’ intentions were good, chaos reigned. Lack of governance and loss of control were the norm.

The same thing is happening with Kubernetes – today’s go-to open source, container orchestration tool. Kubernetes allows you to take containers and put them in the right place and manage them. And because it is open source—and free—lots of people are installing it. Without visibility and control by central IT, three primary problems can occur.

1. The first issue is pretty obvious: security breaches. Numerous analysts have highlighted configuration errors as the primary security risk with Kubernetes. That’s because having many uncontrolled instances makes it nearly impossible to get security right everywhere. Put another way, somebody will get it wrong, possibly grievously wrong!

2. The second problem is more of a hidden one: excess cost. Although the software is free, the resources are not. You still need to run it on something. If you install it on AWS, you need to pay for the AWS services. If you install it on hardware located on premises, you still must pay for that.

3. Lastly, inconsistency is a big issue. And when you have inconsistent processes, lots of problems ensue. In a typical enterprise, each person or group is probably running Kubernetes with different tools. Although the version of Kubernetes that OpenShift provides is the same exact version that AWS provides, there are significant behavioral differences between the two, caused by the Kubernetes configuration and the installed tools (aka, Operators/CSI/CNI plugins, etc.).

Kubernetes is flexible and does not mandate which tools you choose to use. Let’s say one group in your enterprise likes one vendors’ networking tools; another group prefers the storage tools from a different vendor. The applications deployed may have poorly understood dependencies on a given vendors’ networking or storage tools, causing the applications to run differently – if the tools are changed. No one wants to introduce this kind of risk, so they won’t want to change vendors. In other words, if it works, leave it alone.

The result is a big messy food fight between different groups within the enterprise.

Gain control by accepting 4 realities
The primary mandate for any CIO in the midst of this Kubernetes food fight is to gain control. Now keep in mind that controlling something is not the same thing as stopping someone from using it. Gaining control means finding a way to enable people to use the version of Kubernetes they want AND the vendor they want. Yet, IT still must have visibility and control of Kubernetes across all of the groups throughout the entire enterprise.

How do you gain control? You must first accept some key realities:

Reality #1. You will manage multiple versions of Kubernetes.
You will need to provide multiple versions of Kubernetes across multiple private and public installations. Marketing may want Kubernetes version 1.1.3 on Amazon and your finance group may want version 1.6.6 on Azure.
Reality #2. You will need to manage Kubernetes on different platforms.
Different groups within your organization will not only ask for different versions of the Kubernetes distribution, they will want it on the platform of their choice. Some will want it through public cloud providers and some will ask for it within your own datacenter.
Reality #3. You will need to be able to upgrade each of these versions independent of each other. Yes, of course this process will be time-consuming. But it is a reality.
Reality #4. You will need to provide access to data in a consistent manner across your private and public data sources. Even though you have the three realities above—inconsistent versions, platforms, and processes—you must provide consistency.
Before moving forward, every CIO must accept these four realities, or you will waste valuable time and energy fighting them.

How to control the uncontrollable

Given these four realities, what’s a CIO to do? Develop what I like to call a control plane. Just as a control plane in networking is responsible for routing traffic, your IT team must develop a control plane for how you control all the Kubernetes realities.

To prepare yourself, you need to ask questions and really understand what’s going on. Then you need to develop processes that are consistent with your current models.

For example, here’s a starting list of some questions you should be asking.

How many clusters does your control system manage?

How do you consistently maintain identity and access management across multiple vendors’ Kubernetes clusters?
How do you maintain a consistent global namespace for your data fabric across multiple Kubernetes clusters in different geographies on different platforms?
How do you secure applications with different versions of Kubernetes running on different public and private infrastructure?
How does someone unify logging and monitoring across dissimilar platforms and Kubernetes distributions?
As I mentioned, this list is only the beginning. By sitting down with IT and reviewing all control issues, you will likely come up with many more items. Once you identify your issues, then you can start to resolve them. And you accomplish this by developing controls.

Stopping the Kubernetes food fight

It’s time to gain control and stop the Kubernetes food fight. Experts in the container team at Hewlett Packard Enterprise (HPE) have worked with customers all over the world finding solutions that help gain back control. To solve this issue as quickly as possible, organizations find it helpful to work with people who have been there and done that. To learn more, visit the HPE Ezmeral software page. To read more articles by Robert Christiansen, visit HPE Ezmeral: Uncut.


Notify of

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Inline Feedbacks
View all comments
Would love your thoughts, please comment.x