The Challenge of DevOps


Disclaimer: I am not an expert in this area, this is just a blog over difficulties and conflicts created in the search for DevOps. I will go through some fallacies that will make each group look bad, so stick with it if you can. I really would like to get conversations started where they weren’t.

The Phoenix Project. One of the greatest books of this IT age gives a good concept when it comes to how Infrastructure and Development work with each other. However, there are a plethora of definitions for “DevOps”. How do you take a company that has existed just fine with two (if not more) warring parties and get them not only to work with each other, but to help each other.

Fallacy 1: DevOps is Infrastructure working for Developers

This was my first collision with the term. The idea here, again, is basic. Infrastructure and its core individuals exist to give a foundation and support to the developers, and nothing more. A good example of this is a simple ticket request for a new server with specifics of the hardware. Any infrastructure worker worth his salt would look at those requirements and ask “why”, but this adoption of DevOps is “Why not?”. Obviously I’m taking things to an extreme here, but I have seen too many server requests for extreme hardware requirements (really…a virtual machine with 64 gb of ram?) and the inevitable fight that ensues in response to that three letter question “why?” There are so many other examples I’ve seen, from resource adjustments of the ridiculous, to the blatant opening of security holes. Obviously this is in varying environments, but the idea is the same.

This expands past infrastructure into Operations. For example, when there is a “priority one” ticket, and everyone jumps on the call, the first phrase is, “If operations did their job and fixed this we wouldn’t have to deal with this. I don’t even know why we are here.” Which is something I’ve heard this stated multiple times for multiple reasons.

Like all Fallacies there is truth here. Developers are the reason companies make money. Unless you work in the past, Devs work daily to create and stabilize performance that generate monetization for the rest of IT. The greatest example of this is Uber. Of course the top Taxi company in the nation is an app. So without it there is no real company at all.

For IT workers that, will lead them to a mindset of denial. No they’re not in denial, but  the next fallacy.

Fallacy 2: Developers should come to heel for Infrastructure

So ok, here we go. This is so prevalent in my career that I can’t even think of that many specific instances, because it’s almost daily that I hear something along this term.

Developer asked to be able to monitor or help maintain resources with their machine,  “No”. Developer asked to be able to create snaps for code roll, “No”. Developer asks to be local admin on a box, “No”. Developer asks to have a new dev environment that mirrors production, “No.”, and so on.

Its amazing and a marvel to me that people can just shutdown Developers for minimal reasons and yet, this is the normal situation I find myself involved in. I see a ticket and instead of doing investigation or adjustment the immediate response that I hear is either, “No”, or, “We cant do that”, or “Thats not our problem”.

This is just a few examples, and I’m sure everyone could think of their own examples. One thing that has really stirred this up and made my brain hurt is Containers. Developers ask for a container solution from infrastructure, and they get a big fat, “Why? It’s in the OS, so you figure it out.” This is a very frustrating stalemate.

This pushes to an extreme called “Shadow IT”. Shadow IT is basically if infrastructure wont grant the needed support or help getting things off the ground, Dev will use their individual budget and spin up an entire instance in AWS, GCP, Azure, or a basic private cloud. Just a license  for VMware Workstation can create Shadow IT, it’s that easy (don’t get any ideas). I heard a developer talking about the public cloud saying, “It makes me happy knowing no infrastructure people are touching my boxes.”

This fallacy again comes from a smidge of truth. Development doesn’t know everything about infrastructure. Infrastructure spends all their time doing resource management, monitoring, and adjustments trying to keep the infrastructure running in the best way possible, so they should be the main contributors to the how/what/why of infrastructure.

Fallacy 3: Security CONTROLS ALL

With Ransomware and Wannacry still being buzzwords in our time period, this is definitely a big deal. Security is extremely needed this day and age, from blocking things like DNS floods, to removing patch vulnerabilities. There are countless reasons to keep security up to date. Where does this come in for Infrastructure or Development? Well, Security is the Uber-Deny group. Almost every security individual I’ve met has stated something along the terms of, “My job is to make sure we don’t do something stupid.” I feel like the answer to this is both “Yes”, and “No”. I’m sure people do stupid stuff all the time that has nothing to do with security. The proper statement would be something like, “My position is to rectify security vulnerabilities throughout the stack” Or something like that.

Security will try to block where networks are, how they are setup, where infrastructure is placed, and the list goes on. Are there legitimate reasons? Of course! Security is the necessary part of IT that helps keep things in line, away from prying eyes, and malicious intent.

The Common Factor

All of these fallacies are connected to an amount of truth. They all start when the truth is twisted and exploited by individuals to make their position greater than it is.

It’s like the problem with DevOps is even more internal than IT. The problem with DevOps is us.

IMG_0427

People

I love this quite so much, because at its core it grasps the internal hierarchy of DevOps. It all starts with People.

Obviously each group mentioned is run by People. These people are all built around their own past experiences and troubles. Like iron, these experiences and troubles have made us stronger and sharper, but like the fingers of a string musician we become callous, and those callouses help generate better solutions in both good and bad ways.

There is also the problem with Ego. My favorite statement with immediate meetings are, “Everyone check your ego at the door. It has no place here.” If we only were able to do that and bring only the strengths of the department and not the callous overreaching, “where would we go?”

Finally for us, the problem of listening and understanding.

Meetings

I know this picture hits a chord for all us. How many of us have listened to what seems like the dumbest ideas and kept our mouths shut to it? DevOps starts with listening, and by listening, I mean the whole management stack. Is this easy to do? Absolutely not, in fact, it’s extremely hard to listen for nuggets of truth in a whirlwind of ideas. However, thats what we need to do. To start this, the greatest thing we can do is just listen, and if you don’t understand, ask the dumb questions that everyone is too proud to ask, so that you do. You are probably not the only one who has that same question.

Finally: People… Again

End Users…the concept is lost by all groups now and again, but everyone in IT works for end users. The hardest concept to grasp, and the easiest road to DevOps, is how to create a better solution for them. Developing a new patch, a more secure Dev infrastructure, or new storage solution. This all has DIRECT impact on an end user, and each group works individually to create a solution for them. Now how to mix them all together? A good start is a CI/CD pipeline and securing an automated solution for Developers to run continual delivery. It exists on infrastructure, it has to run in the best HA stability, and it has to be secured. This is a great solution that involves all groups together. There is so, so much more, but that’s the journey we are all on.

Call me “Optimistic”, or “Crazy,” Or “Nuts”, Or “Dangerous”, but I believe this is the future of our industry. With the kubernetes, containers, CI/CD buzzphrases, and the dominance of public cloud, the old standards needs to be replaced with the promise of DevOps. Now, how will you do it?

Things I learned this week!

  1. Error 500 in VCenter when deploying OVF.. Verify your VCenter certificate is trusted… Also How to do this on MAC and then trust it by selecting “get-info” and setting it to “always trust” Needed this for LifeCycle Manager and NSX deployment
  2. INSTALL PowerCLI on da MAC (Includes Homebrew, Powershell Core).
  3. Initial setup of vLCM
%d bloggers like this: