The first time I gave a talk titled "Enterprise flight into DevOps space", was in late 2014 at DevConFu conference. After that, I delivered the presentation several times during 2015 at various conferences in Europe. For some time, I considered it a "closed topic" and did not really want to expand on it, until I was asked to deliver it again in 2016, and then again in 2017.
Enterprise is a word that is highly abused. Most people associate it with slow inefficient processes. Nobody wants to be there! At the same time, when I ask the audience: "How many of you work in enterprise?" — over 95% raise their hands. Somehow, human nature makes us prefer simple models of the world, even though, we, of course, totally understand that the world is way more complicated. Still, we are easily trapped into selecting good vs evil, being a start-up vs being an enterprise, developing microservices vs struggling with a monolith. Are most of us really on the evil/wrong side? Probably, not.
I've heard a lot of definitions of DevOps and will most likely hear it even more in upcoming years, but what I heard many times is that DevOps is solely related to start-up environments. I disagree with that statement. I strongly believe that successful start-ups will come to the DevOps culture (kind of) automatically because it's the only way for them to survive and conquer new markets given all the resource/time constraints. So, it's not that interesting to talk about DevOps inside start-ups, since they, by definition, have a different approach to business (until they grow big enough and become slow enterprises, of course).
DevOps addresses many issues of small and big companies involved in IT, which is pretty much everywhere. Introducing DevOps principles inside slowly moving enterprises will uncover inefficiencies, and will help optimize processes and teams. Talking about DevOps in the enterprise is much more important than reflecting on experiences achieved by some high-performing start-ups.
Let me tell you the story of Jack the SysAdmin.
Once upon a time in the enterprise:
Jack knows what to do. Jack starts working.
After a couple of days:
After a few more of days:
After all is done and documented, servers are handed over to the development team. Jack is happy. He did his job well. Requirements are fulfilled completely, documents and forms are filled according to Operations Department guidelines. All e-mails are sent, all animals are fed.
Developers have no idea who Jack is and where he sits in the office. The only way for them to change anything in the environment (in the "official" way) is to report to their manager, who will use the organization's issue tracking system to file a ticket to the Operations Department, where Jack will pick it up. To speed up the process, the project manager also sends an e-mail directly to Jack.
Jack has to obey and implement a blurry change request in the way he understands that. Ka-boom! Developers can't deploy. They start to complain. Jack has to fix the servers. It takes all night.
Things get worse with time. More change requests are entering the system. Also, developers, realizing that infrastructure change process is slow, start making some of the changes themselves to speed things up.
Infrastructure quickly flies into the state that is called THE UNKNOWN:
Somehow developers implement all the functionality and it kind of works in the development environment (and on my computer). It is time to move to the QA phase:
And we add more environments:
Things get complicated. Documentation does not serve the purpose anymore. Nobody knows how to configure the beast. Jack is overwhelmed with work. He does not cope. Solution? Let's throw more people at that:
Again, somehow it is fixed. It's not stable, but tolerable. The "go-live" date is approaching. Production becomes the most important environment:
Production is catching fire every now and then. Monitoring does not help. Operations team monitors everything they usually do: CPU load, the amount of disk space, maybe network I/O, but they have no idea about vital application metrics like the number of incoming customer requests, how many of them are successful, how many active customers the system contains, etc. Therefore, the dashboards are green, hiding the obvious problem with the application:
Project managers get a call from the customer much earlier than the team realizes there is a problem with the application. It takes a few hours or days to find what is going on. Of course, after a few incidents, the system is fixed and becomes better to some extent, or maybe not. In any case, the trust of the customer is lost.
This is the end of the story, well, at least of the first chapter.
When I ask my audience if this story is something they have already seen, I see quite many brave hands and many nodding heads. Some people even desperately scream: "Oh, yeah!".
The problem has nothing to do with technology. The problem has to do with the lack of communication, with the lack of cohesion between Devs and Ops, and with treating the Development and Operations roles as very much isolated phases of the same process.
That's what often happens in many enterprises: communication is broken and it only happens at management level meetings or at issue tracking system level:
New or updated applications are just thrown as-is from development to production. The feedback cycle is slow and/or incomplete. Information, on how bad the application design is or how inconvenient it is to deploy, configure and monitor that application is often not getting back to Developers or is considered not important. Developers continue to produce code that gives long working nights and weekends for Operators whose only task is to make it run.
On the other hand, the Operations department may prefer to hide behind the wall of "standard" platforms and force Developers to create new applications using outdated technologies. The will to standardize may reach its extremes and produce quite rigid structures. That's what the reality turns out to be in bigger companies where departmentalization reached its apogee:
Of course, when Development requirements and Operations capabilities are not aligned with each other or are aligned very late in the process, we get to the situation that nobody wants to be in: the number of unexpected issues grows.
This is where enterprises need the DevOps transformation, which is not about the tools and technologies (even though they definitely help), but about improved communication, reduced feedback cycle and higher empathy level.
A communication model for DevOps should move beyond the 2 (or more) pyramids talking at the top of something, which resembles the following state:
The picture above looks a bit like a football game. The two teams have conflicting targets, but they are playing by the rules, and the real goal is not to win each other (even though it will bring benefits to their department), but to bring joy to the spectators of the game i.e. the customers. Nobody wants to watch a game, where one team scores 20 goals and other one scores nothing; not many want to watch a game, where all players get red cards due to misbehaving. But the game, where equal competitors show their best skills and demonstrate fair-play and respect, will be remembered by the watchers and they would want to see it again.
The analogy with the football game probably cannot be fully transferred to the IT world, but at least some aspects of it seem interesting to me.
Enterprises - break the silos, invest into better communication, increase empathy! Do not replace DevOps transformation with tool selection. In 2017, tooling is not a problem anymore, but DevOps culture still is.
by Ovidiu Mățan
by Ovidiu Mățan
by Mihai Varga
