Entropy, Negentropy and DevOps

Forgive me for I have probably just click baited you.

I've written this post to highlight some similarities between the conversations we have about work in the world of DevOps and how these are not just tech problems but general life problems.

TLRD; Negentropy (negative entropy?) can be thought of as the opposite of chaos, it's discrete and controlled and gives predictable outcomes.

A brief introduction to Entropy

Entropy, the measure of a system’s thermal energy per unit temperature that is unavailable for doing useful work. Because work is obtained from ordered molecular motion, the amount of entropy is also a measure of the molecular disorder, or randomness, of a system. _ Brittanica.co.uk

The concept of entropy dates back to the early 1800s and is contributed to the works of Rudolf Clausius and his study of thermo-dynamics, in particular inefficiencies of the Carnot cycle.

Now I'm not going to pretend I can recall the laws of Equilibrium Thermodynamics (it's been too many years), but in general as the amount of entropy increases in a system, the system eventually plunges into chaos, not too unlike tech debt and big bang (I'm full of physics puns) releases in the world of tech.

Negentropy and organising that chaos

So this post was part inspired by a link to an article on negnentropy that my dad sent to me.

In the opening statements the article talks about those day to day micro moments of opportunity, picking up the sock on the floor, cleaning the dishes before bed etc.

Much like tech debt, the amount of wasted effort, or unused energy ultimately builds up and descends into chaos.

The article draws parallels between social systems and the waste in those systems that ultimately leads to their failures, advocating for a concious effort to think about day to day living in terms of entropy (work left undone) and negentropy (small amounts work done with immediate effect).

The 5 step program

So obviously, when I read an article like this and it immediately gets me excited. Ever since I was introduced to the concepts in DevOps of small unit changes that are continually added to the main line branch, resulting in measurable changes to a system and of the power of making changes with a purpose to solve a hypothesis in order to return value back to the business, I've always long thought about how this isn't anything particularly unique to the industry and could probably be applied to a broad spectrum of disciplines, businesses, industries and institutes.

The article above ends with how one can apply negentropy thinking to their lives in order to stave off the tide of looming chaos which I'll summate:

  • 1: Find the entropy
  • 2: Prioritize the losses
  • 3: Come up with a plan
  • 4: Try it out and pay attention
  • 5: Go beyond fixing and maintenance

So let's take these points and draw those parallels with the way we might do those things in "tech":

Find the entropy

So this one I see as being the identification of the tech debt, the bottle neck in the system, here we are seeking to find the wasted effort, which seems pretty obvious.

Prioritize the losess

To me this is not that dissimilar to prioritizing the backlog, in my humble opinion I always think the most important changes to a system are the ones that will deliver the greatest value back. There's not much value picking up dirty socks if the kitchen has been renderded inoperable with dirty dishes (hell, you might even be guilty of then spending MORE money on food in the form of take away and that's definitely not deliverg value).

Come up with a plan

Okay, so here my immediate thought isn't planning your backlog or the week ahead, this to me isn't about the sprint goal. The plan, for me, is the ability to deliver that change. It's the pipeline, your plan of execution may have multiple stages, certain things have to happen first before you can move onto the next step, and each step brings you closer to the goal, the deliverable at the end.

Your plan is your pipeline.

Try it out and pay attention

As I read that statement, my mind is quiet simply buzzing, I'm energised.

HELLO experimentation and monitoring my old friend.

DevOps advocates that we have good observability and monitoring, but the goal of those aspects of the system should always be to confirm that the hypothesis was met, otherwise it's pretty much wasted effort, meaningless monitoring is waste.

Trying it out, that's your business idea, your department hypothesis, but if your not collecitng the data and analysing the results of that idea, or the change then your just adding to the entropy of the system, not helping to negate it.

Go beyond fixing and maintenance

So in the article the paragraph talks about "maintaing a social system that isn't beneficial no matter how smoothly it works" and that the "small changes don't mean anything if they aren't contributing to the bigger picture" and I love that as a statement.

It's not exactly hard to create a system, write the code, put in the tests, get the pipeline in place, measure the outcome and determine it's worth, but it's ultimately redundant if it's delivering ZERO value, it's just more waste and increased entropy if your work and your life.

But the sentiment of the statement here also hints at something far more important than any line of code and that's culture. If the culture isn't there to support all of that technical greateness then it's almost destined to fail and all of that work, that effort taken to get those things in place again will be wasted when the project ultimately gets abandonded.

Conclusion

Okay in the interest in drawing parallels with science let's just write a little conclusion ( I loved science in school and I still love science and engineering now, my favourite part was always writing the conclusion at the end, so muhc fun to be had in taking the data and determingin the bigger picture).

So, entropy is the meaure of unused or wasted work in a system, that, if left untouched will devolve into chaos. To combat this in our work (and as the phys.org article highlights, our lives) we should look at the bigger picture with an negentropic lens. Find the waste, prioritise the clean up, experiment and learn, but don't forget it's got to deliver value back!

I hope you enjoyed this one.