Proprioception

Proprioception is one of those big complex, compound latin-y words that means “knowing where you’re at”. Hit the wikipedia link for a much deeper definition. The reason I’m writing about it is because it’s something that I think anyone writing software for a distributed system should know about.

This kind of capability may not be a requirement or capability that you’re tracking or thinking about, but you should. Load balances (HAProxy or the various hardware ones like F5) have been using this concept for ages. It is now getting traction in the various services that orchestrate containers. Kelsey Hightower gave a talk about the concept at Monitorama this year. It is more than worth the 30 minutes of the video. I would actually suggest you just stop reading this and go watch it if you’re involved in writing services or building software. It is applicable well outside the space of back-end distributed systems, although his examples are all about building robust server-side software.

Then this morning in DevOps Weekly there was an article from NewRelic on how to use the same kind feature the Docker has recently added: HEALTHCHECK. It’s Docker specific and with excellent examples of why adding this into your service matters, and how you can use it. It’s not just containers either, Consul has a similiar mechanism built in for use with service discovery when using VMs or machines as well.

It’s a concept that I think a lot of folks are using, but without much consistency, and seems to be a good indicator of “Do they get it” when it comes to devops and robustly running large infrastructure. We should go beyond that, and from my perspective it should be an expected delivery, testing and validated for every component of a distributed system. It is an out-of-band channel for any application or service to say “I’m overloaded, or broken, or doing fine” – if you’re writing one, add this in.