Three years ago (April, 2018) Packt published my book Kubernetes for Developers. There weren’t many books related to Kubernetes on the market, and the implementation of Kubernetes was still early – solid, but early. Looking back, I’m pleased with the content I created. It’s still useful today, and for technical content that is pretty darned amazing. I wrote it for an audience of NodeJS and Python developers, and while some of the examples and tasks are now a touch dated, the core concepts remain sound. I originally hoped for at least an 18 month lifespan of the content, and I think it has doubled that.
When I wrote that book, a lot of folks that I interacted with wondered if Kubernetes would stick around, or if it was just a flash-in-the-pan fad. Over the decade prior to writing it, I used with a large variety of devops tools (Puppet, Chef, Ansible, Terraform, and a number of older variations), providing me with experience and understanding. When I saw how Kubernetes represented the infrastructure it manages, I thought it would be a win. The concepts were the “right” primitives, modular and composable, they felt natural, and applied consistently. When I saw AWS “flinch” — responding to the competitive pressures of Google releasing and supporting Kubernetes with cloud services — I knew it was destined for success.
I saw, and still see, a place where this kind of infrastructure can be an incredible boon to a developer or small development team — but with Kubernetes there is a notable downside. While it has great flexibility and composes well, it’s brutally complex, has a very steep learning curve, and can be notably opaque. For a developer looking at the raw output of Kubernetes manifests, it can be a horror show. Values are copied and repeated all over the place, the loose coupling means you have to know the conventions to trace the constructs it makes, and error messages presume in-depth knowledge of of how the infrastructure works. In addition, there are conventions that rule functional operation, and when you step to trying to edit something, it’s astonishingly easy to misconfigure things with an extra space because — well — YAML.
I recently came back to creating application manifests for Kubernetes, and wrote a (simple) helm chart for an application. Coming back after a couple years working on different technology reminded me how opaque and confusing the whole process was. The updated version of Helm (version 3) is an improvement over its predecessor for a number of technical reasons, but it’s no easier to use to develop charts. Creating even a simple chart expects a very deep knowledge of Kubernetes, the options it provides and how to specify them, knowledge of the coupling that’s implicit within the manifests — where data is repeated, and the conventions that rule them. It was very eye opening. The win that I hoped for three years ago when I published the book — that developers could see and have a guide to using the power of Kubernetes to support the development process — hasn’t entirely come to pass. It still could, but needs work.
For developers who are being asked to be responsible for their code running in production, running apps with Kubernetes provides useful levers and feedback loops. Taking advantage of Kubernetes in your development flow is not going to speed up that tight loop where you’re enabling a new HTTP endpoint, but it can make a notable difference when you get the stage of integration, acceptance, and functional testing — the places where you verify correctness, performance analysis, and resiliency testing. As a developer, if you know what Kubernetes is looking for, you can write code to communicate back to the cluster. This in turn lets the cluster manage your code – individually or at scale – far more effectively. Kubernetes has the concept of liveness and readiness probes for any application. By crafting extensions on your application, you can provide direct signals of when things are all good, when there’s trouble, and when your app needs to restart.
The same pattern of interaction that Kubernetes uses to manage its resources is used by observability tools. Once you’re comfortable sending signals to Kubernetes, you can extend that and send metrics, logs, and even distributed tracing – about how your application is working with the details that let you debug, or forecast, how your code operates. Open source observability tools such as Prometheus, Grafana, and Jaeger all comfortably run within Kubernetes, enabling you to quickly provide observability to your apps. The same observability that you use in production can provide you with additional insights to experiment and explore during development. A post I wrote two years ago – Adding tracing with Jaeger to an express application – is still a popular article, and I used that setup to characterize an app by capturing and visualizing traces while running an integration test.
Having been periodically responsible for large and small “dev and test labs” over several decades of my career, it appeals to me that I can create a cluster, deploy the code, run functional and performance tests, and validate the results while also getting a full load of metrics, traces, and logging details. And because it’s both ephemeral and software defined, I can create, run, capture data, and destroy in whatever iterative loop makes the most sense for what I need. In the smallest scaled scenario, where I don’t need a lot of hardware, I can verify everything on a laptop. And when I need scale, I can use a cloud provider to host a cluster, and get the scale that I need, use it, and tear it down again at the end.
Getting back into these recent from-nothing deployments into a cluster, and writing new charts, reminds me that while the primitives are great, the tooling and user-interface for developers working with this has a long, long way to go. My past experience is that developer tools can be among the last to get decent, let alone good, user interfaces. Its often only slightly ahead of the dreaded “enterprise application” tools in terms of thoughtful user-experience or visual design. With work, I think the complexities of Kubernetes could be encapsulated – visible if you needed or wanted to really see. That work should let you focus on how your app works within a cluster, and use the good stuff from Kubernetes and the surrounding community of tools to establish an effective information feedback loop. It could be so much better supporting developers and allowing them to verify, optimize, and analyze their applications.