Kubernetes Controllers via Metatron (Part 2)

Previously in Part 1, I discussed what Kubernetes Controllers are, how they work, and gave some examples of their usage. I also gave a description of Operators and explained how they are just a specific kind of Controller. All this in the service of explaining what my new gem Metatron is and how it is helpful.

Here, in Part 2 of this series, I’ll describe Metacontroller, including what it is, how it is helpful, and how it relates to Metatron for creating new Controllers.

But First…

Before diving into Metacontroller, I think a brief addendum to Part 1 is in order. While I do feel that I adequately described how Controllers work (at least from a high level), I didn’t spend any time on why one might want to create a Controller. Looking back, I believe this is pretty critical in understanding why I made Metatron, so it seems worth spending at least a little time discussing.

Controllers are a fairly foundational component of Kubernetes. They power features like self-healing and high-availability, cloud integrations, storage provisioning, auto-scaling, and more. Put another way, Controllers are what make Kubernetes powerful. Being able to seamlessly add our own Controllers is what makes Kubernetes amazingly flexible.

Beyond the features that I just listed, Controllers form the basis of most add-ons to Kubernetes. If you’re a user of Ingresses (especially with Let’s Encrypt or Vault issued Certificates, probably via cert-manager), you know what I’m talking about. Many vendors offer Operators to assist with installing their products via Custom Resources (vendors like IBM, Oracle, and MongoDB, just to name a few).

My personal favorite use though, is as an alternative (or compliment) to Helm or other mechanisms to deploy your application. Through carefully-crafted, testable Operators, it becomes much simpler to deploy things in a standard way. Plus it solves a common problem with running Kubernetes at-scale: how, after deploying lots of apps, do I propagate better ways to do things without needing to make a PR on every repo? By deploying components of your application via Custom Resources, all you have to do is update the Controller and your changes can be rolled out everywhere.

I won’t go into more detail as YMMV, but I hope this offers some insight into why I want to make creating Controllers as easy as possible.

What Is Metacontroller?

Metacontroller’s purpose is to make it possible (and easy) to write Controllers in whatever language you’d like without the boilerplate to actually interact with the Kubernetes API. From Metacontroller’s own documentation:

Building Operators with Metacontroller frees developers from learning the internal machinery of implementing Kubernetes controllers and APIs, allowing them to focus on solving problems in the application domain. It also means they can take advantage of existing API machinery like shared caches without having to write their Operators in Go.

Put even more succinctly, if you can write a function that can handle JSON and piece together JSON representations of the Kubernetes objects you’d like, you can write a Metacontroller-based Controller. If the JSON you generate would work with kubectl apply, you’re nearly there.

Even if you have the expertise, time, and desire to write your own Controller in Golang, Metacontroller can still offer some features that would make it compelling. As mentioned in the quote above, Metacontroller provides shared caches and manages its own connections to the Kubernetes API. This means that writing lots of Metacontroller-based Controllers will be more efficient (and likely perform better) than writing your own independent Controllers.

How Does Metacontroller Itself Work?

Metacontroller is itself a Kubernetes Controller that implements the Operator pattern. When you install Metacontroller, it listens for Custom Resources (namely CompositeController and DecoratorController). Through these resource, Metacontroller knows which other Kubernetes resources to pay attention to and where to send information about these resources.

The easiest way to understand how Metacontroller works is by closely examining one of these Controller resources. For some context before the example, let’s assume we’re creating a CompositeController that will handle a new, custom resource. Let’s say this resource is to help us run custom blogs on a home-grown blogging platform. We might build the custom resource so it looks like this:

The Blog resource has a name (via metadata.name), has a Docker image that it’ll run (via spec.image), and it has a number of replicas, so we can scale up blogs as they become more popular.

The CustomResourceDefinition for this resource might look something like this:

No code will actually run here; all this is built-in to Kubernetes and is just extending its own API to support a new kind of resource. It won’t actually do anything with these yet, other than track them.

The CompositeController resource is then used to inform Metacontroller about the endpoint that is responsible for events related to a kind of parent resource. These event subscriptions are based on the spec.parentResource and spec.childResources fields. The child resources are the types of resources we’ll want to manage for our Blogs. These are things like Secrets, Ingresses, Deployments, or really any other type. Other than these things, we just need to inform Metacontroller about our Controller code’s endpoint.

A CompositeController might look like this:

From here, we just need to create a web service that understands what to do.

What Happens Next?

What remains is the code to do the actual JSON work. Above, Metacontroller will try to communicate with a Kubernetes Service called blog-controller running in the blog-controller namespace on port 8080 over HTTP. Obviously, adjust this as required.

Metacontroller provides a predictable, well-defined request format for the sync endpoint you’ll need to write. As expected, it also provides a description of the expected response format. In essence, you’ll receive the current state of the cluster, at least from the perspective of some Blog resource (its details, its existing children, its namespace, etc.), and you’ll return what the cluster should look like (new children, removing unnecessary resources, and so on).

Crafting an endpoint to accomplish this isn’t terribly difficult and you can use any language and framework you’d like. If you’re interested in using Ruby, check out Part 3 where I describe Metatron, which is my framework for making this as easy as possible.

Further Reading

• Part 1
• Metacontroller API Reference
• Metacontroller Examples on GitHub