Kubernetes Controllers via Metatron (Part 3)

Previously, in Part 1 I described Kubernetes Controllers and the Operator pattern. In Part 2, I explained why Controllers are important and how Metacontroller makes it easier to build them in your favorite language.

In this, the 3rd and final part of this series, I’ll show off Metatron, which is my Ruby-based approach to building Metacontrollers. We’ll walk through building a basic CompositeController responsible for a custom Blog Kubernetes resource. I’m still working on the gem, but this should be a good demonstration of its capabilities and how easy it is to get started. Much of this post is a recap from Part 2, but it seems worthwhile to present it all together.

Our Requirements

First, let’s go over what we’ll be building. The goal is to be able to create blog sites, but without having to go through all the work of creating all the required Kubernetes resources ourselves. We still want the power of Kubernetes, but without all the complexity. Ideally, this means we’ll just make a single Blog resource with a minimal spec, and our Controller will take care of the rest for us.

We need the Controller to create:

• A database we can connect to for persistence
  • This can be a StatefulSet
  • We’ll also need a Service to allow connecting
  • Plus a Secret for storing user credentials
• A Secret that contains the database connection details (for use by the blog application)
• A Deployment for the application itself
  • We’ll also make a Service for the app’s HTTP port
  • Plus we’ll make an Ingress for exposing the app outside the cluster

That’s a fair amount of Kubernetes resources we’ll create. Let’s think through what these resource will need to create successfully.

For the DB, we can make some assumptions just to get started. We’ll hard-code the Docker image it runs to be mysql:8.0 and we’ll be boring about how we create credentials. We can also derive the name of the DB from the blog name.

For the app, I’m going to go with wordpress:6, since I know how to get it up and running quickly, but we’ll keep that configurable. It listens on port 80 and only needs a few environment variables and one volume.

To keep things simple, let’s provide a single offering to our users:

• 20GB of disk space (5GB for the DB and 15GB for WordPress)
• 1 domain name with free TLS (thanks to Let’s Encrypt)
• 2 app replicas for HA

We’ll make all these configurable though, in case we want to change them later. This means that we’ll want our Custom Resource to look something like this:

With that, I think we know what we’re hoping to achieve and why.

Laying the Groundwork

Before we hack on any Ruby code (or any of its supporting components), we’ll need to inform Kubernetes about this new, custom Blog resource. We can create it via the following:

That might look like a lot, but it is pretty easy to follow. It is essentially just an OpenAPI3 schema for the custom resource we described above.

Apply that to your Kubernetes cluster:

Now we need to create a CompositeController resource that instructs Metacontroller to watch for our new custom resources. This assumes that Metacontroller is already installed. It’s pretty easy to use kustomize to install it, but there are other methods as well.

We can use a slightly modified version of the CompositeController resource from part 2:

Run this to apply the resource:

This instructs Metacontroller that, when it encounters changes to blogs, it should hit the /sync endpoint on the blog-controller service in the blog-controller namespace on port 9292. Metacontroller will provide a list of child resources for any encountered Blog and will expect a response that describes the new desired state of these (or other) resources.

Creating a Metatron Controller

With all that boilerplate Metacontroller stuff out of the way, we can get to work on the Metatron controller. Much of this will be a repeat of what is in the Metatron repo’s README.

A Typical App Skeleton

As with most Ruby/Sinatra-based projects, we’ll need to prepare a basic starting point for building and deploying a Ruby application. This includes things like a Gemfile, a config.ru, and a Dockerfile. If you’re unfamiliar with these, there are tons of great resources that can help explain what they’re doing (plus I add comments to make it clear). Leave a comment if you get stuck.

First, let’s make a directory to get started on our development:

This made a new directory and started us out with a new git repo to track our code changes.

From there, let’s create the Gemfile; this is required to make sure we’re installing the dependencies required to run our Metatron controller:

Yup, that’s really all that we’ll need. Let’s install things:

You should now have a Gemfile.lock file which pins installed libraries to specific versions. The lock file makes sure your development environment is the same as what you deploy to production.

Now, let’s create our config.ru file. This informs rack about how to route incoming requests to our code:

Finally, we’ll need to make a Dockerfile to build a container image with our code in it:

A Metatron Controller

The code for our controller might start out with something like this (don’t worry, we’ll walk through some of it):

I know that looks like a lot of code, but there’s quite a bit going on in there. It generates over 250 lines of JSON (when prettified, at least).

Let’s walk through some of the important code. We’re creating a subclass of Metatron::SyncController, which means this controller can handle /sync requests. This defers to the sync method which looks like this:

Metatron offers some handy helper methods to make things easy. For instance request_body is the parsed request body. There are others, but I have some documentation work to do.

Our goal is to construct a list of desired children and provide them in the response body under the key children. To construct this list of children, we call some other methods we created to provide Template instances based on Metatron-provided templates. Calling render on a Template causes it to provide a ruby Hash that can be converted to JSON.

Put another way, as long as sync provides a top-level Hash that has the keys status and children, where status is a Hash and children is an Array of Hashes, things will work. You don’t need to use Metatron templates; just provide a Hash representation of any Kubernetes resource and it’ll work.

To build the DB-related resources for our blogs, we use construct_db_resources, which looks like this:

We’re building a Secret, a StatefulSet, and a Service, each via their own method. Check out those methods to see how to do that via Metatron templates.

For the app-related resources, we use construct_app_resources, which looks like:

Ruby

def construct_app_resources(parent, db_secret) resources = [] app_db_secret = construct_app_secret(parent["metadata"], db_secret) resources << app_db_secret app_deployment = construct_app_deployment( parent["metadata"], parent["spec"], app_db_secret ) resources << app_deployment app_service = construct_service(parent["metadata"], app_deployment) resources << app_service resources << construct_ingress(parent["metadata"], app_service) resources end

1 2 3 4 5 6 7 8 9 10 11 12 13

def construct_app_resources(parent, db_secret)       resources = []       app_db_secret = construct_app_secret(parent["metadata"], db_secret)       resources << app_db_secret       app_deployment = construct_app_deployment(         parent["metadata"], parent["spec"], app_db_secret       )       resources << app_deployment       app_service = construct_service(parent["metadata"], app_deployment)       resources << app_service       resources << construct_ingress(parent["metadata"], app_service)       resources     end

This creates a Secret, Deployment, a Service, and anIngress. There’s also a relationship between some of the DB resources (like for our app Secret) which you can dive into if you’d like.

With these two helper methods, we’ve got quite a list of Kubernetes resources we’ll create! We just collect those up into desired_children and then return those in sync.

That’s all there is to writing our controller! If you have questions about the code, please leave a comment and I can help. There’s a lot of room to expand on that simple example to build some very powerful capabilities into your controller. For what it’s worth, the above code will actually produce a working WordPress site (though you will no doubt need to tweak the domain we generate for it).

Building Our Metatron Controller

Now that we’ve got our controller code written, we need to build and deploy it so Metacontroller can use it. This means we’ll need to build an image and deploy it to Kubernetes. Luckily, this is pretty easy.

Building an image is a matter of running the right command:

Note that the tag there (blogcontroller:latest) will need to be pushed somewhere that Kubernetes can pull it. This is likely a private registry.

Deploying the Controller

There’s nothing particularly special about deploying the controller. This should look like a typical Kubernetes Deployment. First, make a namespace to run the controller and its Service:

Here’s an example of the Kubernetes resources that will get things up and running (be sure to adjust the container image to match wherever you pished the above image):

Now we just need to apply the Kubernetes YAML:

This should create the Deployment and Service that Metacontroller is looking for.

Using the Controller

All that’s left is to make a new Blog! Taking the original example at the beginning of this post, we just need a namespace to use it:

Now to create the blog:

This should create the Blog and all the child resources. You can check on it with things like these commands:

If things went well, you should be able to hit your new fancy site, all thanks to surprisingly little Ruby code.