Kubernetes Node Affinity and EBS Volumes

Occasionally, Kubernetes workloads require specialized nodes. Sometimes it’s for machine learning, or to save money through burstable node types, or maybe just to lock certain Pods to a dedicated subset of nodes. Thankfully, Kubernetes offers a few useful mechanisms to inform the scheduler how we’d like our workloads distributed: node-based or pod-based affinity rules along with taints and tolerations. I’ll go over how to use these briefly, but I use these frequently at work for numerous reasons. Recently, I realized something interesting about how Persistent Volume Claims (PVCs) work with dynamically provisioned storage like EBS volumes (meaning volumes that are created automatically by Kubernetes based on a StorageClass, rather than referencing existing volumes). The default behavior of a StorageClass is to immediately create a volume as soon as the PVC is created. This can have some consequences when trying to guide how Pods are scheduled.

Continue reading Kubernetes Node Affinity and EBS Volumes

RSpec Testing for Ruby AWS Lambda Functions

Recently, I wrote an AWS Lambda function at work in Ruby but I didn’t have a handy tool for creating a project skeleton like bundle gem does. That means nothing bootstrapped my testing for me. While copy+pasting code into pry proved that my simple function worked, that wasn’t quite good enough. What I really wanted was the ability to use RSpec with my Lambda code. After a cursory search of the Internet for some examples, I was left disappointed with how little I found. So, I rolled up my sleeves and figured it out.

Continue reading RSpec Testing for Ruby AWS Lambda Functions

Meet Bullion, An ACMEv2 Certificate Authority

I’m a huge fan of Let’s Encrypt and what they’ve done to secure the Internet. They’ve made safe communication free and open. Through their ACME protocol (and subsequent ACMEv2 protocol), they have changed PKI and the way we look at automating certificate provisioning for good. All that said, Let’s Encrypt only really helps for public-facing services; for internal domains and services, I want a solution that doesn’t require creating public records. I also don’t want to reinvent all the tooling; I want to be able to use ACMEv2 (through things like cert-manager on Kubernetes or certbot for EC2). Enter Bullion, an unassuming Ruby ACMEv2 Certificate Authority built specifically for internal domains.

Continue reading Meet Bullion, An ACMEv2 Certificate Authority

Connected: A Ruby Graph Search Library

A common problem posed in coding competitions and undergraduate computer science classes is implementing an algorithm for finding the shortest path through a series of interconnected nodes. It can be phrased in many ways, mostly because graph theory (which is the domain of this particular problem) applies to so many areas of life. Things like driving directions, routing packets in a network, shipping logistics, and so many NP problems are examples of the same problem. It turns out that finding the shortest/lowest cost path through a graph is difficult. While Edsger Dijkstra wrote his famous algorithm to solve this problem back in 1956, there aren’t many libraries to actually use it, at least not for Ruby and in a practical sense. Sure, there are plenty of academic demonstrations but I couldn’t find anything that I would want to depend on as a library (it turns out that rgl is pretty great). This inspired me to write connected, which can be used to search weighted, directed, or undirected graphs.

Continue reading Connected: A Ruby Graph Search Library

Rewriting Ruby’s #pack and #unpack Methods in Crystal

In Ruby, especially while writing low-level protocols, you may have encountered Array#pack() and/or String#unpack(). If you’ve ever experimented with them, they may seem mysterious; they take a cryptic string as a parameter (maybe something like C4 or w*) and seem to return gibberish (or, conversely, convert gibberish into something humans can understand). I’m not going to go into tons of detail, nor am I going to cover every format that these methods accept, but I’ll cover a few that I’ve used.

What led me to writing this post was actually some recent experimenting that I’ve been doing with Crystal. I’m trying my hand at writing a simple BER parser/encoder as a part of my journey to writing an LDAP library for Crystal. The lack of an LDAP library is honestly the biggest reason I haven’t used Crystal for more things. Since BER is a binary method of encoding, the Ruby LDAP BER code uses a ton of Array#pack() and String#unpack(). Unfortunately, Crystal doesn’t have analogous methods for its Array class or String struct so I’ve had to write my own.

Here, I’ll describe a few of the formats supported by #pack and write some compatible examples in Crystal.

Continue reading Rewriting Ruby’s #pack and #unpack Methods in Crystal

Redis, Ruby, and Some Surprising Uses

Any developer worth their salt knows that Redis is great for caching. As an in-memory cache, it gets the job done. You certainly don’t have to take my word for it; the major sponsors of Redis (redislabs) wrote a white paper to explain it. What isn’t quite as widely known is that Redis has some other uses worth considering. I’ll list the ones I’m aware of (and have used) which are all available with open-source Redis.

Continue reading Redis, Ruby, and Some Surprising Uses

Updating GoDaddy DNS Entries with Ruby

I bought a new (arguably better) domain for my blog now! If you’re reading this, you’ve probably noticed, but it is therubyist.org, because I’m a fan of Ruby. The old name won’t be going anywhere, at least for the time being. Given the purchase of this new domain, I have several domains I need to maintain. Since I run this blog (and a few other services) from my home server which has a dynamic IP, setting up the domain apex (i.e., “naked domain”) is tricky. I’ve been using a dynamic DNS service called duckdns.org which gets the job done for subdomains since I can just CNAME to my personal DuckDNS subdomain, but it doesn’t solve this apex problem.

I decided that I was tired of doing this manually and that I would try to write a script. It would be appropriate, given my blog’s new domain name, to write it in Ruby. I did a quick DuckDuckGo search and noticed that someone wrote a GoDaddy SDK for Ruby. Yes, I use GoDaddy for my DNS… not necessarily endorsing them, but they seem to work well for my needs, especially now that I’ve discovered that they offer RESTful APIs.

Continue reading Updating GoDaddy DNS Entries with Ruby

Read-only Docker Containers

There are lots of good reasons for and articles recommending running Docker containers read-only, but what I have a difficult time finding are descriptions of how to do this for many popular images. Some software needs to write to a few important and predictable locations. It surprises me how often image providers neglect to offer instructions or details required to run their image this way.

Even setting aside read-only containers, counting on writing to the writable layer just feels wrong. Per the documentation, for the writable layer, both read and write speeds are lower because of the copy-on-write/overlay process through the storage driver. In my experience, docker diff output means I haven’t taken the time to configure my volume declarations, either through tmpfs mounts, volumes, or bind mounts.

Continue reading Read-only Docker Containers

Distributing CLI Tools via Docker

Throughout my career, I’ve seen a couple recurring patterns related to the tools I write: I write a lot of small CLI tools and I like to share them with my coworkers (and whenever possible, the rest of the world).

This has led to several iterations of solving the problem How do I make this tool easy to run? since I don’t want to burden people with understanding the intricacies of all my tools’ dependencies. These tend to be Ruby, some number of gems, and possibly some other common unix utilities. The solutions I’ve come up with have included a lengthy README with detailed instructions, Bundler with Rake tasks to do all the heavy lifting for non-Ruby things, fpm, and even “curl bash piping” (yes, I’m horrible).

Recently I decided to use Docker to solve this problem, since I’m using it so much anyway. Using Docker has some huge benefits for sharing applications of all types: the dependencies list gets whittled down to just Docker, things work on more platforms, testing gets simpler, and it is the new hotness which makes people say “whoa” and that’s fun. That said, the downsides can be frustrating: working with files on your machine gets messy, more typing with the extra Docker-related preamble, things are less straightforward and clear, simple mistakes can lead to lots of images and containers to clean up, and the executable gets significantly larger (since the Docker image is a whole, albeit lightweight, OS userland to run the app). After weighing these pros and cons, I’ve found that telling a coworker to docker pull registry.url/my/app and run it with --help is so much more convenient than the alternatives.

Continue reading Distributing CLI Tools via Docker