Kubernoodles

Kubernoodles

Kubernoodles is a reference architecture to demonstrate a lot of “how to securely devops” things, mostly for actions-runner-controller within a larger business. This is how I’ve built and maintain my demo environment.

Why I’ve made certain design choices is based on experiences shared below:

Here’s what has been done and where we’re going.

  1. Initial cluster setup - Kubernetes cluster setup in a managed provider, installing cilium and hubble to power observability, and actions-runner-controller with default runners in a scaling set.
  2. Testing runner scalability - Create a few Actions to test, scale, and debug our self-hosted runners.
  3. What are your users really doing? - Dive into understanding what’s being run in our self-hosted GitHub Actions runners with eBPF and tetragon.
  4. Continuous delivery for custom runners - Setup the Kubernetes cluster, GitHub, and actions-runner-controller to work together, then make a GitHub Actions workflow to create and remove test deployments from a Helm chart.
  5. Building custom runner images - How to build your own custom images for actions-runner-controller!
  6. Building containers in ARC with Kaniko - Using Kaniko in actions-runner-controller to build containers without privileged pods.
  7. Continuous integration for custom runner images - CI for your CI, or how to test your custom runner images on each change.
  8. Writing tests for Actions runners - Test your enterprise CI images with the same rigor as your other software.

Last updated in January 2024 with the updated versions of Kubernetes, actions-runner-controller, Cilium, etc. that I am currently demonstrating.

Coming soon:

  • Rebuilding and releasing custom runners on a schedule
  • Log streaming
  • More fun with eBPF
  • File caching
  • Fun with metrics!

My environment uses Azure services. They’re currently “free” for me and I’ll try to call out using vanilla k8s as much as possible for portability.