The Arminius Helm Chart

The eventual goal is to have Arminius deployable on Kubernetes. Therefore, we are currently developing a helm chart. This helm chart is not production ready, but just the least amount of helm chart we could write to get something that runs, such that we can start work on all the tooling around kubernetes that will be necessary.

The application chart lives in charts/arminius. Cluster dependencies for local development (Postgres, NATS, Gateway API, optional Zitadel) live in charts/dev-services.

Running the Helm Chart with Kind and Tilt

These instructions assume that you have a Docker daemon running on your system. Additionally, you’ll need kind, tilt, kubernetes, and an array of other packages which are included in the Nix devshell.

First, you’ll want to add the following line to your /etc/hosts :

127.0.0.1 matrix.local zitadel.local openobserve.local

This is needed because Zitadel needs its own hostname, distinct from your Matrix server. If you’re running NixOS, you can make this change permanent with

networking.extraHosts = "127.0.0.1 matrix.local zitadel.local";

Afterwards, you need to use tilt to start the homeserver:

kind create cluster --config kind_config.yaml
helm repo add nats https://nats-io.github.io/k8s/helm/charts/
helm dependency build charts/dev-services
tilt up

To log in, we recommend first opening https://matrix.local/_matrix/federation/v1/version in a Chrome-based browser to create an exception for the self-signed TLS certificate. Afterwards, you will be able to select the homeserver matrix.local on app.element.io. You can log in with the Zitadel admin account, with the username zitadel-admin@zitadel.zitadel.local and password Password1!.

If you want to look at the traces OpenObserve is started at openobserve.local with the username root@example.com and password Complexpass#123.

To delete your cluster again, use the command kind delete cluster.

On macOS

There is no native aarch64-darwin container runtime, so the solution on macOS is generally to run an aarch64-linux VM which you then install kind into.

However, in order to be able to build these images on darwin using the nix flake, you need to use the nix-darwin linux builder. If using nix-darwin, see below:

nix.linux-builder = {
  enable = true;
  config.virtualisation = {
    cores = 6;
    darwin-builder.memorySize = 24 * 1024;
    # The default 20G is not enough: the workspace's target directory alone
    # goes past that.
    darwin-builder.diskSize = 100 * 1024;
  };
};

Check it works with nix build nixpkgs#legacyPackages.aarch64-linux.hello. The first tilt up then builds the whole Rust workspace inside that VM, which takes a while; afterwards only changed crates are rebuilt.

Hacking

Famedly’s Helm Chart Guidelines can be found here

Chart values are validated against charts/arminius/values.schema.json (JSON Schema Draft-07), generated with helm schema --draft 7. After changing values.yaml annotations, regenerate the schema and commit the result.

Troubleshooting, Inspecting Your Cluster

When you have a cluster up and running, you can look at it using kubectl. This is not a complete list of commands available in kubectl. kubernetes is very big and complicated. But if something fails in the previous setup, these are probably the commands you’d run to figure out what.

`kubectl get pods`

The command kubectl get pods prints all pods which are running on the cluster in the selected namespace. Resource names are derived from the helm release name, which the Tiltfile pins to arminius for the application chart. The Arminius pods should look like this, alongside the charts/dev-services pods (Postgres, NATS, the Gateway, and Zitadel):

NAME                                                       READY   STATUS    RESTARTS        AGE
deploy-arminius-admin-7df7dc99c7-2zkvx                     1/1     Running   1 (5m37s ago)   6m9s
deploy-arminius-client-64bd9f4dd-blzv6                     1/1     Running   1 (5m37s ago)   6m9s
deploy-arminius-federation-input-69484689f4-kdwx5          1/1     Running   1 (5m37s ago)   6m9s
deploy-arminius-federation-output-5c9f8b7d64-w4np2         1/1     Running   1 (5m37s ago)   6m9s
deploy-arminius-nuss-6b8c47d5f9-x2mrt                      1/1     Running   0               6m9s
deploy-arminius-scim-7c4d9f8b56-qk8vn                      1/1     Running   1 (5m37s ago)   6m9s
deploy-arminius-rooms-7d75d8f654-dn2gx                     1/1     Running   2 (5m7s ago)    6m9s
deploy-arminius-rooms-7d75d8f654-pqhwc                     1/1     Running   2 (5m7s ago)    6m9s
deploy-arminius-rooms-7d75d8f654-r7tpz                     1/1     Running   1 (5m37s ago)   6m9s
deploy-arminius-rooms-7d75d8f654-z5skk                     1/1     Running   2 (5m7s ago)    6m9s

If the STATUS of some pod is not Running as expected, you can use the kubectl describe pod <NAME> to inspect the state of your pod, and kubectl logs <NAME> to get logs from it.

Rootless Container with Podman

If you run kind with rootless Podman, the cluster cannot bind the privileged ports 80 and 443 that the Gateway needs, so lower the unprivileged port start:

echo 'net.ipv4.ip_unprivileged_port_start=80' | sudo tee /etc/sysctl.d/99-rootless-ports.conf
sudo sysctl --system

The `dev-services` Helmchart

For dev-services an additional Helm-Chart is provided. It provides the same purpose as the other options for development setups and covers the following services:

NATS (for dev purposes)
A CNPG cluster for Arminius (for dev purposes)
Single-Node Postgres for Nuss (for dev purposes)
Zitadel (for dev purposes)
Gateway API configuration resources
OpenObserve (for dev purposes)

Scope of the Dev-Services Helmchart

The Dev-Services shall not be used for production services. It serves the following purposes:

run in local Nix/Tilt driven environments
run in Development Deployments

For flexible integration into local/remote pre-provisioned clusters which come often with some services by default (like Gateway resources), it’s necessary to make all components in the Dev-Services toggleable.

Keyboard shortcuts

Arminius Documentation