Clusters from Secrets

When Lumovi runs in a cluster, it can find its fleet’s clusters in that cluster’s Secrets, each tool’s way:

lumovi: Secrets you make, each with a kubeconfig.
cluster-api: the kubeconfig Cluster API keeps for each cluster it makes.
argocd: Argo CD’s cluster Secrets, for the clusters it deploys to.

Lumovi lists them with its own service account when it starts, and again every 30 seconds (LUMOVI_FLEET_REFRESH_SECONDS), so clusters come and go as their Secrets do.

Turn it on

With the Helm chart, name the sources, and the namespaces their Secrets are in:

values.yaml

fleet:
  secrets: [lumovi, cluster-api, argocd]
  # Where those Secrets are: the release's namespace unless set.
  secretNamespaces: [lumovi, clusters, argocd]

The chart makes a Role in each of those namespaces that lets Lumovi’s service account list Secrets, and binds it, so each namespace must exist before you install. Lumovi can then list every Secret there, not only the ones that describe clusters: keep those namespaces to administrators. Without the chart, set LUMOVI_FLEET_SECRETS (lumovi,cluster-api,argocd) and LUMOVI_FLEET_SECRETS_NAMESPACES (comma-separated, the service account’s namespace unless set), and give the service account list on Secrets there. This works only inside a cluster. Outside one, Lumovi doesn’t start:

LUMOVI_FLEET_SECRETS reads Secrets of the cluster Lumovi runs in, and it isn't running in one (KUBERNETES_SERVICE_HOST isn't set).

Secrets that can’t be listed now keep the clusters they last described, as a kubeconfig file that can’t be read does, and the log says why, once:

Fleet: Secrets for argocd in argocd can't be listed: …

A Secret that doesn’t describe a cluster as it should is still shown, as Misconfigured, saying why: under the Secret’s own name when Lumovi can’t tell the cluster’s. A cluster from a Secret is named in the log by where it came from, like Secret argocd/cluster-prod-us.

Lumovi’s own

A Secret labelled lumovi.dev/cluster (with any value), with a kubeconfig under kubeconfig. Each context of the kubeconfig is a cluster, called by the context’s name, with Lumovi’s settings in its extension if it has them, prefixes included.

apiVersion: v1
kind: Secret
metadata:
  name: prod-eu
  namespace: lumovi
  labels:
    lumovi.dev/cluster: ''
  annotations:
    lumovi.dev/labels: env=production,region=eu-west
type: Opaque
stringData:
  kubeconfig: |
    apiVersion: v1
    kind: Config
    clusters:
      - name: prod-eu
        cluster:
          server: https://prod-eu.example.com:6443
          certificate-authority-data: LS0tLS1CRUdJTi…
    users:
      - name: prod-eu
        user:
          token: eyJhbGciOiJSUzI1NiIs…
    contexts:
      - name: prod-eu
        context: { cluster: prod-eu, user: prod-eu }

Or from a member’s kubeconfig:

kubectl create secret generic prod-eu --namespace lumovi --from-file kubeconfig=prod-eu.yaml
kubectl label secret prod-eu --namespace lumovi lumovi.dev/cluster=

A Secret without a kubeconfig is shown under its own name, saying It has no kubeconfig (data.kubeconfig)., and one whose kubeconfig isn’t one says so, like Secret lumovi/prod-eu isn't a kubeconfig: …. Paths in it would be read in Lumovi’s container, which has none of your files: embed certificates with the *-data fields.

Cluster API’s

Cluster API keeps each cluster’s kubeconfig in a Secret called <cluster>-kubeconfig, under value, in the namespace of its Cluster, and labels it, like the cluster’s other Secrets, with cluster.x-k8s.io/cluster-name. Lumovi reads the -kubeconfig ones, uses their kubeconfig’s first context, and calls each cluster by that label. It skips the cluster’s other Secrets, and a -kubeconfig one that has no value or no context yet. List the namespaces your Cluster objects are in. When Cluster API replaces a kubeconfig, Lumovi reads the new one at its next look.

Cluster API’s kubeconfigs sign in as each cluster’s administrator. Lumovi uses them only to impersonate the people who sign in, so their RBAC applies, but Lumovi then holds administrator credentials for every one of those clusters. For credentials that may only impersonate, add the clusters as members instead.

Argo CD’s

Argo CD keeps each cluster it deploys to in a Secret labelled argocd.argoproj.io/secret-type: cluster, with its name, its server, and a config that says how to sign in. Lumovi calls each cluster by its name, and reads from config:

Field	What Lumovi does with it
`bearerToken`	Signs in with it.
`username`, `password`	Signs in with them.
`tlsClientConfig`	`caData`, `certData`, `keyData`, `serverName` and `insecure`, as Argo CD does.
`execProviderConfig`	Runs its `command`, with its `args` and `env`. The command must be in Lumovi’s image, which has only Node.js and helm: otherwise the card says why it couldn’t be run.
`awsAuthConfig`	Nothing: the cluster’s card says `Argo CD signs in to it with AWS (awsAuthConfig), which Lumovi can't: give it a bearerToken.`

The Secret’s own labels become the cluster’s, except Argo CD’s (argocd.argoproj.io/…). A Secret without a name or a server is shown under its own name, saying It has no name or server., and one whose config isn’t JSON is shown under its own name as Misconfigured. One with none of these credentials, like the entry Argo CD may keep for the cluster it runs in, says Lumovi has no credentials for it…. Argo CD’s credentials are usually its argocd-manager service account’s, which may do anything in the cluster. As with Cluster API’s, Lumovi uses them only to impersonate.

Lumovi’s settings, as annotations

Any of these Secrets can carry Lumovi’s settings for its clusters as annotations, for every cluster it describes. Each one that’s set wins over a cluster’s own setting, from its kubeconfig’s extension or its Argo CD labels.

Annotation	What it sets
`lumovi.dev/labels`	Labels, like `env=production,region=eu-west`. They’re added to the cluster’s own, and win over one with the same key.
`lumovi.dev/groups`	Only people in these groups see the cluster, separated by commas, like `sre, developers`: groups as your provider or proxy names them, without prefixes.
`lumovi.dev/forward-token`	`true`: requests carry each person’s own token, instead of impersonating them. Anything else: Lumovi impersonates them.

A Secret whose lumovi.dev/labels isn’t like this is shown under its own name, as Misconfigured, saying why, like lumovi.dev/labels must be labels like env=production,region=eu, not "staging". There are no annotations for prefixes. A cluster in Lumovi’s own Secret can have its own in its kubeconfig’s extension; Cluster API’s and Argo CD’s take the server’s.

A fleet of clusters

The other ways to describe clusters, and the fleet page.

Helm values

The chart’s fleet values.

Run it in your cluster

Sign-in

Fleet

Operate

Reference

Clusters from Secrets

Turn it on

Lumovi’s own

Cluster API’s

Argo CD’s

Lumovi’s settings, as annotations

A fleet of clusters

Helm values

​Turn it on

​Lumovi’s own

​Cluster API’s

​Argo CD’s

​Lumovi’s settings, as annotations

A fleet of clusters

Helm values

Turn it on

Lumovi’s own

Cluster API’s

Argo CD’s

Lumovi’s settings, as annotations