BlumeOps Docs

Grafana

2 min read

Grafana

Dashboards and visualization for BlumeOps observability.

Quick Reference

PropertyValue
URLhttps://grafana.ops.eblu.me
Tailscale URLhttps://grafana.tail8d86e.ts.net
Namespacemonitoring
DeploymentKustomize (argocd/manifests/grafana-ringtail/)
Imageregistry.ops.eblu.me/blumeops/grafana
Sidecar Imageregistry.ops.eblu.me/blumeops/grafana-sidecar

Authentication

Grafana supports two login methods:

  • SSO via authentik — OIDC login through Authentik (auth.generic_oauth). Members of the Authentik admins group get the Admin role; everyone else gets Viewer (role_attribute_path in grafana.ini).
  • Local admin — break-glass login using the password from 1Password (“Grafana (blumeops)”). Always available if Authentik is down.

The OIDC endpoints live in the [auth.generic_oauth] section of argocd/manifests/grafana-ringtail/grafana.ini (fed to the grafana ConfigMap via configMapGenerator). The OIDC client secret is injected via external-secrets — argocd/manifests/grafana-config-ringtail/external-secret-authentik-oauth.yaml pulls grafana-client-secret from the 1Password “Authentik (blumeops)” item, surfacing it as the grafana-authentik-oauth secret in the monitoring namespace.

Upgrading

To upgrade Grafana:

  1. Update version = "..." in containers/grafana/default.nix (see build-grafana-images) and the grafana entry in service-versions.yaml in the same PR.
  2. Merge — the push to main builds the image at the merge commit and pushes it to the registry; the horkos publisher opens a pin PR with the new tag in argocd/manifests/grafana-ringtail/kustomization.yaml (under images:).
  3. Merge the pin PR — that deploys.

The SQLite PVC is disposable — dashboards come from ConfigMaps and datasources from provisioning config.

Datasources

NameTypeTarget
Prometheusprometheusprometheus.monitoring.svc.cluster.local:9090
Lokilokiloki.monitoring.svc.cluster.local:3100
Tempotempotempo.monitoring.svc.cluster.local:3200
TeslaMatepostgrespg.ops.eblu.me:5434 (TeslaMate’s database on ringtail, via Caddy L4)

Dashboard Provisioning

Dashboards are ConfigMaps with label grafana_dashboard: "1".

Location: argocd/manifests/grafana-config-ringtail/dashboards/

Optional annotation: grafana_folder: "FolderName"

Key Dashboards

Provisioned dashboards live in argocd/manifests/grafana-config-ringtail/dashboards/ (one ConfigMap per dashboard). Coverage as of 2026-06: alerts, borgmatic, CV APM, devpi, docs APM, fly.io proxy, forgejo, frigate, jellyfin, kubernetes, loki, macOS (indri host), postgresql, ringtail, sifaka disks, snowflake proxy, tempo, transmission, zot.

TeslaMate’s dashboards are not in the repo — an init container fetches them from the forge mirror at a pinned tag (TESLAMATE_VERSION in argocd/manifests/grafana-ringtail/deployment.yaml).

Graph View

Backlinks