Prometheus

Metrics storage and querying for BlumeOps infrastructure.

Quick Reference

PropertyValue
URLhttps://prometheus.ops.eblu.me
Namespacemonitoring
Imageregistry.ops.eblu.me/blumeops/prometheus (see argocd/manifests/prometheus-ringtail/kustomization.yaml for current tag)
Storage20Gi PVC
Manifestsargocd/manifests/prometheus-ringtail/
ArgoCD appprometheus-ringtail

Data Sources

Remote Write (from Alloy)

  • Indri system metrics via Alloy remote_write
  • Ringtail Kubernetes metrics (kube-state-metrics, node, kubelet) via the alloy-ringtail DaemonSet
  • Textfile metrics: borgmatic, zot, jellyfin, forgejo
  • flyio-proxy nginx metrics (flyio_nginx_*) via Alloy embedded in the Fly.io container

Scrape Targets

JobTarget
node-exporter-sifakanas.ops.eblu.me:9100 — sifaka NAS (via Caddy L4 on indri)
smartctl-sifakanas.ops.eblu.me:9633 — sifaka NAS (via Caddy L4 on indri)
cnpg-postgresblumeops-pg-metrics.databases.svc:9187, immich-pg-metrics.databases.svc:9187 — postgresql
prometheuslocalhost:9090 — self-monitoring
lokiloki.monitoring.svc:3100
tempotempo.monitoring.svc:3200
argocdargocd-metrics.argocd.svc:8082
frigatenvr.ops.eblu.me/api/metrics
birdnet-gobirdnet-go.birdnet-go.svc:8090

kube-state-metrics is deliberately not scraped here: alloy-ringtail scrapes and remote-writes it instead.