pez-infra/docs
Rasmus Wejlgaard 3212930949 Install Grafana Alloy from Ansible (PESO-158)
copenhagen-a came back from its Proxmox repave with no Alloy at all, so it's
been unmonitored since. Nothing in this repo installs Alloy — every host got it
from Grafana's onboarding one-liner, and the common role only bolted on a
resilience drop-in if the unit already happened to exist. A rebuild silently
loses monitoring.

New alloy role owns the whole thing: the Grafana apt repo, the package, the
config, the credentials drop-in and the resilience override. Runs on every
host, so this can't happen again on the next repave.

Moved the resilience drop-in out of common into the role that owns Alloy.

The rendered config is byte-identical to what's on the hosts today apart from
the managed-by header, so the only effect on the healthy hosts is one restart.
2026-09-18 20:45:33 +01:00
..
hosts Back up the Proxmox VMs to london-b (#151) 2026-09-17 17:12:16 +01:00
architecture.md decomm copenhagen-b and copenhagen-c (#144) 2026-07-28 10:54:23 +01:00
getting-started.md decomm copenhagen-b and copenhagen-c (#144) 2026-07-28 10:54:23 +01:00
monitoring.md Install Grafana Alloy from Ansible (PESO-158) 2026-09-18 20:45:33 +01:00
networking.md decomm copenhagen-b and copenhagen-c (#144) 2026-07-28 10:54:23 +01:00
README.md decomm copenhagen-b and copenhagen-c (#144) 2026-07-28 10:54:23 +01:00
secrets.md docs: catch up with the Cloudflare to Hetzner DNS move, fix secrets/terraform drift (#130) 2026-06-10 20:59:23 +01:00
services.md Copenhagen b and c decomm (#145) 2026-07-28 21:24:59 +01:00

Documentation

Everything you need to understand how this infrastructure works.

Contents

  • Architecture — High-level overview, network topology, traffic flow diagrams
  • Networking — Tailscale mesh, physical networking, DNS and proxy flow
  • Services — Complete service map: what runs where, ports, auth
  • Monitoring — Grafana Cloud, Alloy, synthetic checks, alerting via PagerDuty
  • Secrets — SOPS + age encryption: setup, usage, CI integration
  • Getting Started — How to work with this repo, deploy changes, add services
  • Hosts — Per-host detail (hardware, services, quirks)

Quick Reference

Host Tailscale IP Location Role
helsinki-a 100.67.6.27 Hetzner Cloud (Helsinki) Reverse proxy, SSO, Bitwarden, Forgejo
london-a 100.122.180.98 London Proxmox VE hypervisor
london-b 100.84.65.101 London Storage, media, Docker services
london-c 100.123.72.87 London Raspberry Pi, Octopus Energy exporter
nuremberg-a 100.70.180.24 Hetzner Cloud (Nuremberg) Mail (poste.io)
copenhagen-a 100.91.240.29 Copenhagen Proxmox VE hypervisor