pez-infra

mirror of https://github.com/RWejlgaard/pez-infra.git synced 2026-10-31 19:11:03 +00:00

History

Rasmus "Pez" Wejlgaard 72970e7675 Fix Alloy state ownership after the 1.19 upgrade (#154 ) The 1.19 package runs alloy.service as User=alloy; 1.14 ran it as root. The state left behind in /var/lib/alloy is still root-owned, so the service user can't write its WAL or positions.yml, every remotecfg pipeline fails to build, and the host falls back to a cached config and stops shipping. That took node metrics off london-b, helsinki-a, london-c and nuremberg-a from last night's deploy. I noticed because the ZFS pool state metric vanished off london-b, which is the only ZFS host and so the only visible symptom. Chown the state dir to the service user when anything in it isn't already owned by them, then restart. The detect step keeps it a no-op on hosts that are already correct, so london-a and copenhagen-a don't get churned.	2026-09-19 09:07:01 +01:00
..
main.yml	Fix Alloy state ownership after the 1.19 upgrade (#154 )	2026-09-19 09:07:01 +01:00

Fix Alloy state ownership after the 1.19 upgrade (#154 )

The 1.19 package runs alloy.service as User=alloy; 1.14 ran it as root. The
state left behind in /var/lib/alloy is still root-owned, so the service user
can't write its WAL or positions.yml, every remotecfg pipeline fails to build,
and the host falls back to a cached config and stops shipping.

That took node metrics off london-b, helsinki-a, london-c and nuremberg-a from
last night's deploy. I noticed because the ZFS pool state metric vanished off
london-b, which is the only ZFS host and so the only visible symptom.

Chown the state dir to the service user when anything in it isn't already
owned by them, then restart. The detect step keeps it a no-op on hosts that
are already correct, so london-a and copenhagen-a don't get churned.

2026-09-19 09:07:01 +01:00

main.yml

Fix Alloy state ownership after the 1.19 upgrade (#154 )

2026-09-19 09:07:01 +01:00