The SOPS install + version, the decrypt loop, the OpenTofu version, and
the Backblaze backend-credential extraction were copy-pasted across
terraform.yml (twice), validate-terraform.yml, and _deploy-core.yml.
A version bump meant editing the same string in up to four places and
was easy to do partially.
Pull them into three local composite actions so each is defined once:
- setup-tofu (pins OpenTofu version)
- sops-decrypt (installs SOPS, decrypts *.enc.* in place)
- tofu-backend-creds (exports Backblaze S3 creds to GITHUB_ENV)
Behaviour is unchanged; sops-decrypt also matches *.enc.env everywhere
(previously only _deploy-core did), which is a no-op in terraform/.
The validate-caddyfile workflow fetched the Caddy binary by first hitting
api.github.com/releases/latest to resolve the version tag, then building a
release-asset URL from it. That API call is unauthenticated, so it shares
the 60-requests/hour-per-IP limit across all GitHub-hosted runners and
returns 403 under load. On failure jq emits "null", the URL becomes
caddy_null_linux_amd64.tar.gz, and `curl -sL` silently pipes a 404 page
into tar — a confusing, flaky failure on every PR that touches the Caddyfile.
Switch to Caddy's official download API, which serves the latest linux/amd64
binary directly: one request, no GitHub API, no jq/tar parsing. Add `-f` so
curl fails loudly on an HTTP error instead of writing an error page to disk.
The docker_services and systemd_services roles ran their "start the
service" tasks with `failed_when: false`, so a container or unit that
failed to come up still reported the deploy as green. Drop it from both
start tasks so a broken deploy actually fails CI. The compose/unit *copy*
tasks keep `failed_when: false` — that's load-bearing for the
`item is not failed` filter that skips services without a compose/unit file.
Also:
- Remove a duplicate "Template service .env files" task in docker_services
(second copy used a hardcoded path and didn't register; first one is the
one the start task reads).
- Don't trigger a full fleet deploy on docs/markdown/workflow-only pushes
to main — add docs/**, **/*.md and .github/** to paths-ignore.
- Drop the dangling `update-freebsd` Make target (playbook doesn't exist;
fleet has no FreeBSD hosts).
* chore: add dependabot config
Add Dependabot for the three supported ecosystems in this repo:
GitHub Actions, Terraform (root + grafana/hetzner/pagerduty modules),
and Docker (service compose files + dotfile Dockerfiles). Weekly
schedule with per-ecosystem grouping to keep PR noise down.
* ci: make terraform validation work on dependabot PRs
Dependabot PRs run with no access to repository secrets and a read-only
token, so the SOPS decrypt step (and the PR-comment step) fail. Give
Dependabot a secret-free path: stub the secrets.yaml keys it reads and
run init -backend=false + validate, skipping decrypt/plan/comment. Human
PRs are unchanged and still get a full plan.
* ci: serialize infra runs and enable terraform state locking
Add concurrency guards to the terraform and deploy-on-merge workflows so
two merges in quick succession can't run against the same state or the
same hosts at once (queue, never cancel an in-flight run).
Enable native S3 state locking (use_lockfile) on the Backblaze B2 backend,
which needs OpenTofu 1.10+, so bump the CI tofu version 1.9.0 -> 1.10.10
and the required_version constraint to >= 1.10.0.
* ci: bump tofu to 1.10.10 in the validate workflow too
Missed this one in the last commit — the PR-time validate still pinned
1.9.0, which trips the new required_version >= 1.10.0 constraint.
* ci: drop use_lockfile — Backblaze B2 can't do native state locking
B2's S3 API returns 501 NotImplemented for the conditional PutObject that
use_lockfile relies on, so tofu plan/apply fails to acquire the lock.
Revert the lockfile and the 1.10 version bump it required; rely on the
concurrency guard to serialize applies instead. Left a note in the
backend block so this isn't re-attempted.
Cloudflared tunnels are no longer used. All traffic now routes through
Cloudflare DNS to Caddy on helsinki-a over Tailscale.
- Remove cloudflared systemd unit files (copenhagen-a, london-b)
- Remove cloudflared from media_stack role and copenhagen-a host_vars
- Remove cloudflared references from services README and host docs
- Remove cloudflared deploy trigger from CI workflow
Live service on london-b stopped and disabled. copenhagen-a was
unreachable but the tunnel is unused regardless.
The lint-docker-compose workflow was swallowing all validation errors with
|| true, meaning broken compose files would never fail the check.
- Remove || true and let validation failures propagate
- Add a pre-step that creates empty stubs for referenced env_file entries
(e.g. bitwarden/settings.env) so docker compose config can validate
structure without needing real secrets
- Track per-file pass/fail and exit non-zero if any file fails
Closes PESO-130
- Add copenhagen-a to [docker_hosts] inventory group so the docker role
runs on it in Stage 2
- Add docker_services: [minecraft] to copenhagen-a host_vars
- Add docker_services role to Stage 4d (copenhagen-a) in deploy.yml
- Update deploy-on-merge scope mapping to include copenhagen-a for
docker role changes
Closes PESO-132
* Add systemd_exporter Ansible role and Prometheus scrape config
- Create systemd_exporter role (download binary, create user, deploy service)
- Add scrape job for london-b:9558 and copenhagen-a:9558
- Add systemd_exporter_hosts inventory group
- Add stage 3b to deploy.yml
- Map role to deploy-on-merge scope
Closes PESO-120
* Fix line length lint violations in systemd_exporter tasks
* Fix var-naming lint: use systemd_exporter_ prefix for role variables
Instead of deploying to the entire fleet on every merge, detect which
files changed and limit ansible-playbook to only affected hosts.
Maps ansible roles, services, and host_vars to their target hosts.
Falls back to full fleet deploy for unmapped paths or changes to
shared infrastructure (common role, deploy.yml, inventory).
Closes PESO-108
Both deploy-on-merge.yml and deploy.yml install ansible via pip but
never install the required Galaxy collections (community.docker,
community.general, ansible.posix) from ansible/requirements.yml.
This works by accident because the pip ansible package bundles some
collections, but it's fragile — a pip upgrade or runner image change
could break deploys silently.
Fixes PESO-110
* Add Docker official apt repo to docker role
The docker role was installing docker-compose-plugin which is only
available from Docker's official apt repository. helsinki-a had it
configured manually, but london-b and copenhagen-a did not, causing
deploy failures.
Now the role:
- Adds Docker's GPG key and apt repo (handles both Debian and Ubuntu)
- Installs docker-ce, docker-ce-cli, containerd.io, docker-compose-plugin
- Removes conflicting stock packages (docker.io, docker-compose)
* fix: resolve yamllint violations in docker role
- Remove standalone comment blocks that caused indentation errors
- Collapse multiline repo string to single line
- Ensure document start marker is present
* fix: keep all lines under 160 chars for yamllint
Use set_fact to build the Docker repo line in parts instead of
one long inline string.
* fix: resolve yamllint errors in london-b host_vars and promtail config
- Remove trailing blank line in inventory/host_vars/london-b.yml
- Add missing document start marker to promtail config
- Fix indentation in promtail scrape_configs (indent list items under key)
* Remove ansible-lint on push, keep PR-only
Lint already runs on pull_request — no need to double up on push to main.
