ci(deploy): deploy the stack to Talos on merge to main (closes #175) #176

Merged
not merged 1 commits from ci/175-deploy-on-merge into main 2026-09-18 13:53:59 +00:00
Contributor

What & why

The chart has been deployable by hand since #25 and linted in CI since #168. This makes a
merged PR actually ship it to the Talos VM on the lab server.

.gitea/workflows/deploy.yaml runs on a push to main (a squash-merged PR) and on manual
dispatch:

  1. Tunnel — neither the Kubernetes API nor the in-cluster registry is publicly reachable,
    so 6443, 30500 and 30141 are forwarded over the same SSH hop into the Fedora host that the
    Gitea-runner pipeline uses (ssh -p 6667 user@labs.respellion.tech).
  2. Imagesmake k8s-images K8S_REGISTRY=localhost:30500, pushed through the tunnel.
  3. Deploymake k8s-reseed TALOS_HOST=… K8S_REGISTRY=<vm-ip>:30500, pulled by the node
    from its own NodePort.
  4. Rollrollout restart + rollout status on the nine repo deployments.
  5. SmokeGET /openbaar/register through the openbaar portal.

Three decisions worth the review:

  • One registry, two names. The push target (localhost:30500, the tunnel) and the pull
    target (<vm-ip>:30500, the node's own NodePort) address the same store. The pull name has
    to be the one in the node's registry-mirror patch, which is what makes plain HTTP acceptable.
  • k8s-reseed, not k8s-up. A Job's pod template is immutable, so a chart change to any
    bootstrap Job would otherwise fail the upgrade with cannot patch … with kind Job. The Jobs
    are idempotent by design, so re-running them every deploy is safe and removes that whole
    class of failure. Cost: a few minutes per deploy, and seed-zaaktype needs egress from the VM.
  • No re-run of the checks. PR CI is the merge gate, so main is green by construction.
    Deploys queue (cancel-in-progress: false) — a helm upgrade killed half-way leaves the
    release in pending-upgrade and has to be unwedged by hand.

Settings on the repo (already added): secrets TALOS_SSH_KEY and TALOS_KUBECONFIG
(base64, and its server: must be https://127.0.0.1:6443 — Talos puts 127.0.0.1 in the
apiserver cert SANs, so TLS still verifies through the tunnel); variables TALOS_VM_IP
(default 192.168.122.173) and TALOS_HOST (default localhost).

Closes #175

Definition of Done

  • Linked Gitea issue (above).
  • Failing test committed before the implementation — n/a: this is a deployment
    workflow with no unit under test. Its check is the run itself: rollout status and the
    public-register smoke both have to pass or the job fails. make k8s-lint / make k8s-drift
    (#168) already gate the chart it deploys.
  • Implementation — one workflow file, no production code touched.
  • Conventional Commits referencing the issue (refs #175).
  • CI green — awaiting the run on this PR.
  • docker compose up unaffected — no service, image or compose file is touched.
  • Docs updated — docs/runbooks/kubernetes-talos.md §9 (the tunnel, the two registry
    names, the secrets table, the smoke) and a pointer from docs/runbooks/ci.md.
  • No ADR needed: no new dependency (kubectl/helm/crane are already prerequisites of the
    k8s-* targets), no service boundary moved, no CLAUDE.md §8 rule bent.
  • Demo note — not user-visible.

Notes for reviewers

  • The first deploy is the real test. It cannot be dry-run: the tunnel, the secrets and the
    registry only exist on the lab server. Merging is how we find out; Pods on failure dumps
    get pods,jobs if it doesn't.
  • Known gap — the portals still aren't browsable. PKCE needs a secure context, so a
    NodePort on an IP can't serve them (runbook §5); they need make k8s-portals or an SSH
    forward. Giving the server a hostname + TLS is the follow-up, and is where TALOS_HOST
    stops defaulting to localhost.
  • Databases are emptyDir. Any change to a database pod's template wipes it; the
    k8s-reseed in the deploy re-runs the bootstrap, so the stack recovers, but submitted
    registrations do not. Persistence is runbook §6.

🤖 Generated with Claude Code

## What & why The chart has been deployable by hand since #25 and linted in CI since #168. This makes a merged PR actually ship it to the Talos VM on the lab server. `.gitea/workflows/deploy.yaml` runs on a push to `main` (a squash-merged PR) and on manual dispatch: 1. **Tunnel** — neither the Kubernetes API nor the in-cluster registry is publicly reachable, so 6443, 30500 and 30141 are forwarded over the same SSH hop into the Fedora host that the Gitea-runner pipeline uses (`ssh -p 6667 user@labs.respellion.tech`). 2. **Images** — `make k8s-images K8S_REGISTRY=localhost:30500`, pushed *through* the tunnel. 3. **Deploy** — `make k8s-reseed TALOS_HOST=… K8S_REGISTRY=<vm-ip>:30500`, pulled by the node from its own NodePort. 4. **Roll** — `rollout restart` + `rollout status` on the nine repo deployments. 5. **Smoke** — `GET /openbaar/register` through the openbaar portal. Three decisions worth the review: - **One registry, two names.** The push target (`localhost:30500`, the tunnel) and the pull target (`<vm-ip>:30500`, the node's own NodePort) address the same store. The pull name has to be the one in the node's registry-mirror patch, which is what makes plain HTTP acceptable. - **`k8s-reseed`, not `k8s-up`.** A Job's pod template is immutable, so a chart change to any bootstrap Job would otherwise fail the upgrade with `cannot patch … with kind Job`. The Jobs are idempotent by design, so re-running them every deploy is safe and removes that whole class of failure. Cost: a few minutes per deploy, and `seed-zaaktype` needs egress from the VM. - **No re-run of the checks.** PR CI is the merge gate, so `main` is green by construction. Deploys **queue** (`cancel-in-progress: false`) — a `helm upgrade` killed half-way leaves the release in `pending-upgrade` and has to be unwedged by hand. Settings on the repo (already added): secrets `TALOS_SSH_KEY` and `TALOS_KUBECONFIG` (base64, and its `server:` must be `https://127.0.0.1:6443` — Talos puts `127.0.0.1` in the apiserver cert SANs, so TLS still verifies through the tunnel); variables `TALOS_VM_IP` (default `192.168.122.173`) and `TALOS_HOST` (default `localhost`). Closes #175 ## Definition of Done - [x] Linked Gitea issue (above). - [ ] Failing test committed before the implementation — **n/a**: this is a deployment workflow with no unit under test. Its check is the run itself: `rollout status` and the public-register smoke both have to pass or the job fails. `make k8s-lint` / `make k8s-drift` (#168) already gate the chart it deploys. - [x] Implementation — one workflow file, no production code touched. - [x] Conventional Commits referencing the issue (`refs #175`). - [ ] CI green — awaiting the run on this PR. - [x] `docker compose up` unaffected — no service, image or compose file is touched. - [x] Docs updated — `docs/runbooks/kubernetes-talos.md` §9 (the tunnel, the two registry names, the secrets table, the smoke) and a pointer from `docs/runbooks/ci.md`. - [x] No ADR needed: no new dependency (kubectl/helm/crane are already prerequisites of the `k8s-*` targets), no service boundary moved, no CLAUDE.md §8 rule bent. - [ ] Demo note — not user-visible. ## Notes for reviewers - **The first deploy is the real test.** It cannot be dry-run: the tunnel, the secrets and the registry only exist on the lab server. Merging is how we find out; `Pods on failure` dumps `get pods,jobs` if it doesn't. - **Known gap — the portals still aren't browsable.** PKCE needs a secure context, so a NodePort on an IP can't serve them (runbook §5); they need `make k8s-portals` or an SSH forward. Giving the server a hostname + TLS is the follow-up, and is where `TALOS_HOST` stops defaulting to `localhost`. - **Databases are `emptyDir`.** Any change to a database pod's template wipes it; the `k8s-reseed` in the deploy re-runs the bootstrap, so the stack recovers, but submitted registrations do not. Persistence is runbook §6. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
not added this to the Iteration 6 — Production Posture milestone 2026-09-18 13:41:15 +00:00
not added 1 commit 2026-09-18 13:41:16 +00:00
ci(deploy): deploy the stack to Talos on merge to main (refs #175)
CI / k8s (pull_request) Successful in 5s
CI / lint (pull_request) Successful in 1m25s
CI / build (pull_request) Successful in 1m22s
CI / unit (pull_request) Successful in 1m24s
CI / frontend (pull_request) Successful in 1m45s
CI / mutation (pull_request) Successful in 3m1s
CI / verify-stack (pull_request) Successful in 6m22s
fc036d53d5
The chart has been deployable by hand since #25 and linted in CI since #168;
this makes a merged PR actually ship it to the lab server's Talos VM.

Neither the Kubernetes API nor the in-cluster registry is publicly reachable,
so the job forwards 6443, 30500 and 30141 over the same SSH hop into the Fedora
host that the Gitea-runner pipeline uses. That splits the registry into two
names for one store: images are pushed through the tunnel to localhost:30500,
and the node pulls them from its own NodePort — the address its registry-mirror
patch trusts over plain HTTP.

It deploys with `make k8s-reseed` rather than `make k8s-up`: the bootstrap Jobs
are idempotent, and deleting them first is what stops a changed Job template
from wedging `helm upgrade`. The nine deployments are then rolled explicitly,
because `dev` is a mutable tag and helm sees an unchanged pod template.

PR CI is the merge gate, so this workflow does not re-run the checks. Deploys
queue instead of cancelling: a `helm upgrade` killed half-way leaves the release
in `pending-upgrade` and needs unwedging by hand.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
not added the type:chorearea:infra labels 2026-09-18 13:41:23 +00:00
not merged commit 6cfcc4cf83 into main 2026-09-18 13:53:59 +00:00
Sign in to join this conversation.