Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
  notandClaude Opus 5.5
|
c7f06b35fa |
@@ -207,14 +207,8 @@ jobs:
|
||||
# dispatched (gitea-actions-gotchas.md §7). Default `if: success()` dispatches normally. Cost: a
|
||||
# failing mutation ratchet now skips verify-stack instead of running it anyway; the fix-and-re-push
|
||||
# re-run exercises verify-stack, so we still get the signal.
|
||||
#
|
||||
# Main only, not on PRs: the runner shares the lab node with the deployed stack, and a second
|
||||
# full stack per PR was what got the runner OOM-killed (#182). PRs still gate on every job above;
|
||||
# the live-stack check runs once per merge. A plain event `if` keeps the implicit success(), so it
|
||||
# is not the status-function case from gotchas §7.
|
||||
verify-stack:
|
||||
needs: [mutation]
|
||||
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: https://github.com/actions/checkout@v4
|
||||
|
||||
@@ -31,9 +31,6 @@ jobs:
|
||||
# origin, e.g. https://big-auth.labs.respellion.tech (runbook, "Publishing
|
||||
# through the labs Caddy").
|
||||
KEYCLOAK_URL: ${{ vars.KEYCLOAK_URL }}
|
||||
# `true` fills in the medewerker OTP step for the public demo (chart value
|
||||
# demo.otpAutofill). The fixture secret is committed: demo only.
|
||||
OTP_AUTOFILL: ${{ vars.OTP_AUTOFILL }}
|
||||
steps:
|
||||
- uses: https://github.com/actions/checkout@v4
|
||||
|
||||
@@ -100,7 +97,7 @@ jobs:
|
||||
make k8s-reseed \
|
||||
TALOS_HOST=${TALOS_HOST:-localhost} \
|
||||
K8S_REGISTRY=${TALOS_VM_IP:-192.168.122.173}:30500 \
|
||||
K8S_SET="${KEYCLOAK_URL:+--set keycloakUrl=$KEYCLOAK_URL} --set demo.otpAutofill=${OTP_AUTOFILL:-false}"
|
||||
K8S_SET="${KEYCLOAK_URL:+--set keycloakUrl=$KEYCLOAK_URL}"
|
||||
|
||||
# `dev` is a mutable tag and helm sees an unchanged pod template, so the
|
||||
# new images only land on a restart (pullPolicy is already Always).
|
||||
|
||||
@@ -1,108 +0,0 @@
|
||||
# ADR-0035: The deployed stack is published through the existing labs Caddy
|
||||
|
||||
- **Status:** Accepted
|
||||
- **Date:** 2026-09-25
|
||||
- **Deciders:** Respellion engineering
|
||||
- **Slice:** [#177](https://git.labs.respellion.tech/eho/register-referentie/issues/177) —
|
||||
that issue proposed the opposite (an in-cluster Caddy edge); this ADR records why the
|
||||
host-side option won. Implemented in #179, #180 and #181.
|
||||
|
||||
## Context
|
||||
|
||||
The stack deploys to a single-node Talos VM (ADR-0033, #175). Until now it was only usable
|
||||
through five SSH port-forwards: the portals' OIDC flow uses PKCE, PKCE needs
|
||||
`crypto.subtle`, and browsers expose that only in a **secure context**, meaning HTTPS or a
|
||||
`localhost` origin. A NodePort on the VM's address is neither. We want a URL a demo
|
||||
audience can simply open.
|
||||
|
||||
Three facts about where things run shape the answer:
|
||||
|
||||
- The Talos VM is a libvirt guest on a **Fedora hypervisor in the office**, behind NAT
|
||||
with no public address. The only way in from outside is an existing reverse SSH tunnel
|
||||
(`autossh-reverse-tunnel.service`) into an `openssh-server` container on the labs
|
||||
server.
|
||||
- The **labs server** (public IP) already runs Caddy for `*.labs.respellion.tech`, with
|
||||
the wildcard certificate (DNS-01 via Cloudflare) and ports 80/443. Every other labs
|
||||
service is published there (repo `Infra`, `infra/development/`).
|
||||
- #177 proposed a Caddy **inside the cluster**, fed by a layer-4 forward on the host, so
|
||||
that routing and certificates would be cluster state. That assumes the public IP is on
|
||||
the hypervisor. It isn't: the hypervisor has no inbound path, and 80/443 on the labs
|
||||
server are already taken by the labs Caddy.
|
||||
|
||||
## Decision
|
||||
|
||||
**Publish the portals and Keycloak through the existing labs Caddy. Carry the traffic to
|
||||
the cluster over a second reverse SSH tunnel from the hypervisor.**
|
||||
|
||||
```
|
||||
browser ─https─▶ labs Caddy ─▶ openssh-server:3014x/30180
|
||||
─reverse SSH tunnel─▶ Fedora hypervisor ─▶ Talos NodePorts
|
||||
```
|
||||
|
||||
- **Hostnames** under the existing wildcard: `big-register` (openbaar), `big-mijn`
|
||||
(self-service), `big-behandel`, `big-beheer`, and `big-auth` (Keycloak, with `/admin*`
|
||||
answered 404).
|
||||
- **Tunnel:** `big-portals-tunnel.service` on the hypervisor (repo `Infra`)
|
||||
reverse-forwards the five browser-facing NodePorts into `openssh-server`. It is
|
||||
separate from the access tunnel on `:6667`, so a failed forward can't cut SSH access.
|
||||
Caddy joins the `openssh_default` network to reach the tunnel ends.
|
||||
- **Keycloak's issuer** is the public origin. The chart value `keycloakUrl` replaces
|
||||
`host` + NodePort in one helper, `big.keycloakUrl`, which feeds both `KC_HOSTNAME` and
|
||||
the portals' `config.json` authority, so the two cannot drift (ADR-0010). The deploy
|
||||
workflow sets it from the `KEYCLOAK_URL` repository variable.
|
||||
- **`KC_PROXY_HEADERS=xforwarded`:** `KC_HOSTNAME_BACKCHANNEL_DYNAMIC` builds the token,
|
||||
userinfo and certs URLs from the request. That request reaches Keycloak as plain HTTP,
|
||||
so the URLs came out `http://` and browsers blocked them as mixed content. Trusting
|
||||
Caddy's `X-Forwarded-Proto` keeps them HTTPS. In-cluster calls send no such header and
|
||||
still use `keycloak:8080`.
|
||||
- **Demo MFA (optional):** `demo.otpAutofill` (`OTP_AUTOFILL`) makes the `big-demo` theme
|
||||
(`infra/keycloak/themes/big-demo`) Keycloak's default. Its script fills in and submits
|
||||
the medewerker OTP from the fixture secret (ADR-0031), so the step is visibly enforced
|
||||
without an authenticator. It is off by default.
|
||||
|
||||
### Alternatives considered
|
||||
|
||||
- **In-cluster Caddy edge (#177, PR #178).** It would keep routes and certificates in
|
||||
cluster state. But it needs a public inbound path to the hypervisor that doesn't exist,
|
||||
plus a second certificate authority beside the labs Caddy, which already holds the
|
||||
wildcard. Closed unmerged.
|
||||
- **Port-forward on the office router to the hypervisor.** This opens the office network
|
||||
itself to the internet. Rejected.
|
||||
- **Move the cluster to a host with a public IP.** It would remove the tunnel, but it's a
|
||||
bigger change than publishing one demo. It remains the natural step if the stack
|
||||
outgrows a lab VM.
|
||||
- **Keep the SSH port-forwards.** Fine for one developer, but not something you can send
|
||||
to someone.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive**
|
||||
|
||||
- Real hostnames and HTTPS, so PKCE works in any browser with no client-side setup.
|
||||
- No new certificate handling: the labs Caddy's wildcard covers the new hosts.
|
||||
- The chart stays edge-agnostic. With `keycloakUrl` empty it renders exactly as before,
|
||||
so compose, CI and the `localhost` workflow are untouched.
|
||||
|
||||
**Negative / costs**
|
||||
|
||||
- **Routing lives outside the cluster**, in the Infra repo's Caddyfile. That is exactly
|
||||
what #177 wanted to avoid. Adding a portal means changing three places: a NodePort in
|
||||
the chart, a forward in the tunnel unit, and a host in the Caddyfile.
|
||||
- **Two SSH hops in the data path.** If the hypervisor or the tunnel is down, the
|
||||
portals return 502 even though the cluster is healthy.
|
||||
- **One issuer string.** With `keycloakUrl` set, the `localhost` port-forward workflow
|
||||
(runbook §5) can no longer log in.
|
||||
- **Keycloak trusts `X-Forwarded-*`** from anything that reaches it. Today that is only
|
||||
in-cluster callers and the tunnel. `KC_PROXY_TRUSTED_ADDRESSES` can narrow it if the
|
||||
NodePort is ever exposed more widely.
|
||||
- **The portals are public.** Anyone with the link can log in with the committed test
|
||||
credentials, and with `OTP_AUTOFILL` on, no second factor stands in the way. That is
|
||||
acceptable for synthetic data. Put the labs Caddy's Azure `authorize` in front of the
|
||||
`big-*` hosts if the audience must be restricted.
|
||||
|
||||
**Follow-up**
|
||||
|
||||
- Runbook: `docs/runbooks/kubernetes-talos.md`, "Publishing through the labs Caddy".
|
||||
- Dev-mode Keycloak generates new signing keys on every restart, and the BFF re-fetches
|
||||
them at most every 5 minutes, so expect a few minutes of 401s after a Keycloak restart.
|
||||
Persisting Keycloak's database (runbook §6) would remove that.
|
||||
+1
-1
@@ -21,7 +21,7 @@ and CI cannot drift:
|
||||
| `frontend` | `make frontend` → Nx lint/test/build for the four portals | pnpm + Node |
|
||||
| `k8s` | `make k8s-lint` (render + schema-check the Helm chart) → `make k8s-drift` (chart still describes the same stack as `infra/docker-compose.yml`) | pinned `helm` binary + `docker compose` |
|
||||
| `mutation` | `make mutation` → `dotnet tool restore` → `dotnet stryker` (ACL); uploads the HTML report as an artifact | .NET 10 SDK |
|
||||
| `verify-stack` | **push to `main` only, skipped on PRs** (#182) — the single live-stack stage — steps: `make verify-up` (full stack up + health, the DoD smoke) → `make verify-acl` (ACL ↔ OpenZaak) → `make verify-nrc` (OpenZaak → NRC delivery) → `make down` | container engine + egress (base images, nuget, `selectielijst.openzaak.nl`) |
|
||||
| `verify-stack` | the single live-stack stage — steps: `make verify-up` (full stack up + health, the DoD smoke) → `make verify-acl` (ACL ↔ OpenZaak) → `make verify-nrc` (OpenZaak → NRC delivery) → `make down` | container engine + egress (base images, nuget, `selectielijst.openzaak.nl`) |
|
||||
|
||||
> **Why one `verify-stack` job, not three.** The single self-hosted runner runs jobs
|
||||
> **sequentially**, so booting OpenZaak once (instead of once per check) is the
|
||||
|
||||
@@ -403,8 +403,6 @@ upgrade path.
|
||||
|
||||
## Publishing through the labs Caddy
|
||||
|
||||
Why this route and not an in-cluster edge: [ADR-0035](../architecture/adr-0035-public-access-through-the-labs-caddy.md).
|
||||
|
||||
The portals can be reached on real hostnames through the Caddy that already fronts
|
||||
`*.labs.respellion.tech` (repo `Infra`, `infra/development/`). The chain:
|
||||
|
||||
@@ -432,18 +430,6 @@ make k8s-up TALOS_HOST=localhost K8S_REGISTRY=<TALOS_HOST>:30500 \
|
||||
For deploy-on-merge, set the repository variable `KEYCLOAK_URL` to the same value.
|
||||
With it set, the `localhost` port-forwards (§5) no longer log in: the issuer is one string.
|
||||
|
||||
Staff logins still hit the enforced OTP step. For a demo, set the repository variable
|
||||
`OTP_AUTOFILL=true` (chart value `demo.otpAutofill`): Keycloak then uses the `big-demo`
|
||||
theme, which fills in and submits the code from the fixture secret, so the step is visible
|
||||
but needs no authenticator. Keycloak restarts when the value flips. Demo only — the secret
|
||||
is committed.
|
||||
|
||||
The theme lives in `infra/keycloak/themes/big-demo/` and is seeded as the `rr-kc-theme`
|
||||
ConfigMap by `infra/helm/seed-configmaps.sh` on every deploy. Keycloak runs `start-dev`,
|
||||
which doesn't cache themes, so an edit shows up about a minute after the ConfigMap changes.
|
||||
A *new* theme file also needs a key in the seed script and a path in the keycloak `files`
|
||||
in `values.yaml`.
|
||||
|
||||
One-time setup:
|
||||
|
||||
1. Fedora host: install `infra/development/big-portals-tunnel.service` from the Infra repo
|
||||
|
||||
@@ -94,10 +94,6 @@ volumes:
|
||||
{{- with .defaultMode }}
|
||||
defaultMode: {{ . }}
|
||||
{{- end }}
|
||||
{{- with .items }}
|
||||
items:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- with $w.data }}
|
||||
- name: data
|
||||
@@ -129,17 +125,13 @@ volumes:
|
||||
{{/*
|
||||
Env list from a map. Every value is run through `tpl`, so values.yaml can name
|
||||
cluster-internal hosts ({{ .Release.Namespace }}) and the node address
|
||||
({{ .Values.host }}) without the chart hard-coding either. A value that renders
|
||||
empty is left out, which is how a setting is made conditional on a chart value.
|
||||
({{ .Values.host }}) without the chart hard-coding either.
|
||||
*/}}
|
||||
{{- define "big.env" -}}
|
||||
{{- $root := index . 0 -}}
|
||||
{{- range $k, $v := index . 1 }}
|
||||
{{- $val := tpl (toString $v) $root }}
|
||||
{{- if $val }}
|
||||
- name: {{ $k }}
|
||||
value: {{ $val | quote }}
|
||||
{{- end }}
|
||||
value: {{ tpl (toString $v) $root | quote }}
|
||||
{{- end }}
|
||||
{{- end -}}
|
||||
|
||||
|
||||
@@ -30,12 +30,6 @@ host: 192.168.122.100
|
||||
# portals' authority (runbook, "Publishing through the labs Caddy").
|
||||
keycloakUrl: ""
|
||||
|
||||
demo:
|
||||
# Fill in and submit the medewerker OTP step from the fixture secret, so a public
|
||||
# demo shows MFA enforced without an authenticator: makes the big-demo theme
|
||||
# (infra/keycloak/themes/big-demo) Keycloak's default. Demo only: the secret is committed.
|
||||
otpAutofill: false
|
||||
|
||||
# Set when pulling from a private registry (e.g. the Gitea Container Registry).
|
||||
imagePullSecrets: []
|
||||
|
||||
@@ -283,29 +277,11 @@ workloads:
|
||||
# this issuer back, which is what browser tokens carry (infra/host-browser.yml).
|
||||
KC_HOSTNAME: '{{ include "big.keycloakUrl" . }}'
|
||||
KC_HOSTNAME_BACKCHANNEL_DYNAMIC: "true"
|
||||
# Only rendered with demo.otpAutofill (big.env skips empty values); off, Keycloak
|
||||
# keeps its stock theme and the mounted big-demo theme is unused.
|
||||
KC_SPI_THEME_DEFAULT: '{{ if .Values.demo.otpAutofill }}big-demo{{ end }}'
|
||||
# Behind a TLS proxy (keycloakUrl) the dynamic backchannel URLs — token,
|
||||
# userinfo, certs — take their scheme from the request, which reaches Keycloak
|
||||
# as plain http; trusting X-Forwarded-Proto keeps them https so the browser
|
||||
# doesn't block them as mixed content. In-cluster calls send no such header.
|
||||
KC_PROXY_HEADERS: xforwarded
|
||||
ports: [{ name: http, port: 8080 }]
|
||||
# TCP, not /health/ready on the management port: nothing here gates on realm
|
||||
# import, and a wrong health path would leave the Service with no endpoints.
|
||||
probe: { tcpSocket: { port: 8080 }, initialDelaySeconds: 15 }
|
||||
files:
|
||||
- { configMap: rr-kc-realms, mountPath: /opt/keycloak/data/import }
|
||||
# infra/keycloak/themes/big-demo, seeded by infra/helm/seed-configmaps.sh.
|
||||
- configMap: rr-kc-theme
|
||||
mountPath: /opt/keycloak/themes/big-demo
|
||||
items:
|
||||
- { key: login.properties, path: login/theme.properties }
|
||||
- { key: otp-autofill.js, path: login/resources/js/otp-autofill.js }
|
||||
- { key: account.properties, path: account/theme.properties }
|
||||
- { key: admin.properties, path: admin/theme.properties }
|
||||
- { key: email.properties, path: email/theme.properties }
|
||||
files: [{ configMap: rr-kc-realms, mountPath: /opt/keycloak/data/import }]
|
||||
|
||||
# ── Flowable (S-03) ─────────────────────────────────────────────────────────
|
||||
flowable-db:
|
||||
|
||||
@@ -30,15 +30,6 @@ seed() { # name <kubectl --from-file args...>
|
||||
seed rr-oz-config --from-file="$repo/infra/openzaak/setup_configuration/"
|
||||
seed rr-nrc-config --from-file="$repo/infra/opennotificaties/setup_configuration/"
|
||||
seed rr-kc-realms --from-file="$repo/infra/keycloak/realms/"
|
||||
# The big-demo login theme (demo.otpAutofill). ConfigMap keys are flat, so each
|
||||
# file gets a key here and its path back in the keycloak `files` in values.yaml.
|
||||
theme="$repo/infra/keycloak/themes/big-demo"
|
||||
seed rr-kc-theme \
|
||||
--from-file=login.properties="$theme/login/theme.properties" \
|
||||
--from-file=otp-autofill.js="$theme/login/resources/js/otp-autofill.js" \
|
||||
--from-file=account.properties="$theme/account/theme.properties" \
|
||||
--from-file=admin.properties="$theme/admin/theme.properties" \
|
||||
--from-file=email.properties="$theme/email/theme.properties"
|
||||
seed rr-objecttypen-config --from-file="$repo/infra/objecttypen/setup_configuration/"
|
||||
seed rr-objecten-config --from-file="$repo/infra/objecten/setup_configuration/"
|
||||
# register.py + the RegisterRecord JSON schema (the __pycache__ dir is skipped:
|
||||
|
||||
@@ -1,4 +0,0 @@
|
||||
# The chart makes big-demo the default for every theme type, and Keycloak does not
|
||||
# fall back for a type a theme lacks (the account page then fails), so each type is
|
||||
# declared as a plain child of Keycloak 26's own default.
|
||||
parent=keycloak.v3
|
||||
@@ -1,4 +0,0 @@
|
||||
# The chart makes big-demo the default for every theme type, and Keycloak does not
|
||||
# fall back for a type a theme lacks (the admin page then fails), so each type is
|
||||
# declared as a plain child of Keycloak 26's own default.
|
||||
parent=keycloak.v2
|
||||
@@ -1,4 +0,0 @@
|
||||
# The chart makes big-demo the default for every theme type, and Keycloak does not
|
||||
# fall back for a type a theme lacks (the email page then fails), so each type is
|
||||
# declared as a plain child of Keycloak 26's own default.
|
||||
parent=keycloak
|
||||
@@ -1,24 +0,0 @@
|
||||
// RFC 6238 with Keycloak's default policy (HmacSHA1, 6 digits, 30 s) over the
|
||||
// raw bytes of the medewerker fixture secret — same as tests/e2e/keycloak-login.ts.
|
||||
document.addEventListener('DOMContentLoaded', async () => {
|
||||
const input = document.querySelector('input[name="otp"]');
|
||||
if (!input || !input.form) return;
|
||||
const key = await crypto.subtle.importKey('raw',
|
||||
new TextEncoder().encode('BIGMEDEWERKEROTPSEED'), { name: 'HMAC', hash: 'SHA-1' }, false, ['sign']);
|
||||
// A code is single-use, so a second login in the same window spends the next
|
||||
// counter (Keycloak's look-ahead accepts it). Past that, fill but don't submit,
|
||||
// so a rejected code can't turn into a submit loop.
|
||||
const now = Math.floor(Date.now() / 30000);
|
||||
let last = -1;
|
||||
try { last = Number(sessionStorage.getItem('big-otp-counter')) || -1; } catch {}
|
||||
const counter = Math.max(now, last + 1);
|
||||
const msg = new DataView(new ArrayBuffer(8));
|
||||
msg.setBigUint64(0, BigInt(counter));
|
||||
const mac = new Uint8Array(await crypto.subtle.sign('HMAC', key, msg.buffer));
|
||||
const o = mac[19] & 0x0f;
|
||||
const n = ((mac[o] & 0x7f) << 24 | mac[o + 1] << 16 | mac[o + 2] << 8 | mac[o + 3]) % 1e6;
|
||||
input.value = String(n).padStart(6, '0');
|
||||
if (counter > now + 1) return;
|
||||
try { sessionStorage.setItem('big-otp-counter', String(counter)); } catch {}
|
||||
input.form.requestSubmit();
|
||||
});
|
||||
@@ -1,11 +0,0 @@
|
||||
# Demo login theme for the public Talos deployment: keycloak.v2 plus a script that
|
||||
# fills in and submits the medewerker OTP step from the committed fixture secret
|
||||
# (docs/runbooks/keycloak.md). Only used when the chart's demo.otpAutofill is on —
|
||||
# it then becomes Keycloak's default theme. Never enable it anywhere real.
|
||||
#
|
||||
# Add styles, messages or template overrides here as in any Keycloak theme
|
||||
# (https://www.keycloak.org/ui-customization/themes); new files must also be
|
||||
# listed in infra/helm/seed-configmaps.sh and the keycloak `files` in values.yaml.
|
||||
parent=keycloak.v2
|
||||
import=common/keycloak
|
||||
scripts=js/otp-autofill.js
|
||||
@@ -56,7 +56,6 @@ nav:
|
||||
- "ADR-0032: Werkbak live refresh": architecture/adr-0032-werkbak-live-refresh.md
|
||||
- "ADR-0033: Kubernetes via one Helm chart": architecture/adr-0033-kubernetes-via-one-helm-chart.md
|
||||
- "ADR-0034: Caddy serves the portals": architecture/adr-0034-caddy-serves-the-portals.md
|
||||
- "ADR-0035: Public access through the labs Caddy": architecture/adr-0035-public-access-through-the-labs-caddy.md
|
||||
- FDS-architectuur:
|
||||
- Overzicht: architecture/fds/README.md
|
||||
- Componentview (L3): architecture/fds/c4-component-view.md
|
||||
|
||||
Reference in New Issue
Block a user