Merge branch 'main' into feat/demo-otp-autofill

## What & why

`verify-stack` is being killed by the OOM controller on the shared Talos node, on main (run 827) and on #180 (run 830). The cause is Celery: with `CELERY_WORKER_CONCURRENCY` unset, `oz-celery` and `nrc-celery` each fork one worker per CPU. That's 22 each on the lab node, 49 Celery processes at about 225 MB apiece. Details and the kernel log evidence are in #182.

This sets `CELERY_WORKER_CONCURRENCY: "2"` in the oz and nrc env groups:
- **compose** (`&oz-env`, `&nrc-env`): what `verify-stack` starts inside `dind`.
- **chart** (`envGroups.oz` / `.nrc`): the deployed demo on the same node.

Both images' `/celery_worker.sh` honour the variable; I checked in the running pods. Web, init and beat containers share the anchors and ignore it. `objecten-celery` already defaults to 1.

Closes #182

## Definition of Done

- [x] Linked Gitea issue (above).
- [ ] Failing test committed before the implementation. *(Resource setting; the evidence is the OOM log in #182.)*
- [x] Conventional Commits referencing the issue (`refs #NN`).
- [ ] CI green. This PR's `verify-stack` run is the check.
- [x] `docker compose config` renders the variable into all 7 services on the two anchors.
- [x] Docs: comments next to the setting, following the existing uWSGI notes.

## Notes for reviewers

- `make k8s-lint` and `make k8s-drift` pass.
- **Not applied live.** I couldn't patch the running cluster from my session. After merge, the deploy updates the `oz-env` / `nrc-env` ConfigMaps. The celery pods only pick that up on restart, and the deploy step restarts only this repo's nine services. So run once:
  `kubectl -n big rollout restart deploy/oz-celery deploy/nrc-celery`
- **Why 2 and not 1:** this matches `UWSGI_THREADS: "2"`, and it keeps one notification delivery from blocking behind a slow task. It cuts roughly 40 processes, about 9 GB RSS (less in practice, because forked workers share pages).
- **Longer term:** CI and the demo share one 15 GB VM. Resource requests on the runner, or moving the runner off the node, would stop one from starving the other.

🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #183

.gitea/workflows/deploy.yaml

@@ -27,6 +27,13 @@ jobs:
       # `kubectl port-forward` — runbook §5. Override with repo variables.
       TALOS_VM_IP: ${{ vars.TALOS_VM_IP }}
       TALOS_HOST: ${{ vars.TALOS_HOST }}
+      # Set it when the labs Caddy publishes the portals: Keycloak's public https
+      # origin, e.g. https://big-auth.labs.respellion.tech (runbook, "Publishing
+      # through the labs Caddy").
+      KEYCLOAK_URL: ${{ vars.KEYCLOAK_URL }}
+      # `true` fills in the medewerker OTP step for the public demo (chart value
+      # demo.otpAutofill). The fixture secret is committed: demo only.
+      OTP_AUTOFILL: ${{ vars.OTP_AUTOFILL }}
     steps:
       - uses: https://github.com/actions/checkout@v4
 
@@ -89,7 +96,11 @@ jobs:
       # The jobs are idempotent, and deleting them first is what keeps a changed
       # Job template from wedging the upgrade (`cannot patch … with kind Job`).
       - name: Deploy the chart
-        run: make k8s-reseed TALOS_HOST=${TALOS_HOST:-localhost} K8S_REGISTRY=${TALOS_VM_IP:-192.168.122.173}:30500
+        run: |
+          make k8s-reseed \
+            TALOS_HOST=${TALOS_HOST:-localhost} \
+            K8S_REGISTRY=${TALOS_VM_IP:-192.168.122.173}:30500 \
+            K8S_SET="${KEYCLOAK_URL:+--set keycloakUrl=$KEYCLOAK_URL} --set demo.otpAutofill=${OTP_AUTOFILL:-false}"
 
       # `dev` is a mutable tag and helm sees an unchanged pod template, so the
       # new images only land on a restart (pullPolicy is already Always).

docs/runbooks/kubernetes-talos.md

@@ -401,6 +401,54 @@ Not covered: the portals still need `make k8s-portals` (or an SSH forward) to be
 browser, because PKCE needs a secure context (§5). Giving the server a hostname + TLS is the
 upgrade path.
 
+## Publishing through the labs Caddy
+
+The portals can be reached on real hostnames through the Caddy that already fronts
+`*.labs.respellion.tech` (repo `Infra`, `infra/development/`). The chain:
+
+```
+browser → Caddy (labs server, TLS) → openssh-server:3014x/30180
+        → reverse SSH tunnel → Fedora host → <TALOS_VM_IP>:3014x/30180 (NodePorts)
+```
+
+| URL | NodePort |
+|---|---|
+| `https://big-register.labs.respellion.tech` | 30141 openbaar |
+| `https://big-mijn.labs.respellion.tech` | 30140 self-service |
+| `https://big-behandel.labs.respellion.tech` | 30142 behandel |
+| `https://big-beheer.labs.respellion.tech` | 30143 beheer |
+| `https://big-auth.labs.respellion.tech` | 30180 Keycloak (`/admin` blocked) |
+
+HTTPS makes the portals a secure context, so PKCE works without port-forwards — but
+Keycloak's issuer must be the public origin. Deploy with it:
+
+```bash
+make k8s-up TALOS_HOST=localhost K8S_REGISTRY=<TALOS_HOST>:30500 \
+  K8S_SET="--set keycloakUrl=https://big-auth.labs.respellion.tech"
+```
+
+For deploy-on-merge, set the repository variable `KEYCLOAK_URL` to the same value.
+With it set, the `localhost` port-forwards (§5) no longer log in: the issuer is one string.
+
+Staff logins still hit the enforced OTP step. For a demo, set the repository variable
+`OTP_AUTOFILL=true` (chart value `demo.otpAutofill`): Keycloak then uses the `big-demo`
+theme, which fills in and submits the code from the fixture secret, so the step is visible
+but needs no authenticator. Keycloak restarts when the value flips. Demo only — the secret
+is committed.
+
+The theme lives in `infra/keycloak/themes/big-demo/` and is seeded as the `rr-kc-theme`
+ConfigMap by `infra/helm/seed-configmaps.sh` on every deploy. Keycloak runs `start-dev`,
+which doesn't cache themes, so an edit shows up about a minute after the ConfigMap changes.
+A *new* theme file also needs a key in the seed script and a path in the keycloak `files`
+in `values.yaml`.
+
+One-time setup:
+
+1. Fedora host: install `infra/development/big-portals-tunnel.service` from the Infra repo
+   (instructions in the file).
+2. Labs server: deploy the Infra `Caddyfile` + `compose.yml` (Caddy joins the
+   `openssh_default` network to reach the tunnel ends).
+
 ## What is not ported
 
 - **Observability** (Tempo, Prometheus, Grafana) is defined but disabled — those are built

infra/docker-compose.yml

@@ -57,6 +57,9 @@ services:
       # share this anchor and ignore it — they don't run uwsgi.
       UWSGI_PROCESSES: "1"
       UWSGI_THREADS: "2"
+      # Same lever for oz-celery: unset, the worker forks one process per CPU (22 on the lab node,
+      # ~225 MB each), which OOM-killed the shared runner mid-verify-stack. Only celery reads it.
+      CELERY_WORKER_CONCURRENCY: "2"
       DJANGO_SETTINGS_MODULE: openzaak.conf.docker
       SECRET_KEY: ${OZ_SECRET_KEY:-dev-only-not-for-production}
       DB_HOST: oz-db
@@ -144,6 +147,8 @@ services:
       # 1 uWSGI worker, not the image default of 4×4 (#147) — see the oz-env note above.
       UWSGI_PROCESSES: "1"
       UWSGI_THREADS: "2"
+      # Two celery workers, not one per CPU — see the oz-env note above.
+      CELERY_WORKER_CONCURRENCY: "2"
       DJANGO_SETTINGS_MODULE: nrc.conf.docker
       SECRET_KEY: ${NRC_SECRET_KEY:-dev-only-not-for-production}
       DB_HOST: nrc-db

infra/helm/big-reference/templates/_helpers.tpl

@@ -94,6 +94,10 @@ volumes:
       {{- with .defaultMode }}
       defaultMode: {{ . }}
       {{- end }}
+      {{- with .items }}
+      items:
+{{- toYaml . | nindent 8 }}
+      {{- end }}
 {{- end }}
 {{- with $w.data }}
   - name: data
@@ -125,14 +129,26 @@ volumes:
 {{/*
 Env list from a map. Every value is run through `tpl`, so values.yaml can name
 cluster-internal hosts ({{ .Release.Namespace }}) and the node address
-({{ .Values.host }}) without the chart hard-coding either.
+({{ .Values.host }}) without the chart hard-coding either. A value that renders
+empty is left out, which is how a setting is made conditional on a chart value.
 */}}
 {{- define "big.env" -}}
 {{- $root := index . 0 -}}
 {{- range $k, $v := index . 1 }}
+{{- $val := tpl (toString $v) $root }}
+{{- if $val }}
 - name: {{ $k }}
-  value: {{ tpl (toString $v) $root | quote }}
+  value: {{ $val | quote }}
 {{- end }}
+{{- end }}
+{{- end -}}
+
+{{/*
+The origin a browser reaches Keycloak on: the issuer Keycloak pins and the
+authority the portals use, from one place so they cannot drift (ADR-0010).
+*/}}
+{{- define "big.keycloakUrl" -}}
+{{- .Values.keycloakUrl | default (printf "http://%s:%v" .Values.host (index .Values.nodePorts "keycloak")) -}}
 {{- end -}}
 
 {{- define "big.labels" -}}

infra/helm/big-reference/templates/config.yaml

@@ -40,5 +40,5 @@ metadata:
 {{- include "big.labels" (dict "root" $ "name" (printf "portal-config-%s" $realm)) | nindent 4 }}
 data:
   config.json: |
-    { "authority": "{{ printf "http://%s:%v" $.Values.host (index $.Values.nodePorts "keycloak") }}/realms/{{ $realm }}" }
+    { "authority": "{{ include "big.keycloakUrl" $ }}/realms/{{ $realm }}" }
 {{- end }}

infra/helm/big-reference/templates/deployments.yaml

@@ -28,7 +28,7 @@ spec:
       {{- range $w.files }}
       {{- if hasPrefix "portal-config-" .configMap }}
       annotations:
-        checksum/portal-config: {{ printf "%s|%v" $.Values.host (index $.Values.nodePorts "keycloak") | sha256sum }}
+        checksum/portal-config: {{ include "big.keycloakUrl" $ | sha256sum }}
       {{- end }}
       {{- end }}
       labels:

infra/helm/big-reference/values.yaml

@@ -25,6 +25,17 @@
 # string, so browser tokens and the BFF's discovered issuer agree.
 host: 192.168.122.100
 
+# Set when a TLS proxy outside the cluster publishes Keycloak: the full origin, no
+# trailing slash. It replaces `host` + Keycloak's NodePort as the issuer and the
+# portals' authority (runbook, "Publishing through the labs Caddy").
+keycloakUrl: ""
+
+demo:
+  # Fill in and submit the medewerker OTP step from the fixture secret, so a public
+  # demo shows MFA enforced without an authenticator: makes the big-demo theme
+  # (infra/keycloak/themes/big-demo) Keycloak's default. Demo only: the secret is committed.
+  otpAutofill: false
+
 # Set when pulling from a private registry (e.g. the Gitea Container Registry).
 imagePullSecrets: []
 
@@ -71,6 +82,7 @@ envGroups:
   oz:
     UWSGI_PROCESSES: "1"
     UWSGI_THREADS: "2"
+    CELERY_WORKER_CONCURRENCY: "2"
     DJANGO_SETTINGS_MODULE: openzaak.conf.docker
     SECRET_KEY: dev-only-not-for-production
     DB_HOST: oz-db
@@ -93,6 +105,7 @@ envGroups:
   nrc:
     UWSGI_PROCESSES: "1"
     UWSGI_THREADS: "2"
+    CELERY_WORKER_CONCURRENCY: "2"
     DJANGO_SETTINGS_MODULE: nrc.conf.docker
     SECRET_KEY: dev-only-not-for-production
     DB_HOST: nrc-db
@@ -268,13 +281,26 @@ workloads:
       # Pin the issuer to the address the browser uses, and let backchannel calls
       # keep using keycloak:8080 — the BFF discovers metadata in-cluster and gets
       # this issuer back, which is what browser tokens carry (infra/host-browser.yml).
-      KC_HOSTNAME: "http://{{ .Values.host }}:{{ index .Values.nodePorts \"keycloak\" }}"
+      KC_HOSTNAME: '{{ include "big.keycloakUrl" . }}'
       KC_HOSTNAME_BACKCHANNEL_DYNAMIC: "true"
+      # Only rendered with demo.otpAutofill (big.env skips empty values); off, Keycloak
+      # keeps its stock theme and the mounted big-demo theme is unused.
+      KC_SPI_THEME_DEFAULT: '{{ if .Values.demo.otpAutofill }}big-demo{{ end }}'
     ports: [{ name: http, port: 8080 }]
     # TCP, not /health/ready on the management port: nothing here gates on realm
     # import, and a wrong health path would leave the Service with no endpoints.
     probe: { tcpSocket: { port: 8080 }, initialDelaySeconds: 15 }
-    files: [{ configMap: rr-kc-realms, mountPath: /opt/keycloak/data/import }]
+    files:
+      - { configMap: rr-kc-realms, mountPath: /opt/keycloak/data/import }
+      # infra/keycloak/themes/big-demo, seeded by infra/helm/seed-configmaps.sh.
+      - configMap: rr-kc-theme
+        mountPath: /opt/keycloak/themes/big-demo
+        items:
+          - { key: login.properties, path: login/theme.properties }
+          - { key: otp-autofill.js, path: login/resources/js/otp-autofill.js }
+          - { key: account.properties, path: account/theme.properties }
+          - { key: admin.properties, path: admin/theme.properties }
+          - { key: email.properties, path: email/theme.properties }
 
   # ── Flowable (S-03) ─────────────────────────────────────────────────────────
   flowable-db:

infra/helm/seed-configmaps.sh

@@ -30,6 +30,15 @@ seed() {  # name  <kubectl --from-file args...>
 seed rr-oz-config            --from-file="$repo/infra/openzaak/setup_configuration/"
 seed rr-nrc-config           --from-file="$repo/infra/opennotificaties/setup_configuration/"
 seed rr-kc-realms            --from-file="$repo/infra/keycloak/realms/"
+# The big-demo login theme (demo.otpAutofill). ConfigMap keys are flat, so each
+# file gets a key here and its path back in the keycloak `files` in values.yaml.
+theme="$repo/infra/keycloak/themes/big-demo"
+seed rr-kc-theme \
+  --from-file=login.properties="$theme/login/theme.properties" \
+  --from-file=otp-autofill.js="$theme/login/resources/js/otp-autofill.js" \
+  --from-file=account.properties="$theme/account/theme.properties" \
+  --from-file=admin.properties="$theme/admin/theme.properties" \
+  --from-file=email.properties="$theme/email/theme.properties"
 seed rr-objecttypen-config   --from-file="$repo/infra/objecttypen/setup_configuration/"
 seed rr-objecten-config      --from-file="$repo/infra/objecten/setup_configuration/"
 # register.py + the RegisterRecord JSON schema (the __pycache__ dir is skipped:

infra/keycloak/themes/big-demo/account/theme.properties

@@ -0,0 +1,4 @@
+# The chart makes big-demo the default for every theme type, and Keycloak does not
+# fall back for a type a theme lacks (the account page then fails), so each type is
+# declared as a plain child of Keycloak 26's own default.
+parent=keycloak.v3

infra/keycloak/themes/big-demo/admin/theme.properties

@@ -0,0 +1,4 @@
+# The chart makes big-demo the default for every theme type, and Keycloak does not
+# fall back for a type a theme lacks (the admin page then fails), so each type is
+# declared as a plain child of Keycloak 26's own default.
+parent=keycloak.v2

infra/keycloak/themes/big-demo/email/theme.properties

@@ -0,0 +1,4 @@
+# The chart makes big-demo the default for every theme type, and Keycloak does not
+# fall back for a type a theme lacks (the email page then fails), so each type is
+# declared as a plain child of Keycloak 26's own default.
+parent=keycloak

infra/keycloak/themes/big-demo/login/resources/js/otp-autofill.js

@@ -0,0 +1,24 @@
+// RFC 6238 with Keycloak's default policy (HmacSHA1, 6 digits, 30 s) over the
+// raw bytes of the medewerker fixture secret — same as tests/e2e/keycloak-login.ts.
+document.addEventListener('DOMContentLoaded', async () => {
+  const input = document.querySelector('input[name="otp"]');
+  if (!input || !input.form) return;
+  const key = await crypto.subtle.importKey('raw',
+    new TextEncoder().encode('BIGMEDEWERKEROTPSEED'), { name: 'HMAC', hash: 'SHA-1' }, false, ['sign']);
+  // A code is single-use, so a second login in the same window spends the next
+  // counter (Keycloak's look-ahead accepts it). Past that, fill but don't submit,
+  // so a rejected code can't turn into a submit loop.
+  const now = Math.floor(Date.now() / 30000);
+  let last = -1;
+  try { last = Number(sessionStorage.getItem('big-otp-counter')) || -1; } catch {}
+  const counter = Math.max(now, last + 1);
+  const msg = new DataView(new ArrayBuffer(8));
+  msg.setBigUint64(0, BigInt(counter));
+  const mac = new Uint8Array(await crypto.subtle.sign('HMAC', key, msg.buffer));
+  const o = mac[19] & 0x0f;
+  const n = ((mac[o] & 0x7f) << 24 | mac[o + 1] << 16 | mac[o + 2] << 8 | mac[o + 3]) % 1e6;
+  input.value = String(n).padStart(6, '0');
+  if (counter > now + 1) return;
+  try { sessionStorage.setItem('big-otp-counter', String(counter)); } catch {}
+  input.form.requestSubmit();
+});

infra/keycloak/themes/big-demo/login/theme.properties

@@ -0,0 +1,11 @@
+# Demo login theme for the public Talos deployment: keycloak.v2 plus a script that
+# fills in and submits the medewerker OTP step from the committed fixture secret
+# (docs/runbooks/keycloak.md). Only used when the chart's demo.otpAutofill is on —
+# it then becomes Keycloak's default theme. Never enable it anywhere real.
+#
+# Add styles, messages or template overrides here as in any Keycloak theme
+# (https://www.keycloak.org/ui-customization/themes); new files must also be
+# listed in infra/helm/seed-configmaps.sh and the keycloak `files` in values.yaml.
+parent=keycloak.v2
+import=common/keycloak
+scripts=js/otp-autofill.js