docs(portals): ADR-0034 — Caddy serves the portals (refs #166)

Records the decision, the directive-order footgun that shapes the Caddyfiles, and
the measured cost (the images grew 75.7 MB → 90.6 MB). Also updates the three
frontend-decisions entries and the two other docs that named nginx.
This commit is contained in:
not
2026-09-04 17:25:29 +02:00
parent 162e495f29
commit 0fe4813388
4 changed files with 114 additions and 9 deletions
@@ -18,7 +18,7 @@ Three forces shape the mechanism:
- **Nothing notifies anyone.** The trigger lives in Flowable. The domain does not publish - **Nothing notifies anyone.** The trigger lives in Flowable. The domain does not publish
task events, and there is no bus between the domain and the BFF. task events, and there is no bus between the domain and the BFF.
- **The BFF is stateless** and sits behind each portal's nginx. - **The BFF is stateless** and sits behind each portal's reverse proxy.
- **This is the repo's first live-updating view**, so the choice sets a precedent. - **This is the repo's first live-updating view**, so the choice sets a precedent.
## Decision ## Decision
@@ -40,7 +40,7 @@ werkbak is readable at all.
Neither buys freshness here, because **nothing notifies the BFF either**: Neither buys freshness here, because **nothing notifies the BFF either**:
- **SSE** (`text/event-stream`) would mean a new streaming endpoint whose handler polls the - **SSE** (`text/event-stream`) would mean a new streaming endpoint whose handler polls the
domain and forwards diffs — the same latency, plus connection lifecycle, nginx domain and forwards diffs — the same latency, plus connection lifecycle, proxy
buffering, and auth on a long-lived connection. buffering, and auth on a long-lived connection.
- **WebSocket/SignalR** adds a dependency (CLAUDE.md §13) and makes the BFF stateful and - **WebSocket/SignalR** adds a dependency (CLAUDE.md §13) and makes the BFF stateful and
sticky-session-bound. A genuine push path would *also* need the domain to publish task sticky-session-bound. A genuine push path would *also* need the domain to publish task
@@ -0,0 +1,103 @@
# ADR-0034: The portals are served by Caddy, not nginx
- **Status:** Accepted
- **Date:** 2026-09-04
- **Deciders:** Respellion engineering
- **Slice:** _(none yet — raised directly alongside the Kubernetes deployment, ADR-0033)_
## Context
Each portal ships as one image that does two jobs: serve the built Angular app, and
reverse-proxy *its own* BFF endpoint group so the browser calls a single origin (no CORS,
and the DigiD/medewerker token rides along — ADR-0010, ADR-0013). Until now that was nginx
with a hand-written `nginx.conf` per app.
Two workarounds had accumulated around nginx's resolver, both for the same root cause:
**nginx resolves a variable `proxy_pass` upstream itself**, using only the `resolver`
directive, and never the search domains in `/etc/resolv.conf`.
1. `resolver 127.0.0.11` (Docker's embedded DNS) is wrong on rootless podman, which uses a
network-specific aardvark address — so `apps/portal-nginx-resolver.sh` rewrote the
directive at container start by reading the pod's actual nameserver.
2. On Kubernetes the bare `bff` name cannot resolve at all without the `svc.cluster.local`
search domain, so the same script gained a `BFF_HOST` override that the Helm chart set
per portal (ADR-0033).
Both existed only to tell the proxy how to resolve one hostname.
## Decision
**Serve the portals with `caddy:2-alpine` and a small `Caddyfile` per app, replacing the
nginx runtime stage, the four `nginx.conf` files, and the resolver workaround.**
Caddy dials its upstream per request through Go's resolver, which reads
`/etc/resolv.conf` — nameserver *and* search domains. So `reverse_proxy bff:8080` resolves
correctly under Docker, rootless podman and Kubernetes with no per-engine configuration,
and it still starts before the BFF exists and picks up its restarts (the property the
variable `proxy_pass` was there to buy). `apps/portal-nginx-resolver.sh`, its unit test and
the chart's `BFF_HOST` env are deleted.
The Caddyfile uses `handle` blocks rather than a bare `try_files`:
```
handle /behandel/* { reverse_proxy bff:8080 }
handle { root * /usr/share/caddy; try_files {path} /index.html; file_server }
```
`handle` blocks are mutually exclusive and matched most-specific-first. This matters:
Caddy's default directive order puts rewrites (`try_files`) *before* `reverse_proxy`, so a
top-level `try_files {path} /index.html` would rewrite every API path to `/index.html`
before the proxy ever saw it — the SPA fallback would silently eat the API. The `handle`
form makes the routing explicit instead of relying on directive-order trivia.
`infra/test_portal_caddyfiles.py` (in `make unit`) asserts each portal proxies exactly its
own endpoint groups and keeps the SPA fallback. The four files are near-identical, so a
copy-paste slip is cheap to make and expensive to find: proxying another portal's group
hands a browser an endpoint its token isn't for, and the failure surfaces as a 401 three
services away.
### Alternatives considered
- **Keep nginx.** Zero migration, and it works — but the resolver workaround stays, and it
had already grown a second head for Kubernetes. Both heads are nginx-specific.
- **Keep nginx, hard-code the FQDN.** Would need a different config per deployment target
(compose vs Kubernetes), which is exactly the fork the chart was written to avoid.
- **Drop the proxy and use CORS.** Turns the same-origin design (ADR-0010) inside out:
CORS preflights, an explicit origin allowlist in the BFF, and a token attached
cross-origin. Not a serving decision — an architectural regression.
- **Kubernetes Ingress in front of the portals.** Solves nothing about compose, adds a
controller, and the portals would still need something to serve static files.
- ponytail ceiling: plain HTTP on `:80`, no compression, no cache headers beyond Caddy's
defaults, and Caddy's automatic HTTPS deliberately unused (there is no hostname to get a
certificate for). Upgrade path: `encode zstd gzip` and a cache policy for immutable
Angular bundles; a real hostname makes TLS a one-line `Caddyfile` change, which is the
main reason this is worth having in place.
## Consequences
**Positive**
- One resolver behaviour across compose, podman and Kubernetes; a script, a unit test and a
chart env var are deleted rather than maintained.
- The images gain `curl` for free (the alpine nginx image had only busybox `wget`), which
the compose healthchecks can use.
- Routing intent is readable: one `handle` block per endpoint group, one for the app.
- TLS later is a one-line change instead of a new component.
**Negative / costs**
- A new runtime dependency in four images (CLAUDE.md §13): Caddy replaces nginx rather than
joining it, so the count is unchanged, but it is a less familiar config language for
anyone who has only read nginx configs.
- The images grew: 90.6 MB against nginx's 75.7 MB, because `caddy:2-alpine` carries a
bigger static binary than nginx's. Measured, not estimated.
- Caddy's directive-order rule is a genuine footgun (see above); the `handle` form and the
Caddyfile comments exist to keep the next person out of it.
- Any operational note that says "the portal's nginx" is now wrong; the ones in `docs/` were
updated with this ADR.
## Coupling rules touched (CLAUDE.md §8)
None. §8.3 is unchanged and unchanged in kind: the portals still talk only to the BFF, and
the proxy is still the thing that makes that same-origin.
+1 -1
View File
@@ -389,7 +389,7 @@ make verify-e2e # → login as jan-burger → submit → "ontvangen" co
open http://localhost:8140 open http://localhost:8140
``` ```
> The portal is served same-origin with the BFF (nginx proxies `/self-service` + `/openbaar`), so no > The portal is served same-origin with the BFF (Caddy proxies `/self-service` + `/openbaar`), so no
> CORS; the OIDC authority comes from `/config.json` at runtime. See `docs/frontend-decisions.md`. > CORS; the OIDC authority comes from `/config.json` at runtime. See `docs/frontend-decisions.md`.
--- ---
+8 -6
View File
@@ -77,11 +77,13 @@ with the submit form (S-08c, #67); any deviation from NL DS will be recorded her
## Serving + e2e (S-08d, #68) ## Serving + e2e (S-08d, #68)
- **Served by nginx, same-origin as the BFF.** The compose `self-service` image serves the built app - **Served by Caddy, same-origin as the BFF.** The compose `self-service` image serves the built app
and **reverse-proxies** `/self-service/*` + `/openbaar/*` to the `bff` service. Because the and **reverse-proxies** `/self-service/*` + `/openbaar/*` to the `bff` service. Because the
api-client uses **relative URLs**, the browser calls the app's own origin → nginx forwards to the api-client uses **relative URLs**, the browser calls the app's own origin → Caddy forwards to the
BFF: **no CORS**, and the DigiD token (same-origin) is attached by the interceptor. nginx resolves BFF: **no CORS**, and the DigiD token (same-origin) is attached by the interceptor. Caddy dials
the BFF at request time (a `resolver` + variable `proxy_pass`) so it starts before the BFF is up. the BFF per request through the system resolver, so it starts before the BFF is up, picks up its
restarts, and resolves the bare `bff` name on every engine — compose, podman and Kubernetes
(ADR-0034; the `Caddyfile` sits next to each app's `Dockerfile`).
- **Runtime config.** The app fetches `/config.json` before bootstrap (`main.ts`); `appConfig` is a - **Runtime config.** The app fetches `/config.json` before bootstrap (`main.ts`); `appConfig` is a
factory. The dev default (`public/config.json`) points at `localhost:8180`; the Docker image bakes factory. The dev default (`public/config.json`) points at `localhost:8180`; the Docker image bakes
the compose value (`keycloak:8080`). One build, per-environment OIDC authority. the compose value (`keycloak:8080`). One build, per-environment OIDC authority.
@@ -110,7 +112,7 @@ with the submit form (S-08c, #67); any deviation from NL DS will be recorded her
`angular-auth-oidc-client`, no interceptor, and no `config.json` — `main.ts` bootstraps `appConfig` `angular-auth-oidc-client`, no interceptor, and no `config.json` — `main.ts` bootstraps `appConfig`
directly with just `provideHttpClient` + `provideRouter`. This is the deliberate contrast to directly with just `provideHttpClient` + `provideRouter`. This is the deliberate contrast to
self-service and keeps the app trivially cacheable/CDN-able. self-service and keeps the app trivially cacheable/CDN-able.
- **Same-origin via nginx, like self-service.** The compose `openbaar` image serves the built app and - **Same-origin via Caddy, like self-service.** The compose `openbaar` image serves the built app and
reverse-proxies `/openbaar` to the BFF; the api-client's relative calls stay same-origin (no CORS). reverse-proxies `/openbaar` to the BFF; the api-client's relative calls stay same-origin (no CORS).
Served on `:8141`, health-checked over IPv4 (`127.0.0.1`), no Keycloak dependency. Served on `:8141`, health-checked over IPv4 (`127.0.0.1`), no Keycloak dependency.
- **Public-safe by construction.** The portal only ever sees the BFF's `OpenbaarProjection.PublicView` - **Public-safe by construction.** The portal only ever sees the BFF's `OpenbaarProjection.PublicView`
@@ -138,7 +140,7 @@ frontend work is the medewerker realm auth and the werkbak/decide page. Wiring r
**BFF remains the security boundary** (`behandelaar` policy, 401/403 on `/behandel/*`, ADR-0013); **BFF remains the security boundary** (`behandelaar` policy, 401/403 on `/behandel/*`, ADR-0013);
the frontend role signal is for display/UX, and the werkbak page surfaces a load failure (e.g. a the frontend role signal is for display/UX, and the werkbak page surfaces a load failure (e.g. a
403 for a non-behandelaar) rather than swallowing it. 403 for a non-behandelaar) rather than swallowing it.
- **Same-origin via nginx, like the other portals.** The compose `behandel` image serves the built - **Same-origin via Caddy, like the other portals.** The compose `behandel` image serves the built
app and reverse-proxies `/behandel` to the BFF (relative calls, no CORS). Served on `:8142`, app and reverse-proxies `/behandel` to the BFF (relative calls, no CORS). Served on `:8142`,
health-checked over IPv4 (`127.0.0.1`), depends on Keycloak for the medewerker realm. health-checked over IPv4 (`127.0.0.1`), depends on Keycloak for the medewerker realm.
- **Werkbak = decide-and-refresh.** `WerkbakPage` loads `GET /behandel/werkbak` on open and renders a - **Werkbak = decide-and-refresh.** `WerkbakPage` loads `GET /behandel/werkbak` on open and renders a