Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4568b682a6 |
@@ -34,9 +34,6 @@ jobs:
|
||||
# `true` fills in the medewerker OTP step for the public demo (chart value
|
||||
# demo.otpAutofill). The fixture secret is committed: demo only.
|
||||
OTP_AUTOFILL: ${{ vars.OTP_AUTOFILL }}
|
||||
# Tempo for the services' traces, e.g. http://tempo.monitoring.svc:4317 (the
|
||||
# cluster monitoring stack, Infra repo). Empty = the chart default.
|
||||
OTEL_ENDPOINT: ${{ vars.OTEL_ENDPOINT }}
|
||||
steps:
|
||||
- uses: https://github.com/actions/checkout@v4
|
||||
|
||||
@@ -103,7 +100,7 @@ jobs:
|
||||
make k8s-reseed \
|
||||
TALOS_HOST=${TALOS_HOST:-localhost} \
|
||||
K8S_REGISTRY=${TALOS_VM_IP:-192.168.122.173}:30500 \
|
||||
K8S_SET="${KEYCLOAK_URL:+--set keycloakUrl=$KEYCLOAK_URL} --set demo.otpAutofill=${OTP_AUTOFILL:-false}${OTEL_ENDPOINT:+ --set otelEndpoint=$OTEL_ENDPOINT}"
|
||||
K8S_SET="${KEYCLOAK_URL:+--set keycloakUrl=$KEYCLOAK_URL} --set demo.otpAutofill=${OTP_AUTOFILL:-false}"
|
||||
|
||||
# `dev` is a mutable tag and helm sees an unchanged pod template, so the
|
||||
# new images only land on a restart (pullPolicy is already Always).
|
||||
|
||||
@@ -1,108 +0,0 @@
|
||||
# ADR-0035: The deployed stack is published through the existing labs Caddy
|
||||
|
||||
- **Status:** Accepted
|
||||
- **Date:** 2026-09-25
|
||||
- **Deciders:** Respellion engineering
|
||||
- **Slice:** [#177](https://git.labs.respellion.tech/eho/register-referentie/issues/177) —
|
||||
that issue proposed the opposite (an in-cluster Caddy edge); this ADR records why the
|
||||
host-side option won. Implemented in #179, #180 and #181.
|
||||
|
||||
## Context
|
||||
|
||||
The stack deploys to a single-node Talos VM (ADR-0033, #175). Until now it was only usable
|
||||
through five SSH port-forwards: the portals' OIDC flow uses PKCE, PKCE needs
|
||||
`crypto.subtle`, and browsers expose that only in a **secure context**, meaning HTTPS or a
|
||||
`localhost` origin. A NodePort on the VM's address is neither. We want a URL a demo
|
||||
audience can simply open.
|
||||
|
||||
Three facts about where things run shape the answer:
|
||||
|
||||
- The Talos VM is a libvirt guest on a **Fedora hypervisor in the office**, behind NAT
|
||||
with no public address. The only way in from outside is an existing reverse SSH tunnel
|
||||
(`autossh-reverse-tunnel.service`) into an `openssh-server` container on the labs
|
||||
server.
|
||||
- The **labs server** (public IP) already runs Caddy for `*.labs.respellion.tech`, with
|
||||
the wildcard certificate (DNS-01 via Cloudflare) and ports 80/443. Every other labs
|
||||
service is published there (repo `Infra`, `infra/development/`).
|
||||
- #177 proposed a Caddy **inside the cluster**, fed by a layer-4 forward on the host, so
|
||||
that routing and certificates would be cluster state. That assumes the public IP is on
|
||||
the hypervisor. It isn't: the hypervisor has no inbound path, and 80/443 on the labs
|
||||
server are already taken by the labs Caddy.
|
||||
|
||||
## Decision
|
||||
|
||||
**Publish the portals and Keycloak through the existing labs Caddy. Carry the traffic to
|
||||
the cluster over a second reverse SSH tunnel from the hypervisor.**
|
||||
|
||||
```
|
||||
browser ─https─▶ labs Caddy ─▶ openssh-server:3014x/30180
|
||||
─reverse SSH tunnel─▶ Fedora hypervisor ─▶ Talos NodePorts
|
||||
```
|
||||
|
||||
- **Hostnames** under the existing wildcard: `big-register` (openbaar), `big-mijn`
|
||||
(self-service), `big-behandel`, `big-beheer`, and `big-auth` (Keycloak, with `/admin*`
|
||||
answered 404).
|
||||
- **Tunnel:** `big-portals-tunnel.service` on the hypervisor (repo `Infra`)
|
||||
reverse-forwards the five browser-facing NodePorts into `openssh-server`. It is
|
||||
separate from the access tunnel on `:6667`, so a failed forward can't cut SSH access.
|
||||
Caddy joins the `openssh_default` network to reach the tunnel ends.
|
||||
- **Keycloak's issuer** is the public origin. The chart value `keycloakUrl` replaces
|
||||
`host` + NodePort in one helper, `big.keycloakUrl`, which feeds both `KC_HOSTNAME` and
|
||||
the portals' `config.json` authority, so the two cannot drift (ADR-0010). The deploy
|
||||
workflow sets it from the `KEYCLOAK_URL` repository variable.
|
||||
- **`KC_PROXY_HEADERS=xforwarded`:** `KC_HOSTNAME_BACKCHANNEL_DYNAMIC` builds the token,
|
||||
userinfo and certs URLs from the request. That request reaches Keycloak as plain HTTP,
|
||||
so the URLs came out `http://` and browsers blocked them as mixed content. Trusting
|
||||
Caddy's `X-Forwarded-Proto` keeps them HTTPS. In-cluster calls send no such header and
|
||||
still use `keycloak:8080`.
|
||||
- **Demo MFA (optional):** `demo.otpAutofill` (`OTP_AUTOFILL`) makes the `big-demo` theme
|
||||
(`infra/keycloak/themes/big-demo`) Keycloak's default. Its script fills in and submits
|
||||
the medewerker OTP from the fixture secret (ADR-0031), so the step is visibly enforced
|
||||
without an authenticator. It is off by default.
|
||||
|
||||
### Alternatives considered
|
||||
|
||||
- **In-cluster Caddy edge (#177, PR #178).** It would keep routes and certificates in
|
||||
cluster state. But it needs a public inbound path to the hypervisor that doesn't exist,
|
||||
plus a second certificate authority beside the labs Caddy, which already holds the
|
||||
wildcard. Closed unmerged.
|
||||
- **Port-forward on the office router to the hypervisor.** This opens the office network
|
||||
itself to the internet. Rejected.
|
||||
- **Move the cluster to a host with a public IP.** It would remove the tunnel, but it's a
|
||||
bigger change than publishing one demo. It remains the natural step if the stack
|
||||
outgrows a lab VM.
|
||||
- **Keep the SSH port-forwards.** Fine for one developer, but not something you can send
|
||||
to someone.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive**
|
||||
|
||||
- Real hostnames and HTTPS, so PKCE works in any browser with no client-side setup.
|
||||
- No new certificate handling: the labs Caddy's wildcard covers the new hosts.
|
||||
- The chart stays edge-agnostic. With `keycloakUrl` empty it renders exactly as before,
|
||||
so compose, CI and the `localhost` workflow are untouched.
|
||||
|
||||
**Negative / costs**
|
||||
|
||||
- **Routing lives outside the cluster**, in the Infra repo's Caddyfile. That is exactly
|
||||
what #177 wanted to avoid. Adding a portal means changing three places: a NodePort in
|
||||
the chart, a forward in the tunnel unit, and a host in the Caddyfile.
|
||||
- **Two SSH hops in the data path.** If the hypervisor or the tunnel is down, the
|
||||
portals return 502 even though the cluster is healthy.
|
||||
- **One issuer string.** With `keycloakUrl` set, the `localhost` port-forward workflow
|
||||
(runbook §5) can no longer log in.
|
||||
- **Keycloak trusts `X-Forwarded-*`** from anything that reaches it. Today that is only
|
||||
in-cluster callers and the tunnel. `KC_PROXY_TRUSTED_ADDRESSES` can narrow it if the
|
||||
NodePort is ever exposed more widely.
|
||||
- **The portals are public.** Anyone with the link can log in with the committed test
|
||||
credentials, and with `OTP_AUTOFILL` on, no second factor stands in the way. That is
|
||||
acceptable for synthetic data. Put the labs Caddy's Azure `authorize` in front of the
|
||||
`big-*` hosts if the audience must be restricted.
|
||||
|
||||
**Follow-up**
|
||||
|
||||
- Runbook: `docs/runbooks/kubernetes-talos.md`, "Publishing through the labs Caddy".
|
||||
- Dev-mode Keycloak generates new signing keys on every restart, and the BFF re-fetches
|
||||
them at most every 5 minutes, so expect a few minutes of 401s after a Keycloak restart.
|
||||
Persisting Keycloak's database (runbook §6) would remove that.
|
||||
@@ -60,9 +60,6 @@ make mutation # dotnet tool restore + dotnet stryker on the ACL
|
||||
Config lives in [`services/acl/stryker-config.json`](../../services/acl/stryker-config.json).
|
||||
It runs in **solution mode** against `Acl.slnx`, mutating the two projects under test
|
||||
(`Acl.Application`, `Acl.Infrastructure`); `Acl.Api` has no tests and is skipped.
|
||||
`Acl.slnx` leaves out `Acl.IntegrationTests`: it needs a live OpenZaak, and Stryker
|
||||
runs every test project in the solution, so keeping it in makes 8 tests fail in the
|
||||
initial run (#174).
|
||||
|
||||
**Baseline (the ratchet):** the ACL is the first service with branching logic, so it
|
||||
sets the repo-wide baseline. Observed score **95%**; enforced `break` threshold **90%**
|
||||
|
||||
@@ -403,8 +403,6 @@ upgrade path.
|
||||
|
||||
## Publishing through the labs Caddy
|
||||
|
||||
Why this route and not an in-cluster edge: [ADR-0035](../architecture/adr-0035-public-access-through-the-labs-caddy.md).
|
||||
|
||||
The portals can be reached on real hostnames through the Caddy that already fronts
|
||||
`*.labs.respellion.tech` (repo `Infra`, `infra/development/`). The chain:
|
||||
|
||||
|
||||
@@ -30,12 +30,6 @@ host: 192.168.122.100
|
||||
# portals' authority (runbook, "Publishing through the labs Caddy").
|
||||
keycloakUrl: ""
|
||||
|
||||
# Where the .NET services send traces (OTLP gRPC). The default is the chart's own
|
||||
# `tempo` workload (off by default, like compose). Point it at a Tempo outside the
|
||||
# release, e.g. the cluster monitoring stack's http://tempo.monitoring.svc:4317 —
|
||||
# with no Tempo at all, every export fails and is counted as a .NET exception.
|
||||
otelEndpoint: http://tempo:4317
|
||||
|
||||
demo:
|
||||
# Fill in and submit the medewerker OTP step from the fixture secret, so a public
|
||||
# demo shows MFA enforced without an authenticator: makes the big-demo theme
|
||||
@@ -166,11 +160,10 @@ envGroups:
|
||||
NOTIFICATIONS_DISABLED: "false"
|
||||
RUN_SETUP_CONFIG: "true"
|
||||
|
||||
# Traces for the .NET services. Always set, like compose. With no Tempo behind
|
||||
# `otelEndpoint` the exporter fails quietly but throws on every batch, which
|
||||
# shows up as HttpRequestException/SocketException in dotnet_exceptions_total.
|
||||
# Traces for the .NET services. Always set, like compose: the exporter fails
|
||||
# harmlessly when Tempo is absent (services/*/Program.cs).
|
||||
otel:
|
||||
OTEL_EXPORTER_OTLP_ENDPOINT: '{{ .Values.otelEndpoint }}'
|
||||
OTEL_EXPORTER_OTLP_ENDPOINT: http://tempo:4317
|
||||
OTEL_EXPORTER_OTLP_PROTOCOL: grpc
|
||||
|
||||
# ── Workloads ──────────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -56,7 +56,6 @@ nav:
|
||||
- "ADR-0032: Werkbak live refresh": architecture/adr-0032-werkbak-live-refresh.md
|
||||
- "ADR-0033: Kubernetes via one Helm chart": architecture/adr-0033-kubernetes-via-one-helm-chart.md
|
||||
- "ADR-0034: Caddy serves the portals": architecture/adr-0034-caddy-serves-the-portals.md
|
||||
- "ADR-0035: Public access through the labs Caddy": architecture/adr-0035-public-access-through-the-labs-caddy.md
|
||||
- FDS-architectuur:
|
||||
- Overzicht: architecture/fds/README.md
|
||||
- Componentview (L3): architecture/fds/c4-component-view.md
|
||||
|
||||
@@ -1,9 +1,7 @@
|
||||
<Solution>
|
||||
<!-- Stryker-only. Acl.IntegrationTests is left out on purpose: it needs a live
|
||||
OpenZaak, so in the mutation job it fails its initial run (#174). The root
|
||||
register-referentie.slnx still builds and lints it. -->
|
||||
<Project Path="Acl.Api/Acl.Api.csproj" />
|
||||
<Project Path="Acl.Application/Acl.Application.csproj" />
|
||||
<Project Path="Acl.Infrastructure/Acl.Infrastructure.csproj" />
|
||||
<Project Path="Acl.IntegrationTests/Acl.IntegrationTests.csproj" />
|
||||
<Project Path="Acl.Tests/Acl.Tests.csproj" />
|
||||
</Solution>
|
||||
|
||||
Reference in New Issue
Block a user