Compare commits

..
7 Commits
Author SHA1 Message Date
not e6aaed7c8c docs(k8s): ADR-0033 + the Talos deployment runbook (refs #25)
CI / build (pull_request) Successful in 1m9s
CI / lint (pull_request) Successful in 1m27s
CI / unit (pull_request) Successful in 1m37s
CI / frontend (pull_request) Successful in 3m16s
CI / mutation (pull_request) Successful in 6m20s
CI / verify-stack (pull_request) Successful in 9m58s
ADR-0033 records why one values-driven chart rather than 30 subcharts, the four
platform-forced deviations from compose, and the alternatives (kompose, bitnami
subcharts, ingress-nginx, Helm hooks for ordering, a laptop-side registry).

The runbook is the walkthrough as actually performed on a single-node Talos v1.14
VM under virt-manager, including the parts that bite: virt-manager ejecting the
install ISO on first shutdown, Talos 1.14 moving the install disk into its own
config document, the control-plane taint, and why the portals must be reached
over localhost (crypto.subtle needs a secure context for PKCE).
2026-09-04 17:51:21 +02:00
not 7a5840149c feat(k8s): Helm chart for the whole stack on a single-node cluster (refs #25)
One chart whose values.yaml is a near-literal transcription of
infra/docker-compose.yml, rendered by three generic templates (Deployment, Job,
Service) over a `workloads` map — so the two stacks can be diffed by eye instead
of by archaeology, and adding a service is a values edit.

Platform-forced deviations, each commented where it appears:
- `args`, never `command`: compose replaces the image CMD, Kubernetes replaces the
  ENTRYPOINT. The chart fails to render on `command`, because the symptom (postgres
  refusing to run as root, Keycloak exec-ing `start-dev`) is nothing like the cause.
- The four Django services apply their own setup_configuration in the web pod
  rather than in a separate init Job: both scripts migrate, and without compose's
  depends_on they race the same database.
- OpenZaak and Objecten are addressed by service FQDN, because Django rejects a
  single-label host in a URL — the reason compose passes container IPs around.
- NodePorts, no ingress; databases are emptyDir until persistence.storageClass is
  set, so the stack comes up on a cluster with no CSI driver.

The upstream config inputs stay in the repo and become ConfigMaps via
infra/helm/seed-configmaps.sh — the Kubernetes sibling of infra/seed-config.sh —
so the compose stack and the chart cannot fork. infra/helm/registry.yaml runs an
in-cluster registry because Talos cannot side-load an image and a laptop-side one
needs a root-level firewall change.
2026-09-04 17:51:21 +02:00
not 916d671d49 test(k8s): gate the Helm chart with a render + schema check (refs #25)
`make k8s-lint` runs `helm lint` plus a full `helm template`, so a values typo or a
malformed resource is caught without a cluster — the only automated check the chart
can have while CI has no Kubernetes to deploy into.

Red: there is no chart to lint yet.
2026-09-04 17:51:21 +02:00
not 2d40c84e2c docs(portals): ADR-0034 — Caddy serves the portals (refs #166)
Records the decision, the directive-order footgun that shapes the Caddyfiles, and
the measured cost (the images grew 75.7 MB → 90.6 MB). Also updates the three
frontend-decisions entries and the two other docs that named nginx.
2026-09-04 17:51:21 +02:00
not 4edcf00267 feat(portals): serve each portal with Caddy instead of nginx (refs #166)
nginx resolves a variable `proxy_pass` upstream itself, using only the `resolver`
directive and never the search domains in /etc/resolv.conf. That cost two
workarounds in one script: rewriting the resolver address for rootless podman
(Docker's 127.0.0.11 is wrong there), and injecting a full FQDN so the bare `bff`
name could resolve on Kubernetes at all.

Caddy dials its upstream per request through the system resolver, which reads
nameserver *and* search domains, so `reverse_proxy bff:8080` resolves on every
engine with no per-engine configuration — and it still starts before the BFF
exists and picks up its restarts. Both workarounds are deleted with the script.

Routing uses mutually-exclusive `handle` blocks, not a bare `try_files`: Caddy
sorts rewrites *before* reverse_proxy, so a top-level SPA fallback would rewrite
every API path to /index.html before the proxy saw it.
2026-09-04 17:51:21 +02:00
not 51d99855d1 test(portals): assert each portal proxies only its own endpoint group (refs #166)
The four portal proxy configs are near-identical, so a copy-paste slip is cheap to
introduce and expensive to find: proxying another portal's endpoint group hands a
browser an endpoint its token is not for, and the failure surfaces as a 401 three
services away. Asserts each portal proxies exactly its own groups to the BFF and
keeps the SPA fallback for Angular's client-side routes.

Red: the Caddyfiles it reads do not exist yet.
2026-09-04 17:50:50 +02:00
not d6b3f9764f fix(e2e): bound the Playwright run and make a failed login say why (closes #161) (#165)
CI / build (push) Successful in 1m9s
CI / lint (push) Successful in 1m27s
CI / unit (push) Successful in 1m32s
CI / frontend (push) Successful in 3m19s
CI / mutation (push) Successful in 6m24s
CI / verify-stack (push) Successful in 9m33s
## What & why

#161 is really two defects, and the second one is why the first was undiagnosable.

**A wedged suite consumed the job, and took the post-mortem with it.** Nothing bounded the
Playwright run, so CI stopped the job mid-suite — and `if: always()` does not survive that. Run
739's job metadata shows every step after the e2e as a **0-second failure** stamped at the kill:

```
14 failure  09:48:17 -> 10:14:54  Self-service e2e (Playwright …)
15 failure  10:14:54 -> 10:14:54  verify-stack check summary       ← if: always()
16 failure  10:14:54 -> 10:14:54  e2e spec summary                 ← if: always()
17 failure  10:14:54 -> 10:14:54  Dump container logs on failure   ← if: failure()
18 failure  10:14:54 -> 10:14:54  Tear down                        ← if: always()
```

So the per-spec summary, the container-log dump and the teardown never ran, and the log lost
whatever the killed process had buffered — leaving the single `✘` line the issue was filed from.
`globalTimeout` now makes Playwright stop and *report*: the JSON report is written and those steps
still get their turn. (A `timeout-minutes` on the job would have reproduced the same failure, so
there isn't one.) The "~24-minute gap" is that kill, not necessarily a hang — note run 739 shows
`run_attempt: 2`, and `concurrency.cancel-in-progress` kills an in-flight run on any re-run or push.

**A login that never got its form ate the 90-second test timeout.** Playwright actions auto-wait
until the *test* timeout, not `expect.timeout` — so a portal that serves its page but never
bootstraps (its `config.json` fetch or the OIDC discovery behind `authorize()` failed; `main.ts`
only `console.error`s) spent 90s to report `locator.fill: Test timeout of 90000ms exceeded`: the
symptom, not the cause. That is catalogus.spec's 1.8 minutes. Both Keycloak forms are now asserted
visible first, with a 20s budget and a message naming the step that never happened.

Verified against a real blank-bootstrap portal — the beheer image served with a `config.json` that
is not JSON — which fails in **20.2s** with *"the Keycloak login form never appeared — the portal
did not reach Keycloak (check its config.json fetch and the OIDC discovery …)"*.

**And the summary now says why.** The per-spec table (#136) rendered a verdict icon and nothing
else, so even a surviving summary cost a log dive. Failing specs now carry their first error,
flattened for a table cell (ANSI stripped, newlines collapsed, `|` escaped, clipped) — shape
verified against a real @playwright/test 1.61 failing report, with a stdlib assert self-check on
`make unit`.

Closes #161

## Definition of Done

- [x] Linked Gitea issue (above).
- [x] Failing test committed before the implementation.
- [x] Implementation makes the test pass; refactor commit follows (login helper dedup).
- [x] Conventional Commits referencing the issue (`refs #161`).
- [ ] CI green — all Gitea Actions jobs.
- [x] `docker compose up` from a fresh clone reaches green health checks within 3 minutes (untouched).
- [x] Docs updated — `docs/runbooks/gitea-actions-gotchas.md` §9.
- [x] ADR — not needed: no boundary, dependency or coupling rule touched (test/CI infra only).
- [x] Demo note — not applicable: nothing user-visible.

## Notes for reviewers

**What this does not do: identify why the beheerder login failed that once.** The evidence to do
that was destroyed by defect 2, which is what this PR fixes. The suite ran green here five times
today (catalogus.spec 1.1–5.3s each) — but a local box is not the loaded CI runner, so that is weak
evidence and I am not claiming the flake is gone. What changes is that the next occurrence is
bounded and self-describing: it fails in 20s naming the failing step, the JSON report survives, and
the summary prints the error. Please keep #161 in mind rather than treating this as proof.

**Two follow-ups I did not pull into this PR:**
- *All four portals show a permanently blank page if their startup fetch fails* — `main.ts` does
  `fetch('config.json').then(bootstrap).catch(console.error)`, one shot, no UI and no recovery. That
  is a real product gap (the deliberately-broken portal above is exactly what a user would see) and
  wants its own slice, not a test-infra PR.
- `retries: 1` is untouched. CLAUDE.md §15 says flaky tests are fixed rather than retried, but
  removing retries while a real flake is unexplained would trade a rare red for a frequent one.
  Worth revisiting once #161 recurs (or doesn't) with the new diagnostics.

The login-helper rename (`medewerker-login.ts` → `keycloak-login.ts`, citizen logins routed through
`loginBurger`) is its own no-behaviour-change commit: the three citizen specs each duplicated the
same three-line login, so guarding the login path once meant routing them through it first.Reviewed-on: #165
2026-09-04 10:53:35 +00:00
14 changed files with 326 additions and 75 deletions
+3
View File
@@ -71,8 +71,11 @@ build:
## unit: run unit tests (excludes the container-backed Integration lane)
# TRX per test project (→ TestResults/) feeds the CI per-service summary (#136); harmless locally.
# The CI reporting scripts are stdlib Python with their own assert-based self-checks (#161) — they
# ride this lane so a broken job summary is caught by CI rather than by the next red pipeline.
unit:
dotnet test $(SLN) -c Release --filter "Category!=Integration" --logger trx --results-directory TestResults
python3 infra/test_playwright_summary.py
python3 infra/test_portal_caddyfiles.py
## mutation: run the Stryker.NET ratchet on each service with branching logic (fails below baseline)
+44
View File
@@ -245,3 +245,47 @@ the verify-stack check table, and per-spec e2e results (`infra/playwright-summar
- Getting a report out of the e2e container: Playwright writes `playwright-report.json`
inside the container; `infra/run-e2e-check.sh` `docker cp`s it back to the host
(capturing the test exit code first) so the summary step can read it.
---
## 9. `if: always()` does not survive the job being killed — bound the work itself
`if: always()` makes a step run when an *earlier step failed*. It does **not** help when
the job as a whole is stopped: the run's remaining steps are simply never dispatched.
That is how #161 lost its diagnosis. `verify-stack` entered `make verify-e2e` at 09:48:17
and the job ended at 10:14:54 — 26½ minutes later, mid-suite. Every step after the e2e
shows a **0-second `failure`** stamped at that same instant:
```
14 failure 09:48:17 -> 10:14:54 Self-service e2e (Playwright, login → submit → success)
15 failure 10:14:54 -> 10:14:54 verify-stack check summary ← if: always()
16 failure 10:14:54 -> 10:14:54 e2e spec summary ← if: always()
17 failure 10:14:54 -> 10:14:54 Dump container logs on failure ← if: failure()
18 failure 10:14:54 -> 10:14:54 Tear down ← if: always()
```
So the per-spec summary, the container-log dump and the teardown never ran, and the job
log — which also loses whatever the killed process had buffered — ended at a single `✘`
line. A job that dies takes its own post-mortem with it.
**Read the step timings, not just the log.** `GET /api/v1/repos/{owner}/{repo}/actions/jobs/{id}`
returns every step with `started_at`/`completed_at`; a row of identical zero-length
steps at the end means *killed*, not *silent*. (Job ids come from
`…/actions/runs/{run}/jobs`, and that route returns only the **latest attempt** — a
re-run hides the failed one, so keep the failing job id from the original report. Logs:
`…/actions/jobs/{id}/logs`, see also `gitea-ci-logs`.)
**Conventions that follow:**
- **Bound long-running work inside the tool**, where it can still report. Playwright's
`globalTimeout` (`tests/e2e/playwright.config.ts`) ends the run, writes the JSON
report and exits, so the summary and log-dump steps still get their turn. A
`timeout-minutes` on the job would reproduce the very failure above.
- **Never let an auto-waiting action be the timeout.** Playwright actions (`fill`,
`click`) inherit the *test* timeout, not `expect.timeout`, so a missing element costs
the full 90 s and reports `locator.fill: Test timeout …` — the symptom. Assert the
element visible first with its own budget and a message (`tests/e2e/keycloak-login.ts`).
- Remember `concurrency.cancel-in-progress: true` in `ci.yaml`: a new push to the same
ref, or a re-run, kills the in-flight run the same way. Check `run_attempt` before
concluding a job hung.
+20
View File
@@ -0,0 +1,20 @@
# Overlay: make the CI compose stack usable from a HOST browser.
# Same two mechanisms infra/docker-compose.local.yml already uses — pin Keycloak's issuer to the
# host-published address, and point each portal's runtime config.json at it. The BFF needs no
# change: it discovers metadata over keycloak:8080 and the discovered issuer is the pinned
# localhost:8180, which is what browser tokens carry.
services:
keycloak:
environment:
KC_HOSTNAME: http://localhost:8180
KC_HOSTNAME_BACKCHANNEL_DYNAMIC: "true"
self-service:
volumes:
- ./local-config/self-service.config.json:/usr/share/caddy/config.json:ro,z
behandel:
volumes:
- ./local-config/behandel.config.json:/usr/share/caddy/config.json:ro,z
# beheer is the same medewerker realm as behandel, so it reuses behandel's config verbatim.
beheer:
volumes:
- ./local-config/behandel.config.json:/usr/share/caddy/config.json:ro,z
+37 -5
View File
@@ -7,10 +7,34 @@ redirects it into $GITHUB_STEP_SUMMARY. Stdlib only.
"""
import json
import os
import re
import sys
STATUS_ICON = {"expected": "", "unexpected": "", "skipped": "⏭️", "flaky": "⚠️"}
# A verdict alone still costs a log dive, and a killed or truncated job leaves no log to dive into
# (#161) — so a failing spec carries its first error into the table. Playwright errors are multi-line
# with a "Call log:", which a markdown table cell cannot hold, so they are flattened and clipped.
ERROR_CLIP = 300
def first_error(spec):
"""The first error message across a spec's test results, flattened for one table cell."""
for test in spec.get("tests", []):
for result in test.get("results", []):
for error in result.get("errors", []):
message = (error.get("message") or "").strip()
if not message:
continue
# Strip ANSI colour, collapse to one line, and keep it inside the cell.
message = re.sub(r"\x1b\[[0-9;]*m", "", message)
message = " ".join(message.split())
if len(message) > ERROR_CLIP:
message = message[:ERROR_CLIP - 1].rstrip() + ""
# `|` would end the cell early.
return message.replace("|", "\\|")
return ""
def walk(suite, out):
for spec in suite.get("specs", []):
@@ -22,7 +46,8 @@ def walk(suite, out):
else "expected" if spec.get("ok", False)
else "unexpected")
out.append({"file": spec.get("file") or suite.get("file") or suite.get("title", ""),
"title": spec.get("title", ""), "status": status})
"title": spec.get("title", ""), "status": status,
"error": first_error(spec) if status in ("unexpected", "flaky") else ""})
for child in suite.get("suites", []):
walk(child, out)
@@ -46,10 +71,17 @@ def main(path):
if not specs:
print("_No specs ran._")
return 0
print("| Spec | Result |")
print("| ---- | :----: |")
for s in specs:
print(f"| {s['file']} {s['title']} | {STATUS_ICON.get(s['status'], '')} |")
# The failure column only earns its width when something failed.
if any(s["error"] for s in specs):
print("| Spec | Result | Why |")
print("| ---- | :----: | --- |")
for s in specs:
print(f"| {s['file']} {s['title']} | {STATUS_ICON.get(s['status'], '')} | {s['error']} |")
else:
print("| Spec | Result |")
print("| ---- | :----: |")
for s in specs:
print(f"| {s['file']} {s['title']} | {STATUS_ICON.get(s['status'], '')} |")
return 0
+108
View File
@@ -0,0 +1,108 @@
#!/usr/bin/env python3
"""Self-check for infra/playwright-summary.py — stdlib asserts, no framework.
Run: python3 infra/test_playwright_summary.py (also runs in `make unit`).
A red e2e is only useful if the job summary says WHY it failed: #161 lost a 36-minute
verify-stack job whose only surviving output was one ✘ line with no assertion detail.
"""
import importlib.util
import io
import json
import os
import tempfile
from contextlib import redirect_stdout
# The script's filename is not a valid module name, so load it by path.
spec = importlib.util.spec_from_file_location(
"playwright_summary",
os.path.join(os.path.dirname(os.path.abspath(__file__)), "playwright-summary.py"),
)
summary = importlib.util.module_from_spec(spec)
spec.loader.exec_module(summary)
def render(report):
"""Run the renderer over a report dict and return its markdown."""
with tempfile.NamedTemporaryFile("w", suffix=".json", delete=False) as fh:
json.dump(report, fh)
path = fh.name
try:
out = io.StringIO()
with redirect_stdout(out):
summary.main(path)
return out.getvalue()
finally:
os.unlink(path)
def spec_entry(title, status, errors=()):
return {
"title": title,
"file": "catalogus.spec.ts",
"ok": status == "expected",
"tests": [{"status": status, "results": [{"errors": [{"message": m} for m in errors]}]}],
}
def test_failing_spec_reports_why():
md = render({
"stats": {"expected": 4, "unexpected": 1, "flaky": 0, "skipped": 0, "duration": 108_000},
"suites": [{"file": "catalogus.spec.ts", "specs": [
spec_entry("a beheerder sees the published zaaktypen in the catalogus", "unexpected",
["locator.fill: Test timeout of 90000ms exceeded.\n"
"Call log:\n - waiting for locator('#username')\n"]),
]}],
})
assert "" in md, md
# The point of the slice: the summary names the cause, not just the verdict.
assert "Test timeout of 90000ms exceeded" in md, md
assert "waiting for locator('#username')" in md, md
# A multi-line Playwright error must not break out of its table row.
assert not any(line.startswith("Call log:") for line in md.splitlines()), md
def test_real_playwright_error_is_flattened():
# A real report's message is multi-line and ANSI-coloured, and embeds the source snippet with
# `|` gutters — all three would break the table cell. Shape verified against an actual
# @playwright/test 1.61 JSON report.
md = render({
"stats": {"expected": 0, "unexpected": 1, "flaky": 0, "skipped": 0, "duration": 1_000},
"suites": [{"file": "catalogus.spec.ts", "specs": [
spec_entry("a beheerder sees the catalogus", "unexpected",
["Error: expect(locator).toBeVisible() failed\n\n"
"\x1b[2mLocator: \x1b[22mgetByRole('heading')\n"
" 12 | await login(page);\n> 13 | await expect(heading).toBeVisible();\n"]),
]}],
})
row = [line for line in md.splitlines() if line.startswith("| catalogus.spec.ts")][0]
assert "\x1b" not in row, row
assert "Locator: getByRole('heading')" in row, row
# Every literal `|` from the snippet gutters is escaped, so the row keeps exactly 3 cells.
assert row.count("|") - row.count("\\|") == 4, row
def test_passing_run_stays_quiet():
md = render({
"stats": {"expected": 1, "unexpected": 0, "flaky": 0, "skipped": 0, "duration": 5_000},
"suites": [{"file": "catalogus.spec.ts",
"specs": [spec_entry("a beheerder sees the catalogus", "expected")]}],
})
assert "" in md, md
assert "timeout" not in md.lower(), md
def test_missing_report_is_not_a_crash():
out = io.StringIO()
with redirect_stdout(out):
rc = summary.main("/nonexistent/playwright-report.json")
assert rc == 0
assert "did not reach the e2e step" in out.getvalue()
if __name__ == "__main__":
for name, fn in sorted(globals().items()):
if name.startswith("test_") and callable(fn):
fn()
print(f" ok {name}")
print("playwright-summary self-check passed")
+1 -1
View File
@@ -1,5 +1,5 @@
import { expect, test } from '@playwright/test';
import { loginMedewerker } from './medewerker-login';
import { loginMedewerker } from './keycloak-login';
// S-15a walking skeleton: a beheerder logs in to the beheer portal (medewerker realm) and sees the
// read-only ZTC catalogus. The verify stack seeds and publishes the BIG-REGISTRATIE zaaktype (the
+1 -1
View File
@@ -1,5 +1,5 @@
import { expect, test } from '@playwright/test';
import { loginMedewerker } from './medewerker-login';
import { loginMedewerker } from './keycloak-login';
// S-15b: a beheerder edits the ACL default-fill in the beheer portal and gets a saved confirmation.
// Runs against the shared verify stack; it edits + saves (the ACL store is in-memory, ADR-0026) and
@@ -1,5 +1,5 @@
import { expect, test } from '@playwright/test';
import { OTP_PERIOD_MS, nextUnusedCounter } from './medewerker-login';
import { OTP_PERIOD_MS, nextUnusedCounter } from './keycloak-login';
// Pure check of the TOTP counter guard in loginMedewerker — no browser, no stack. Keycloak refuses
// a code it has already accepted (its otpPolicyCodeReusable defaults to false), so two logins as
+99
View File
@@ -0,0 +1,99 @@
import { createHmac } from 'node:crypto';
import { readFileSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { expect, type Page } from '@playwright/test';
// Every portal login in the suite goes through this module — citizen realms (mock DigiD) and the
// medewerker realm alike — so the shared Keycloak form handling lives in exactly one place.
// The medewerker realm enforces MFA (S-15c), so a staff login is two steps: password, then a TOTP
// code. The realm export seeds every medewerker with this fixture secret — Keycloak HMACs the raw
// secret bytes — so the e2e can compute a valid code instead of enrolling an authenticator.
const OTP_SECRET = 'BIGMEDEWERKEROTPSEED';
export const OTP_PERIOD_MS = 30_000;
/**
* How long a Keycloak form gets to appear. Generous enough for a cold first browser launch and a
* loaded stack, far short of the 90-second test timeout an auto-waiting action would otherwise eat.
*/
const FORM_TIMEOUT_MS = 20_000;
const FORM_NEVER_APPEARED =
'the Keycloak login form never appeared — the portal did not reach Keycloak (check its ' +
'config.json fetch and the OIDC discovery on the authority it was built with)';
const OTP_NEVER_APPEARED =
'the Keycloak OTP form never appeared — the password step did not complete (check the ' +
'medewerker realm seeded this user with both a password and a TOTP credential)';
// RFC 6238 TOTP: HMAC-SHA1 over the 30-second counter, dynamically truncated to 6 digits.
export function totp(secret = OTP_SECRET, at = Date.now()): string {
const counter = Buffer.alloc(8);
counter.writeBigUInt64BE(BigInt(Math.floor(at / OTP_PERIOD_MS)));
const mac = createHmac('sha1', secret).update(counter).digest();
const offset = mac[mac.length - 1] & 0x0f;
return String((mac.readUInt32BE(offset) & 0x7fffffff) % 1_000_000).padStart(6, '0');
}
// Keycloak refuses a TOTP code it has already accepted (its otpPolicyCodeReusable defaults to
// false), so two logins as the same medewerker inside one 30-second window would both submit the
// same code and the second is rejected. Spend the first counter this medewerker has left.
export function nextUnusedCounter(now: number, spent: number): number {
return Math.max(Math.floor(now / OTP_PERIOD_MS), spent + 1);
}
// The spent counter lives on disk rather than in module state: Playwright starts a fresh worker
// process for a retry, which would otherwise forget it and resubmit the rejected code.
function spendCounter(username: string): number {
const file = join(tmpdir(), `otp-counter-${username}`);
let spent = -1;
try {
spent = Number(readFileSync(file, 'utf8')) || -1;
} catch {
// first login as this medewerker in this run
}
const counter = nextUnusedCounter(Date.now(), spent);
writeFileSync(file, String(counter));
return counter;
}
/**
* Fill Keycloak's login form. Every portal is guarded, so the first navigation redirects here; the
* form ids are stable across themes.
*
* The form is asserted visible *before* it is filled. A portal that never reaches Keycloak — its
* runtime `config.json` fetch or the OIDC discovery behind `authorize()` failed, so it never
* bootstrapped and shows a blank page (main.ts only logs to the console) — would otherwise leave
* `fill()` auto-waiting until the whole test times out: 90 seconds spent to report
* `locator.fill: Test timeout of 90000ms exceeded`, naming the symptom and not the cause. That is
* how #161's catalogus.spec burned 1.8 minutes. This fails in a quarter of the time and says which
* step never happened.
*/
async function submitPassword(page: Page, username: string): Promise<void> {
await expect(page.locator('#username'), FORM_NEVER_APPEARED).toBeVisible({ timeout: FORM_TIMEOUT_MS });
await page.locator('#username').fill(username);
await page.locator('#password').fill('test123');
await page.locator('#kc-login').click();
}
/** A citizen login on a mock-DigiD realm — no second factor (ADR-0031). */
export async function loginBurger(page: Page, username: string): Promise<void> {
await submitPassword(page, username);
}
/** A staff login on the medewerker realm: password, then the enforced TOTP second factor. */
export async function loginMedewerker(page: Page, username: string): Promise<void> {
await submitPassword(page, username);
// Keycloak's conditional-OTP step. Same reasoning as the password form above: assert it arrived
// rather than letting `fill()` swallow the test timeout.
await expect(page.locator('#otp'), OTP_NEVER_APPEARED).toBeVisible({ timeout: FORM_TIMEOUT_MS });
// Wait out the rest of the window if the counter we may spend is still in the future; Keycloak's
// lookAheadWindow would accept the code a moment early, but only by one counter — waiting keeps a
// third login in the same window valid too.
const counter = spendCounter(username);
await page.waitForTimeout(Math.max(0, counter * OTP_PERIOD_MS - Date.now()));
await page.locator('#otp').fill(totp(OTP_SECRET, counter * OTP_PERIOD_MS));
await page.locator('#kc-login').click();
}
-57
View File
@@ -1,57 +0,0 @@
import { createHmac } from 'node:crypto';
import { readFileSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import type { Page } from '@playwright/test';
// The medewerker realm enforces MFA (S-15c), so a staff login is two steps: password, then a TOTP
// code. The realm export seeds every medewerker with this fixture secret — Keycloak HMACs the raw
// secret bytes — so the e2e can compute a valid code instead of enrolling an authenticator.
const OTP_SECRET = 'BIGMEDEWERKEROTPSEED';
export const OTP_PERIOD_MS = 30_000;
// RFC 6238 TOTP: HMAC-SHA1 over the 30-second counter, dynamically truncated to 6 digits.
export function totp(secret = OTP_SECRET, at = Date.now()): string {
const counter = Buffer.alloc(8);
counter.writeBigUInt64BE(BigInt(Math.floor(at / OTP_PERIOD_MS)));
const mac = createHmac('sha1', secret).update(counter).digest();
const offset = mac[mac.length - 1] & 0x0f;
return String((mac.readUInt32BE(offset) & 0x7fffffff) % 1_000_000).padStart(6, '0');
}
// Keycloak refuses a TOTP code it has already accepted (its otpPolicyCodeReusable defaults to
// false), so two logins as the same medewerker inside one 30-second window would both submit the
// same code and the second is rejected. Spend the first counter this medewerker has left.
export function nextUnusedCounter(now: number, spent: number): number {
return Math.max(Math.floor(now / OTP_PERIOD_MS), spent + 1);
}
// The spent counter lives on disk rather than in module state: Playwright starts a fresh worker
// process for a retry, which would otherwise forget it and resubmit the rejected code.
function spendCounter(username: string): number {
const file = join(tmpdir(), `otp-counter-${username}`);
let spent = -1;
try {
spent = Number(readFileSync(file, 'utf8')) || -1;
} catch {
// first login as this medewerker in this run
}
const counter = nextUnusedCounter(Date.now(), spent);
writeFileSync(file, String(counter));
return counter;
}
export async function loginMedewerker(page: Page, username: string): Promise<void> {
await page.locator('#username').fill(username);
await page.locator('#password').fill('test123');
await page.locator('#kc-login').click();
// Keycloak's conditional-OTP step. Wait out the rest of the window if the counter we may spend is
// still in the future; its lookAheadWindow would accept the code a moment early, but only by one
// counter — waiting keeps a third login in the same window valid too.
const counter = spendCounter(username);
await page.waitForTimeout(Math.max(0, counter * OTP_PERIOD_MS - Date.now()));
await page.locator('#otp').fill(totp(OTP_SECRET, counter * OTP_PERIOD_MS));
await page.locator('#kc-login').click();
}
+6
View File
@@ -15,6 +15,12 @@ export default defineConfig({
timeout: 90_000,
expect: { timeout: 15_000 },
retries: 1,
// Bound the whole run, not just each test (#161). A wedged suite used to run until CI killed the
// job — which also killed the `if: always()` steps that would have said why: the per-spec summary
// and the container-log dump never ran, leaving a 36-minute job whose entire surviving output was
// one ✘ line. On `globalTimeout` Playwright stops and *reports*, so the JSON report is written and
// those steps still run. Generous over the ~1-minute suite: this is a backstop, not a budget.
globalTimeout: 12 * 60_000,
// Run the specs serially. Each spec drives a full `channel: 'chromium'` browser, and the e2e
// shares an 8 GB runner with the entire compose stack (OpenZaak, NRC, Keycloak, Flowable, 4×
// Postgres, every service + 3 portals). Two parallel browsers exhaust memory and the renderer is
+2 -4
View File
@@ -1,5 +1,5 @@
import { expect, request, test } from '@playwright/test';
import { loginMedewerker } from './medewerker-login';
import { loginBurger, loginMedewerker } from './keycloak-login';
// Walking-skeleton happy path (S-08d + S-09 + S-09b + S-12 + S-10a + S-19b-2): a zorgprofessional
// logs in via mock DigiD and submits through the self-service portal → BFF → domain; the entry
@@ -23,9 +23,7 @@ test('DigiD submit → public INGEDIEND → documenten → behandelaar goedkeurt
// checks submit as jan-burger (bsn 123456782) before the e2e runs on the shared stack, and
// resume-on-load (S-26) would otherwise restore one of those on login — so each self-service spec
// uses a dedicated citizen no other actor touches.
await page.locator('#username').fill('emma-burger');
await page.locator('#password').fill('test123');
await page.locator('#kc-login').click();
await loginBurger(page, 'emma-burger');
// Back on the portal, authenticated.
await expect(page.getByRole('heading', { name: /Zelfservice/i })).toBeVisible();
+2 -3
View File
@@ -1,4 +1,5 @@
import { expect, test } from '@playwright/test';
import { loginBurger } from './keycloak-login';
// S-26: a zorgprofessional submits, then reloads the self-service portal. On load the portal asks the
// BFF for the caller's current open registration (owner-scoped by the DigiD token's bsn) and restores
@@ -9,9 +10,7 @@ test('DigiD submit → reload → self-service restores the existing registratio
// Its own DigiD user (like every self-service spec): on the shared verify stack, resume-on-load
// (S-26) restores any open registration for the bsn, so each spec uses a dedicated citizen that no
// other spec or verify-* check touches. This one in particular leaves an open registration.
await page.locator('#username').fill('sanne-burger');
await page.locator('#password').fill('test123');
await page.locator('#kc-login').click();
await loginBurger(page, 'sanne-burger');
await expect(page.getByRole('heading', { name: /Zelfservice/i })).toBeVisible();
await page.getByRole('button', { name: /indienen/i }).click();
+2 -3
View File
@@ -1,4 +1,5 @@
import { expect, test } from '@playwright/test';
import { loginBurger } from './keycloak-login';
// S-11 (Flow 3): a zorgprofessional logs in via mock DigiD, submits a registration, then withdraws
// it ("trek aanvraag in") from the self-service portal. The withdrawal goes portal → BFF (owner-
@@ -10,9 +11,7 @@ test('DigiD submit → trek aanvraag in → self-service confirms ingetrokken',
// Its own DigiD user — isolated from the verify-* checks (jan-burger/123456782) so resume-on-load
// (S-26) can't restore someone else's registration on the shared stack.
await page.locator('#username').fill('lars-burger');
await page.locator('#password').fill('test123');
await page.locator('#kc-login').click();
await loginBurger(page, 'lars-burger');
await expect(page.getByRole('heading', { name: /Zelfservice/i })).toBeVisible();