Compare commits

..
Author SHA1 Message Date
notandClaude Opus 5 3628efc17a test(e2e): open the werkbak before the registration is ready for beoordeling (refs #162)
CI / build (pull_request) Successful in 1m4s
CI / lint (pull_request) Successful in 1m23s
CI / unit (pull_request) Successful in 1m24s
CI / frontend (pull_request) Successful in 3m8s
CI / mutation (pull_request) Successful in 6m18s
CI / verify-stack (pull_request) Successful in 10m15s
The happy path visited the behandel portal only after the documents were
supplied, so the row was already in the werkbak at page load — dropping the
reload proved nothing. Now the behandelaar logs in first, asserts the row is
NOT there yet, and only then does the citizen supply the documents that route
it to Beoordelen. The row can therefore only reach that already-open,
never-reloaded page by the werkbak refreshing itself.

Verified both ways against a live stack: with the interval stubbed out the
spec fails at "Goedkeuren <ref> … element(s) not found" after 30s; with it,
the behandel nginx logs the poll that delivers the row.

The werkbak page is foregrounded before the assertion — Chromium throttles
timers in a hidden tab.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 11:12:30 +02:00
notandClaude Opus 5 fc843a419a docs(portal-behandel): ADR-0032 + demo note for the self-refreshing werkbak (refs #162)
Records why the werkbak polls rather than being pushed to (proposal #163):
nothing notifies the BFF either, so SSE/WebSockets would poll the domain
inside the BFF for the same freshness plus connection state, a dependency and
a stateful BFF. Names the ceiling (fixed 5s interval, per open page, keeps
polling in a hidden tab) and the upgrade path (publish task events, then swap
the interval for a stream).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 11:00:21 +02:00
notandClaude Opus 5 3f8b438500 feat(portal-behandel): let a successful refresh clear the load failure (refs #162)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 10:46:41 +02:00
notandClaude Opus 5 d906e2c111 test(portal-behandel): a refresh must clear a stale load failure (refs #162)
If the first read fails the werkbak shows its error until the behandelaar
reloads — the very thing this slice removes. Fails with "expected <p
utrecht-paragraph …> to be null".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 10:46:28 +02:00
notandClaude Opus 5 e38bf62b43 feat(portal-behandel): refresh the werkbak on an interval while it is open (refs #162)
`interval(WERKBAK_REFRESH_MS)` re-reads the existing BFF werkbak endpoint,
scoped to the page's lifetime with `takeUntilDestroyed()`. A background read
leaves the rows and states on screen alone until it has an answer, so a tick
neither flashes the loading state over rows being read nor swaps the list for
the failure alert on a blip — only a foreground read (on open, after a
decision) speaks for whether the werkbak is readable at all.

No new endpoint, dependency or server state: rxjs and the endpoint are both
already here (ADR-0032).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 10:46:02 +02:00
notandClaude Opus 5 cd3faaf638 test(portal-behandel): the werkbak must show a new registration without a reload (refs #162)
A registration only reaches the Beoordelen task after the citizen supplies its
documents, so it lands in an already-open werkbak asynchronously — today a
behandelaar has to reload to see it.

Component spec: one interval passes and the werkbak has re-read itself, the new
row is on screen, and the loading state has not flashed over the rows being
read. Fails with "expected vi.fn() to be called 2 times, but got 1 times".
Two guards go with it — a failing background poll must not replace the list
with the load-failure alert, and the refresh must stop when the page is
destroyed.

E2E: the happy path now waits for the werkbak row WITHOUT reloading the page.
The removed `staff.reload()` is the assertion.

Refs the accepted mechanism in #163 (ADR-0032, landing with the implementation).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 10:45:17 +02:00
18 changed files with 336 additions and 350 deletions
-3
View File
@@ -71,11 +71,8 @@ build:
## unit: run unit tests (excludes the container-backed Integration lane)
# TRX per test project (→ TestResults/) feeds the CI per-service summary (#136); harmless locally.
# The CI reporting scripts are stdlib Python with their own assert-based self-checks (#161) — they
# ride this lane so a broken job summary is caught by CI rather than by the next red pipeline.
unit:
dotnet test $(SLN) -c Release --filter "Category!=Integration" --logger trx --results-directory TestResults
python3 infra/test_playwright_summary.py
## mutation: run the Stryker.NET ratchet on each service with branching logic (fails below baseline)
# Stryker is pinned as a local dotnet tool (.config/dotnet-tools.json); `tool restore`
@@ -4,7 +4,7 @@ import { of, throwError } from 'rxjs';
import { BffApiV1Service, type WerkbakItem } from 'api-client';
import { AuthService } from 'auth';
import { axe } from 'vitest-axe';
import { WerkbakPage } from './werkbak-page';
import { WERKBAK_REFRESH_MS, WerkbakPage } from './werkbak-page';
const sample: WerkbakItem[] = [
{ registrationId: 'reg-1', bsn: '123456782', status: 'InBehandeling' },
@@ -81,6 +81,94 @@ describe('WerkbakPage', () => {
});
});
it('picks up a newly submitted registration without a reload', async () => {
// S-26 (#162): a registration reaches Beoordelen asynchronously, after the citizen supplies
// documents — so the werkbak must refresh itself rather than wait for the behandelaar to reload.
vi.useFakeTimers();
try {
const getBehandelWerkbak = vi
.fn()
.mockReturnValueOnce(of([sample[0]]))
.mockReturnValue(of(sample));
const { providers } = setup({ getBehandelWerkbak });
const { detectChanges } = await render(WerkbakPage, { providers });
expect(screen.getByText('reg-1')).toBeTruthy();
expect(screen.queryByText('reg-2')).toBeNull();
vi.advanceTimersByTime(WERKBAK_REFRESH_MS);
detectChanges();
expect(getBehandelWerkbak).toHaveBeenCalledTimes(2);
expect(screen.getByText('reg-2')).toBeTruthy();
// A background refresh must not flash the loading state over the rows the behandelaar is reading.
expect(screen.queryByText(/bezig met laden/i)).toBeNull();
} finally {
vi.useRealTimers();
}
});
it('keeps the rows on screen when a background refresh fails', async () => {
// A blip on a background poll must not replace the list with the load-failure alert; the next
// tick recovers. Only the first load speaks for whether the werkbak is readable at all.
vi.useFakeTimers();
try {
const getBehandelWerkbak = vi
.fn()
.mockReturnValueOnce(of(sample))
.mockReturnValue(throwError(() => new Error('503')));
const { providers } = setup({ getBehandelWerkbak });
const { detectChanges } = await render(WerkbakPage, { providers });
vi.advanceTimersByTime(WERKBAK_REFRESH_MS);
detectChanges();
expect(screen.getByText('reg-1')).toBeTruthy();
expect(screen.queryByText(/kon de werkbak niet laden/i)).toBeNull();
} finally {
vi.useRealTimers();
}
});
it('stops refreshing once the page is destroyed', async () => {
vi.useFakeTimers();
try {
const { getBehandelWerkbak, providers } = setup();
const { fixture } = await render(WerkbakPage, { providers });
fixture.destroy();
vi.advanceTimersByTime(WERKBAK_REFRESH_MS * 3);
expect(getBehandelWerkbak).toHaveBeenCalledTimes(1);
} finally {
vi.useRealTimers();
}
});
it('clears a load failure once a refresh succeeds', async () => {
// Without this the werkbak stays stuck on the error until the behandelaar reloads — the very
// thing this slice removes. A recovered read must put the rows back.
vi.useFakeTimers();
try {
const getBehandelWerkbak = vi
.fn()
.mockReturnValueOnce(throwError(() => new Error('503')))
.mockReturnValue(of(sample));
const { providers } = setup({ getBehandelWerkbak });
const { detectChanges } = await render(WerkbakPage, { providers });
expect(screen.getByText(/kon de werkbak niet laden/i)).toBeTruthy();
vi.advanceTimersByTime(WERKBAK_REFRESH_MS);
detectChanges();
expect(screen.queryByText(/kon de werkbak niet laden/i)).toBeNull();
expect(screen.getByText('reg-1')).toBeTruthy();
} finally {
vi.useRealTimers();
}
});
it('shows an empty state when the werkbak has no items', async () => {
const { providers } = setup({ getBehandelWerkbak: vi.fn().mockReturnValue(of([])) });
await render(WerkbakPage, { providers });
+34 -3
View File
@@ -1,7 +1,15 @@
import { Component, inject, signal } from '@angular/core';
import { takeUntilDestroyed } from '@angular/core/rxjs-interop';
import { interval } from 'rxjs';
import { BffApiV1Service, type WerkbakItem } from 'api-client';
import { UtrechtComponentsModule } from 'ui';
/**
* How often an open werkbak re-reads itself (S-26/#162, ADR-0032). Exported so the spec advances the
* clock by exactly one interval instead of hard-coding the number.
*/
export const WERKBAK_REFRESH_MS = 5_000;
/** The two decisions a behandelaar can make; the BFF validates these exact values (ADR-0013). */
type Besluit = 'goedkeuren' | 'afwijzen';
@@ -10,6 +18,11 @@ type Besluit = 'goedkeuren' | 'afwijzen';
* Flowable `Beoordelen` tasks, read through the domain) and decides each — goedkeuren or afwijzen. A
* decision posts to the BFF, which applies the domain transition and completes the workflow task
* (ADR-0013; S-12). After a decision the werkbak refreshes so the handled item drops off the list.
*
* The page also re-reads itself every {@link WERKBAK_REFRESH_MS} while it is open, so a registration
* that reaches beoordeling after the behandelaar opened the werkbak shows up on its own — no reload
* (S-26/#162). Polling rather than a pushed stream: nothing notifies the BFF either, so a stream
* would poll the domain in the BFF instead and add connection state for the same freshness (ADR-0032).
*/
@Component({
selector: 'app-werkbak-page',
@@ -27,19 +40,37 @@ export class WerkbakPage {
constructor() {
this.load();
// ponytail: a fixed interval, polled while the page lives — it keeps refreshing in a background
// tab. Gate on `document.visibilityState` if the request volume ever matters.
interval(WERKBAK_REFRESH_MS)
.pipe(takeUntilDestroyed())
.subscribe(() => this.load({ background: true }));
}
load(): void {
this.loading.set(true);
this.failed.set(false);
/**
* Read the werkbak. A `background` read is the interval refresh: it leaves the rows and the states
* the behandelaar is looking at alone until it has an answer — no loading flash on every tick, and
* a blip does not swap the list for the failure alert (the next tick recovers). Only a foreground
* read — on open, or after a decision — speaks for whether the werkbak is readable at all.
*/
load(options: { background?: boolean } = {}): void {
const background = options.background ?? false;
if (!background) {
this.loading.set(true);
this.failed.set(false);
}
this.bff.getBehandelWerkbak().subscribe({
next: (rows: WerkbakItem[]) => {
this.items.set(rows);
this.loading.set(false);
this.loaded.set(true);
// A read that came back is the answer, so a refresh also clears an earlier failure — the
// werkbak recovers on its own instead of showing the error until someone reloads.
this.failed.set(false);
},
// Surface the failure (e.g. 403 for a non-behandelaar) instead of swallowing it.
error: () => {
if (background) return;
this.items.set([]);
this.loading.set(false);
this.loaded.set(true);
@@ -0,0 +1,79 @@
# ADR-0032: The werkbak refreshes itself by polling, not by a pushed stream
- **Status:** Accepted
- **Date:** 2026-09-04
- **Deciders:** Respellion engineering
- **Slice:** #162 (proposal #163). The issue titles it S-26; that id already belongs to
the self-service resume slice (#111), so #162 is the identifier that counts.
## Context
The werkbak (S-12) is a read of the open Flowable `Beoordelen` tasks: portal → BFF
`GET /behandel/werkbak` → domain `Werkbak` query → workflow engine, each task enriched
from its aggregate. A registration reaches `Beoordelen` **asynchronously**, only once the
citizen supplies its documents and the DMN routes it (S-10a) — so it appears in a werkbak
that is already open, and until now a behandelaar had to reload the page to see it.
Three forces shape the mechanism:
- **Nothing notifies anyone.** The trigger lives in Flowable. The domain does not publish
task events, and there is no bus between the domain and the BFF.
- **The BFF is stateless** and sits behind each portal's nginx.
- **This is the repo's first live-updating view**, so the choice sets a precedent.
## Decision
**The werkbak page re-reads the existing BFF endpoint on a fixed interval
(`WERKBAK_REFRESH_MS`, 5 s) while it is open. No new endpoint, dependency or server-side
state.**
The refresh is a *background* read: it leaves the rows and the loading/failure states
untouched until it has an answer, so a tick never flashes a spinner over rows a
behandelaar is reading and a single failed poll never swaps the list for the error alert.
A read that comes back also clears an earlier failure, so the view recovers on its own —
the same reload this slice set out to remove would otherwise be needed to escape a
transient error. Only a foreground read (on open, after a decision) speaks for whether the
werkbak is readable at all.
### Why not SSE or WebSockets
Neither buys freshness here, because **nothing notifies the BFF either**:
- **SSE** (`text/event-stream`) would mean a new streaming endpoint whose handler polls the
domain and forwards diffs — the same latency, plus connection lifecycle, nginx
buffering, and auth on a long-lived connection.
- **WebSocket/SignalR** adds a dependency (CLAUDE.md §13) and makes the BFF stateful and
sticky-session-bound. A genuine push path would *also* need the domain to publish task
events. Warranted by high-frequency, bidirectional or fan-out-heavy traffic; the werkbak
is none of those.
Polling meets the acceptance ("a registration can be seen in the werkbak once it is ready
for review") in a handful of lines inside one component.
- ponytail ceiling: a fixed 5 s interval, per open page, that keeps polling in a
background tab. Each tick costs one Flowable task query plus a store read per open task.
- Upgrade path: publish task events from the domain, then swap the component's `interval`
for a stream. The endpoint contract and the component's rendering stay as they are;
gate on `document.visibilityState` first if request volume is the concern.
## Consequences
**Positive**
- The outcome is delivered with no new endpoint, dependency, or server-side state, and no
service boundary moves.
- Self-healing: a transient read failure no longer strands the view until a manual reload.
- The e2e got *simpler* — the happy path waits for the werkbak row without reloading the
page, which is itself the live-refresh assertion.
**Negative / costs**
- Staleness is bounded by one interval (≤5 s) rather than instant.
- One `GET /behandel/werkbak` per open werkbak per interval, including in hidden tabs.
- The precedent is polling; a future view with genuinely high-frequency updates will have
to revisit this (see the upgrade path above).
## Coupling rules touched (CLAUDE.md §8)
None. The poll reuses the existing portal → BFF → domain read path: §8.3 (portals talk
only to the BFF) and §8.2 (only the Workflow Client talks to Flowable) are unchanged.
+34
View File
@@ -5,6 +5,40 @@ copy-pasteable walkthrough against a local `make up` stack.
---
## S-26/#162 — the werkbak refreshes itself (ADR-0032)
**Outcome:** a registration that reaches beoordeling while a behandelaar already has the werkbak open
**appears on its own** — no reload. The page re-reads `GET /behandel/werkbak` every 5 seconds; a
background refresh swaps the rows in without flashing the loading state, and a transient failure no
longer strands the view on its error message until someone reloads.
```bash
# 1. Two windows. Left: the behandel werkbak, already open and idle.
python3 infra/keycloak/check_realms.py otp # a code, valid right now
open http://localhost:8142 # merel-behandelaar / test123 + that code
#
# 2. Right: submit a registration and supply its documents (this is what routes it to Beoordelen).
open http://localhost:8140 # jan-burger / test123 → indienen → upload a PDF
#
# 3. Watch the left window. Within ~5 seconds the new reference appears in the werkbak — the page was
# never reloaded and never left the werkbak.
#
# 4. Automated, end to end: the happy path now waits for the werkbak row WITHOUT reloading, so the
# absence of the reload IS the assertion.
make verify-e2e # → registration.spec: "… → behandelaar goedkeurt → public INGESCHREVEN"
#
# 5. Component level (background refresh, failure recovery, teardown):
pnpm nx test behandel # → "picks up a newly submitted registration without a reload" (+3 guards)
```
**The path:** unchanged — portal → BFF `GET /behandel/werkbak` → domain `Werkbak` → Flowable. Only the
page's cadence is new: `interval(WERKBAK_REFRESH_MS)` scoped to the page with `takeUntilDestroyed()`.
**Not push:** nothing notifies the BFF either, so SSE/WebSockets would poll the domain inside the BFF
for the same freshness plus connection state — see ADR-0032 for the trade-off and the upgrade path.
---
## S-19a — approval writes the register record to Objecten (#149, ADR-0028)
**Outcome:** approving a registration no longer only moves the ZGW zaak to its eindstatus — it also
-44
View File
@@ -245,47 +245,3 @@ the verify-stack check table, and per-spec e2e results (`infra/playwright-summar
- Getting a report out of the e2e container: Playwright writes `playwright-report.json`
inside the container; `infra/run-e2e-check.sh` `docker cp`s it back to the host
(capturing the test exit code first) so the summary step can read it.
---
## 9. `if: always()` does not survive the job being killed — bound the work itself
`if: always()` makes a step run when an *earlier step failed*. It does **not** help when
the job as a whole is stopped: the run's remaining steps are simply never dispatched.
That is how #161 lost its diagnosis. `verify-stack` entered `make verify-e2e` at 09:48:17
and the job ended at 10:14:54 — 26½ minutes later, mid-suite. Every step after the e2e
shows a **0-second `failure`** stamped at that same instant:
```
14 failure 09:48:17 -> 10:14:54 Self-service e2e (Playwright, login → submit → success)
15 failure 10:14:54 -> 10:14:54 verify-stack check summary ← if: always()
16 failure 10:14:54 -> 10:14:54 e2e spec summary ← if: always()
17 failure 10:14:54 -> 10:14:54 Dump container logs on failure ← if: failure()
18 failure 10:14:54 -> 10:14:54 Tear down ← if: always()
```
So the per-spec summary, the container-log dump and the teardown never ran, and the job
log — which also loses whatever the killed process had buffered — ended at a single `✘`
line. A job that dies takes its own post-mortem with it.
**Read the step timings, not just the log.** `GET /api/v1/repos/{owner}/{repo}/actions/jobs/{id}`
returns every step with `started_at`/`completed_at`; a row of identical zero-length
steps at the end means *killed*, not *silent*. (Job ids come from
`…/actions/runs/{run}/jobs`, and that route returns only the **latest attempt** — a
re-run hides the failed one, so keep the failing job id from the original report. Logs:
`…/actions/jobs/{id}/logs`, see also `gitea-ci-logs`.)
**Conventions that follow:**
- **Bound long-running work inside the tool**, where it can still report. Playwright's
`globalTimeout` (`tests/e2e/playwright.config.ts`) ends the run, writes the JSON
report and exits, so the summary and log-dump steps still get their turn. A
`timeout-minutes` on the job would reproduce the very failure above.
- **Never let an auto-waiting action be the timeout.** Playwright actions (`fill`,
`click`) inherit the *test* timeout, not `expect.timeout`, so a missing element costs
the full 90 s and reports `locator.fill: Test timeout …` — the symptom. Assert the
element visible first with its own budget and a message (`tests/e2e/keycloak-login.ts`).
- Remember `concurrency.cancel-in-progress: true` in `ci.yaml`: a new push to the same
ref, or a re-run, kills the in-flight run the same way. Check `run_attempt` before
concluding a job hung.
-20
View File
@@ -1,20 +0,0 @@
# Overlay: make the CI compose stack usable from a HOST browser.
# Same two mechanisms infra/docker-compose.local.yml already uses — pin Keycloak's issuer to the
# host-published address, and point each portal's runtime config.json at it. The BFF needs no
# change: it discovers metadata over keycloak:8080 and the discovered issuer is the pinned
# localhost:8180, which is what browser tokens carry.
services:
keycloak:
environment:
KC_HOSTNAME: http://localhost:8180
KC_HOSTNAME_BACKCHANNEL_DYNAMIC: "true"
self-service:
volumes:
- ./local-config/self-service.config.json:/usr/share/nginx/html/config.json:ro,z
behandel:
volumes:
- ./local-config/behandel.config.json:/usr/share/nginx/html/config.json:ro,z
# beheer is the same medewerker realm as behandel, so it reuses behandel's config verbatim.
beheer:
volumes:
- ./local-config/behandel.config.json:/usr/share/nginx/html/config.json:ro,z
+5 -37
View File
@@ -7,34 +7,10 @@ redirects it into $GITHUB_STEP_SUMMARY. Stdlib only.
"""
import json
import os
import re
import sys
STATUS_ICON = {"expected": "", "unexpected": "", "skipped": "⏭️", "flaky": "⚠️"}
# A verdict alone still costs a log dive, and a killed or truncated job leaves no log to dive into
# (#161) — so a failing spec carries its first error into the table. Playwright errors are multi-line
# with a "Call log:", which a markdown table cell cannot hold, so they are flattened and clipped.
ERROR_CLIP = 300
def first_error(spec):
"""The first error message across a spec's test results, flattened for one table cell."""
for test in spec.get("tests", []):
for result in test.get("results", []):
for error in result.get("errors", []):
message = (error.get("message") or "").strip()
if not message:
continue
# Strip ANSI colour, collapse to one line, and keep it inside the cell.
message = re.sub(r"\x1b\[[0-9;]*m", "", message)
message = " ".join(message.split())
if len(message) > ERROR_CLIP:
message = message[:ERROR_CLIP - 1].rstrip() + ""
# `|` would end the cell early.
return message.replace("|", "\\|")
return ""
def walk(suite, out):
for spec in suite.get("specs", []):
@@ -46,8 +22,7 @@ def walk(suite, out):
else "expected" if spec.get("ok", False)
else "unexpected")
out.append({"file": spec.get("file") or suite.get("file") or suite.get("title", ""),
"title": spec.get("title", ""), "status": status,
"error": first_error(spec) if status in ("unexpected", "flaky") else ""})
"title": spec.get("title", ""), "status": status})
for child in suite.get("suites", []):
walk(child, out)
@@ -71,17 +46,10 @@ def main(path):
if not specs:
print("_No specs ran._")
return 0
# The failure column only earns its width when something failed.
if any(s["error"] for s in specs):
print("| Spec | Result | Why |")
print("| ---- | :----: | --- |")
for s in specs:
print(f"| {s['file']} {s['title']} | {STATUS_ICON.get(s['status'], '')} | {s['error']} |")
else:
print("| Spec | Result |")
print("| ---- | :----: |")
for s in specs:
print(f"| {s['file']} {s['title']} | {STATUS_ICON.get(s['status'], '')} |")
print("| Spec | Result |")
print("| ---- | :----: |")
for s in specs:
print(f"| {s['file']} {s['title']} | {STATUS_ICON.get(s['status'], '')} |")
return 0
-108
View File
@@ -1,108 +0,0 @@
#!/usr/bin/env python3
"""Self-check for infra/playwright-summary.py — stdlib asserts, no framework.
Run: python3 infra/test_playwright_summary.py (also runs in `make unit`).
A red e2e is only useful if the job summary says WHY it failed: #161 lost a 36-minute
verify-stack job whose only surviving output was one ✘ line with no assertion detail.
"""
import importlib.util
import io
import json
import os
import tempfile
from contextlib import redirect_stdout
# The script's filename is not a valid module name, so load it by path.
spec = importlib.util.spec_from_file_location(
"playwright_summary",
os.path.join(os.path.dirname(os.path.abspath(__file__)), "playwright-summary.py"),
)
summary = importlib.util.module_from_spec(spec)
spec.loader.exec_module(summary)
def render(report):
"""Run the renderer over a report dict and return its markdown."""
with tempfile.NamedTemporaryFile("w", suffix=".json", delete=False) as fh:
json.dump(report, fh)
path = fh.name
try:
out = io.StringIO()
with redirect_stdout(out):
summary.main(path)
return out.getvalue()
finally:
os.unlink(path)
def spec_entry(title, status, errors=()):
return {
"title": title,
"file": "catalogus.spec.ts",
"ok": status == "expected",
"tests": [{"status": status, "results": [{"errors": [{"message": m} for m in errors]}]}],
}
def test_failing_spec_reports_why():
md = render({
"stats": {"expected": 4, "unexpected": 1, "flaky": 0, "skipped": 0, "duration": 108_000},
"suites": [{"file": "catalogus.spec.ts", "specs": [
spec_entry("a beheerder sees the published zaaktypen in the catalogus", "unexpected",
["locator.fill: Test timeout of 90000ms exceeded.\n"
"Call log:\n - waiting for locator('#username')\n"]),
]}],
})
assert "" in md, md
# The point of the slice: the summary names the cause, not just the verdict.
assert "Test timeout of 90000ms exceeded" in md, md
assert "waiting for locator('#username')" in md, md
# A multi-line Playwright error must not break out of its table row.
assert not any(line.startswith("Call log:") for line in md.splitlines()), md
def test_real_playwright_error_is_flattened():
# A real report's message is multi-line and ANSI-coloured, and embeds the source snippet with
# `|` gutters — all three would break the table cell. Shape verified against an actual
# @playwright/test 1.61 JSON report.
md = render({
"stats": {"expected": 0, "unexpected": 1, "flaky": 0, "skipped": 0, "duration": 1_000},
"suites": [{"file": "catalogus.spec.ts", "specs": [
spec_entry("a beheerder sees the catalogus", "unexpected",
["Error: expect(locator).toBeVisible() failed\n\n"
"\x1b[2mLocator: \x1b[22mgetByRole('heading')\n"
" 12 | await login(page);\n> 13 | await expect(heading).toBeVisible();\n"]),
]}],
})
row = [line for line in md.splitlines() if line.startswith("| catalogus.spec.ts")][0]
assert "\x1b" not in row, row
assert "Locator: getByRole('heading')" in row, row
# Every literal `|` from the snippet gutters is escaped, so the row keeps exactly 3 cells.
assert row.count("|") - row.count("\\|") == 4, row
def test_passing_run_stays_quiet():
md = render({
"stats": {"expected": 1, "unexpected": 0, "flaky": 0, "skipped": 0, "duration": 5_000},
"suites": [{"file": "catalogus.spec.ts",
"specs": [spec_entry("a beheerder sees the catalogus", "expected")]}],
})
assert "" in md, md
assert "timeout" not in md.lower(), md
def test_missing_report_is_not_a_crash():
out = io.StringIO()
with redirect_stdout(out):
rc = summary.main("/nonexistent/playwright-report.json")
assert rc == 0
assert "did not reach the e2e step" in out.getvalue()
if __name__ == "__main__":
for name, fn in sorted(globals().items()):
if name.startswith("test_") and callable(fn):
fn()
print(f" ok {name}")
print("playwright-summary self-check passed")
+1 -1
View File
@@ -1,5 +1,5 @@
import { expect, test } from '@playwright/test';
import { loginMedewerker } from './keycloak-login';
import { loginMedewerker } from './medewerker-login';
// S-15a walking skeleton: a beheerder logs in to the beheer portal (medewerker realm) and sees the
// read-only ZTC catalogus. The verify stack seeds and publishes the BIG-REGISTRATIE zaaktype (the
+1 -1
View File
@@ -1,5 +1,5 @@
import { expect, test } from '@playwright/test';
import { loginMedewerker } from './keycloak-login';
import { loginMedewerker } from './medewerker-login';
// S-15b: a beheerder edits the ACL default-fill in the beheer portal and gets a saved confirmation.
// Runs against the shared verify stack; it edits + saves (the ACL store is in-memory, ADR-0026) and
-99
View File
@@ -1,99 +0,0 @@
import { createHmac } from 'node:crypto';
import { readFileSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { expect, type Page } from '@playwright/test';
// Every portal login in the suite goes through this module — citizen realms (mock DigiD) and the
// medewerker realm alike — so the shared Keycloak form handling lives in exactly one place.
// The medewerker realm enforces MFA (S-15c), so a staff login is two steps: password, then a TOTP
// code. The realm export seeds every medewerker with this fixture secret — Keycloak HMACs the raw
// secret bytes — so the e2e can compute a valid code instead of enrolling an authenticator.
const OTP_SECRET = 'BIGMEDEWERKEROTPSEED';
export const OTP_PERIOD_MS = 30_000;
/**
* How long a Keycloak form gets to appear. Generous enough for a cold first browser launch and a
* loaded stack, far short of the 90-second test timeout an auto-waiting action would otherwise eat.
*/
const FORM_TIMEOUT_MS = 20_000;
const FORM_NEVER_APPEARED =
'the Keycloak login form never appeared — the portal did not reach Keycloak (check its ' +
'config.json fetch and the OIDC discovery on the authority it was built with)';
const OTP_NEVER_APPEARED =
'the Keycloak OTP form never appeared — the password step did not complete (check the ' +
'medewerker realm seeded this user with both a password and a TOTP credential)';
// RFC 6238 TOTP: HMAC-SHA1 over the 30-second counter, dynamically truncated to 6 digits.
export function totp(secret = OTP_SECRET, at = Date.now()): string {
const counter = Buffer.alloc(8);
counter.writeBigUInt64BE(BigInt(Math.floor(at / OTP_PERIOD_MS)));
const mac = createHmac('sha1', secret).update(counter).digest();
const offset = mac[mac.length - 1] & 0x0f;
return String((mac.readUInt32BE(offset) & 0x7fffffff) % 1_000_000).padStart(6, '0');
}
// Keycloak refuses a TOTP code it has already accepted (its otpPolicyCodeReusable defaults to
// false), so two logins as the same medewerker inside one 30-second window would both submit the
// same code and the second is rejected. Spend the first counter this medewerker has left.
export function nextUnusedCounter(now: number, spent: number): number {
return Math.max(Math.floor(now / OTP_PERIOD_MS), spent + 1);
}
// The spent counter lives on disk rather than in module state: Playwright starts a fresh worker
// process for a retry, which would otherwise forget it and resubmit the rejected code.
function spendCounter(username: string): number {
const file = join(tmpdir(), `otp-counter-${username}`);
let spent = -1;
try {
spent = Number(readFileSync(file, 'utf8')) || -1;
} catch {
// first login as this medewerker in this run
}
const counter = nextUnusedCounter(Date.now(), spent);
writeFileSync(file, String(counter));
return counter;
}
/**
* Fill Keycloak's login form. Every portal is guarded, so the first navigation redirects here; the
* form ids are stable across themes.
*
* The form is asserted visible *before* it is filled. A portal that never reaches Keycloak — its
* runtime `config.json` fetch or the OIDC discovery behind `authorize()` failed, so it never
* bootstrapped and shows a blank page (main.ts only logs to the console) — would otherwise leave
* `fill()` auto-waiting until the whole test times out: 90 seconds spent to report
* `locator.fill: Test timeout of 90000ms exceeded`, naming the symptom and not the cause. That is
* how #161's catalogus.spec burned 1.8 minutes. This fails in a quarter of the time and says which
* step never happened.
*/
async function submitPassword(page: Page, username: string): Promise<void> {
await expect(page.locator('#username'), FORM_NEVER_APPEARED).toBeVisible({ timeout: FORM_TIMEOUT_MS });
await page.locator('#username').fill(username);
await page.locator('#password').fill('test123');
await page.locator('#kc-login').click();
}
/** A citizen login on a mock-DigiD realm — no second factor (ADR-0031). */
export async function loginBurger(page: Page, username: string): Promise<void> {
await submitPassword(page, username);
}
/** A staff login on the medewerker realm: password, then the enforced TOTP second factor. */
export async function loginMedewerker(page: Page, username: string): Promise<void> {
await submitPassword(page, username);
// Keycloak's conditional-OTP step. Same reasoning as the password form above: assert it arrived
// rather than letting `fill()` swallow the test timeout.
await expect(page.locator('#otp'), OTP_NEVER_APPEARED).toBeVisible({ timeout: FORM_TIMEOUT_MS });
// Wait out the rest of the window if the counter we may spend is still in the future; Keycloak's
// lookAheadWindow would accept the code a moment early, but only by one counter — waiting keeps a
// third login in the same window valid too.
const counter = spendCounter(username);
await page.waitForTimeout(Math.max(0, counter * OTP_PERIOD_MS - Date.now()));
await page.locator('#otp').fill(totp(OTP_SECRET, counter * OTP_PERIOD_MS));
await page.locator('#kc-login').click();
}
@@ -1,5 +1,5 @@
import { expect, test } from '@playwright/test';
import { OTP_PERIOD_MS, nextUnusedCounter } from './keycloak-login';
import { OTP_PERIOD_MS, nextUnusedCounter } from './medewerker-login';
// Pure check of the TOTP counter guard in loginMedewerker — no browser, no stack. Keycloak refuses
// a code it has already accepted (its otpPolicyCodeReusable defaults to false), so two logins as
+57
View File
@@ -0,0 +1,57 @@
import { createHmac } from 'node:crypto';
import { readFileSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import type { Page } from '@playwright/test';
// The medewerker realm enforces MFA (S-15c), so a staff login is two steps: password, then a TOTP
// code. The realm export seeds every medewerker with this fixture secret — Keycloak HMACs the raw
// secret bytes — so the e2e can compute a valid code instead of enrolling an authenticator.
const OTP_SECRET = 'BIGMEDEWERKEROTPSEED';
export const OTP_PERIOD_MS = 30_000;
// RFC 6238 TOTP: HMAC-SHA1 over the 30-second counter, dynamically truncated to 6 digits.
export function totp(secret = OTP_SECRET, at = Date.now()): string {
const counter = Buffer.alloc(8);
counter.writeBigUInt64BE(BigInt(Math.floor(at / OTP_PERIOD_MS)));
const mac = createHmac('sha1', secret).update(counter).digest();
const offset = mac[mac.length - 1] & 0x0f;
return String((mac.readUInt32BE(offset) & 0x7fffffff) % 1_000_000).padStart(6, '0');
}
// Keycloak refuses a TOTP code it has already accepted (its otpPolicyCodeReusable defaults to
// false), so two logins as the same medewerker inside one 30-second window would both submit the
// same code and the second is rejected. Spend the first counter this medewerker has left.
export function nextUnusedCounter(now: number, spent: number): number {
return Math.max(Math.floor(now / OTP_PERIOD_MS), spent + 1);
}
// The spent counter lives on disk rather than in module state: Playwright starts a fresh worker
// process for a retry, which would otherwise forget it and resubmit the rejected code.
function spendCounter(username: string): number {
const file = join(tmpdir(), `otp-counter-${username}`);
let spent = -1;
try {
spent = Number(readFileSync(file, 'utf8')) || -1;
} catch {
// first login as this medewerker in this run
}
const counter = nextUnusedCounter(Date.now(), spent);
writeFileSync(file, String(counter));
return counter;
}
export async function loginMedewerker(page: Page, username: string): Promise<void> {
await page.locator('#username').fill(username);
await page.locator('#password').fill('test123');
await page.locator('#kc-login').click();
// Keycloak's conditional-OTP step. Wait out the rest of the window if the counter we may spend is
// still in the future; its lookAheadWindow would accept the code a moment early, but only by one
// counter — waiting keeps a third login in the same window valid too.
const counter = spendCounter(username);
await page.waitForTimeout(Math.max(0, counter * OTP_PERIOD_MS - Date.now()));
await page.locator('#otp').fill(totp(OTP_SECRET, counter * OTP_PERIOD_MS));
await page.locator('#kc-login').click();
}
-6
View File
@@ -15,12 +15,6 @@ export default defineConfig({
timeout: 90_000,
expect: { timeout: 15_000 },
retries: 1,
// Bound the whole run, not just each test (#161). A wedged suite used to run until CI killed the
// job — which also killed the `if: always()` steps that would have said why: the per-spec summary
// and the container-log dump never ran, leaving a 36-minute job whose entire surviving output was
// one ✘ line. On `globalTimeout` Playwright stops and *reports*, so the JSON report is written and
// those steps still run. Generous over the ~1-minute suite: this is a backstop, not a budget.
globalTimeout: 12 * 60_000,
// Run the specs serially. Each spec drives a full `channel: 'chromium'` browser, and the e2e
// shares an 8 GB runner with the entire compose stack (OpenZaak, NRC, Keycloak, Flowable, 4×
// Postgres, every service + 3 portals). Two parallel browsers exhaust memory and the renderer is
+29 -22
View File
@@ -1,5 +1,5 @@
import { expect, request, test } from '@playwright/test';
import { loginBurger, loginMedewerker } from './keycloak-login';
import { loginMedewerker } from './medewerker-login';
// Walking-skeleton happy path (S-08d + S-09 + S-09b + S-12 + S-10a + S-19b-2): a zorgprofessional
// logs in via mock DigiD and submits through the self-service portal → BFF → domain; the entry
@@ -23,7 +23,9 @@ test('DigiD submit → public INGEDIEND → documenten → behandelaar goedkeurt
// checks submit as jan-burger (bsn 123456782) before the e2e runs on the shared stack, and
// resume-on-load (S-26) would otherwise restore one of those on login — so each self-service spec
// uses a dedicated citizen no other actor touches.
await loginBurger(page, 'emma-burger');
await page.locator('#username').fill('emma-burger');
await page.locator('#password').fill('test123');
await page.locator('#kc-login').click();
// Back on the portal, authenticated.
await expect(page.getByRole('heading', { name: /Zelfservice/i })).toBeVisible();
@@ -57,12 +59,32 @@ test('DigiD submit → public INGEDIEND → documenten → behandelaar goedkeurt
await expect(staff.getByRole('row', { name: reference }).getByRole('cell', { name: 'INGEDIEND' }))
.toBeVisible();
// A behandelaar opens the behandel-portal werkbak and approves the registration (goedkeuren) — the
// S-12 flow that replaces the temporary admin endpoint. The staff tab switches to the medewerker
// realm (a different Keycloak realm than the citizen's digid session).
//
// The werkbak is opened BEFORE the citizen supplies the documents that route the registration to
// Beoordelen, so its row cannot be there at page load: the only thing that can deliver it to this
// already-open page is the werkbak refreshing itself (S-26/#162, ADR-0032). This spec used to
// `staff.reload()` in a poll loop here; the absence of that reload is the live-refresh assertion.
await staff.goto('http://behandel/');
// That realm enforces MFA (S-15c), so the behandelaar logs in with password + TOTP.
await loginMedewerker(staff, 'merel-behandelaar');
await expect(staff.getByRole('heading', { name: /Werkbak/i })).toBeVisible();
// Target the decide button by reference (not a generic "Goedkeuren"): the shared verify stack holds
// other open tasks, so a positional match could act on someone else's registration.
const goedkeuren = staff.getByRole('button', { name: `Goedkeuren ${reference}` });
await expect(goedkeuren, 'the registration is not awaiting beoordeling yet').toBeHidden();
// Provide the documents the registration is waiting for (S-10a), on the still-open self-service tab.
// The process parks at WachtOpDocumenten only after the zaak is opened; the INGEDIEND row above proves
// the zaak exists — so the OpenZaak worker has completed and the process is now at the wait — which is
// why we supply the documents here rather than right after submit, when the trigger would race the
// wait and no-op. (S-10b turns this into a real file upload; here it is the trigger that unblocks
// beoordeling.)
await page.bringToFront();
await page.setInputFiles('#diploma', {
name: 'diploma.pdf',
mimeType: 'application/pdf',
@@ -71,26 +93,11 @@ test('DigiD submit → public INGEDIEND → documenten → behandelaar goedkeurt
await page.getByRole('button', { name: /documenten aanleveren/i }).click();
await expect(page.getByText(/documenten zijn aangeleverd/i)).toBeVisible();
// A behandelaar picks the registration up in the behandel-portal werkbak and approves it (goedkeuren)
// — the S-12 flow that replaces the temporary admin endpoint. The staff tab switches to the
// medewerker realm (a different Keycloak realm than the citizen's digid session).
await staff.goto('http://behandel/');
// That realm enforces MFA (S-15c), so the behandelaar logs in with password + TOTP.
await loginMedewerker(staff, 'merel-behandelaar');
await expect(staff.getByRole('heading', { name: /Werkbak/i })).toBeVisible();
// The registration reaches the Beoordelen user task only after its documents are provided (above), so
// it appears in the werkbak asynchronously — reload until this reference's row shows up. Target the
// decide button by reference (not a generic "Goedkeuren"): the shared verify stack holds other open
// tasks, so a positional match could act on someone else's registration.
const goedkeuren = staff.getByRole('button', { name: `Goedkeuren ${reference}` });
await expect
.poll(async () => {
await staff.reload();
return goedkeuren.count();
}, { timeout: 30_000, intervals: [1_000, 2_000, 3_000, 5_000] })
.toBeGreaterThan(0);
// Back to the werkbak — untouched since login, never reloaded. The row arrives on its own once the
// DMN routes the registration to Beoordelen. (Foregrounded so Chromium doesn't throttle the page's
// refresh timer as a hidden tab.)
await staff.bringToFront();
await expect(goedkeuren).toBeVisible({ timeout: 30_000 });
// Click and wait for the decide POST to finish (204) BEFORE leaving the page. `click()` only
// dispatches the request; navigating away immediately cancels it in flight (nginx logs a 499) and
+3 -2
View File
@@ -1,5 +1,4 @@
import { expect, test } from '@playwright/test';
import { loginBurger } from './keycloak-login';
// S-26: a zorgprofessional submits, then reloads the self-service portal. On load the portal asks the
// BFF for the caller's current open registration (owner-scoped by the DigiD token's bsn) and restores
@@ -10,7 +9,9 @@ test('DigiD submit → reload → self-service restores the existing registratio
// Its own DigiD user (like every self-service spec): on the shared verify stack, resume-on-load
// (S-26) restores any open registration for the bsn, so each spec uses a dedicated citizen that no
// other spec or verify-* check touches. This one in particular leaves an open registration.
await loginBurger(page, 'sanne-burger');
await page.locator('#username').fill('sanne-burger');
await page.locator('#password').fill('test123');
await page.locator('#kc-login').click();
await expect(page.getByRole('heading', { name: /Zelfservice/i })).toBeVisible();
await page.getByRole('button', { name: /indienen/i }).click();
+3 -2
View File
@@ -1,5 +1,4 @@
import { expect, test } from '@playwright/test';
import { loginBurger } from './keycloak-login';
// S-11 (Flow 3): a zorgprofessional logs in via mock DigiD, submits a registration, then withdraws
// it ("trek aanvraag in") from the self-service portal. The withdrawal goes portal → BFF (owner-
@@ -11,7 +10,9 @@ test('DigiD submit → trek aanvraag in → self-service confirms ingetrokken',
// Its own DigiD user — isolated from the verify-* checks (jan-burger/123456782) so resume-on-load
// (S-26) can't restore someone else's registration on the shared stack.
await loginBurger(page, 'lars-burger');
await page.locator('#username').fill('lars-burger');
await page.locator('#password').fill('test123');
await page.locator('#kc-login').click();
await expect(page.getByRole('heading', { name: /Zelfservice/i })).toBeVisible();