Compare commits
6
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6e00deb6ee | ||
|
|
ee8301f39f | ||
|
|
eade6b0a73 | ||
|
|
0fe4813388 | ||
|
|
162e495f29 | ||
|
|
d07aa2f64f |
@@ -64,9 +64,6 @@ frontend:
|
|||||||
## lint: verify formatting (no changes)
|
## lint: verify formatting (no changes)
|
||||||
lint:
|
lint:
|
||||||
dotnet format $(SLN) --verify-no-changes
|
dotnet format $(SLN) --verify-no-changes
|
||||||
# Only pages in mkdocs.yml's nav are published, and mkdocs keeps a build green
|
|
||||||
# when one is missing — so the nav is checked here rather than not at all.
|
|
||||||
python3 infra/check-docs-nav.py
|
|
||||||
|
|
||||||
## build: release build
|
## build: release build
|
||||||
build:
|
build:
|
||||||
@@ -74,11 +71,8 @@ build:
|
|||||||
|
|
||||||
## unit: run unit tests (excludes the container-backed Integration lane)
|
## unit: run unit tests (excludes the container-backed Integration lane)
|
||||||
# TRX per test project (→ TestResults/) feeds the CI per-service summary (#136); harmless locally.
|
# TRX per test project (→ TestResults/) feeds the CI per-service summary (#136); harmless locally.
|
||||||
# The CI reporting scripts are stdlib Python with their own assert-based self-checks (#161) — they
|
|
||||||
# ride this lane so a broken job summary is caught by CI rather than by the next red pipeline.
|
|
||||||
unit:
|
unit:
|
||||||
dotnet test $(SLN) -c Release --filter "Category!=Integration" --logger trx --results-directory TestResults
|
dotnet test $(SLN) -c Release --filter "Category!=Integration" --logger trx --results-directory TestResults
|
||||||
python3 infra/test_playwright_summary.py
|
|
||||||
python3 infra/test_portal_caddyfiles.py
|
python3 infra/test_portal_caddyfiles.py
|
||||||
|
|
||||||
## mutation: run the Stryker.NET ratchet on each service with branching logic (fails below baseline)
|
## mutation: run the Stryker.NET ratchet on each service with branching logic (fails below baseline)
|
||||||
|
|||||||
@@ -3,8 +3,8 @@
|
|||||||
- **Status:** Accepted
|
- **Status:** Accepted
|
||||||
- **Date:** 2026-09-04
|
- **Date:** 2026-09-04
|
||||||
- **Deciders:** Respellion engineering
|
- **Deciders:** Respellion engineering
|
||||||
- **Slice:** #25 (S-24) — raised directly as a deployment-target request and matched to
|
- **Slice:** _(none yet — raised directly as a deployment-target request; see
|
||||||
that issue afterwards; see the "Process note" at the end
|
"Process note" at the end)_
|
||||||
|
|
||||||
## Context
|
## Context
|
||||||
|
|
||||||
|
|||||||
@@ -14,8 +14,6 @@ should teach.
|
|||||||
In Dutch; the strategic framing lives in `Respellion/innovation-lab`.
|
In Dutch; the strategic framing lives in `Respellion/innovation-lab`.
|
||||||
- **[Working in Gitea](gitea-workflow.md)** — issues, milestones, branches, PRs.
|
- **[Working in Gitea](gitea-workflow.md)** — issues, milestones, branches, PRs.
|
||||||
- **[CI runbook](runbooks/ci.md)** — the pipeline and the `make ci` local gate.
|
- **[CI runbook](runbooks/ci.md)** — the pipeline and the `make ci` local gate.
|
||||||
- **[Kubernetes on Talos](runbooks/kubernetes-talos.md)** — the second deployment target:
|
|
||||||
one Helm chart, a single-node cluster, and the parts that bite (ADR-0033).
|
|
||||||
|
|
||||||
## Quickstart
|
## Quickstart
|
||||||
|
|
||||||
|
|||||||
@@ -245,47 +245,3 @@ the verify-stack check table, and per-spec e2e results (`infra/playwright-summar
|
|||||||
- Getting a report out of the e2e container: Playwright writes `playwright-report.json`
|
- Getting a report out of the e2e container: Playwright writes `playwright-report.json`
|
||||||
inside the container; `infra/run-e2e-check.sh` `docker cp`s it back to the host
|
inside the container; `infra/run-e2e-check.sh` `docker cp`s it back to the host
|
||||||
(capturing the test exit code first) so the summary step can read it.
|
(capturing the test exit code first) so the summary step can read it.
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 9. `if: always()` does not survive the job being killed — bound the work itself
|
|
||||||
|
|
||||||
`if: always()` makes a step run when an *earlier step failed*. It does **not** help when
|
|
||||||
the job as a whole is stopped: the run's remaining steps are simply never dispatched.
|
|
||||||
|
|
||||||
That is how #161 lost its diagnosis. `verify-stack` entered `make verify-e2e` at 09:48:17
|
|
||||||
and the job ended at 10:14:54 — 26½ minutes later, mid-suite. Every step after the e2e
|
|
||||||
shows a **0-second `failure`** stamped at that same instant:
|
|
||||||
|
|
||||||
```
|
|
||||||
14 failure 09:48:17 -> 10:14:54 Self-service e2e (Playwright, login → submit → success)
|
|
||||||
15 failure 10:14:54 -> 10:14:54 verify-stack check summary ← if: always()
|
|
||||||
16 failure 10:14:54 -> 10:14:54 e2e spec summary ← if: always()
|
|
||||||
17 failure 10:14:54 -> 10:14:54 Dump container logs on failure ← if: failure()
|
|
||||||
18 failure 10:14:54 -> 10:14:54 Tear down ← if: always()
|
|
||||||
```
|
|
||||||
|
|
||||||
So the per-spec summary, the container-log dump and the teardown never ran, and the job
|
|
||||||
log — which also loses whatever the killed process had buffered — ended at a single `✘`
|
|
||||||
line. A job that dies takes its own post-mortem with it.
|
|
||||||
|
|
||||||
**Read the step timings, not just the log.** `GET /api/v1/repos/{owner}/{repo}/actions/jobs/{id}`
|
|
||||||
returns every step with `started_at`/`completed_at`; a row of identical zero-length
|
|
||||||
steps at the end means *killed*, not *silent*. (Job ids come from
|
|
||||||
`…/actions/runs/{run}/jobs`, and that route returns only the **latest attempt** — a
|
|
||||||
re-run hides the failed one, so keep the failing job id from the original report. Logs:
|
|
||||||
`…/actions/jobs/{id}/logs`, see also `gitea-ci-logs`.)
|
|
||||||
|
|
||||||
**Conventions that follow:**
|
|
||||||
|
|
||||||
- **Bound long-running work inside the tool**, where it can still report. Playwright's
|
|
||||||
`globalTimeout` (`tests/e2e/playwright.config.ts`) ends the run, writes the JSON
|
|
||||||
report and exits, so the summary and log-dump steps still get their turn. A
|
|
||||||
`timeout-minutes` on the job would reproduce the very failure above.
|
|
||||||
- **Never let an auto-waiting action be the timeout.** Playwright actions (`fill`,
|
|
||||||
`click`) inherit the *test* timeout, not `expect.timeout`, so a missing element costs
|
|
||||||
the full 90 s and reports `locator.fill: Test timeout …` — the symptom. Assert the
|
|
||||||
element visible first with its own budget and a message (`tests/e2e/keycloak-login.ts`).
|
|
||||||
- Remember `concurrency.cancel-in-progress: true` in `ci.yaml`: a new push to the same
|
|
||||||
ref, or a re-run, kills the in-flight run the same way. Check `run_attempt` before
|
|
||||||
concluding a job hung.
|
|
||||||
|
|||||||
@@ -1,30 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Fail when a page under docs/ is missing from mkdocs.yml's nav.
|
|
||||||
|
|
||||||
docs/ is the source of truth (CLAUDE.md §12), but only the pages listed in the nav
|
|
||||||
are published — and mkdocs' own `omitted_files: warn` keeps a build green while
|
|
||||||
silently dropping them, which is how every ADR after 0010 and every runbook but
|
|
||||||
ci.md fell off the site.
|
|
||||||
|
|
||||||
ponytail: a substring test, not a YAML parse — a page's path either appears in
|
|
||||||
mkdocs.yml or it doesn't, and that needs no dependency.
|
|
||||||
"""
|
|
||||||
|
|
||||||
import sys
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[1]
|
|
||||||
nav = (ROOT / "mkdocs.yml").read_text()
|
|
||||||
|
|
||||||
missing = sorted(
|
|
||||||
str(page.relative_to(ROOT / "docs"))
|
|
||||||
for page in (ROOT / "docs").rglob("*.md")
|
|
||||||
if str(page.relative_to(ROOT / "docs")) not in nav
|
|
||||||
)
|
|
||||||
|
|
||||||
if missing:
|
|
||||||
print(f"{len(missing)} page(s) under docs/ are not in mkdocs.yml's nav:")
|
|
||||||
print("\n".join(f" {m}" for m in missing))
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
print("docs nav complete: every page under docs/ is published")
|
|
||||||
@@ -1,20 +0,0 @@
|
|||||||
# Overlay: make the CI compose stack usable from a HOST browser.
|
|
||||||
# Same two mechanisms infra/docker-compose.local.yml already uses — pin Keycloak's issuer to the
|
|
||||||
# host-published address, and point each portal's runtime config.json at it. The BFF needs no
|
|
||||||
# change: it discovers metadata over keycloak:8080 and the discovered issuer is the pinned
|
|
||||||
# localhost:8180, which is what browser tokens carry.
|
|
||||||
services:
|
|
||||||
keycloak:
|
|
||||||
environment:
|
|
||||||
KC_HOSTNAME: http://localhost:8180
|
|
||||||
KC_HOSTNAME_BACKCHANNEL_DYNAMIC: "true"
|
|
||||||
self-service:
|
|
||||||
volumes:
|
|
||||||
- ./local-config/self-service.config.json:/usr/share/caddy/config.json:ro,z
|
|
||||||
behandel:
|
|
||||||
volumes:
|
|
||||||
- ./local-config/behandel.config.json:/usr/share/caddy/config.json:ro,z
|
|
||||||
# beheer is the same medewerker realm as behandel, so it reuses behandel's config verbatim.
|
|
||||||
beheer:
|
|
||||||
volumes:
|
|
||||||
- ./local-config/behandel.config.json:/usr/share/caddy/config.json:ro,z
|
|
||||||
@@ -7,34 +7,10 @@ redirects it into $GITHUB_STEP_SUMMARY. Stdlib only.
|
|||||||
"""
|
"""
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
import re
|
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
STATUS_ICON = {"expected": "✅", "unexpected": "❌", "skipped": "⏭️", "flaky": "⚠️"}
|
STATUS_ICON = {"expected": "✅", "unexpected": "❌", "skipped": "⏭️", "flaky": "⚠️"}
|
||||||
|
|
||||||
# A verdict alone still costs a log dive, and a killed or truncated job leaves no log to dive into
|
|
||||||
# (#161) — so a failing spec carries its first error into the table. Playwright errors are multi-line
|
|
||||||
# with a "Call log:", which a markdown table cell cannot hold, so they are flattened and clipped.
|
|
||||||
ERROR_CLIP = 300
|
|
||||||
|
|
||||||
|
|
||||||
def first_error(spec):
|
|
||||||
"""The first error message across a spec's test results, flattened for one table cell."""
|
|
||||||
for test in spec.get("tests", []):
|
|
||||||
for result in test.get("results", []):
|
|
||||||
for error in result.get("errors", []):
|
|
||||||
message = (error.get("message") or "").strip()
|
|
||||||
if not message:
|
|
||||||
continue
|
|
||||||
# Strip ANSI colour, collapse to one line, and keep it inside the cell.
|
|
||||||
message = re.sub(r"\x1b\[[0-9;]*m", "", message)
|
|
||||||
message = " ".join(message.split())
|
|
||||||
if len(message) > ERROR_CLIP:
|
|
||||||
message = message[:ERROR_CLIP - 1].rstrip() + "…"
|
|
||||||
# `|` would end the cell early.
|
|
||||||
return message.replace("|", "\\|")
|
|
||||||
return ""
|
|
||||||
|
|
||||||
|
|
||||||
def walk(suite, out):
|
def walk(suite, out):
|
||||||
for spec in suite.get("specs", []):
|
for spec in suite.get("specs", []):
|
||||||
@@ -46,8 +22,7 @@ def walk(suite, out):
|
|||||||
else "expected" if spec.get("ok", False)
|
else "expected" if spec.get("ok", False)
|
||||||
else "unexpected")
|
else "unexpected")
|
||||||
out.append({"file": spec.get("file") or suite.get("file") or suite.get("title", ""),
|
out.append({"file": spec.get("file") or suite.get("file") or suite.get("title", ""),
|
||||||
"title": spec.get("title", ""), "status": status,
|
"title": spec.get("title", ""), "status": status})
|
||||||
"error": first_error(spec) if status in ("unexpected", "flaky") else ""})
|
|
||||||
for child in suite.get("suites", []):
|
for child in suite.get("suites", []):
|
||||||
walk(child, out)
|
walk(child, out)
|
||||||
|
|
||||||
@@ -71,17 +46,10 @@ def main(path):
|
|||||||
if not specs:
|
if not specs:
|
||||||
print("_No specs ran._")
|
print("_No specs ran._")
|
||||||
return 0
|
return 0
|
||||||
# The failure column only earns its width when something failed.
|
print("| Spec | Result |")
|
||||||
if any(s["error"] for s in specs):
|
print("| ---- | :----: |")
|
||||||
print("| Spec | Result | Why |")
|
for s in specs:
|
||||||
print("| ---- | :----: | --- |")
|
print(f"| {s['file']} › {s['title']} | {STATUS_ICON.get(s['status'], '❔')} |")
|
||||||
for s in specs:
|
|
||||||
print(f"| {s['file']} › {s['title']} | {STATUS_ICON.get(s['status'], '❔')} | {s['error']} |")
|
|
||||||
else:
|
|
||||||
print("| Spec | Result |")
|
|
||||||
print("| ---- | :----: |")
|
|
||||||
for s in specs:
|
|
||||||
print(f"| {s['file']} › {s['title']} | {STATUS_ICON.get(s['status'], '❔')} |")
|
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -1,108 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Self-check for infra/playwright-summary.py — stdlib asserts, no framework.
|
|
||||||
|
|
||||||
Run: python3 infra/test_playwright_summary.py (also runs in `make unit`).
|
|
||||||
|
|
||||||
A red e2e is only useful if the job summary says WHY it failed: #161 lost a 36-minute
|
|
||||||
verify-stack job whose only surviving output was one ✘ line with no assertion detail.
|
|
||||||
"""
|
|
||||||
import importlib.util
|
|
||||||
import io
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import tempfile
|
|
||||||
from contextlib import redirect_stdout
|
|
||||||
|
|
||||||
# The script's filename is not a valid module name, so load it by path.
|
|
||||||
spec = importlib.util.spec_from_file_location(
|
|
||||||
"playwright_summary",
|
|
||||||
os.path.join(os.path.dirname(os.path.abspath(__file__)), "playwright-summary.py"),
|
|
||||||
)
|
|
||||||
summary = importlib.util.module_from_spec(spec)
|
|
||||||
spec.loader.exec_module(summary)
|
|
||||||
|
|
||||||
|
|
||||||
def render(report):
|
|
||||||
"""Run the renderer over a report dict and return its markdown."""
|
|
||||||
with tempfile.NamedTemporaryFile("w", suffix=".json", delete=False) as fh:
|
|
||||||
json.dump(report, fh)
|
|
||||||
path = fh.name
|
|
||||||
try:
|
|
||||||
out = io.StringIO()
|
|
||||||
with redirect_stdout(out):
|
|
||||||
summary.main(path)
|
|
||||||
return out.getvalue()
|
|
||||||
finally:
|
|
||||||
os.unlink(path)
|
|
||||||
|
|
||||||
|
|
||||||
def spec_entry(title, status, errors=()):
|
|
||||||
return {
|
|
||||||
"title": title,
|
|
||||||
"file": "catalogus.spec.ts",
|
|
||||||
"ok": status == "expected",
|
|
||||||
"tests": [{"status": status, "results": [{"errors": [{"message": m} for m in errors]}]}],
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def test_failing_spec_reports_why():
|
|
||||||
md = render({
|
|
||||||
"stats": {"expected": 4, "unexpected": 1, "flaky": 0, "skipped": 0, "duration": 108_000},
|
|
||||||
"suites": [{"file": "catalogus.spec.ts", "specs": [
|
|
||||||
spec_entry("a beheerder sees the published zaaktypen in the catalogus", "unexpected",
|
|
||||||
["locator.fill: Test timeout of 90000ms exceeded.\n"
|
|
||||||
"Call log:\n - waiting for locator('#username')\n"]),
|
|
||||||
]}],
|
|
||||||
})
|
|
||||||
assert "❌" in md, md
|
|
||||||
# The point of the slice: the summary names the cause, not just the verdict.
|
|
||||||
assert "Test timeout of 90000ms exceeded" in md, md
|
|
||||||
assert "waiting for locator('#username')" in md, md
|
|
||||||
# A multi-line Playwright error must not break out of its table row.
|
|
||||||
assert not any(line.startswith("Call log:") for line in md.splitlines()), md
|
|
||||||
|
|
||||||
|
|
||||||
def test_real_playwright_error_is_flattened():
|
|
||||||
# A real report's message is multi-line and ANSI-coloured, and embeds the source snippet with
|
|
||||||
# `|` gutters — all three would break the table cell. Shape verified against an actual
|
|
||||||
# @playwright/test 1.61 JSON report.
|
|
||||||
md = render({
|
|
||||||
"stats": {"expected": 0, "unexpected": 1, "flaky": 0, "skipped": 0, "duration": 1_000},
|
|
||||||
"suites": [{"file": "catalogus.spec.ts", "specs": [
|
|
||||||
spec_entry("a beheerder sees the catalogus", "unexpected",
|
|
||||||
["Error: expect(locator).toBeVisible() failed\n\n"
|
|
||||||
"\x1b[2mLocator: \x1b[22mgetByRole('heading')\n"
|
|
||||||
" 12 | await login(page);\n> 13 | await expect(heading).toBeVisible();\n"]),
|
|
||||||
]}],
|
|
||||||
})
|
|
||||||
row = [line for line in md.splitlines() if line.startswith("| catalogus.spec.ts")][0]
|
|
||||||
assert "\x1b" not in row, row
|
|
||||||
assert "Locator: getByRole('heading')" in row, row
|
|
||||||
# Every literal `|` from the snippet gutters is escaped, so the row keeps exactly 3 cells.
|
|
||||||
assert row.count("|") - row.count("\\|") == 4, row
|
|
||||||
|
|
||||||
|
|
||||||
def test_passing_run_stays_quiet():
|
|
||||||
md = render({
|
|
||||||
"stats": {"expected": 1, "unexpected": 0, "flaky": 0, "skipped": 0, "duration": 5_000},
|
|
||||||
"suites": [{"file": "catalogus.spec.ts",
|
|
||||||
"specs": [spec_entry("a beheerder sees the catalogus", "expected")]}],
|
|
||||||
})
|
|
||||||
assert "✅" in md, md
|
|
||||||
assert "timeout" not in md.lower(), md
|
|
||||||
|
|
||||||
|
|
||||||
def test_missing_report_is_not_a_crash():
|
|
||||||
out = io.StringIO()
|
|
||||||
with redirect_stdout(out):
|
|
||||||
rc = summary.main("/nonexistent/playwright-report.json")
|
|
||||||
assert rc == 0
|
|
||||||
assert "did not reach the e2e step" in out.getvalue()
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
for name, fn in sorted(globals().items()):
|
|
||||||
if name.startswith("test_") and callable(fn):
|
|
||||||
fn()
|
|
||||||
print(f" ok {name}")
|
|
||||||
print("playwright-summary self-check passed")
|
|
||||||
-31
@@ -32,30 +32,6 @@ nav:
|
|||||||
- "ADR-0008: Read projection store": architecture/adr-0008-read-projection-store.md
|
- "ADR-0008: Read projection store": architecture/adr-0008-read-projection-store.md
|
||||||
- "ADR-0009: External-task job worker": architecture/adr-0009-external-task-job-worker.md
|
- "ADR-0009: External-task job worker": architecture/adr-0009-external-task-job-worker.md
|
||||||
- "ADR-0010: BFF OIDC validation": architecture/adr-0010-bff-oidc.md
|
- "ADR-0010: BFF OIDC validation": architecture/adr-0010-bff-oidc.md
|
||||||
- "ADR-0011: Approval status flow": architecture/adr-0011-approval-status-flow.md
|
|
||||||
- "ADR-0012: Citizen reference correlation": architecture/adr-0012-citizen-reference-correlation.md
|
|
||||||
- "ADR-0013: Behandel-portal wiring": architecture/adr-0013-behandel-portal-wiring.md
|
|
||||||
- "ADR-0014: Withdrawal cancels the process": architecture/adr-0014-withdrawal-cancels-the-process.md
|
|
||||||
- "ADR-0015: Beoordeling escalation": architecture/adr-0015-beoordeling-escalation.md
|
|
||||||
- "ADR-0016: Diploma eligibility DMN": architecture/adr-0016-diploma-eligibility-dmn.md
|
|
||||||
- "ADR-0017: Document-wait timeout": architecture/adr-0017-document-wait-timeout-cancellation.md
|
|
||||||
- "ADR-0018: Diploma upload via the ACL": architecture/adr-0018-diploma-upload-via-acl-documenten.md
|
|
||||||
- "ADR-0019: Zaak cancellation on timeout": architecture/adr-0019-zaak-cancellation-on-timeout.md
|
|
||||||
- "ADR-0020: Local stack self-seeds": architecture/adr-0020-local-stack-self-seeds.md
|
|
||||||
- "ADR-0021: Zaaktype by identificatie": architecture/adr-0021-acl-resolves-zaaktype-by-identificatie.md
|
|
||||||
- "ADR-0022: Quartz scheduler": architecture/adr-0022-quartz-scheduler.md
|
|
||||||
- "ADR-0023: Observability stack": architecture/adr-0023-observability-stack.md
|
|
||||||
- "ADR-0024: Prometheus AspNetCore exporter": architecture/adr-0024-prometheus-aspnetcore-exporter.md
|
|
||||||
- "ADR-0025: BFF reads catalogus via the ACL": architecture/adr-0025-bff-reads-catalogus-via-acl.md
|
|
||||||
- "ADR-0026: Mutable default-fill store": architecture/adr-0026-mutable-default-fill-store.md
|
|
||||||
- "ADR-0027: RegisterRecord objecttype": architecture/adr-0027-registerrecord-objecttype-schema.md
|
|
||||||
- "ADR-0028: Objecten holds the register": architecture/adr-0028-objecten-holds-the-register.md
|
|
||||||
- "ADR-0029: Objecten publishes to NRC": architecture/adr-0029-objecten-publishes-to-nrc.md
|
|
||||||
- "ADR-0030: Projection sourced from the register": architecture/adr-0030-projection-sourced-from-the-register.md
|
|
||||||
- "ADR-0031: MFA on the medewerker realm": architecture/adr-0031-mfa-on-the-medewerker-realm.md
|
|
||||||
- "ADR-0032: Werkbak live refresh": architecture/adr-0032-werkbak-live-refresh.md
|
|
||||||
- "ADR-0033: Kubernetes via one Helm chart": architecture/adr-0033-kubernetes-via-one-helm-chart.md
|
|
||||||
- "ADR-0034: Caddy serves the portals": architecture/adr-0034-caddy-serves-the-portals.md
|
|
||||||
- FDS-architectuur:
|
- FDS-architectuur:
|
||||||
- Overzicht: architecture/fds/README.md
|
- Overzicht: architecture/fds/README.md
|
||||||
- Componentview (L3): architecture/fds/c4-component-view.md
|
- Componentview (L3): architecture/fds/c4-component-view.md
|
||||||
@@ -70,15 +46,8 @@ nav:
|
|||||||
- Working in Gitea: gitea-workflow.md
|
- Working in Gitea: gitea-workflow.md
|
||||||
- Frontend decisions: frontend-decisions.md
|
- Frontend decisions: frontend-decisions.md
|
||||||
- Demo script: demo-script.md
|
- Demo script: demo-script.md
|
||||||
- Synthetic data: synthetic-data.md
|
|
||||||
- Runbooks:
|
- Runbooks:
|
||||||
- CI: runbooks/ci.md
|
- CI: runbooks/ci.md
|
||||||
- OpenZaak: runbooks/openzaak.md
|
|
||||||
- Open Notificaties (NRC): runbooks/opennotificaties.md
|
|
||||||
- Keycloak: runbooks/keycloak.md
|
|
||||||
- Flowable: runbooks/flowable.md
|
|
||||||
- Kubernetes on Talos: runbooks/kubernetes-talos.md
|
|
||||||
- Gitea Actions gotchas: runbooks/gitea-actions-gotchas.md
|
|
||||||
|
|
||||||
markdown_extensions:
|
markdown_extensions:
|
||||||
- admonition
|
- admonition
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
import { expect, test } from '@playwright/test';
|
import { expect, test } from '@playwright/test';
|
||||||
import { loginMedewerker } from './keycloak-login';
|
import { loginMedewerker } from './medewerker-login';
|
||||||
|
|
||||||
// S-15a walking skeleton: a beheerder logs in to the beheer portal (medewerker realm) and sees the
|
// S-15a walking skeleton: a beheerder logs in to the beheer portal (medewerker realm) and sees the
|
||||||
// read-only ZTC catalogus. The verify stack seeds and publishes the BIG-REGISTRATIE zaaktype (the
|
// read-only ZTC catalogus. The verify stack seeds and publishes the BIG-REGISTRATIE zaaktype (the
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
import { expect, test } from '@playwright/test';
|
import { expect, test } from '@playwright/test';
|
||||||
import { loginMedewerker } from './keycloak-login';
|
import { loginMedewerker } from './medewerker-login';
|
||||||
|
|
||||||
// S-15b: a beheerder edits the ACL default-fill in the beheer portal and gets a saved confirmation.
|
// S-15b: a beheerder edits the ACL default-fill in the beheer portal and gets a saved confirmation.
|
||||||
// Runs against the shared verify stack; it edits + saves (the ACL store is in-memory, ADR-0026) and
|
// Runs against the shared verify stack; it edits + saves (the ACL store is in-memory, ADR-0026) and
|
||||||
|
|||||||
@@ -1,99 +0,0 @@
|
|||||||
import { createHmac } from 'node:crypto';
|
|
||||||
import { readFileSync, writeFileSync } from 'node:fs';
|
|
||||||
import { tmpdir } from 'node:os';
|
|
||||||
import { join } from 'node:path';
|
|
||||||
import { expect, type Page } from '@playwright/test';
|
|
||||||
|
|
||||||
// Every portal login in the suite goes through this module — citizen realms (mock DigiD) and the
|
|
||||||
// medewerker realm alike — so the shared Keycloak form handling lives in exactly one place.
|
|
||||||
|
|
||||||
// The medewerker realm enforces MFA (S-15c), so a staff login is two steps: password, then a TOTP
|
|
||||||
// code. The realm export seeds every medewerker with this fixture secret — Keycloak HMACs the raw
|
|
||||||
// secret bytes — so the e2e can compute a valid code instead of enrolling an authenticator.
|
|
||||||
const OTP_SECRET = 'BIGMEDEWERKEROTPSEED';
|
|
||||||
|
|
||||||
export const OTP_PERIOD_MS = 30_000;
|
|
||||||
|
|
||||||
/**
|
|
||||||
* How long a Keycloak form gets to appear. Generous enough for a cold first browser launch and a
|
|
||||||
* loaded stack, far short of the 90-second test timeout an auto-waiting action would otherwise eat.
|
|
||||||
*/
|
|
||||||
const FORM_TIMEOUT_MS = 20_000;
|
|
||||||
const FORM_NEVER_APPEARED =
|
|
||||||
'the Keycloak login form never appeared — the portal did not reach Keycloak (check its ' +
|
|
||||||
'config.json fetch and the OIDC discovery on the authority it was built with)';
|
|
||||||
const OTP_NEVER_APPEARED =
|
|
||||||
'the Keycloak OTP form never appeared — the password step did not complete (check the ' +
|
|
||||||
'medewerker realm seeded this user with both a password and a TOTP credential)';
|
|
||||||
|
|
||||||
// RFC 6238 TOTP: HMAC-SHA1 over the 30-second counter, dynamically truncated to 6 digits.
|
|
||||||
export function totp(secret = OTP_SECRET, at = Date.now()): string {
|
|
||||||
const counter = Buffer.alloc(8);
|
|
||||||
counter.writeBigUInt64BE(BigInt(Math.floor(at / OTP_PERIOD_MS)));
|
|
||||||
const mac = createHmac('sha1', secret).update(counter).digest();
|
|
||||||
const offset = mac[mac.length - 1] & 0x0f;
|
|
||||||
return String((mac.readUInt32BE(offset) & 0x7fffffff) % 1_000_000).padStart(6, '0');
|
|
||||||
}
|
|
||||||
|
|
||||||
// Keycloak refuses a TOTP code it has already accepted (its otpPolicyCodeReusable defaults to
|
|
||||||
// false), so two logins as the same medewerker inside one 30-second window would both submit the
|
|
||||||
// same code and the second is rejected. Spend the first counter this medewerker has left.
|
|
||||||
export function nextUnusedCounter(now: number, spent: number): number {
|
|
||||||
return Math.max(Math.floor(now / OTP_PERIOD_MS), spent + 1);
|
|
||||||
}
|
|
||||||
|
|
||||||
// The spent counter lives on disk rather than in module state: Playwright starts a fresh worker
|
|
||||||
// process for a retry, which would otherwise forget it and resubmit the rejected code.
|
|
||||||
function spendCounter(username: string): number {
|
|
||||||
const file = join(tmpdir(), `otp-counter-${username}`);
|
|
||||||
let spent = -1;
|
|
||||||
try {
|
|
||||||
spent = Number(readFileSync(file, 'utf8')) || -1;
|
|
||||||
} catch {
|
|
||||||
// first login as this medewerker in this run
|
|
||||||
}
|
|
||||||
const counter = nextUnusedCounter(Date.now(), spent);
|
|
||||||
writeFileSync(file, String(counter));
|
|
||||||
return counter;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Fill Keycloak's login form. Every portal is guarded, so the first navigation redirects here; the
|
|
||||||
* form ids are stable across themes.
|
|
||||||
*
|
|
||||||
* The form is asserted visible *before* it is filled. A portal that never reaches Keycloak — its
|
|
||||||
* runtime `config.json` fetch or the OIDC discovery behind `authorize()` failed, so it never
|
|
||||||
* bootstrapped and shows a blank page (main.ts only logs to the console) — would otherwise leave
|
|
||||||
* `fill()` auto-waiting until the whole test times out: 90 seconds spent to report
|
|
||||||
* `locator.fill: Test timeout of 90000ms exceeded`, naming the symptom and not the cause. That is
|
|
||||||
* how #161's catalogus.spec burned 1.8 minutes. This fails in a quarter of the time and says which
|
|
||||||
* step never happened.
|
|
||||||
*/
|
|
||||||
async function submitPassword(page: Page, username: string): Promise<void> {
|
|
||||||
await expect(page.locator('#username'), FORM_NEVER_APPEARED).toBeVisible({ timeout: FORM_TIMEOUT_MS });
|
|
||||||
await page.locator('#username').fill(username);
|
|
||||||
await page.locator('#password').fill('test123');
|
|
||||||
await page.locator('#kc-login').click();
|
|
||||||
}
|
|
||||||
|
|
||||||
/** A citizen login on a mock-DigiD realm — no second factor (ADR-0031). */
|
|
||||||
export async function loginBurger(page: Page, username: string): Promise<void> {
|
|
||||||
await submitPassword(page, username);
|
|
||||||
}
|
|
||||||
|
|
||||||
/** A staff login on the medewerker realm: password, then the enforced TOTP second factor. */
|
|
||||||
export async function loginMedewerker(page: Page, username: string): Promise<void> {
|
|
||||||
await submitPassword(page, username);
|
|
||||||
|
|
||||||
// Keycloak's conditional-OTP step. Same reasoning as the password form above: assert it arrived
|
|
||||||
// rather than letting `fill()` swallow the test timeout.
|
|
||||||
await expect(page.locator('#otp'), OTP_NEVER_APPEARED).toBeVisible({ timeout: FORM_TIMEOUT_MS });
|
|
||||||
|
|
||||||
// Wait out the rest of the window if the counter we may spend is still in the future; Keycloak's
|
|
||||||
// lookAheadWindow would accept the code a moment early, but only by one counter — waiting keeps a
|
|
||||||
// third login in the same window valid too.
|
|
||||||
const counter = spendCounter(username);
|
|
||||||
await page.waitForTimeout(Math.max(0, counter * OTP_PERIOD_MS - Date.now()));
|
|
||||||
await page.locator('#otp').fill(totp(OTP_SECRET, counter * OTP_PERIOD_MS));
|
|
||||||
await page.locator('#kc-login').click();
|
|
||||||
}
|
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
import { expect, test } from '@playwright/test';
|
import { expect, test } from '@playwright/test';
|
||||||
import { OTP_PERIOD_MS, nextUnusedCounter } from './keycloak-login';
|
import { OTP_PERIOD_MS, nextUnusedCounter } from './medewerker-login';
|
||||||
|
|
||||||
// Pure check of the TOTP counter guard in loginMedewerker — no browser, no stack. Keycloak refuses
|
// Pure check of the TOTP counter guard in loginMedewerker — no browser, no stack. Keycloak refuses
|
||||||
// a code it has already accepted (its otpPolicyCodeReusable defaults to false), so two logins as
|
// a code it has already accepted (its otpPolicyCodeReusable defaults to false), so two logins as
|
||||||
@@ -0,0 +1,57 @@
|
|||||||
|
import { createHmac } from 'node:crypto';
|
||||||
|
import { readFileSync, writeFileSync } from 'node:fs';
|
||||||
|
import { tmpdir } from 'node:os';
|
||||||
|
import { join } from 'node:path';
|
||||||
|
import type { Page } from '@playwright/test';
|
||||||
|
|
||||||
|
// The medewerker realm enforces MFA (S-15c), so a staff login is two steps: password, then a TOTP
|
||||||
|
// code. The realm export seeds every medewerker with this fixture secret — Keycloak HMACs the raw
|
||||||
|
// secret bytes — so the e2e can compute a valid code instead of enrolling an authenticator.
|
||||||
|
const OTP_SECRET = 'BIGMEDEWERKEROTPSEED';
|
||||||
|
|
||||||
|
export const OTP_PERIOD_MS = 30_000;
|
||||||
|
|
||||||
|
// RFC 6238 TOTP: HMAC-SHA1 over the 30-second counter, dynamically truncated to 6 digits.
|
||||||
|
export function totp(secret = OTP_SECRET, at = Date.now()): string {
|
||||||
|
const counter = Buffer.alloc(8);
|
||||||
|
counter.writeBigUInt64BE(BigInt(Math.floor(at / OTP_PERIOD_MS)));
|
||||||
|
const mac = createHmac('sha1', secret).update(counter).digest();
|
||||||
|
const offset = mac[mac.length - 1] & 0x0f;
|
||||||
|
return String((mac.readUInt32BE(offset) & 0x7fffffff) % 1_000_000).padStart(6, '0');
|
||||||
|
}
|
||||||
|
|
||||||
|
// Keycloak refuses a TOTP code it has already accepted (its otpPolicyCodeReusable defaults to
|
||||||
|
// false), so two logins as the same medewerker inside one 30-second window would both submit the
|
||||||
|
// same code and the second is rejected. Spend the first counter this medewerker has left.
|
||||||
|
export function nextUnusedCounter(now: number, spent: number): number {
|
||||||
|
return Math.max(Math.floor(now / OTP_PERIOD_MS), spent + 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
// The spent counter lives on disk rather than in module state: Playwright starts a fresh worker
|
||||||
|
// process for a retry, which would otherwise forget it and resubmit the rejected code.
|
||||||
|
function spendCounter(username: string): number {
|
||||||
|
const file = join(tmpdir(), `otp-counter-${username}`);
|
||||||
|
let spent = -1;
|
||||||
|
try {
|
||||||
|
spent = Number(readFileSync(file, 'utf8')) || -1;
|
||||||
|
} catch {
|
||||||
|
// first login as this medewerker in this run
|
||||||
|
}
|
||||||
|
const counter = nextUnusedCounter(Date.now(), spent);
|
||||||
|
writeFileSync(file, String(counter));
|
||||||
|
return counter;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function loginMedewerker(page: Page, username: string): Promise<void> {
|
||||||
|
await page.locator('#username').fill(username);
|
||||||
|
await page.locator('#password').fill('test123');
|
||||||
|
await page.locator('#kc-login').click();
|
||||||
|
|
||||||
|
// Keycloak's conditional-OTP step. Wait out the rest of the window if the counter we may spend is
|
||||||
|
// still in the future; its lookAheadWindow would accept the code a moment early, but only by one
|
||||||
|
// counter — waiting keeps a third login in the same window valid too.
|
||||||
|
const counter = spendCounter(username);
|
||||||
|
await page.waitForTimeout(Math.max(0, counter * OTP_PERIOD_MS - Date.now()));
|
||||||
|
await page.locator('#otp').fill(totp(OTP_SECRET, counter * OTP_PERIOD_MS));
|
||||||
|
await page.locator('#kc-login').click();
|
||||||
|
}
|
||||||
@@ -15,12 +15,6 @@ export default defineConfig({
|
|||||||
timeout: 90_000,
|
timeout: 90_000,
|
||||||
expect: { timeout: 15_000 },
|
expect: { timeout: 15_000 },
|
||||||
retries: 1,
|
retries: 1,
|
||||||
// Bound the whole run, not just each test (#161). A wedged suite used to run until CI killed the
|
|
||||||
// job — which also killed the `if: always()` steps that would have said why: the per-spec summary
|
|
||||||
// and the container-log dump never ran, leaving a 36-minute job whose entire surviving output was
|
|
||||||
// one ✘ line. On `globalTimeout` Playwright stops and *reports*, so the JSON report is written and
|
|
||||||
// those steps still run. Generous over the ~1-minute suite: this is a backstop, not a budget.
|
|
||||||
globalTimeout: 12 * 60_000,
|
|
||||||
// Run the specs serially. Each spec drives a full `channel: 'chromium'` browser, and the e2e
|
// Run the specs serially. Each spec drives a full `channel: 'chromium'` browser, and the e2e
|
||||||
// shares an 8 GB runner with the entire compose stack (OpenZaak, NRC, Keycloak, Flowable, 4×
|
// shares an 8 GB runner with the entire compose stack (OpenZaak, NRC, Keycloak, Flowable, 4×
|
||||||
// Postgres, every service + 3 portals). Two parallel browsers exhaust memory and the renderer is
|
// Postgres, every service + 3 portals). Two parallel browsers exhaust memory and the renderer is
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
import { expect, request, test } from '@playwright/test';
|
import { expect, request, test } from '@playwright/test';
|
||||||
import { loginBurger, loginMedewerker } from './keycloak-login';
|
import { loginMedewerker } from './medewerker-login';
|
||||||
|
|
||||||
// Walking-skeleton happy path (S-08d + S-09 + S-09b + S-12 + S-10a + S-19b-2): a zorgprofessional
|
// Walking-skeleton happy path (S-08d + S-09 + S-09b + S-12 + S-10a + S-19b-2): a zorgprofessional
|
||||||
// logs in via mock DigiD and submits through the self-service portal → BFF → domain; the entry
|
// logs in via mock DigiD and submits through the self-service portal → BFF → domain; the entry
|
||||||
@@ -23,7 +23,9 @@ test('DigiD submit → public INGEDIEND → documenten → behandelaar goedkeurt
|
|||||||
// checks submit as jan-burger (bsn 123456782) before the e2e runs on the shared stack, and
|
// checks submit as jan-burger (bsn 123456782) before the e2e runs on the shared stack, and
|
||||||
// resume-on-load (S-26) would otherwise restore one of those on login — so each self-service spec
|
// resume-on-load (S-26) would otherwise restore one of those on login — so each self-service spec
|
||||||
// uses a dedicated citizen no other actor touches.
|
// uses a dedicated citizen no other actor touches.
|
||||||
await loginBurger(page, 'emma-burger');
|
await page.locator('#username').fill('emma-burger');
|
||||||
|
await page.locator('#password').fill('test123');
|
||||||
|
await page.locator('#kc-login').click();
|
||||||
|
|
||||||
// Back on the portal, authenticated.
|
// Back on the portal, authenticated.
|
||||||
await expect(page.getByRole('heading', { name: /Zelfservice/i })).toBeVisible();
|
await expect(page.getByRole('heading', { name: /Zelfservice/i })).toBeVisible();
|
||||||
|
|||||||
@@ -1,5 +1,4 @@
|
|||||||
import { expect, test } from '@playwright/test';
|
import { expect, test } from '@playwright/test';
|
||||||
import { loginBurger } from './keycloak-login';
|
|
||||||
|
|
||||||
// S-26: a zorgprofessional submits, then reloads the self-service portal. On load the portal asks the
|
// S-26: a zorgprofessional submits, then reloads the self-service portal. On load the portal asks the
|
||||||
// BFF for the caller's current open registration (owner-scoped by the DigiD token's bsn) and restores
|
// BFF for the caller's current open registration (owner-scoped by the DigiD token's bsn) and restores
|
||||||
@@ -10,7 +9,9 @@ test('DigiD submit → reload → self-service restores the existing registratio
|
|||||||
// Its own DigiD user (like every self-service spec): on the shared verify stack, resume-on-load
|
// Its own DigiD user (like every self-service spec): on the shared verify stack, resume-on-load
|
||||||
// (S-26) restores any open registration for the bsn, so each spec uses a dedicated citizen that no
|
// (S-26) restores any open registration for the bsn, so each spec uses a dedicated citizen that no
|
||||||
// other spec or verify-* check touches. This one in particular leaves an open registration.
|
// other spec or verify-* check touches. This one in particular leaves an open registration.
|
||||||
await loginBurger(page, 'sanne-burger');
|
await page.locator('#username').fill('sanne-burger');
|
||||||
|
await page.locator('#password').fill('test123');
|
||||||
|
await page.locator('#kc-login').click();
|
||||||
|
|
||||||
await expect(page.getByRole('heading', { name: /Zelfservice/i })).toBeVisible();
|
await expect(page.getByRole('heading', { name: /Zelfservice/i })).toBeVisible();
|
||||||
await page.getByRole('button', { name: /indienen/i }).click();
|
await page.getByRole('button', { name: /indienen/i }).click();
|
||||||
|
|||||||
@@ -1,5 +1,4 @@
|
|||||||
import { expect, test } from '@playwright/test';
|
import { expect, test } from '@playwright/test';
|
||||||
import { loginBurger } from './keycloak-login';
|
|
||||||
|
|
||||||
// S-11 (Flow 3): a zorgprofessional logs in via mock DigiD, submits a registration, then withdraws
|
// S-11 (Flow 3): a zorgprofessional logs in via mock DigiD, submits a registration, then withdraws
|
||||||
// it ("trek aanvraag in") from the self-service portal. The withdrawal goes portal → BFF (owner-
|
// it ("trek aanvraag in") from the self-service portal. The withdrawal goes portal → BFF (owner-
|
||||||
@@ -11,7 +10,9 @@ test('DigiD submit → trek aanvraag in → self-service confirms ingetrokken',
|
|||||||
|
|
||||||
// Its own DigiD user — isolated from the verify-* checks (jan-burger/123456782) so resume-on-load
|
// Its own DigiD user — isolated from the verify-* checks (jan-burger/123456782) so resume-on-load
|
||||||
// (S-26) can't restore someone else's registration on the shared stack.
|
// (S-26) can't restore someone else's registration on the shared stack.
|
||||||
await loginBurger(page, 'lars-burger');
|
await page.locator('#username').fill('lars-burger');
|
||||||
|
await page.locator('#password').fill('test123');
|
||||||
|
await page.locator('#kc-login').click();
|
||||||
|
|
||||||
await expect(page.getByRole('heading', { name: /Zelfservice/i })).toBeVisible();
|
await expect(page.getByRole('heading', { name: /Zelfservice/i })).toBeVisible();
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user