feat(openzaak): one-command UI-OpenZaak bridge, real BSN fix, flake mitigation

scripts/openzaak-ui-up.sh brings up the root app and the OpenZaak harness
together, wires them onto one docker network, seeds the catalogus, grants
the container-alias zaaktype scope, and verifies a real aanvraag submitted
through the UI lands in OpenZaak.

Along the way: DocumentStore.DemoOwner was reusing the seeded doctor's
11-digit BIG-nummer as a stand-in BSN, which isn't a valid 9-digit BSN shape
— OpenZaak rejects it, breaking both submit's rol-creation step and the
citizen's own applications list under Zgw:Enabled=true. Fixed to a real
elfproef-valid BSN.

Also adds mitigation for a still-unexplained per-container flake (every
outbound ZGW POST fails as if the body were empty, for that container's
whole lifetime) that correlates with host memory pressure: the script now
warns when host swap is heavily used, and an opt-in ZgwDiagnosticHandler
(ZGW_DEBUG_HTTP=1) logs Content-Length vs. actual bytes sent so the next
reproduction can confirm or rule out client-side body corruption.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

backend/openzaak/README.md

@@ -80,6 +80,42 @@ BFF_AUTH='<the same secret>' ./verify-notificatie.sh   # proves a real delivery,
 To go back to the fast, no-notifications default: `docker compose -f docker-compose.openzaak.yml
 up -d --remove-orphans` (drops the celery worker, restores `NOTIFICATIONS_DISABLED: 'true'`).
 
+## Testing the Angular UI against this harness
+
+The base harness above and the app's own root `docker-compose.yml` are independent projects on
+purpose (see the top of this file) — this is the opt-in bridge between them, for when you want
+to click through the real UI and see an aanvraag land in a real OpenZaak instead of just
+running `dotnet test`. One command from the repo root:
+
+```bash
+scripts/openzaak-ui-up.sh
+```
+
+It brings up the root app (so its docker network exists), brings up this harness plus
+`docker-compose.openzaak.bff.yml` (gives this harness's `web` service a dotted alias,
+`openzaak.local`, on its own network — the root project's `api` container joins THIS network,
+in the opposite direction from the notifications overlay below, to avoid a real alias
+collision: the root project's frontend service is also called `web`. The alias needs a dot
+because Django's URLValidator rejects a bare hostname in a URL field; this environment's
+rootless Podman also can't route container→host-port traffic through `host.docker.internal`,
+so container-to-container is the only reliable path either way — see that file's header
+comment for the full, empirically-confirmed reasoning), seeds the catalogus, additively
+replaces the zrc authorization grant to match the alias (ZGW authorization is scoped by the
+*exact* zaaktype URL string, not just the resource; see `scripts/openzaak-ui-up.sh`'s own
+comment for why this is a replace, not an add), and brings the root app back up pointed at
+OpenZaak (`docker-compose.openzaak.yml` at the repo root). A final self-check submits a
+throwaway aanvraag and confirms it actually lands in OpenZaak, restarting `api` (up to 5
+times) if not — see that script for a caveat about an intermittent per-container networking
+flake this environment can hit under memory pressure (the script now warns if host swap is
+already high going in; `ZGW_DEBUG_HTTP=1` on `api`, see `docker-compose.openzaak.yml`, logs
+diagnostics to help nail the cause next time it reproduces). Prints the URLs to check
+afterward and the teardown commands.
+
+Two caveats, both non-fatal (WP-60 catches and flags rather than surfacing an error):
+**only `herregistratie` has a seeded zaaktype** here, so submit that wizard to prove a real
+write; and **no Documenten content is seeded**, so a document upload's ZGW half no-ops (pick
+"per post" in the wizard's document step, or ignore it).
+
 ## Tear down
 
 ```bash

backend/openzaak/docker-compose.openzaak.bff.yml

@@ -0,0 +1,38 @@
+# Opt-in overlay, layered ON TOP of docker-compose.openzaak.yml (never alone):
+#
+#   docker compose -f docker-compose.openzaak.yml -f docker-compose.openzaak.bff.yml up -d
+#
+# Gives this harness's `web` (the OpenZaak API) an extra, dotted hostname alias
+# (`openzaak.local`) on its OWN network, so the root project's `api` container (joined in via
+# `docker-compose.openzaak.yml` at the repo root, as an EXTERNAL network) can reach it. See
+# `scripts/openzaak-ui-up.sh` for the one-command version that brings both projects up
+# together, seeds the catalogus, and grants the extra authorization scope this alias needs.
+#
+# Three real things this works around, each discovered empirically (curl against the
+# running containers), not guessed:
+#
+# 1. Why container-to-container instead of `http://localhost:8000`: this dev environment's
+#    rootless Podman drops container→host-port traffic through `host.docker.internal`
+#    (confirmed for the WP-58 notifications overlay's celery worker — DNS resolves it, every
+#    TCP connect times out).
+#
+# 2. Why the ROOT project's `api` joins INTO this project's network (below), not the other way
+#    around: the root project's frontend service is also called `web`. Docker Compose always
+#    adds a service's own name as a network alias on every network it joins — so if THIS `web`
+#    joined the root project's network, "web" would resolve to two different containers there.
+#    Only `api` crosses into this network, under its own already-unique name.
+#
+# 3. Why the alias has a dot in it (`openzaak.local`, not e.g. `openzaak`): Django's built-in
+#    URLValidator rejects a bare, dotless hostname in a URL field (it special-cases exactly
+#    "localhost"; anything else needs a dot or to be a valid IP). OpenZaak's `zaaktype` field
+#    (and others) run through this validator — confirmed with a POST referencing
+#    `http://<dotless-alias>:8000/...` failing "Voer een geldige URL in" (enter a valid URL)
+#    before any authorization check even runs.
+services:
+  web:
+    environment:
+      # Django rejects any request whose Host header isn't in ALLOWED_HOSTS.
+      ALLOWED_HOSTS: localhost,127.0.0.1,web,openzaak.local
+    networks:
+      default:
+        aliases: [openzaak.local]

backend/src/BigRegister.Api/Data/DocumentStore.cs

@@ -36,8 +36,16 @@ public sealed record AuditEntry(DateTimeOffset At, string Action, string Documen
 /// </summary>
 public static class DocumentStore
 {
-  /// The single seeded user (the demo has no real auth; ownership = this id).
+  /// The single seeded user (the demo has no real auth; ownership = this id) — a real,
-  public const string DemoOwner = "19012345601";
+  /// elfproef-valid 9-digit BSN (src/app/shared/kernel/bsn.ts's own checksum), distinct from
+  /// SeedData.Registration.BigNummer ("19012345601", 11 digits — the seeded doctor's BIG-nummer,
+  /// a different Dutch identifier scheme). Previously this constant reused that BigNummer value
+  /// as a stand-in BSN, which is invalid Dutch-BSN shape: harmless against the local store, but
+  /// a real OpenZaak instance rejects it outright — GET /api/v1/applications 500s (`inpBsn` query
+  /// filter validation) and every submit's rol-creation POST fails (`inpBsn` max_length) once
+  /// Zgw:Enabled=true. Not "111222333" or "999888777" — both already mean a different fixture
+  /// identity (the OpenZaak-harness/unit-test caller, and ApplicationTests' "other citizen").
+  public const string DemoOwner = "123456782";
 
   private static readonly object _gate = new();
 

backend/src/BigRegister.Api/Program.cs

@@ -61,9 +61,18 @@ if (zgw.Enabled)
   // WP-60: a bounded client timeout matters once ZgwHttpClient retries — without one, the
   // sources' sync-over-async call (no CancellationToken threaded through) could block a
   // thread-pool thread for HttpClient's 100s default times 3 attempts.
-  builder.Services.AddHttpClient<IZaakSource, OpenZaakZaakSource>(c => c.Timeout = TimeSpan.FromSeconds(15));
+  var zaakClientBuilder = builder.Services.AddHttpClient<IZaakSource, OpenZaakZaakSource>(c => c.Timeout = TimeSpan.FromSeconds(15));
   // WP-51: the documents (Documenten API / DRC) seam — same pattern as IZaakSource above.
-  builder.Services.AddHttpClient<IDocumentSource, OpenZaakDocumentSource>(c => c.Timeout = TimeSpan.FromSeconds(15));
+  var documentClientBuilder = builder.Services.AddHttpClient<IDocumentSource, OpenZaakDocumentSource>(c => c.Timeout = TimeSpan.FromSeconds(15));
+
+  // Opt-in diagnostic for the still-unexplained per-container flake (see
+  // scripts/openzaak-ui-up.sh's header comment) — off by default, zero cost unless set.
+  if (Environment.GetEnvironmentVariable("ZGW_DEBUG_HTTP") == "1")
+  {
+    builder.Services.AddTransient<ZgwDiagnosticHandler>();
+    zaakClientBuilder.AddHttpMessageHandler<ZgwDiagnosticHandler>();
+    documentClientBuilder.AddHttpMessageHandler<ZgwDiagnosticHandler>();
+  }
 }
 else
 {

backend/src/BigRegister.Api/Zgw/ZgwDiagnosticHandler.cs

@@ -0,0 +1,25 @@
+namespace BigRegister.Api.Zgw;
+
+/// <summary>
+/// Opt-in only (wired in <c>Program.cs</c> behind <c>ZGW_DEBUG_HTTP=1</c>) — chases the
+/// still-unexplained flake where a freshly-(re)started `api` container has every outbound ZGW
+/// POST fail with what looks like an empty body reaching OpenZaak (see
+/// <c>scripts/openzaak-ui-up.sh</c>'s header comment). Logs the one signal that would actually
+/// distinguish "client built an empty body" from "something ate it after send": the declared
+/// Content-Length vs. the byte count actually read from the request right before it goes out.
+/// A mismatch here would prove client-side corruption; agreement would point downstream instead.
+/// </summary>
+public sealed class ZgwDiagnosticHandler(ILogger<ZgwDiagnosticHandler> logger) : DelegatingHandler
+{
+  protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
+  {
+    if (request.Content is not null)
+    {
+      var bytes = await request.Content.ReadAsByteArrayAsync(cancellationToken);
+      logger.LogInformation(
+          "ZGW {Method} {Url}: Content-Length={ContentLength} actualBytes={Actual}",
+          request.Method, request.RequestUri, request.Content.Headers.ContentLength, bytes.Length);
+    }
+    return await base.SendAsync(request, cancellationToken);
+  }
+}

backend/tests/BigRegister.Tests/ZgwDiagnosticHandlerTests.cs

@@ -0,0 +1,28 @@
+using System.Net.Http.Json;
+using BigRegister.Api.Zgw;
+using Microsoft.Extensions.Logging.Abstractions;
+
+namespace BigRegister.Tests;
+
+/// <summary>
+/// The opt-in diagnostic handler (see ZgwDiagnosticHandler's own doc comment) must be inert on
+/// the request/response — it only observes. This doesn't catch the flake itself (that needs a
+/// real repro with ZGW_DEBUG_HTTP=1), just proves the hook doesn't alter what's sent or break
+/// the response passthrough.
+/// </summary>
+public class ZgwDiagnosticHandlerTests
+{
+  [Fact]
+  public async Task Passes_request_and_response_through_unchanged()
+  {
+    var stub = new ZgwStubHandler(_ => """{ "ok": true }""");
+    var diagnostic = new ZgwDiagnosticHandler(NullLogger<ZgwDiagnosticHandler>.Instance) { InnerHandler = stub };
+    using var client = new HttpClient(diagnostic);
+
+    var response = await client.PostAsJsonAsync("https://oz.example/zaken", new { foo = "bar" });
+
+    Assert.True(response.IsSuccessStatusCode);
+    Assert.Single(stub.Requests);
+    Assert.Contains("\"foo\":\"bar\"", stub.BodyOf("https://oz.example/zaken"));
+  }
+}

docker-compose.openzaak.yml

@@ -0,0 +1,50 @@
+# Opt-in overlay, layered ON TOP of docker-compose.yml (never alone):
+#
+#   docker compose -f docker-compose.yml -f docker-compose.openzaak.yml up -d
+#
+# Points the containerized BFF (`api`) at a real OpenZaak instead of the local SQLite store.
+# `api` joins the OpenZaak project's OWN network (external, below) to reach its `web` service
+# by the dotted alias `docker-compose.openzaak.bff.yml` (backend/openzaak/) gives it —
+# `openzaak.local`, not a bare `openzaak`: Django's URLValidator rejects a dotless hostname
+# embedded in a URL field (confirmed empirically — see that file's header comment for the full
+# reasoning, including why the join runs in this direction and not the reverse). See
+# `scripts/openzaak-ui-up.sh` for the one command that brings both projects up together, seeds
+# the catalogus, grants the extra scope this alias needs, and fills in OPENZAAK_ZAAKTYPE_URL.
+#
+# ClientId/Secret/RSINs match exactly what backend/openzaak/bootstrap-catalogus.sh provisions.
+# Only `herregistratie` gets a ZaaktypeUrls entry: the harness seeds exactly one zaaktype
+# ("Herregistratie arts") — a registratie/intake submission would hit an unconfigured zaaktype,
+# caught by WP-60's retry/flagging (Aanvraag.ZgwError), not surfaced as a UI error. DrcBaseUrl/
+# InformatieobjecttypeUrls are deliberately left unset: this harness seeds no Documenten
+# content or scope, so a document upload's ZGW half just no-ops (also caught since WP-60).
+#
+# ZGW authorization scopes a zaaktype write by the EXACT zaaktype URL string an Applicatie was
+# granted for (confirmed empirically: the same zaaktype, referenced via a different hostname
+# string, 403s even though the URL itself resolves fine) — bootstrap-catalogus.sh only ever
+# grants the `http://localhost:8000/...` form (it runs on the host). scripts/openzaak-ui-up.sh
+# additively grants the SAME scope again for the `openzaak.local:8000` form this file's `api`
+# actually presents, without touching bootstrap-catalogus.sh's own (host-usable) grant.
+services:
+  api:
+    environment:
+      - Zgw__Enabled=true
+      - Zgw__ZrcBaseUrl=http://openzaak.local:8000/zaken/api/v1
+      - Zgw__ZtcBaseUrl=http://openzaak.local:8000/catalogi/api/v1
+      # Uncomment to chase the per-container flake (scripts/openzaak-ui-up.sh's header
+      # comment): logs Content-Length vs. actual bytes sent for every outbound ZGW POST.
+      # - ZGW_DEBUG_HTTP=1
+      - Zgw__ClientId=bigregister-test
+      - Zgw__Secret=bigregister-test-secret
+      - Zgw__UserId=bigregister-test
+      - Zgw__UserRepresentation=Docker compose OpenZaak test
+      - Zgw__Bronorganisatie=123443210
+      - Zgw__VerantwoordelijkeOrganisatie=123443210
+      - Zgw__ZaaktypeUrls__herregistratie=${OPENZAAK_ZAAKTYPE_URL:?run backend/openzaak/bootstrap-catalogus.sh and export OPENZAAK_ZAAKTYPE_URL first — see scripts/openzaak-ui-up.sh for the one-command version}
+    networks:
+      default: {}
+      oz: {}
+
+networks:
+  oz:
+    name: openzaak_default
+    external: true

scripts/openzaak-ui-up.sh

@@ -0,0 +1,150 @@
+#!/usr/bin/env bash
+# One command to test the Angular UI end-to-end against a real OpenZaak instead of the local
+# SQLite store: brings up the OpenZaak harness (backend/openzaak/), seeds its catalogus, wires
+# it onto the root app's docker network (docker-compose.openzaak.bff.yml /
+# docker-compose.openzaak.yml — see those files for the full "why", it's non-obvious), grants
+# the extra authorization scope the container alias needs, and brings the FE+BFF up pointed at
+# OpenZaak.
+#
+# Safe to re-run: every step this chains is already idempotent (bootstrap-catalogus.sh,
+# `docker compose up -d`, and the grant-replace below).
+#
+# Also confirmed empirically, many repeated trials: some freshly-(re)started `api`
+# containers have EVERY outbound ZGW POST fail with what looks like an empty body reaching
+# OpenZaak ("all fields required"), for that container's entire lifetime — while a plain curl
+# to the exact same URL, from inside the exact same container, never fails, even hammered in a
+# loop. Ruled out as the cause: HttpClient connection pooling settings, Expect-100-Continue,
+# content pre-buffering — none of it made a measurable difference. Best lead so far, and
+# reproduced live on this dev host (7.5/8GB swap from long-idle unrelated containers): this
+# correlates with the HOST being under heavy memory pressure — a heavier managed runtime
+# (dotnet's JIT + GC) is plausibly far more sensitive to that than a lightweight one-shot
+# `curl` process. If you hit this, try freeing host memory (stop unrelated containers) before
+# assuming it's a code regression. The preflight check and self-check below warn about and
+# retry around it either way; set ZGW_DEBUG_HTTP=1 on the `api` container (see
+# docker-compose.openzaak.yml) next time it reproduces to log Content-Length vs. actual bytes
+# sent, which would confirm (or rule out) client-side body corruption.
+set -euo pipefail
+cd "$(dirname "$0")/.."
+
+step() { printf '\n\033[1;36m▶ %s\033[0m\n' "$1"; }
+
+check_memory_pressure() {
+  local total used pct
+  read -r total used <<< "$(free -m | awk '/^Swap:/{print $2, $3}')"
+  [ "${total:-0}" -gt 0 ] || return 0
+  pct=$(( used * 100 / total ))
+  if [ "$pct" -ge 50 ]; then
+    echo "⚠ host swap ${pct}% used (${used}MiB/${total}MiB) — this correlates with the" >&2
+    echo "  known per-container ZGW flake (see comment up top). Consider 'docker ps' and" >&2
+    echo "  stopping unrelated long-running stacks before continuing." >&2
+  fi
+}
+check_memory_pressure
+
+step "root app (creates the docker network the OpenZaak harness joins below)"
+docker compose up -d
+
+step "OpenZaak harness + bff overlay"
+( cd backend/openzaak && docker compose -f docker-compose.openzaak.yml -f docker-compose.openzaak.bff.yml up -d )
+
+step "seed catalogus/zaaktype/zaak (idempotent)"
+bootstrap_output=$(cd backend/openzaak && ./bootstrap-catalogus.sh)
+echo "$bootstrap_output"
+
+zaaktype_line=$(printf '%s\n' "$bootstrap_output" | grep -A1 '^Zaaktype (concept)\.\.\.$' | tail -n1)
+zaaktype_url=$(printf '%s\n' "$zaaktype_line" | sed -E 's/^ *(exists|created): //')
+zaaktype_uuid=$(printf '%s\n' "$zaaktype_url" | sed 's#.*/##')
+if [ -z "$zaaktype_uuid" ]; then
+  echo "Could not find the seeded zaaktype's URL in bootstrap-catalogus.sh's output — see above." >&2
+  exit 1
+fi
+# bootstrap-catalogus.sh queried (and granted scope) via http://localhost:8000 (run from the
+# host); the containerized BFF reaches the same resource via the `openzaak.local` alias
+# (docker-compose.openzaak.bff.yml) instead — only the UUID suffix is what actually matters.
+container_zaaktype_url="http://openzaak.local:8000/catalogi/api/v1/zaaktypen/${zaaktype_uuid}"
+export OPENZAAK_ZAAKTYPE_URL="$container_zaaktype_url"
+
+step "grant the container-alias zaaktype scope (REPLACES bootstrap-catalogus.sh's own localhost-scoped grant, doesn't add to it — OpenZaak's own zaken-list authorization filter 500s with a RuntimeError, 'are you sure that all paths point to the same resource?', when an Applicatie has two zrc grants for what's really the same zaaktype under two different hostnames; confirmed empirically. One consequence: dotnet test --filter Category=Integration needs bootstrap-catalogus.sh rerun afterward to restore its localhost-scoped grant — it's idempotent, so that's a plain rerun, not a reset)"
+( cd backend/openzaak && docker compose -f docker-compose.openzaak.yml exec -T --workdir /app/src web python manage.py shell ) <<PY
+from vng_api_common.authorizations.models import Applicatie
+app = Applicatie.objects.get(client_ids__contains=["bigregister-test"])
+app.autorisaties.filter(component="zrc").delete()
+app.autorisaties.create(
+    component="zrc",
+    scopes=["zaken.aanmaken", "zaken.bijwerken", "zaken.lezen"],
+    zaaktype="$container_zaaktype_url",
+    max_vertrouwelijkheidaanduiding="openbaar",
+)
+print("granted:", "$container_zaaktype_url")
+PY
+
+step "root app again (now pointed at OpenZaak)"
+docker compose -f docker-compose.yml -f docker-compose.openzaak.yml up -d
+
+step "verify the BFF can actually reach OpenZaak (retries api if not — see the flake note up top)"
+wait_for_api() {
+  # -f: a response has to actually be 2xx — dotnet run's own build/start (and, on a retry, a
+  # full restart) can leave the port accepting-but-erroring for a few seconds first, which a
+  # plain curl (no -f) would misread as "ready".
+  for _ in $(seq 1 60); do
+    curl -sS -f -m 2 -o /dev/null http://localhost:5000/api/v1/me && return 0
+    sleep 1
+  done
+  return 1
+}
+# A throwaway herregistratie submit: does the resulting zaak actually show up in OpenZaak
+# (via the BFF's own /admin/cases)?
+zgw_write_reaches_openzaak() {
+  local create id ref cases
+  create=$(curl -sS -m 10 -X POST http://localhost:5000/api/v1/applications -H "Content-Type: application/json" -d '{"type":"herregistratie"}') || return 1
+  id=$(printf '%s' "$create" | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])') || return 1
+  ref=$(curl -sS -m 15 -X POST "http://localhost:5000/api/v1/applications/$id/submit" -H "Content-Type: application/json" -d '{"uren":1}' | python3 -c 'import json,sys; print(json.load(sys.stdin)["referentie"])') || return 1
+  cases=$(curl -sS -m 10 -H "X-Role: admin" http://localhost:5000/api/v1/admin/cases)
+  printf '%s' "$cases" | python3 -c "
+import json, sys
+refs = [c['status']['referentie'] for c in json.load(sys.stdin)]
+sys.exit(0 if '$ref' in refs else 1)
+"
+}
+verified=false
+attempts=5
+for attempt in $(seq 1 "$attempts"); do
+  wait_for_api || { echo "api never came up listening — see 'docker logs atomic-design-poc-api-1'." >&2; break; }
+  if zgw_write_reaches_openzaak; then
+    verified=true
+    break
+  fi
+  echo "  attempt $attempt/$attempts: a write didn't reach OpenZaak — restarting api and giving the network a moment to settle (known flake, see comment up top)..."
+  sleep 3
+  docker compose -f docker-compose.yml -f docker-compose.openzaak.yml restart api
+done
+if [ "$verified" = true ]; then
+  echo "  verified: a write reaches OpenZaak."
+else
+  check_memory_pressure
+  echo "  Could not verify after $attempts tries. This is the known flake, not necessarily a real failure —" >&2
+  echo "  keep retrying by hand: 'docker compose -f docker-compose.yml -f docker-compose.openzaak.yml restart api'," >&2
+  echo "  wait for 'Application started' in 'docker logs -f atomic-design-poc-api-1', then try the UI again." >&2
+fi
+
+printf '\n\033[1;32m✔ up\033[0m\n'
+cat <<EOF
+
+App:               http://localhost:4200
+Admin cases (proof an aanvraag landed in OpenZaak): http://localhost:4200/beheer/zaken?role=admin
+Audit trail (should show no zgw:divergence rows):   http://localhost:4200/beheer/audit?role=admin
+OpenZaak directly:  http://localhost:8000
+
+Only "herregistratie" has a seeded zaaktype in this harness — submit that wizard to see a real
+write land in OpenZaak. registratie/intake submissions still succeed locally but their ZGW
+write is flagged (Aanvraag.ZgwError / a zgw:divergence audit row), not surfaced as a UI error.
+Document uploads also won't register in ZGW (no Documenten content seeded) — pick "per post"
+in the wizard's document step to keep the demo clean, or ignore it, it's non-fatal.
+
+Tear down (plain "down", no -f overlay needed — overlay files only add env/network config to
+services docker-compose.yml already defines, and the required-var syntax in
+docker-compose.openzaak.yml would otherwise block "down" too once OPENZAAK_ZAAKTYPE_URL falls
+out of your shell):
+  docker compose down
+  ( cd backend/openzaak && docker compose -f docker-compose.openzaak.yml down -v )
+EOF