## What & why S-19b-2, closing out ADR-0028's stated direction: **the read projection is now derived from the `RegisterRecord` in Objecten, not from ZGW zaak events.** Until now the subscriber listened on `zaken` and *inferred* register state from case events — a `zaak/create` meant INGEDIEND, and any `status/create` was assumed to be the approval (it may not read OpenZaak, so it could not tell statustypen apart). The reference wasn't in the notification at all, so every projection made a second hop to the ACL. The register — a fact about a person — was being reconstructed by guessing at the lifecycle of the case that produced it. - The subscriber's abonnement moves to the `objecten` kanaal (S-19b-1 made it publish). - An Objecten notification carries **no record data**, only the object URL, so the record is read back through the ACL (`POST /register-records/read`) — §8.1 applies to Objecten exactly as ADR-0028 established. - The record carries `id`, `status` and `reference`, so the row *is* the record: `IsZaakCreated`, `IsZaakStatusSet`, `ZaakUrl`, `ZaakId` and `ToEntry`'s `Resource == "status"` inference are all gone, and so is the ACL enrichment hop. - **The ACL now writes an INGEDIEND record on submit.** Without it, re-sourcing would silently drop every submitted registration from the public register, since only approval wrote a record. - `processed_notifications` holds the projected row (`register_id`, `status`, `reference`) instead of the ZGW event, so a rebuild is a replay with no mapping rules and no upstream reads at all. **ADR-0030** records it. ADR-0028's open caveat — record written but not yet read, "the two must agree" — is closed: there is one source now. Closes #153 ## Definition of Done - [x] Linked Gitea issue (above). - [x] Failing tests committed before the implementation — two red/green pairs, ACL side (06c0444→566ef7d) and subscriber side (142ed45→8af09b2). - [x] Refactor commit follows (b496ac9). - [x] Conventional Commits referencing the issue (`refs #153`). - [x] CI green — all six jobs onb30fa66, `verify-stack` end to end including the e2e. - [x] `docker compose up` from a fresh clone reaches green health checks within 3 minutes (`verify-stack`'s bring-up step — see the wait-healthy fix below). - [x] Docs updated — ADR-0030 added, ADR-0028's consequence + caveat annotated, BACKLOG.md, e2e header comment. - [x] ADR added in `docs/architecture/`. - [x] Demo note in `docs/demo-script.md` — n/a: no user-visible change. The openbaar register shows the same two statuses for the same registrations; only where they come from changed. ## Notes for reviewers **The decision I'd most like a second opinion on** is the one the issue didn't settle: what happens to INGEDIEND. Objecten held only INGESCHREVEN records, so re-sourcing forced a choice between (a) the ACL also writing on submit, (b) a public register that lists only actual registrations, or (c) a hybrid keeping both kanalen. I took (a): visible behaviour is unchanged and the register holds the whole lifecycle. (b) is arguably the better *semantics* for a public register but narrows what the portal shows and reads against PRD §68 ("~50 register entries with diverse statuses"); (c) leaves the projection half-derived from ZGW, which is the coupling ADR-0028 set out to remove. All three are laid out in ADR-0030. **The dedup key is the projected row**, `objecten:object:{url}:{status}:{reference}` — not the object URL (the ACL upserts *one object per registration*, so submit and approval notify about the same URL and the approval would be swallowed as a duplicate) and not URL+actie (a retried approval is a second `update`). Redeliveries collapse, genuine state changes don't. §8.6. **The migration drops columns rather than renaming them.** EF scaffolded renames — `resource` → `register_id`, `zaak_id` → `status` — which would have carried ZGW values into columns meaning something else, and a rebuild would then have projected that garbage. It also empties both tables: a pre-slice row describes a zaak event the new projector can't reproject, and those registrations have no RegisterRecord in Objecten either, so they're not re-derivable from the new source. Stated as a ceiling in the ADR — fine while stacks are ephemeral, backfill from Objecten if a long-lived environment ever needs it. **`run-projection-check.sh` now opens its zaak through the ACL** instead of straight against OpenZaak, because the ACL is what writes the record. A zaak created behind the ACL's back produces no projection row — that's the re-source working, not a gap. ## Three fixes CI found, none of them in the projection logic 1. **`wait-healthy.sh` matched the wrong container** (744f91a). Bring-up timed out with `TIMEOUT: 'objecten' not healthy (status=none)` while the `docker ps` it dumps showed objecten `Up 9 minutes (healthy)`. `--filter name=` is a substring match, so `objecten` also matches `objecten-db`/`objecten-redis`/`objecten-celery`, and `head -1` took whichever docker listed first — the celery worker has no healthcheck, hence `status=none`. Latent since those services landed and decided purely by listing order; `objecttypen` matches `objecttypen-db` the same way. Anchored on the compose replica suffix, which the verify scripts already do. 2. **The ACL had to be repointed at OpenZaak's IP** (7e0897a). Opening the zaak through the ACL put this check in the same bind run-domain-check.sh already handles: `400 {"name":"zaaktype","code":"bad-url","reason":"Voer een geldige URL in."}`. OpenZaak reflects the request Host into the zaaktype URL and then rejects it on zaak-create when single-label — the mechanism compose already documents on `ACL_OPENZAAK_BASEURL`. 3. **Approval arrives as `partial_update`, not `update`** (0dd26a7→b30fa66) — the one real bug in the slice. The ACL upserts with PATCH; DRF routes it through the notifying `update()` but names the action `partial_update`, so the projector dropped every approval. Only the e2e could catch it: `verify-projection` drives a submit, and per ADR-0028 the e2e is the only check that drives a *real* approval. `verify-tracing` also failed once (run 722) on a path this PR doesn't touch, and passed on a plain re-run of the same commit. Tempo logged `pusher failed to consume trace data` / `distributor_pool failing healthcheck` — it dropped spans under runner load rather than the trace chain being broken. Filed as **#156** rather than absorbed here. **Correction to the #152 PR notes:** I wrote there that celery concurrency was "the next knob" if verify-stack got tight. It isn't — `CELERY_WORKER_CONCURRENCY` already defaults to 1 in the Maykin image, so `objecten-celery` is already a single-process worker. Noted in #156. **Possible follow-up, deliberately not done here:** an `openzaak.local` network alias mirroring `objecten.local` would remove the ACL-repoint dance from both run-domain-check.sh and run-projection-check.sh. It changes the host in every zaak URL the system produces, which is too broad a ripple to land inside an unrelated slice — worth its own issue. **Known costs, all in the ADR:** submission is now two writes across two modules and eventually consistent (same posture ADR-0028 accepted for approval); projecting now depends on the ACL being reachable on the main path, not just for enrichment (NRC retries, so it converges); and OpenZaak still publishes to `zaken` with nothing in the product listening — kept because `verify-nrc` asserts that path.Reviewed-on: #155
84 lines
3.9 KiB
Python
Executable File
84 lines
3.9 KiB
Python
Executable File
#!/usr/bin/env python3
|
|
"""Local-stack bootstrap (S-B04, #110, ADR-0020) — register the NRC abonnement.
|
|
|
|
Runs as the `nrc-subscribe` init container of infra/docker-compose.local.yml. Registers an
|
|
abonnement on the `objecten` kanaal pointing at the event-subscriber's /notifications callback, so
|
|
the register writes the ACL makes (INGEDIEND on submit, INGESCHREVEN on approval) reach the
|
|
projection — without this the openbaar (public) register stays empty. Since S-19b-2 the projection
|
|
is sourced from the register in Objecten, not from ZGW zaak events (ADR-0030).
|
|
|
|
The callback host is the event-subscriber's resolved **container IP**, not `event-subscriber`, because
|
|
NRC validates callbackUrl with Django's URLValidator (a single-label host is rejected — same reason the
|
|
zaaktype seed uses OpenZaak's IP). Idempotent + restart-safe: it removes any stale /notifications
|
|
abonnement first, then registers one for the current IP. Stdlib only.
|
|
|
|
Env: NRC_BASE, SINK_HOST, SINK_PORT, SINK_AUTH, OZ_CLIENT_ID, OZ_SECRET.
|
|
"""
|
|
import base64, hashlib, hmac, json, os, socket, sys, time, urllib.error, urllib.request
|
|
|
|
NRC = os.environ.get("NRC_BASE", "http://nrc-web:8000").rstrip("/")
|
|
SINK_HOST = os.environ.get("SINK_HOST", "event-subscriber")
|
|
SINK_PORT = os.environ.get("SINK_PORT", "8080")
|
|
SINK_AUTH = os.environ.get("SINK_AUTH", "Bearer big-reference-notifications")
|
|
CID = os.environ.get("OZ_CLIENT_ID", "big-reference-seed")
|
|
SECRET = os.environ.get("OZ_SECRET", "insecure-dev-secret-change-me")
|
|
# The projection is sourced from the register in Objecten, not from ZGW zaak events (S-19b-2).
|
|
KANAAL = "objecten"
|
|
|
|
|
|
def token():
|
|
b64 = lambda b: base64.urlsafe_b64encode(b).rstrip(b"=")
|
|
seg = (
|
|
b64(json.dumps({"alg": "HS256", "typ": "JWT"}, separators=(",", ":")).encode())
|
|
+ b"."
|
|
+ b64(json.dumps(
|
|
{"iss": CID, "iat": int(time.time()), "client_id": CID,
|
|
"user_id": "local-seed", "user_representation": "local-seed"},
|
|
separators=(",", ":")).encode())
|
|
)
|
|
return (seg + b"." + b64(hmac.new(SECRET.encode(), seg, hashlib.sha256).digest())).decode()
|
|
|
|
|
|
def call(method, url, body=None):
|
|
data = json.dumps(body).encode() if body is not None else None
|
|
req = urllib.request.Request(url, data=data, method=method, headers={
|
|
"Authorization": "Bearer " + token(),
|
|
"Content-Type": "application/json", "Accept": "application/json"})
|
|
try:
|
|
with urllib.request.urlopen(req, timeout=30) as r:
|
|
raw = r.read()
|
|
return r.status, (json.loads(raw) if raw else None)
|
|
except urllib.error.HTTPError as e:
|
|
raw = e.read()
|
|
return e.code, (json.loads(raw) if raw else None)
|
|
|
|
|
|
def main():
|
|
ip = socket.gethostbyname(SINK_HOST)
|
|
callback = f"http://{ip}:{SINK_PORT}/notifications"
|
|
|
|
# Restart-safe: drop any prior /notifications abonnement (its IP may be stale) before creating a
|
|
# fresh one for the current event-subscriber IP.
|
|
status, body = call("GET", f"{NRC}/api/v1/abonnement")
|
|
for ab in (body or []) if status == 200 else []:
|
|
if str(ab.get("callbackUrl", "")).endswith("/notifications"):
|
|
# The kanaal is part of "current": an abonnement left over from before S-19b-2 points at
|
|
# the right callback but listens on `zaken`, and would never be replaced on IP alone.
|
|
kanalen = [k.get("naam") for k in ab.get("kanalen", [])]
|
|
if ab.get("callbackUrl") == callback and kanalen == [KANAAL]:
|
|
print(f"abonnement already current: {ab['url']}")
|
|
return
|
|
call("DELETE", ab["url"])
|
|
print(f"removed stale abonnement {ab['url']}")
|
|
|
|
status, ab = call("POST", f"{NRC}/api/v1/abonnement", {
|
|
"callbackUrl": callback, "auth": SINK_AUTH,
|
|
"kanalen": [{"naam": KANAAL, "filters": {}}]})
|
|
if status != 201:
|
|
sys.exit(f"create abonnement -> {status}: {json.dumps(ab)}")
|
|
print(f"abonnement registered: {ab['url']} -> {callback}")
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|