feat: read projection sourced from the register in Objecten (closes #153) #155

Merged

not merged 12 commits from feat/153-projection-sourced-from-objecten into main 2026-09-01 07:26:34 +00:00

Conversation 0 Commits 12 Files Changed 30

+821 -251

not commented 2026-08-28 10:38:58 +00:00

Contributor

Copy Link

Copy Source

What & why

S-19b-2, closing out ADR-0028's stated direction: the read projection is now derived from the
RegisterRecord in Objecten, not from ZGW zaak events.

Until now the subscriber listened on zaken and inferred register state from case events — a
zaak/create meant INGEDIEND, and any status/create was assumed to be the approval (it may not
read OpenZaak, so it could not tell statustypen apart). The reference wasn't in the notification
at all, so every projection made a second hop to the ACL. The register — a fact about a person —
was being reconstructed by guessing at the lifecycle of the case that produced it.

• The subscriber's abonnement moves to the objecten kanaal (S-19b-1 made it publish).
• An Objecten notification carries no record data, only the object URL, so the record is read
  back through the ACL (POST /register-records/read) — §8.1 applies to Objecten exactly as
  ADR-0028 established.
• The record carries id, status and reference, so the row is the record: IsZaakCreated,
  IsZaakStatusSet, ZaakUrl, ZaakId and ToEntry's Resource == "status" inference are all
  gone, and so is the ACL enrichment hop.
• The ACL now writes an INGEDIEND record on submit. Without it, re-sourcing would silently
  drop every submitted registration from the public register, since only approval wrote a record.
• processed_notifications holds the projected row (register_id, status, reference) instead
  of the ZGW event, so a rebuild is a replay with no mapping rules and no upstream reads at all.

ADR-0030 records it. ADR-0028's open caveat — record written but not yet read, "the two must
agree" — is closed: there is one source now.

Closes #153

Definition of Done

• Linked Gitea issue (above).
• Failing tests committed before the implementation — two red/green pairs, ACL side
  (06c0444 → 566ef7d) and subscriber side (142ed45 → 8af09b2).
• Refactor commit follows (b496ac9).
• Conventional Commits referencing the issue (refs #153).
• CI green — all six jobs on b30fa66, verify-stack end to end including the e2e.
• docker compose up from a fresh clone reaches green health checks within 3 minutes
  (verify-stack's bring-up step — see the wait-healthy fix below).
• Docs updated — ADR-0030 added, ADR-0028's consequence + caveat annotated, BACKLOG.md,
  e2e header comment.
• ADR added in docs/architecture/.
• Demo note in docs/demo-script.md — n/a: no user-visible change. The openbaar register
  shows the same two statuses for the same registrations; only where they come from changed.

Notes for reviewers

The decision I'd most like a second opinion on is the one the issue didn't settle: what
happens to INGEDIEND. Objecten held only INGESCHREVEN records, so re-sourcing forced a choice
between (a) the ACL also writing on submit, (b) a public register that lists only actual
registrations, or (c) a hybrid keeping both kanalen. I took (a): visible behaviour is unchanged
and the register holds the whole lifecycle. (b) is arguably the better semantics for a public
register but narrows what the portal shows and reads against PRD §68 ("~50 register entries with
diverse statuses"); (c) leaves the projection half-derived from ZGW, which is the coupling
ADR-0028 set out to remove. All three are laid out in ADR-0030.

The dedup key is the projected row, objecten:object:{url}:{status}:{reference} — not the
object URL (the ACL upserts one object per registration, so submit and approval notify about
the same URL and the approval would be swallowed as a duplicate) and not URL+actie (a retried
approval is a second update). Redeliveries collapse, genuine state changes don't. §8.6.

The migration drops columns rather than renaming them. EF scaffolded renames — resource →
register_id, zaak_id → status — which would have carried ZGW values into columns meaning
something else, and a rebuild would then have projected that garbage. It also empties both
tables: a pre-slice row describes a zaak event the new projector can't reproject, and those
registrations have no RegisterRecord in Objecten either, so they're not re-derivable from the new
source. Stated as a ceiling in the ADR — fine while stacks are ephemeral, backfill from Objecten
if a long-lived environment ever needs it.

run-projection-check.sh now opens its zaak through the ACL instead of straight against
OpenZaak, because the ACL is what writes the record. A zaak created behind the ACL's back
produces no projection row — that's the re-source working, not a gap.

Three fixes CI found, none of them in the projection logic

1. wait-healthy.sh matched the wrong container (744f91a). Bring-up timed out with
   TIMEOUT: 'objecten' not healthy (status=none) while the docker ps it dumps showed
   objecten Up 9 minutes (healthy). --filter name= is a substring match, so objecten also
   matches objecten-db/objecten-redis/objecten-celery, and head -1 took whichever docker
   listed first — the celery worker has no healthcheck, hence status=none. Latent since those
   services landed and decided purely by listing order; objecttypen matches objecttypen-db
   the same way. Anchored on the compose replica suffix, which the verify scripts already do.
2. The ACL had to be repointed at OpenZaak's IP (7e0897a). Opening the zaak through the ACL
   put this check in the same bind run-domain-check.sh already handles:
   400 {"name":"zaaktype","code":"bad-url","reason":"Voer een geldige URL in."}. OpenZaak
   reflects the request Host into the zaaktype URL and then rejects it on zaak-create when
   single-label — the mechanism compose already documents on ACL_OPENZAAK_BASEURL.
3. Approval arrives as partial_update, not update (0dd26a7 → b30fa66) — the one real bug
   in the slice. The ACL upserts with PATCH; DRF routes it through the notifying update() but
   names the action partial_update, so the projector dropped every approval. Only the e2e could
   catch it: verify-projection drives a submit, and per ADR-0028 the e2e is the only check that
   drives a real approval.

verify-tracing also failed once (run 722) on a path this PR doesn't touch, and passed on a
plain re-run of the same commit. Tempo logged pusher failed to consume trace data /
distributor_pool failing healthcheck — it dropped spans under runner load rather than the trace
chain being broken. Filed as #156 rather than absorbed here.

Correction to the #152 PR notes: I wrote there that celery concurrency was "the next knob" if
verify-stack got tight. It isn't — CELERY_WORKER_CONCURRENCY already defaults to 1 in the Maykin
image, so objecten-celery is already a single-process worker. Noted in #156.

Possible follow-up, deliberately not done here: an openzaak.local network alias mirroring
objecten.local would remove the ACL-repoint dance from both run-domain-check.sh and
run-projection-check.sh. It changes the host in every zaak URL the system produces, which is too
broad a ripple to land inside an unrelated slice — worth its own issue.

Known costs, all in the ADR: submission is now two writes across two modules and eventually
consistent (same posture ADR-0028 accepted for approval); projecting now depends on the ACL being
reachable on the main path, not just for enrichment (NRC retries, so it converges); and OpenZaak
still publishes to zaken with nothing in the product listening — kept because verify-nrc
asserts that path.

## What & why S-19b-2, closing out ADR-0028's stated direction: **the read projection is now derived from the `RegisterRecord` in Objecten, not from ZGW zaak events.** Until now the subscriber listened on `zaken` and *inferred* register state from case events — a `zaak/create` meant INGEDIEND, and any `status/create` was assumed to be the approval (it may not read OpenZaak, so it could not tell statustypen apart). The reference wasn't in the notification at all, so every projection made a second hop to the ACL. The register — a fact about a person — was being reconstructed by guessing at the lifecycle of the case that produced it. - The subscriber's abonnement moves to the `objecten` kanaal (S-19b-1 made it publish). - An Objecten notification carries **no record data**, only the object URL, so the record is read back through the ACL (`POST /register-records/read`) — §8.1 applies to Objecten exactly as ADR-0028 established. - The record carries `id`, `status` and `reference`, so the row *is* the record: `IsZaakCreated`, `IsZaakStatusSet`, `ZaakUrl`, `ZaakId` and `ToEntry`'s `Resource == "status"` inference are all gone, and so is the ACL enrichment hop. - **The ACL now writes an INGEDIEND record on submit.** Without it, re-sourcing would silently drop every submitted registration from the public register, since only approval wrote a record. - `processed_notifications` holds the projected row (`register_id`, `status`, `reference`) instead of the ZGW event, so a rebuild is a replay with no mapping rules and no upstream reads at all. **ADR-0030** records it. ADR-0028's open caveat — record written but not yet read, "the two must agree" — is closed: there is one source now. Closes #153 ## Definition of Done - [x] Linked Gitea issue (above). - [x] Failing tests committed before the implementation — two red/green pairs, ACL side (06c0444 → 566ef7d) and subscriber side (142ed45 → 8af09b2). - [x] Refactor commit follows (b496ac9). - [x] Conventional Commits referencing the issue (`refs #153`). - [x] CI green — all six jobs on b30fa66, `verify-stack` end to end including the e2e. - [x] `docker compose up` from a fresh clone reaches green health checks within 3 minutes (`verify-stack`'s bring-up step — see the wait-healthy fix below). - [x] Docs updated — ADR-0030 added, ADR-0028's consequence + caveat annotated, BACKLOG.md, e2e header comment. - [x] ADR added in `docs/architecture/`. - [x] Demo note in `docs/demo-script.md` — n/a: no user-visible change. The openbaar register shows the same two statuses for the same registrations; only where they come from changed. ## Notes for reviewers **The decision I'd most like a second opinion on** is the one the issue didn't settle: what happens to INGEDIEND. Objecten held only INGESCHREVEN records, so re-sourcing forced a choice between (a) the ACL also writing on submit, (b) a public register that lists only actual registrations, or (c) a hybrid keeping both kanalen. I took (a): visible behaviour is unchanged and the register holds the whole lifecycle. (b) is arguably the better *semantics* for a public register but narrows what the portal shows and reads against PRD §68 ("~50 register entries with diverse statuses"); (c) leaves the projection half-derived from ZGW, which is the coupling ADR-0028 set out to remove. All three are laid out in ADR-0030. **The dedup key is the projected row**, `objecten:object:{url}:{status}:{reference}` — not the object URL (the ACL upserts *one object per registration*, so submit and approval notify about the same URL and the approval would be swallowed as a duplicate) and not URL+actie (a retried approval is a second `update`). Redeliveries collapse, genuine state changes don't. §8.6. **The migration drops columns rather than renaming them.** EF scaffolded renames — `resource` → `register_id`, `zaak_id` → `status` — which would have carried ZGW values into columns meaning something else, and a rebuild would then have projected that garbage. It also empties both tables: a pre-slice row describes a zaak event the new projector can't reproject, and those registrations have no RegisterRecord in Objecten either, so they're not re-derivable from the new source. Stated as a ceiling in the ADR — fine while stacks are ephemeral, backfill from Objecten if a long-lived environment ever needs it. **`run-projection-check.sh` now opens its zaak through the ACL** instead of straight against OpenZaak, because the ACL is what writes the record. A zaak created behind the ACL's back produces no projection row — that's the re-source working, not a gap. ## Three fixes CI found, none of them in the projection logic 1. **`wait-healthy.sh` matched the wrong container** (744f91a). Bring-up timed out with `TIMEOUT: 'objecten' not healthy (status=none)` while the `docker ps` it dumps showed objecten `Up 9 minutes (healthy)`. `--filter name=` is a substring match, so `objecten` also matches `objecten-db`/`objecten-redis`/`objecten-celery`, and `head -1` took whichever docker listed first — the celery worker has no healthcheck, hence `status=none`. Latent since those services landed and decided purely by listing order; `objecttypen` matches `objecttypen-db` the same way. Anchored on the compose replica suffix, which the verify scripts already do. 2. **The ACL had to be repointed at OpenZaak's IP** (7e0897a). Opening the zaak through the ACL put this check in the same bind run-domain-check.sh already handles: `400 {"name":"zaaktype","code":"bad-url","reason":"Voer een geldige URL in."}`. OpenZaak reflects the request Host into the zaaktype URL and then rejects it on zaak-create when single-label — the mechanism compose already documents on `ACL_OPENZAAK_BASEURL`. 3. **Approval arrives as `partial_update`, not `update`** (0dd26a7 → b30fa66) — the one real bug in the slice. The ACL upserts with PATCH; DRF routes it through the notifying `update()` but names the action `partial_update`, so the projector dropped every approval. Only the e2e could catch it: `verify-projection` drives a submit, and per ADR-0028 the e2e is the only check that drives a *real* approval. `verify-tracing` also failed once (run 722) on a path this PR doesn't touch, and passed on a plain re-run of the same commit. Tempo logged `pusher failed to consume trace data` / `distributor_pool failing healthcheck` — it dropped spans under runner load rather than the trace chain being broken. Filed as **#156** rather than absorbed here. **Correction to the #152 PR notes:** I wrote there that celery concurrency was "the next knob" if verify-stack got tight. It isn't — `CELERY_WORKER_CONCURRENCY` already defaults to 1 in the Maykin image, so `objecten-celery` is already a single-process worker. Noted in #156. **Possible follow-up, deliberately not done here:** an `openzaak.local` network alias mirroring `objecten.local` would remove the ACL-repoint dance from both run-domain-check.sh and run-projection-check.sh. It changes the host in every zaak URL the system produces, which is too broad a ripple to land inside an unrelated slice — worth its own issue. **Known costs, all in the ADR:** submission is now two writes across two modules and eventually consistent (same posture ADR-0028 accepted for approval); projecting now depends on the ACL being reachable on the main path, not just for enrichment (NRC retries, so it converges); and OpenZaak still publishes to `zaken` with nothing in the product listening — kept because `verify-nrc` asserts that path.

not added this to the Iteration 4 — Objecten milestone 2026-08-28 10:38:58 +00:00

not added 8 commits 2026-08-28 10:38:58 +00:00

test(acl): submit writes an INGEDIEND record, and records are readable back (refs #153) ... 06c0444859

Ports and failing tests for the ACL half of S-19b-2, ahead of the implementation.

Once the projection is sourced from Objecten (ADR-0028's stated direction), a submitted
registration has to exist in the register the moment the zaak is opened — otherwise
re-sourcing silently drops every INGEDIEND row, since today only approval writes a record.
So `OpenZaakAsync` gains a second write, and approval upserts that same record to
INGESCHREVEN.

The subscriber gets only an object URL on an `objecten` notification (the payload carries no
record data) and may not read Objecten itself (§8.1), so `IRegisterRecordGateway` gains a
read and `AclService` exposes it.

Red:
- AclService does not yet write on open → the record assertion fails on an empty list.
- ObjectenGateway.GetAsync is a shell throwing NotImplementedException; its tests pin the
  contract: fetch the object URL directly (no objecttype resolution, no search), the CRS
  header a geo API requires, static Token auth, and a 404 read as "nothing to project"
  rather than an error (§8.6).

feat(acl): write the INGEDIEND record on submit and read records back (refs #153) ... 566ef7dd64

- OpenZaakAsync upserts a RegisterRecord with status INGEDIEND after opening the zaak,
  keyed on the same zaak id approval later upserts to INGESCHREVEN. The reference comes
  from the registration, so this path needs no ZGW read-back.
- ObjectenGateway.GetAsync fetches an object by the URL a notification carried — no
  objecttype resolution, no search — and reads 404 as "no record" rather than an error.
- POST /register-records/read exposes it to the Event Subscriber, which may not talk to
  Objecten itself (§8.1).

test(event-subscriber): the projection is sourced from register records (refs #153) ... 142ed454aa

Ports, schema and failing tests for the subscriber half of S-19b-2, ahead of the
implementation.

The subscriber now listens on the `objecten` kanaal instead of `zaken`. An Objecten
notification carries no record data — only the object URL — so the record is read back
through the ACL (§8.1), and the zaak-shaped surface goes away: IsZaakCreated /
IsZaakStatusSet / ZaakUrl / ZaakId and ToEntry's `Resource == "status"` mapping are replaced
by IsRegisterRecordWritten + ObjectUrl.

The notification log now holds the projected row itself (register id, status, reference),
so a rebuild is a replay with no mapping rules and no upstream reads. The migration drops
the old columns rather than renaming them — EF scaffolded renames that would have carried
ZGW values into columns meaning something else — and empties both tables, since a
pre-slice row is neither reprojectable nor re-derivable from the new source.

Red: HandleAsync recognises a register write but does not yet read or project it, so the
seven projection assertions fail on an empty store.

feat(event-subscriber): project register records read back through the ACL (refs #153) ... 8af09b2c92

HandleAsync reads the record at the notification's object URL through the ACL and writes it
to the projection verbatim — the record already carries id, status and reference, so there
is no mapping and no enrichment hop.

The dedup key is the object plus the state that write projects. It cannot be the object URL
alone (the ACL upserts one object per registration, so submit and approval notify about the
same URL and the approval would be swallowed), nor include the actie (a retried approval is
a second `update`). Keying on the projected row collapses redeliveries and lets genuine
state changes through — §8.6.

feat(infra): subscribe the projection to the objecten kanaal (refs #153) ... ceb65991de

Completes the re-source (ADR-0030): the Event Subscriber's abonnement moves from `zaken` to
`objecten`, in both the local stack's `nrc-subscribe` and the CI projection check. The
OpenZaak → NRC check keeps its own `zaken` abonnement — OpenZaak still publishes, nothing
in the product listens.

- register-abonnement.py subscribes to `objecten`, and now treats the kanaal as part of
  "already current" — an abonnement left from before this slice points at the right callback
  but the wrong kanaal, and would never have been replaced on IP alone.
- run-projection-check.sh opens its zaak *through the ACL* instead of straight against
  OpenZaak, because the ACL is what writes the register record the projection is now derived
  from. A zaak created behind the ACL's back produces no row — which is the re-source working.
- The acceptance scenario is restated in register terms and gains the approval case: the same
  row moving INGEDIEND → INGESCHREVEN is now one registration's record being updated, not two
  unrelated ZGW events.

docs: ADR-0030 — the read projection is sourced from the register (refs #153) ... 62fb986701

Records the re-source and the three decisions inside it: the ACL writing an INGEDIEND record
on submit (without which re-sourcing silently drops every submitted registration), the dedup
key being the projected row rather than the notification, and the notification log holding
the row rather than the event.

Closes out ADR-0028's stated direction and the caveat it left open — the register record was
written but not yet read, and the two had to agree; there is now one source.

docs(e2e): the happy path's public statuses now come from the register (refs #153) ... 88a601123b

Comment only — the assertions were already reference-matched and hold unchanged. Names the
new chain (ACL → Objecten → NRC → event-subscriber → projection) so the INGEDIEND assertion
reads as the proof of the re-source that it now is.

refactor(event-subscriber): drop the hoofdObject fallback (refs #153) ... b496ac9477

For a `resource: object` notification Objecten sends the object as both hoofdObject and
resourceUrl — the object *is* the main resource — so `HoofdObject ?? ResourceUrl` was a
branch that can never take its left side and that no test could distinguish. It came across
from the zaken path, where hoofdObject genuinely differed (the zaak behind a status).

Tests unchanged and green.

not added the area:aclarea:event-subscriberarea:projectiontype:slice labels 2026-08-28 10:39:11 +00:00

not added 1 commit 2026-08-28 11:01:14 +00:00

fix(infra): anchor wait-healthy's container lookup on the compose replica suffix (refs #153) ... 744f91a2b2

Bring-up timed out with

  TIMEOUT: 'objecten' not healthy (status=none)

while the very `docker ps` it dumps showed infra-objecten-1 "Up 9 minutes (healthy)".

`--filter name=` is a substring match, so `objecten` also matches objecten-db,
objecten-redis and (since #152) objecten-celery. `head -1` took whichever docker listed
first; the celery worker declares no healthcheck, so it inspected as status=none and the
wait sat there until the deadline.

Not objecten-specific — `objecttypen` matches objecttypen-db the same way. The bug has been
latent since those services landed and was decided by listing order, which is why it only
surfaced now. Anchored on the replica suffix, matching both docker compose and
podman-compose naming — the same anchoring the verify check scripts already use.

not added 1 commit 2026-08-28 11:18:04 +00:00

fix(infra): repoint the acl at OpenZaak's IP before opening a zaak (refs #153) ... 7e0897a41e

The projection check now opens its zaak through the ACL, which puts it in the same bind
run-domain-check.sh already handles:

  400 {"name":"zaaktype","code":"bad-url","reason":"Voer een geldige URL in."}

OpenZaak reflects the request Host into the zaaktype `url` it returns and then rejects that
same URL on zaak-create when the host is single-label. The stack's ACL is configured with
`http://openzaak:8000/`, so it has to be recreated with ACL_OPENZAAK_BASEURL pointed at
OpenZaak's container IP first — the mechanism compose already documents on that variable.

Same class of constraint as the objecten.local alias (ADR-0029), and the third module now
known to reflect a request Host into data another module validates.

not added 2 commits 2026-08-28 11:40:04 +00:00

test(event-subscriber): approval arrives as partial_update, not update (refs #153) ... 0dd26a711a

The e2e reached INGEDIEND but never INGESCHREVEN. NRC's own log says why:

  {"event": "notification_received", "action": "partial_update",
   "resource_url": "http://objecten.local:8000/api/v2/objects/a9a7f125-..."}

The ACL PATCHes the object on approval. DRF routes a PATCH through the notifying `update()`
but reports the action as `partial_update`, so accepting only `create`/`update` drops every
approval on the floor — the exact state change the slice exists to project.

Red: the approval case is now a Theory over both acties, and the partial_update one fails.

feat(event-subscriber): accept partial_update as a register write (refs #153) ... b30fa664d8

The ACL upserts with PATCH, so every approval notification carries actie `partial_update`.
Accepting it makes the INGEDIEND → INGESCHREVEN transition project. `update` stays accepted
so a PUT-shaped write behaves the same; `destroy` deliberately does not — removing a
registration from the public register is its own decision, not a side effect of this one.

ADR-0030 records why the actie list is what it is.

not referenced this pull request 2026-08-28 12:30:22 +00:00

verify-tracing is flaky — Tempo intermittently drops spans under runner load #156

not merged commit 94742a261f into main 2026-09-01 07:26:34 +00:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

area:acl

area:bff

area:docs

area:domain

area:event-subscriber

area:infra

area:portal-behandel

area:portal-beheer

area:portal-openbaar

area:portal-self-service

area:projection

area:workflow

type:adr-proposal

type:bug

type:chore

type:slice

No labels area:acl area:event-subscriber area:projection type:slice

Milestone

No items

No Milestone Iteration 4 — Objecten

Projects

Clear projects

No projects

Assignees

Clear assignees

eho (Edwin van den Houdt)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: eho/register-referentie#155