feat: read projection sourced from the register in Objecten (closes #153) (#155)
CI / build (push) Successful in 1m7s
CI / lint (push) Successful in 1m26s
CI / unit (push) Successful in 1m37s
CI / frontend (push) Successful in 3m36s
CI / mutation (push) Successful in 6m42s
CI / verify-stack (push) Failing after 11m26s

## What & why

S-19b-2, closing out ADR-0028's stated direction: **the read projection is now derived from the
`RegisterRecord` in Objecten, not from ZGW zaak events.**

Until now the subscriber listened on `zaken` and *inferred* register state from case events — a
`zaak/create` meant INGEDIEND, and any `status/create` was assumed to be the approval (it may not
read OpenZaak, so it could not tell statustypen apart). The reference wasn't in the notification
at all, so every projection made a second hop to the ACL. The register — a fact about a person —
was being reconstructed by guessing at the lifecycle of the case that produced it.

- The subscriber's abonnement moves to the `objecten` kanaal (S-19b-1 made it publish).
- An Objecten notification carries **no record data**, only the object URL, so the record is read
  back through the ACL (`POST /register-records/read`) — §8.1 applies to Objecten exactly as
  ADR-0028 established.
- The record carries `id`, `status` and `reference`, so the row *is* the record: `IsZaakCreated`,
  `IsZaakStatusSet`, `ZaakUrl`, `ZaakId` and `ToEntry`'s `Resource == "status"` inference are all
  gone, and so is the ACL enrichment hop.
- **The ACL now writes an INGEDIEND record on submit.** Without it, re-sourcing would silently
  drop every submitted registration from the public register, since only approval wrote a record.
- `processed_notifications` holds the projected row (`register_id`, `status`, `reference`) instead
  of the ZGW event, so a rebuild is a replay with no mapping rules and no upstream reads at all.

**ADR-0030** records it. ADR-0028's open caveat — record written but not yet read, "the two must
agree" — is closed: there is one source now.

Closes #153

## Definition of Done

- [x] Linked Gitea issue (above).
- [x] Failing tests committed before the implementation — two red/green pairs, ACL side
      (06c0444566ef7d) and subscriber side (142ed458af09b2).
- [x] Refactor commit follows (b496ac9).
- [x] Conventional Commits referencing the issue (`refs #153`).
- [x] CI green — all six jobs on b30fa66, `verify-stack` end to end including the e2e.
- [x] `docker compose up` from a fresh clone reaches green health checks within 3 minutes
      (`verify-stack`'s bring-up step — see the wait-healthy fix below).
- [x] Docs updated — ADR-0030 added, ADR-0028's consequence + caveat annotated, BACKLOG.md,
      e2e header comment.
- [x] ADR added in `docs/architecture/`.
- [x] Demo note in `docs/demo-script.md` — n/a: no user-visible change. The openbaar register
      shows the same two statuses for the same registrations; only where they come from changed.

## Notes for reviewers

**The decision I'd most like a second opinion on** is the one the issue didn't settle: what
happens to INGEDIEND. Objecten held only INGESCHREVEN records, so re-sourcing forced a choice
between (a) the ACL also writing on submit, (b) a public register that lists only actual
registrations, or (c) a hybrid keeping both kanalen. I took (a): visible behaviour is unchanged
and the register holds the whole lifecycle. (b) is arguably the better *semantics* for a public
register but narrows what the portal shows and reads against PRD §68 ("~50 register entries with
diverse statuses"); (c) leaves the projection half-derived from ZGW, which is the coupling
ADR-0028 set out to remove. All three are laid out in ADR-0030.

**The dedup key is the projected row**, `objecten:object:{url}:{status}:{reference}` — not the
object URL (the ACL upserts *one object per registration*, so submit and approval notify about
the same URL and the approval would be swallowed as a duplicate) and not URL+actie (a retried
approval is a second `update`). Redeliveries collapse, genuine state changes don't. §8.6.

**The migration drops columns rather than renaming them.** EF scaffolded renames — `resource` →
`register_id`, `zaak_id` → `status` — which would have carried ZGW values into columns meaning
something else, and a rebuild would then have projected that garbage. It also empties both
tables: a pre-slice row describes a zaak event the new projector can't reproject, and those
registrations have no RegisterRecord in Objecten either, so they're not re-derivable from the new
source. Stated as a ceiling in the ADR — fine while stacks are ephemeral, backfill from Objecten
if a long-lived environment ever needs it.

**`run-projection-check.sh` now opens its zaak through the ACL** instead of straight against
OpenZaak, because the ACL is what writes the record. A zaak created behind the ACL's back
produces no projection row — that's the re-source working, not a gap.

## Three fixes CI found, none of them in the projection logic

1. **`wait-healthy.sh` matched the wrong container** (744f91a). Bring-up timed out with
   `TIMEOUT: 'objecten' not healthy (status=none)` while the `docker ps` it dumps showed
   objecten `Up 9 minutes (healthy)`. `--filter name=` is a substring match, so `objecten` also
   matches `objecten-db`/`objecten-redis`/`objecten-celery`, and `head -1` took whichever docker
   listed first — the celery worker has no healthcheck, hence `status=none`. Latent since those
   services landed and decided purely by listing order; `objecttypen` matches `objecttypen-db`
   the same way. Anchored on the compose replica suffix, which the verify scripts already do.
2. **The ACL had to be repointed at OpenZaak's IP** (7e0897a). Opening the zaak through the ACL
   put this check in the same bind run-domain-check.sh already handles:
   `400 {"name":"zaaktype","code":"bad-url","reason":"Voer een geldige URL in."}`. OpenZaak
   reflects the request Host into the zaaktype URL and then rejects it on zaak-create when
   single-label — the mechanism compose already documents on `ACL_OPENZAAK_BASEURL`.
3. **Approval arrives as `partial_update`, not `update`** (0dd26a7b30fa66) — the one real bug
   in the slice. The ACL upserts with PATCH; DRF routes it through the notifying `update()` but
   names the action `partial_update`, so the projector dropped every approval. Only the e2e could
   catch it: `verify-projection` drives a submit, and per ADR-0028 the e2e is the only check that
   drives a *real* approval.

`verify-tracing` also failed once (run 722) on a path this PR doesn't touch, and passed on a
plain re-run of the same commit. Tempo logged `pusher failed to consume trace data` /
`distributor_pool failing healthcheck` — it dropped spans under runner load rather than the trace
chain being broken. Filed as **#156** rather than absorbed here.

**Correction to the #152 PR notes:** I wrote there that celery concurrency was "the next knob" if
verify-stack got tight. It isn't — `CELERY_WORKER_CONCURRENCY` already defaults to 1 in the Maykin
image, so `objecten-celery` is already a single-process worker. Noted in #156.

**Possible follow-up, deliberately not done here:** an `openzaak.local` network alias mirroring
`objecten.local` would remove the ACL-repoint dance from both run-domain-check.sh and
run-projection-check.sh. It changes the host in every zaak URL the system produces, which is too
broad a ripple to land inside an unrelated slice — worth its own issue.

**Known costs, all in the ADR:** submission is now two writes across two modules and eventually
consistent (same posture ADR-0028 accepted for approval); projecting now depends on the ACL being
reachable on the main path, not just for enrichment (NRC retries, so it converges); and OpenZaak
still publishes to `zaken` with nothing in the product listening — kept because `verify-nrc`
asserts that path.Reviewed-on: #155
This commit was merged in pull request #155.
This commit is contained in:
not
2026-09-01 07:26:33 +00:00
parent 2125fb0cfd
commit 94742a261f
30 changed files with 821 additions and 251 deletions
@@ -2,40 +2,39 @@ namespace EventSubscriber.Application;
/// <summary>
/// An inbound NRC (Open Notificaties) notification, as Open Notificaties POSTs it to an
/// abonnement callback. Only the fields the projection needs are modelled; the full ZGW
/// "Notificatie" resource also carries <c>aanmaakdatum</c> and <c>kenmerken</c> which the
/// minimal projection ignores (bsn is deferred — see ADR-0008). For a <c>zaken</c>/<c>zaak</c>/<c>create</c>
/// notification <c>hoofdObject</c> and <c>resourceUrl</c> are both the created zaak's URL.
/// abonnement callback. Only the fields the projection needs are modelled.
/// </summary>
/// <remarks>
/// Since S-19b-2 the subscriber listens on the <c>objecten</c> kanaal, not <c>zaken</c>: the
/// register record in Objecten is what the projection is derived from (ADR-0030), so the
/// projection is a cache of the register rather than a re-derivation of the case system. An
/// Objecten notification carries <b>no record data</b> — only the object URL (as both
/// <c>hoofdObject</c> and <c>resourceUrl</c>) and the objecttype as a kenmerk — so the record
/// itself is read back through the ACL.
/// </remarks>
public sealed record Notification(
string Kanaal,
string Resource,
string Actie,
Uri ResourceUrl,
Uri? HoofdObject = null)
Uri ResourceUrl)
{
/// <summary>A zaak being created — projected as INGEDIEND.</summary>
public bool IsZaakCreated =>
Kanaal == "zaken" && Resource == "zaak" && Actie == "create";
/// <summary>A status being set on a zaak — the approval, projected as INGESCHREVEN (S-09b). In the
/// walking skeleton the only status ever set after creation is the approval, and the subscriber may
/// not read OpenZaak (§8.1), so any status-create is taken as the approval.</summary>
public bool IsZaakStatusSet =>
Kanaal == "zaken" && Resource == "status" && Actie == "create";
/// <summary>The zaak URL this notification concerns — <c>hoofdObject</c> (the zaak) for a status
/// notification, else the resource URL (which, for a zaak-create, is the zaak).</summary>
public Uri ZaakUrl => HoofdObject ?? ResourceUrl;
/// <summary>The zaak UUID used as the projection key — the trailing segment of <see cref="ZaakUrl"/>.</summary>
public string ZaakId => ZaakUrl.Segments[^1].Trim('/');
/// <summary>
/// A deterministic dedup key. Open Notificaties carries no notification id and may
/// redeliver, so the key is derived from the immutable notification content: two
/// deliveries of the same zaak-create collapse to one. (NRC may also deliver
/// out of order; the projector tolerates that — order does not change the outcome.)
/// A register record written to Objecten — <c>create</c> on submit and <c>partial_update</c> on
/// approval, since the ACL upserts the same object for a registration (§8.6).
/// </summary>
public string IdempotencyKey => $"{Kanaal}:{Resource}:{Actie}:{ResourceUrl}";
/// <remarks>
/// <c>partial_update</c> is what a PATCH actually reports: DRF routes it through the notifying
/// <c>update()</c> but names the action <c>partial_update</c>, and that is what Objecten puts in
/// the notification. <c>update</c> is accepted too, so a PUT-shaped write would project the same
/// way. <c>destroy</c> is deliberately not: removing a registration from the public register is
/// its own decision, not a side effect of this one.
/// </remarks>
public bool IsRegisterRecordWritten =>
Kanaal == "objecten" && Resource == "object"
&& Actie is "create" or "update" or "partial_update";
/// <summary>The object holding the register record. For a <c>resource: object</c> notification
/// Objecten sends the object as both <c>hoofdObject</c> and <c>resourceUrl</c> — the object is
/// the main resource — so the notification's own <c>hoofdObject</c> is not modelled.</summary>
public Uri ObjectUrl => ResourceUrl;
}
@@ -3,21 +3,27 @@ namespace EventSubscriber.Application;
/// <summary>
/// Projects inbound NRC notifications into the read projection. Tolerates duplicate and
/// out-of-order deliveries (CLAUDE.md §8.6): the notification log dedups, and the projection
/// upsert is idempotent on the zaak id. Rebuilds the projection by replaying the log.
/// upsert is idempotent on the register id. Rebuilds the projection by replaying the log.
/// </summary>
public sealed class NotificationProjector(INotificationLog log, IProjectionStore store, IAclClient acl)
{
/// <summary>Handle one inbound notification. Reacts to a zaak being created (INGEDIEND) and a
/// status being set (INGESCHREVEN); ignores everything else. Enriches the row with the zaak's
/// reference via the ACL (§8.1) and records it so a rebuild needs no ZGW access (#78).</summary>
/// <summary>Handle one inbound notification. Reacts to a register record being written to
/// Objecten (S-19b-2, ADR-0030) and ignores everything else. The notification carries only the
/// object URL, so the record is read back through the ACL (§8.1) and becomes the row verbatim.</summary>
public async Task HandleAsync(Notification notification, CancellationToken ct = default)
{
if (!notification.IsZaakCreated && !notification.IsZaakStatusSet)
ArgumentNullException.ThrowIfNull(notification);
if (!notification.IsRegisterRecordWritten)
return;
var record = await acl.GetRegisterRecordAsync(notification.ObjectUrl, ct);
// The object is gone, or holds no register record — nothing to project (§8.6).
if (record is null)
return;
var reference = await acl.GetZaakReferenceAsync(notification.ZaakUrl, ct);
var recorded = new RecordedNotification(
notification.IdempotencyKey, notification.Actie, notification.ZaakId, notification.Resource, reference);
KeyFor(notification.ObjectUrl, record), record.Id, record.Status, record.Reference);
// Atomic record-or-skip: a duplicate (or concurrent) delivery is recognised and dropped
// before it touches the projection, so the projection stays a faithful derived artefact.
@@ -27,6 +33,20 @@ public sealed class NotificationProjector(INotificationLog log, IProjectionStore
await store.UpsertAsync(ToEntry(recorded), ct);
}
/// <summary>
/// A deterministic dedup key: the object, plus the state that write puts in the projection.
/// </summary>
/// <remarks>
/// Open Notificaties carries no notification id and may redeliver, so the key is derived from
/// content. It cannot be the object URL alone — the ACL upserts one object per registration, so
/// submit and approval both notify about the *same* URL and the approval would be swallowed as a
/// duplicate. Nor can it include the actie: a retried approval would be a second `update`. Keying
/// on the projected row means a redelivery collapses and a genuine state change does not, which
/// is exactly the property §8.6 asks for.
/// </remarks>
private static string KeyFor(Uri objectUrl, RegisterRecord record)
=> $"objecten:object:{objectUrl}:{record.Status}:{record.Reference}";
/// <summary>Rebuild the projection from the durable notification log (PRD §8.4).</summary>
public async Task RebuildAsync(CancellationToken ct = default)
{
@@ -35,11 +55,9 @@ public sealed class NotificationProjector(INotificationLog log, IProjectionStore
await store.UpsertAsync(ToEntry(recorded), ct);
}
/// <summary>The projection row for an accepted notification: a status-set maps to INGESCHREVEN,
/// a zaak-create to INGEDIEND. bsn/naam are deferred (ADR-0008).</summary>
/// <summary>The projection row for an accepted notification. The log already holds exactly the
/// row's fields, so a rebuild needs no mapping rules and no upstream reads. bsn/naam stay
/// deferred — the register record is public-safe by construction (ADR-0027).</summary>
private static RegisterEntry ToEntry(RecordedNotification recorded)
=> new(
recorded.ZaakId,
recorded.Resource == "status" ? RegistrationStatus.Ingeschreven : RegistrationStatus.Ingediend,
Reference: recorded.Reference);
=> new(recorded.RegisterId, recorded.Status, recorded.Reference);
}
@@ -4,7 +4,7 @@ namespace EventSubscriber.Application;
/// The durable log of notifications the subscriber has accepted. It is both the idempotency
/// guard (a replayed notification is recognised and dropped) and the rebuild source: the
/// projection is a derived artefact (PRD §8.4) regenerated by replaying this log, so a rebuild
/// needs no access to OpenZaak (CLAUDE.md §8.1). Implemented in Infrastructure over Postgres.
/// needs no access to Objecten or ZGW (CLAUDE.md §8.1). Implemented in Infrastructure over Postgres.
/// </summary>
public interface INotificationLog
{
@@ -19,22 +19,29 @@ public interface INotificationLog
Task<IReadOnlyList<RecordedNotification>> AllAsync(CancellationToken ct = default);
}
/// <summary>A notification that has been accepted, retaining what a rebuild needs to recompute its
/// projection row — the ZGW <c>resource</c> (zaak-create → INGEDIEND vs status-set → INGESCHREVEN) and
/// the zaak <c>reference</c> (identificatie), so a rebuild reproduces the row without re-reading ZGW (#78).</summary>
public sealed record RecordedNotification(string Key, string Actie, string ZaakId, string Resource, string? Reference);
/// <summary>
/// An accepted notification, retaining exactly the projection row it produced — so a rebuild
/// reproduces the row by replaying the log, without re-reading Objecten (S-19b-2, ADR-0030).
/// </summary>
public sealed record RecordedNotification(string Key, string RegisterId, string Status, string? Reference);
/// <summary>
/// Port to the Anti-Corruption Layer. The subscriber enriches the projection with the zaak's
/// public-safe reference (its identificatie) by asking the ACL — the only code that may read ZGW
/// (§8.1) — rather than reading OpenZaak itself (adr-proposal #78).
/// Port to the Anti-Corruption Layer. An Objecten notification carries only the object URL, so the
/// subscriber reads the register record back through the ACL — the only code that may talk to
/// Objecten (§8.1, ADR-0028) — rather than reading Objecten itself.
/// </summary>
public interface IAclClient
{
/// <summary>The zaak's reference (identificatie) for the read projection.</summary>
Task<string> GetZaakReferenceAsync(Uri zaakUrl, CancellationToken ct = default);
/// <summary>The register record the object at <paramref name="objectUrl"/> holds, or
/// <c>null</c> if it holds none — the object may be gone by the time a redelivered
/// notification is handled, which is not an error (§8.6).</summary>
Task<RegisterRecord?> GetRegisterRecordAsync(Uri objectUrl, CancellationToken ct = default);
}
/// <summary>The public-safe register record as the ACL returns it — the RegisterRecord objecttype's
/// schema (ADR-0027). No bsn, no name: the register is world-readable.</summary>
public sealed record RegisterRecord(string Id, string Status, string? Reference);
/// <summary>The read projection store. Owned by the projection bounded context (ADR-0008); the
/// subscriber writes to it and the projection-api reads it.</summary>
public interface IProjectionStore