atomic-design-poc/docs/project/archive/backlog/WP-58-openzaak-notifications.md

# WP-58 — Real notifications (celery + scripted abonnement)

Status: done
Phase: 10 — OpenZaak production hardening

## Why

The WP-54 harness deliberately trims celery/celery-beat/celery-flower and nginx, and sets
`NOTIFICATIONS_DISABLED: 'true'` — without a Celery worker, OpenZaak 500s and rolls back on
every write to a notified resource. Fine for a fixture-driven integration harness; a real
deployment that wants live Notificaties (WP-52's webhook) needs the workers running and the
`abonnement` (subscription) actually registered against the BFF's public callback URL —
today that registration step is manual.

## Read first

- `backend/openzaak/docker-compose.openzaak.yml` (top-of-file ponytail note)
- [openzaak-integration.md](../reference/openzaak-integration.md) (Notificaties section)
- [WP-52](WP-52-openzaak-notificaties.md)

## Decisions (pre-made, don't relitigate)

- Add celery + celery-beat as additional compose services (same `openzaak/open-zaak`
  image, different command), pointed at the same redis broker already in the harness.
- Registering the `abonnement` becomes a scripted, idempotent step — not a manual
  admin-UI action — parameterized by the BFF's real public URL.
- Keep the existing WP-54 harness variant (`NOTIFICATIONS_DISABLED: 'true'`) available for
  fast local iteration where a live webhook round-trip isn't needed; this WP is additive
  (a "with notifications" profile/override), not a replacement.

## Files

- `backend/openzaak/docker-compose.openzaak.yml` (or an override file)
- New script/config for `abonnement` registration
- `docs/reference/openzaak-integration.md`

## Steps

1. Add celery/celery-beat services to a notifications-enabled compose profile.
2. Flip `NOTIFICATIONS_DISABLED` off for that profile.
3. Script the `abonnement` registration (POST to the NRC, pointed at the BFF's
   `/zgw/notificaties` endpoint from WP-52), idempotent on rerun.
4. Verify a real write (e.g. a status change) triggers a live webhook delivery to the BFF.

## Acceptance criteria

- [x] A notifications-enabled harness profile runs a celery worker and delivers a real
      notification end-to-end to the BFF's webhook.
- [x] Provisioning is a script, re-runnable without erroring on an already-configured target.

## What actually happened

The Decisions block assumed OpenZaak itself could be pointed at, celery-wired, and made to
deliver to a subscribed `abonnement` — checking the running image (`grep -ril abonnement` inside
the `web` container) found nothing: **OpenZaak does not serve the Notificaties API.** It's a
separate application (`openzaak/open-notificaties`, its own image/DB/celery/beat stack).
Standing one up for real `abonnement`/kanaal-filtered routing would mean ~5 new services (a
second Postgres, web, worker, beat, plus the NRC↔AC authorization chain) for a benefit this
harness doesn't need — there is exactly one subscriber (this repo's own BFF), never N. Re-scoped
before writing any code (confirmed with the user): OpenZaak's own `NotificationsConfig` points
straight at the BFF's webhook via a `zgw_consumers.Service` (`auth_type=api_key`) instead — no
NRC, no `abonnement`, same delivery proof (a real write → OpenZaak's celery worker → a real HTTP
POST → the BFF's audit trail). The two "no `abonnement`" acceptance-criteria words above were
edited out for the same reason.

- `docker-compose.openzaak.notificaties.yml` — an opt-in overlay (not `profiles:`, matching
  WP-55's prod-override precedent) adding one celery worker (not celery-beat: `send_notification`
  is a plain async task fired on save, not a scheduled one — beat only matters on a real NRC's
  polling side) and flipping `NOTIFICATIONS_DISABLED` off. The two changes are inseparable:
  `NOTIFICATIONS_GUARANTEE_DELIVERY` defaults true, so the moment that flag is false, every write
  to a notified resource 500s-and-rolls-back unless `NotificationsConfig` already has a client —
  hence `bootstrap-notificaties.sh` configuring it is not a separate step.
- Reaching the BFF from the worker turned out to be the real obstacle, not the Django/celery
  wiring. `extra_hosts: host.docker.internal:host-gateway` (the plan's first choice) resolves
  fine but every TCP connect through it timed out — confirmed live: this environment's rootless
  Podman doesn't route container→host-port traffic that way. Fix: join the overlay's `celery`
  service to the repo root's own `docker compose up` network (`external: true`, by the
  `atomic-design-poc_default` name compose derives from the repo directory) and reach the BFF by
  its container name (`api`) instead — container-to-container, which this exact stack already
  proved reliable (`celery` already talks to `db`/`redis` that way). One more trap on that path:
  `docker compose run --name api ...` does **not** register the `api` DNS alias other containers
  need (only `docker compose up -d api` does) — cost a debugging round-trip before switching to
  `up -d` (via a temporary, uncommitted `docker-compose.override.yml`) for the live verification.
- `bootstrap-notificaties.sh` — `update_or_create` on the `Service`'s fixed slug (idempotent);
  preflights the BFF's webhook with a synthetic notification body first (204 required) so a
  misconfigured target fails before touching OpenZaak, not after (a later write would otherwise
  500-and-rollback with no obvious cause).
- `verify-notificatie.sh` — the runnable end-to-end check. First attempt triggered the write via
  a second `statussen` POST (the "final" status) — 403'd: WP-57's narrowed `zaken.aanmaken` scope
  permits exactly **one** status per zaak ("Met de 'zaken.aanmaken' scope mag je slechts 1 status
  zetten"). Switched the trigger to a zaak `PATCH` (`toelichting`), covered by the already-granted
  `zaken.bijwerken` and trivially repeatable. Second attempt used the _final_ statustype anyway
  for a different reason and got a 400 ("Zaak has no resultaat") — OpenZaak requires a `resultaat`
  before the closing status; the `PATCH` sidesteps that precondition entirely too.
- Verified for real, twice: `bootstrap-catalogus.sh` (idempotent re-run, all "exists") →
  `bootstrap-notificaties.sh` (preflight 204, `Service` configured) → `verify-notificatie.sh`
  (PATCH → polled `/admin/audit` → found the delivered `zgw:notificatie`/`allow` row) → reran
  both WP-58 scripts again under the same running harness (still idempotent, delivered again).
  Also confirmed the negative case directly: `POST /zgw/notificaties` with no `Authorization`
  header, and with a wrong one, both 401 — the shared-secret gate isn't just accepting anything.
  Backend suite stayed green throughout (159/159, `dotnet test --filter Category!=Integration`).
  Test infrastructure (the temporary `docker-compose.override.yml`, the manually-created `api`
  container) was torn down / reconciled back to the pre-session baseline afterward.

## Verification

Bring up the notifications-enabled profile (`backend/openzaak/README.md`'s "Notifications-enabled
profile" section); run `./bootstrap-catalogus.sh && ./bootstrap-notificaties.sh &&
./verify-notificatie.sh`. The last script fails loudly (with celery/worker log diagnostics) if no
delivered notification shows up in the BFF's `/admin/audit` within 60s.

## Out of scope

Cache invalidation on notification receipt (flagged separately in
`openzaak-integration.md` as a `ponytail:` marker, not part of this slice);
celery-flower/monitoring UI. A real Notificaties API (NRC) + `abonnement`/kanaal-filtered
routing (see "What actually happened") — add one if a later WP needs more than this harness's
single subscriber.

## Risks

Celery adds real operational surface (another process to keep alive) — scope this WP to
"works, documented," not a fully monitored deployment. The direct-to-BFF shortcut means this
harness doesn't exercise real `abonnement`/kanaal-filter validation — a production deployment's
NRC-based path (documented in `openzaak-integration.md`) is untested by this harness by
construction.

Depends on: WP-56 (provisioning mechanism this extends).