Files
register-referentie/.gitea/workflows/ci.yaml
T
not 2125fb0cfd
CI / build (push) Successful in 1m14s
CI / lint (push) Successful in 1m28s
CI / unit (push) Successful in 1m28s
CI / frontend (push) Successful in 3m8s
CI / mutation (push) Successful in 6m35s
CI / verify-stack (push) Successful in 9m27s
feat(infra): Objecten publishes register events to NRC (closes #152) (#154)
## What & why

S-19b-1. A write to the Objecten API now produces a **delivered** notification on the
`objecten` kanaal in Open Notificaties. ADR-0028 switched Objecten's notifications off on
purpose — there was no broker, worker, kanaal or abonnement, so wiring only the client side
would have dropped every message on the floor. This slice builds the real path and turns it
back on.

- `objecten-celery` worker (mirrors `oz-celery`) + `CELERY_BROKER_URL`/`RESULT_BACKEND` on
  objecten-redis db 1 (db 0 is already the cache). `notifications_api_common` only *queues*
  the send; without a worker every register write is silently undelivered.
- `nrc` service + `notifications_config` in Objecten's `setup_configuration`, reusing the
  `big-reference-seed` credential OpenZaak publishes with (NRC authorizes it via OpenZaak's
  AC, which grants it `heeft_alle_autorisaties` — no second credential needed).
- The `objecten` kanaal in NRC's `setup_configuration`. The name is fixed by the Objects API
  (`NOTIFICATIONS_KANAAL`), not chosen here; publishing to an unregistered kanaal is exactly
  what the red check reported first.
- `NOTIFICATIONS_DISABLED: "false"` in both compose files.
- Writers address Objecten as `objecten.local` — see *Notes for reviewers*.
- `make verify-objecten-notifications` — registers an abonnement on `objecten` pointing at a
  throwaway sink, writes a `RegisterRecord` exactly as the ACL does on approval, asserts the
  delivery. One assertion covering the whole chain: Objecten -> objecten-celery -> NRC ->
  nrc-beat -> callback. Wired into the CI `verify-stack` job and the summary table.

**ADR-0029** records the decisions; ADR-0028's ceiling now points at it.

Closes #152

## Definition of Done

- [x] Linked Gitea issue (above).
- [x] Failing test committed before the implementation (dc9ca2c, red at the first hop:
      `NRC POST /api/v1/abonnement -> 400 "Kanaal met deze naam bestaat niet."`).
- [x] Implementation makes the test pass (4488962, + two fixes found by CI, below).
- [x] Conventional Commits referencing the issue (`refs #152`).
- [x] CI green — all six jobs on a5fd47e, including `verify-stack` end to end (e2e included).
- [x] `docker compose up` from a fresh clone reaches green health checks within 3 minutes
      (`verify-stack`'s bring-up step).
- [x] Docs updated — ADR-0029 added, ADR-0028's ceiling annotated, BACKLOG.md split.
- [x] ADR added in `docs/architecture/`.
- [x] Demo note in `docs/demo-script.md` if user-visible — n/a, infrastructure only; nothing
      consumes the kanaal until S-19b-2 (#153).

## Notes for reviewers

**The one genuinely non-obvious bit: writers address Objecten as `objecten.local:8000`, not
`objecten:8000`.** NRC types a notification's `hoofdObject`/`resourceUrl` as DRF `URLField`,
so Django's `URLValidator` runs on them — and it rejects a **single-label** host. Objecten
fills both from the object url DRF built with `request.build_absolute_uri`, i.e. *the Host
the caller used*. Writing via the plain service name returns 201 and then fails every
publish in the background, forever, with

```
400 {"hoofdObject":["Voer een geldige URL in."],"resourceUrl":["Voer een geldige URL in."]}
```

So the `objecten` service carries an `objecten.local` network alias and every writer uses it
— `Acl__Objecten__BaseUrl`, `ObjectenGatewayIntegrationTests`, this slice's verify driver.
An alias rather than a bare dotted `SITE_DOMAIN` so the host still *resolves*: a subscriber
following `resourceUrl` reaches the record, which S-19b-2 will do. Readers keep the plain
name. Same class of constraint as ADR-0028's Objecttypen base-URL rule.

**Ceiling, stated in the ADR:** nothing enforces the alias — a future writer using
`objecten:8000` gets a 201 and silently no notification. If a second writer ever appears,
rename the compose service rather than adding a lint.

**Two CI-only failures on the way here**, both worth knowing:
1. `SITE_DOMAIN` was my first guess at the mechanism and is simply not what builds those
   URLs — dropped in d76abf2.
2. The check correlated the delivery on the `reference` inside the record it wrote. An NRC
   notification carries `kanaal`/`resource`/`kenmerken`/`hoofdObject`/`resourceUrl` and
   **never the record data**, so it correlates on the object URL now (a5fd47e).

**Cost:** one more long-running container on the memory-tight runner. It inherits the capped
`UWSGI_PROCESSES: "1"` env, which the celery command ignores; if `verify-stack` gets tight
again, celery concurrency is the next knob.

**Follow-up:** S-19b-2 (#153) sources the projection from these events. Nothing subscribes to
the `objecten` kanaal in the product yet — only the verify check does.Reviewed-on: #154
2026-08-28 10:11:54 +00:00

297 lines
14 KiB
YAML

name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
permissions:
contents: read
# Supersede stale runs: a new push to the same branch/PR cancels the previous run, so the runner's
# concurrency slots aren't spent on commits nobody is waiting for (refs #127).
concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
# Self-hosted runner — see docs/runbooks/ci.md for the runner setup.
# `uses:` are absolute, tag-pinned URLs (CLAUDE.md §8.7 / §15).
# Each job calls a `make` target — the same one developers run locally
# (`make ci`). The Makefile is the single source of truth; see docs/runbooks/ci.md.
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: https://github.com/actions/checkout@v4
- uses: https://github.com/actions/setup-dotnet@v4
with:
dotnet-version: '10.0.x'
# Cache the NuGet package store so each .NET job restores from disk, not the network. There are
# no lock files (so setup-dotnet's built-in cache doesn't apply); key on the project files. @v3
# avoids the GHES guard that breaks @v4 on Gitea (gitea-actions-gotchas.md); cache is best-effort
# — a miss just restores from the network. See issue #73.
- uses: https://github.com/actions/cache@v3
with:
path: ~/.nuget/packages
key: nuget-${{ runner.os }}-${{ hashFiles('**/*.csproj') }}
restore-keys: |
nuget-${{ runner.os }}-
- run: make lint
build:
runs-on: ubuntu-latest
steps:
- uses: https://github.com/actions/checkout@v4
- uses: https://github.com/actions/setup-dotnet@v4
with:
dotnet-version: '10.0.x'
- uses: https://github.com/actions/cache@v3
with:
path: ~/.nuget/packages
key: nuget-${{ runner.os }}-${{ hashFiles('**/*.csproj') }}
restore-keys: |
nuget-${{ runner.os }}-
- run: make build
unit:
runs-on: ubuntu-latest
steps:
- uses: https://github.com/actions/checkout@v4
- uses: https://github.com/actions/setup-dotnet@v4
with:
dotnet-version: '10.0.x'
- uses: https://github.com/actions/cache@v3
with:
path: ~/.nuget/packages
key: nuget-${{ runner.os }}-${{ hashFiles('**/*.csproj') }}
restore-keys: |
nuget-${{ runner.os }}-
- run: make unit
# Job summary (#136): a per-service pass/fail table from the TRX `make unit` wrote.
- name: Unit test summary
if: always()
run: |
[ -n "${GITHUB_STEP_SUMMARY:-}" ] || exit 0
python3 infra/trx-summary.py TestResults >> "$GITHUB_STEP_SUMMARY"
# Frontend (Nx/Angular) lane: install with pnpm, then Nx lint + test + build.
frontend:
runs-on: ubuntu-latest
steps:
- uses: https://github.com/actions/checkout@v4
- uses: https://github.com/pnpm/action-setup@v4
with:
version: 11
- uses: https://github.com/actions/setup-node@v4
with:
node-version: '24'
cache: 'pnpm'
- run: make frontend
# Job summary (#136): a per-frontend (app) pass/fail table from the vitest JSON each app wrote.
- name: Frontend test summary
if: always()
run: |
[ -n "${GITHUB_STEP_SUMMARY:-}" ] || exit 0
python3 infra/vitest-summary.py test-output >> "$GITHUB_STEP_SUMMARY"
mutation:
runs-on: ubuntu-latest
steps:
- uses: https://github.com/actions/checkout@v4
- uses: https://github.com/actions/setup-dotnet@v4
with:
dotnet-version: '10.0.x'
- uses: https://github.com/actions/cache@v3
with:
path: ~/.nuget/packages
key: nuget-${{ runner.os }}-${{ hashFiles('**/*.csproj') }}
restore-keys: |
nuget-${{ runner.os }}-
- run: make mutation
# Job summary (#136): render each service's Stryker Markdown report on the run page (Gitea
# 1.27 $GITHUB_STEP_SUMMARY). `if: always()` so a ratchet break still reports — and because
# `make mutation` stops at the first break, the summary also shows exactly where it stopped.
# Guarded so it no-ops on a runner/server without summary support. Strips the report's UTF-8 BOM.
- name: Mutation score summary
if: always()
run: |
[ -n "${GITHUB_STEP_SUMMARY:-}" ] || exit 0
{
echo "## 🧬 Mutation testing"
echo
for svc in acl event-subscriber domain bff; do
echo "### $svc"
echo
report=$(ls services/"$svc"/StrykerOutput/*/reports/mutation-report.md 2>/dev/null | sort | tail -1)
if [ -n "$report" ]; then
sed '1s/^\xef\xbb\xbf//' "$report"
else
echo "_No report — \`make mutation\` stopped before \`$svc\` (earlier ratchet break)._"
fi
echo
done
} >> "$GITHUB_STEP_SUMMARY"
# Publish the Stryker HTML reports. `if: always()` uploads them even when the
# ratchet fails — that is exactly when you want to inspect the survivors.
# `continue-on-error` keeps the upload best-effort: the mutation *gate* is the
# ratchet (make mutation's exit code), not the report, so a Gitea artifact-backend
# 500 must not fail the job (gitea-actions-gotchas.md §4). Glob handles Stryker's
# non-deterministic StrykerOutput/<timestamp>/ dir. Pinned @v3: @v4's bundled
# @actions/artifact hard-aborts on non-github.com (GHES guard) — see the runbook.
- uses: https://github.com/actions/upload-artifact@v3
if: always()
continue-on-error: true
with:
name: acl-mutation-report
path: services/acl/StrykerOutput/**/reports/mutation-report.html
if-no-files-found: warn
- uses: https://github.com/actions/upload-artifact@v3
if: always()
continue-on-error: true
with:
name: event-subscriber-mutation-report
path: services/event-subscriber/StrykerOutput/**/reports/mutation-report.html
if-no-files-found: warn
- uses: https://github.com/actions/upload-artifact@v3
if: always()
continue-on-error: true
with:
name: domain-mutation-report
path: services/domain/StrykerOutput/**/reports/mutation-report.html
if-no-files-found: warn
- uses: https://github.com/actions/upload-artifact@v3
if: always()
continue-on-error: true
with:
name: bff-mutation-report
path: services/bff/StrykerOutput/**/reports/mutation-report.html
if-no-files-found: warn
# One stage for every check that needs the live stack. Booting OpenZaak once (instead
# of once per job) is the cheapest layout (issue #58). No setup-dotnet: the ACL test runs
# in a built image and everything reaches services by container IP. Needs Docker + egress
# (base images, nuget, selectielijst.openzaak.nl).
#
# `needs: [mutation]` is NOT a data dependency — it serialises the two memory-heavy jobs so
# they never co-schedule now the runner has capacity >1. A concurrent Stryker run + full-stack
# bring-up + Playwright browser on one host is what OOMs the e2e (commit d5e5fa2, #126). The
# light .NET/frontend jobs have no `needs`, so they still parallelise up to runner capacity.
#
# No `if: ${{ !cancelled() }}` here (removed in #134): on Gitea 1.27 + act_runner 2.0.0, a job
# gated by a status-function `if` (always()/cancelled()) on top of `needs` routes through the new
# transitional "Cancelling" state + capability negotiation and never leaves `waiting` — it's never
# dispatched (gitea-actions-gotchas.md §7). Default `if: success()` dispatches normally. Cost: a
# failing mutation ratchet now skips verify-stack instead of running it anyway; the fix-and-re-push
# re-run exercises verify-stack, so we still get the signal.
verify-stack:
needs: [mutation]
runs-on: ubuntu-latest
steps:
- uses: https://github.com/actions/checkout@v4
# Bring the full stack up + wait for health — this also is the DoD "compose up
# reaches green health" smoke (it replaces the old compose-smoke job).
# Each check carries an `id` so the summary step below can report its per-check outcome (#136).
# A failed check skips the rest (no step `if:`), so the table shows exactly where it stopped.
- name: Bring up the full stack & wait for health
id: up
run: make verify-up
- name: Observability backplane (Grafana + Tempo + Prometheus datasources)
id: obs
run: OBS_TIMEOUT=180 make verify-observability
- name: Objecttypen API up + token authenticates
id: objecttypen
run: OBJECTTYPEN_TIMEOUT=120 make verify-objecttypen
- name: Objecten API up + token authenticates + trusts Objecttypen
id: objecten
run: OBJECTEN_TIMEOUT=120 make verify-objecten
- name: RegisterRecord objecttype registered + published
id: registerrecord
run: REGISTERRECORD_TIMEOUT=120 make verify-registerrecord
- name: ACL ↔ OpenZaak integration tests
id: acl
run: make verify-acl
- name: OpenZaak → NRC notification delivery
id: nrc
run: make verify-nrc
- name: OpenZaak → NRC → Event Subscriber → projection-api
id: projection
run: make verify-projection
- name: Objecten → NRC notification delivery
id: objecten_nrc
run: make verify-objecten-notifications
- name: Domain → Flowable → ACL → OpenZaak
id: domain
run: make verify-domain
- name: BFF → Keycloak + domain + projection
id: bff
run: make verify-bff
- name: Distributed traces reach Tempo (one connected trace across services)
id: tracing
run: TRACING_TIMEOUT=120 make verify-tracing
- name: Golden-signal metrics scraped by Prometheus (/metrics on every service)
id: metrics
run: METRICS_TIMEOUT=120 make verify-metrics
- name: Self-service e2e (Playwright, login → submit → success)
id: e2e
run: make verify-e2e
# Job summary (#136): a pass/fail table of every live-stack check, so a red verify-stack shows
# which check failed at a glance. `if: always()` (step-level — safe on runner 2.0.0, unlike the
# job-level status-function `if` of #134) so it renders even after a check fails.
- name: verify-stack check summary
if: always()
env:
UP: ${{ steps.up.outcome }}
OBS: ${{ steps.obs.outcome }}
OBJECTTYPEN: ${{ steps.objecttypen.outcome }}
OBJECTEN: ${{ steps.objecten.outcome }}
REGISTERRECORD: ${{ steps.registerrecord.outcome }}
OBJECTEN_NOTIFICATIONS: ${{ steps.objecten_nrc.outcome }}
ACL: ${{ steps.acl.outcome }}
NRC: ${{ steps.nrc.outcome }}
PROJECTION: ${{ steps.projection.outcome }}
DOMAIN: ${{ steps.domain.outcome }}
BFF: ${{ steps.bff.outcome }}
TRACING: ${{ steps.tracing.outcome }}
METRICS: ${{ steps.metrics.outcome }}
E2E: ${{ steps.e2e.outcome }}
run: |
[ -n "${GITHUB_STEP_SUMMARY:-}" ] || exit 0
icon() { case "$1" in success) echo "✅";; failure) echo "❌";; skipped) echo "⏭️";; cancelled) echo "🚫";; *) echo "❔ ${1:-—}";; esac; }
{
echo "## 🔌 verify-stack checks"
echo
echo "| Check | Result |"
echo "| ----- | :----: |"
echo "| Bring up + health | $(icon "$UP") |"
echo "| Observability backplane | $(icon "$OBS") |"
echo "| Objecttypen API + token | $(icon "$OBJECTTYPEN") |"
echo "| Objecten API + token | $(icon "$OBJECTEN") |"
echo "| RegisterRecord objecttype | $(icon "$REGISTERRECORD") |"
echo "| Objecten → NRC | $(icon "$OBJECTEN_NOTIFICATIONS") |"
echo "| ACL ↔ OpenZaak | $(icon "$ACL") |"
echo "| OpenZaak → NRC | $(icon "$NRC") |"
echo "| NRC → Event Subscriber → projection | $(icon "$PROJECTION") |"
echo "| Domain → Flowable → ACL → OpenZaak | $(icon "$DOMAIN") |"
echo "| BFF → Keycloak + domain + projection | $(icon "$BFF") |"
echo "| Distributed traces (Tempo) | $(icon "$TRACING") |"
echo "| Golden-signal metrics (Prometheus) | $(icon "$METRICS") |"
echo "| Self-service e2e (Playwright) | $(icon "$E2E") |"
} >> "$GITHUB_STEP_SUMMARY"
# Job summary (#136): per-spec Playwright results, from the JSON report run-e2e-check.sh copied
# out of the e2e container. Turns a red e2e into a one-glance "which spec" instead of a log dive.
- name: e2e spec summary
if: always()
run: |
[ -n "${GITHUB_STEP_SUMMARY:-}" ] || exit 0
python3 infra/playwright-summary.py tests/e2e/playwright-report.json >> "$GITHUB_STEP_SUMMARY"
# Log dump must precede teardown (which removes the containers).
- name: Dump container logs on failure
if: failure()
run: docker compose -f infra/docker-compose.yml logs --no-color --tail=100 oz-init openzaak nrc-init nrc-web nrc-celery nrc-beat flowable-db flowable-rest flowable-init keycloak acl bff domain projection-db event-subscriber projection-api self-service openbaar behandel beheer objecttypen-db objecttypen-redis objecttypen-init objecttypen objecten-db objecten-redis objecten-init objecten objecten-celery registerrecord-init tempo prometheus grafana 2>&1 || true
- name: Tear down
if: always()
run: make down