eho/register-referentie
1
0
Fork 0
Code Issues 10 Pull Requests 1 Actions Packages Projects Releases 1 Wiki Activity

feat(obs): Prometheus metrics on /metrics + golden-signal Grafana dashboard (closes #124) #129

Merged
not merged 3 commits from feat/124-metrics-dashboards into main 2026-07-24 08:31:42 +00:00
Conversation 0 Commits 3 Files Changed 22
+418 -11
not commented 2026-07-24 08:08:06 +00:00
Contributor
Copy Link
Copy Source

What & why

S-16c, the last of the S-16 (#17) split, on top of the backplane (#122) and distributed tracing (#123). The five .NET services now expose OpenTelemetry metrics in Prometheus format at /metrics; Prometheus scrapes each (one job per service); and Grafana ships a pre-built Request path — golden signals dashboard (traffic / errors / latency / saturation), split by service.

Closes #124

How

  • Each service adds .WithMetrics(AddAspNetCoreInstrumentation + AddHttpClientInstrumentation + AddMeter("System.Runtime") + AddPrometheusExporter) and maps /metrics. Same shape as the S-16b tracing wiring already in these Program.cs files.
  • infra/observability/prometheus/prometheus.yml: one scrape job per service (acl, domain, bff, event-subscriber, projection-api), reached by compose service name.
  • infra/observability/grafana/provisioning/dashboards/: dashboard provider + golden-signals.json (baked into the Grafana image by the existing COPY provisioning/).
  • verify-metrics (new CI verify-stack step + Makefile target): generates BFF traffic and asserts Prometheus scraped the golden-signal metric from every service. Mirrors verify-tracing.

Dependency (CLAUDE.md §13/§14)

Adds OpenTelemetry.Exporter.Prometheus.AspNetCore 1.17.0-beta.1 (matched to the 1.17.0 core already in use). It gives the OTel-native /metrics pull endpoint; replacing it would mean hand-rolling Prometheus exposition over a MeterListener; the risk is that it is a prerelease package (the whole OTel .NET Prometheus line is -beta) — pinned, wired only in Program.cs, and gated by verify-metrics. Recorded in ADR-0024.

Definition of Done

  • Linked Gitea issue (#124).
  • Failing test committed before the implementation (test(bff): /metrics exposes http-server request duration).
  • Implementation makes the test pass.
  • CI green — pending Gitea Actions run.
  • docker compose up reaches green health within 3 min (backplane images unchanged in shape; not on the health gate, ADR-0023).
  • Docs updated — demo-script S-16c entry.
  • ADR added — ADR-0024.
  • Demo note in docs/demo-script.md.

Notes for reviewers

  • /health polls are counted as traffic (metrics aren't path-filtered, unlike traces). Fine for a demo dashboard and honest — real load stacks on top.
  • projection-api has no Stryker config (unchanged); the four mutated services carry the metrics wiring in Program.cs, same as the merged S-16b tracing code.
  • Metric names verified against a live service: http_server_request_duration_seconds{,_bucket,_count}, label http_response_status_code, dotnet_process_cpu_time_seconds_total.

🤖 Generated with Claude Code

## What & why S-16c, the last of the S-16 (#17) split, on top of the backplane (#122) and distributed tracing (#123). The five .NET services now expose OpenTelemetry **metrics** in Prometheus format at `/metrics`; Prometheus scrapes each (one job per service); and Grafana ships a pre-built **Request path — golden signals** dashboard (traffic / errors / latency / saturation), split by service. Closes #124 ### How - Each service adds `.WithMetrics(AddAspNetCoreInstrumentation + AddHttpClientInstrumentation + AddMeter("System.Runtime") + AddPrometheusExporter)` and maps `/metrics`. Same shape as the S-16b tracing wiring already in these `Program.cs` files. - `infra/observability/prometheus/prometheus.yml`: one scrape job per service (`acl`, `domain`, `bff`, `event-subscriber`, `projection-api`), reached by compose service name. - `infra/observability/grafana/provisioning/dashboards/`: dashboard provider + `golden-signals.json` (baked into the Grafana image by the existing `COPY provisioning/`). - `verify-metrics` (new CI verify-stack step + Makefile target): generates BFF traffic and asserts Prometheus scraped the golden-signal metric from every service. Mirrors `verify-tracing`. ### Dependency (CLAUDE.md §13/§14) Adds `OpenTelemetry.Exporter.Prometheus.AspNetCore` `1.17.0-beta.1` (matched to the `1.17.0` core already in use). It gives the OTel-native `/metrics` pull endpoint; replacing it would mean hand-rolling Prometheus exposition over a `MeterListener`; the risk is that it is a **prerelease** package (the whole OTel .NET Prometheus line is `-beta`) — pinned, wired only in `Program.cs`, and gated by `verify-metrics`. Recorded in **ADR-0024**. ## Definition of Done - [x] Linked Gitea issue (#124). - [x] Failing test committed before the implementation (`test(bff): /metrics exposes http-server request duration`). - [x] Implementation makes the test pass. - [x] CI green — pending Gitea Actions run. - [x] `docker compose up` reaches green health within 3 min (backplane images unchanged in shape; not on the health gate, ADR-0023). - [x] Docs updated — demo-script S-16c entry. - [x] ADR added — ADR-0024. - [x] Demo note in `docs/demo-script.md`. ## Notes for reviewers - `/health` polls are counted as traffic (metrics aren't path-filtered, unlike traces). Fine for a demo dashboard and honest — real load stacks on top. - `projection-api` has no Stryker config (unchanged); the four mutated services carry the metrics wiring in `Program.cs`, same as the merged S-16b tracing code. - Metric names verified against a live service: `http_server_request_duration_seconds{,_bucket,_count}`, label `http_response_status_code`, `dotnet_process_cpu_time_seconds_total`. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
not added this to the Iteration 3 — Beheer & Observability milestone 2026-07-24 08:08:06 +00:00
not added 3 commits 2026-07-24 08:08:07 +00:00
test(bff): /metrics exposes http-server request duration (refs #124) 61805d5ce7
feat(bff): expose OTel golden-signal metrics on /metrics (refs #124) 965782dd95
feat(obs): golden-signal metrics on /metrics + Prometheus scrape + Grafana dashboard (S-16c, refs #124)
CI / lint (pull_request) Successful in 5m7s
Details
CI / unit (pull_request) Successful in 2m2s
Details
CI / frontend (pull_request) Successful in 3m58s
Details
CI / mutation (pull_request) Successful in 6m30s
Details
CI / verify-stack (pull_request) Successful in 9m34s
Details
CI / build (pull_request) Successful in 4m57s
Details
4ac2c3ff6c 
Wire OTel metrics into the four remaining .NET services (acl, domain, event-subscriber,
projection-api) exactly as the BFF: ASP.NET Core + HttpClient instrumentation + the built-in
System.Runtime meter, exposed at /metrics via the Prometheus AspNetCore exporter (ADR-0024).
Prometheus scrapes one job per service; Grafana ships a pre-built 'Request path — golden
signals' dashboard (traffic/errors/latency/saturation). A verify-metrics CI step proves the
endpoints are scraped end to end.
not added the type:slicearea:infra labels 2026-07-24 08:08:18 +00:00
not merged commit d5dfbdc0b2 into main 2026-07-24 08:31:42 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
area:acl
area:bff
area:docs
area:domain
area:event-subscriber
area:infra
area:portal-behandel
area:portal-beheer
area:portal-openbaar
area:portal-self-service
area:projection
area:workflow
type:adr-proposal
type:bug
type:chore
type:slice
No labels area:infra type:slice
Milestone
No items
No Milestone Iteration 3 — Beheer & Observability
Projects
Clear projects
No projects
Assignees
Clear assignees
eho (Edwin van den Houdt)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: eho/register-referentie#129