From 849bf4723baca71d3c5e95afbd3115f5d108ff8c Mon Sep 17 00:00:00 2001 From: Niek Otten Date: Fri, 24 Jul 2026 11:59:27 +0000 Subject: [PATCH] ci: unstick verify-stack on Gitea 1.27 + act_runner 2.0.0 (closes #134) (#135) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## What & why After the Gitea 1.27 + act_runner 2.0.0 upgrade, `verify-stack` never starts: the run sits in `waiting` forever with no logs for that job, while the other five jobs pass — so `main` stays pending/red (P0). See #134. Closes #134 ### Root cause `verify-stack` was the only job gated by a status-function `if` on top of `needs`: ```yaml verify-stack: needs: [mutation] if: ${{ !cancelled() }} ``` Gitea 1.27 reworked cancellation/aggregation so that `always()`/`cancelled()`-gated `needs` jobs route through a new transitional **`Cancelling`** state + server↔runner **capability negotiation** ("Requires Gitea Runner 2.0.0"). On this 1.27 + 2.0.0 pairing that handshake doesn't resolve, so the job is never dispatched and never leaves `waiting`. Plain jobs (no `if`/`needs`) are unaffected — exactly the observed pattern. It worked pre-upgrade (old runner). ### Fix Drop the `if: ${{ !cancelled() }}`; keep `needs: [mutation]`. Default `if: success()` dispatches normally and still serialises the two memory-heavy jobs (OOM avoidance, #126). **Trade-off:** the `!cancelled()` (added in #127) let verify-stack run even when the mutation ratchet fails. Now a failing mutation skips verify-stack; the fix-and-re-push re-run exercises it, so the signal isn't lost — just deferred to the green-mutation run. If we later want both signals on one run, serialise via a `concurrency` group rather than `needs` + `always()`. Documented as §7 in `docs/runbooks/gitea-actions-gotchas.md`. ## Note on the stuck run Run 582 (the #133 merge) will **not** clear itself and must be force-cancelled from the Actions UI (plain cancel can also stall on this version, gitea#35782). This PR's own run is the first real test of the fix — if `verify-stack` dispatches and runs here, the fix holds. ## Definition of Done - [x] Linked issue (#134). - [x] Conventional Commit referencing the issue. - [ ] CI green — this PR's run is the verification (verify-stack must dispatch). - [x] Runbook updated (gotchas §7). - [ ] Closed by the merging PR. 🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: https://git.labs.respellion.tech/eho/register-referentie/pulls/135 --- .gitea/workflows/ci.yaml | 10 +++++++--- docs/runbooks/gitea-actions-gotchas.md | 25 +++++++++++++++++++++++++ 2 files changed, 32 insertions(+), 3 deletions(-) diff --git a/.gitea/workflows/ci.yaml b/.gitea/workflows/ci.yaml index a08a5b4..ea39123 100644 --- a/.gitea/workflows/ci.yaml +++ b/.gitea/workflows/ci.yaml @@ -144,11 +144,15 @@ jobs: # they never co-schedule now the runner has capacity >1. A concurrent Stryker run + full-stack # bring-up + Playwright browser on one host is what OOMs the e2e (commit d5e5fa2, #126). The # light .NET/frontend jobs have no `needs`, so they still parallelise up to runner capacity. - # `if: !cancelled()` keeps verify-stack running even when the mutation ratchet fails (so we don't - # lose its signal) while still honouring run cancellation from the concurrency group above. + # + # No `if: ${{ !cancelled() }}` here (removed in #134): on Gitea 1.27 + act_runner 2.0.0, a job + # gated by a status-function `if` (always()/cancelled()) on top of `needs` routes through the new + # transitional "Cancelling" state + capability negotiation and never leaves `waiting` — it's never + # dispatched (gitea-actions-gotchas.md §7). Default `if: success()` dispatches normally. Cost: a + # failing mutation ratchet now skips verify-stack instead of running it anyway; the fix-and-re-push + # re-run exercises verify-stack, so we still get the signal. verify-stack: needs: [mutation] - if: ${{ !cancelled() }} runs-on: ubuntu-latest steps: - uses: https://github.com/actions/checkout@v4 diff --git a/docs/runbooks/gitea-actions-gotchas.md b/docs/runbooks/gitea-actions-gotchas.md index 29f97d2..0be41d3 100644 --- a/docs/runbooks/gitea-actions-gotchas.md +++ b/docs/runbooks/gitea-actions-gotchas.md @@ -196,3 +196,28 @@ service name; the notif verify harness also registers the sink callback by IP. abonnement is registered and refuses it (`no-auth-on-callback-url`) unless it returns **401** without the configured `Authorization`. The verify sink (`infra/notification-sink.py`) enforces a bearer token for exactly this reason. + +--- + +## 7. A job with `if: ${{ !cancelled() }}` (or `always()`) + `needs` sticks in "waiting" + +**Symptom** — after upgrading to **Gitea 1.27** + **act_runner 2.0.0**, one job never +starts: the run sits in state `waiting` forever, the job has **no logs** (never +dispatched to a runner), and the other jobs finish normally. `main` stays pending/red. +Seen on the `verify-stack` job (#134). + +**Why** — Gitea 1.27 reworked cancellation/aggregation: a job gated by a +**status-function `if`** (`always()` / `cancelled()` / `!cancelled()`) on top of +`needs` now routes through a new transitional **`Cancelling`** job state plus a +server↔runner **capability negotiation** ("Requires Gitea Runner 2.0.0"). On the +1.27 + 2.0.0 pairing that handshake doesn't resolve for such a job, so it's never +offered to a runner and never leaves `waiting`. Jobs with no `if`/`needs` are +unaffected. (Related upstream: go-gitea/gitea#31074, #27116, #35782.) + +**Fix** — don't gate a `needs` job with a status-function `if`. Use the default +`if: success()` (i.e. omit the `if`). If you need "run even when an upstream job +fails", prefer serialising with a `concurrency` group over `needs` + `always()`. + +**Also** — a run already stuck this way will **not** clear itself; force-cancel it +from the Actions UI (plain cancel can also stall on this version, #35782). Push the +workflow fix to produce a fresh run.