Merge pull request 'Fix #18: migratie-ledger-sync + deploy-proof Azure SSO' (#19) from fix/issue-18-migration-ledger-sso into main

Reviewed-on: #19

AI_AGENT.md

@@ -137,6 +137,9 @@ The platform ships a global chatbot avatar called **R42**, rendered as the Respe
 *   **PocketBase auto-cancellation is OFF.** Set in `src/lib/pb.js`; never re-enable.
 *   **Go through `callLLM`.** Never call the Anthropic proxy directly; you lose retry, schema validation, and telemetry.
 *   **AI token budget.** Truncation surfaces as `LLMTruncatedError` (`stop_reason: max_tokens`). For extraction, tighten the prompt's topic cap before raising `max_tokens`.
+*   **PocketBase migrations & the `_migrations` ledger (issue #18).** The deployed Labs/prod databases were originally provisioned WITHOUT `--migrationsDir` (schema came from `scripts/setup-pb-collections.mjs`), so their ledger started empty. `pb_migrations/1000000000_baseline_ledger_sync.js` marks the historical files as applied on such databases — never remove it, never rename an applied migration file (the ledger is filename-based), and give new migrations a timestamp that sorts after the existing ones. A failed migration aborts `pocketbase serve` → container crash-loop → 502 on every `/api/*` call; the deploy playbooks gate on `/api/health` to catch this.
+*   **OIDC provider config lives in a hook, not a migration.** `pb_hooks/entra_oidc.pb.js` reconciles the Entra provider from `ENTRA_*` env on every bootstrap + cron tick. Don't move provider credentials back into a one-shot migration — an apply while the env is empty would permanently disable OAuth2.
+*   **JSVM hook scope.** PocketBase runs each hook callback as an isolated program: top-level functions in a `*.pb.js` file are NOT in scope inside callbacks. Shared helpers live in `pb_hooks/utils.js` and are pulled in per-callback via `require(`${__hooks}/utils.js`)`.
 
 ## 13. 26-Week Per-User Curriculum System
 The platform uses a **26-week perpetual curriculum cycle**. Every employee covers the knowledge base in focused, thematic weekly blocks, **starting whenever they enroll**.

Caddyfile

@@ -46,6 +46,13 @@
     }
 
     handle_errors {
+        # Don't mask API errors with the SPA shell — return a proper
+        # JSON error so the frontend can display a meaningful message.
+        @api path /api/*
+        respond @api `{"code":{err.status_code},"message":"Backend unavailable"}` {err.status_code} {
+            Content-Type application/json
+        }
+
         rewrite * /index.html
         file_server
     }

docker-compose.yml

@@ -12,7 +12,13 @@ services:
     container_name: pocketbase-learning
     restart: unless-stopped
     working_dir: /pb
+    env_file:
+      # Optional: absent .env.local must not block `docker compose up` (issue #18).
+      - path: .env.local
+        required: false
     command: ["serve", "--http=0.0.0.0:8090", "--dir=/pb/pb_data", "--migrationsDir=/pb/pb_migrations"]
+    ports:
+      - "8090:8090"
     volumes:
       - pb_data:/pb/pb_data
       - ./pb_migrations:/pb/pb_migrations

docs/auth-spec.md

@@ -1,6 +1,7 @@
 # Authentication — Azure (Entra ID) login
 
 > Implemented for issue #16. Replaces the previous client-side PIN login.
+> Hardened for issue #18 (migratie-ledger-sync + env-onafhankelijke provider-reconciliatie).
 
 ## Doel
 
@@ -40,13 +41,19 @@ Browser (SPA)                PocketBase (auth)                 Entra ID
 | 003 | Provisioning + admin-rol via `pb_hooks/team_members.pb.js` | Rol-logica server-side, niet in de client |
 | 004 | Admin-rol via e-mail-allowlist (`ENTRA_ADMIN_EMAILS`), re-synced elke login | Geen handmatig beheer; allowlist-wijziging werkt bij volgende login |
 | 005 | API-rules → `@request.auth.id != ""` op alle app-collecties | Geen anonieme toegang meer; admins/users zijn beide geauthenticeerd |
+| 006 | Baseline-ledger-sync migratie voor out-of-band geprovisionede DB's | Labs/prod draaide historisch zonder `--migrationsDir`; replay van de historie crashte PB (issue #18). De vroegst-sorterende migratie markeert de historie als applied wanneer schema bestaat maar de ledger leeg is |
+| 007 | Migratie = structuur, hook = configuratie | De OIDC-provider wordt bij elke start (en per cron-minuut) gereconcilieerd vanuit `ENTRA_*` env door `pb_hooks/entra_oidc.pb.js`; een one-shot migratie die van deploy-time env afhangt kan OAuth2 anders permanent uitschakelen |
+| 008 | Hook-helpers via `require(`${__hooks}/utils.js`)` | De JSVM draait elke hook-callback als geïsoleerd programma; top-level functies zijn níet in scope op call-time |
 
 ## Bestanden
 
 | Bestand | Rol |
 |---|---|
-| `pb_migrations/1781000000_team_members_to_auth.js` | Dropt de oude PIN-collection, maakt `team_members` als auth-collection met de OIDC-provider (creds uit env). |
+| `pb_migrations/1000000000_baseline_ledger_sync.js` | Detecteert een out-of-band geprovisionede DB (schema bestaat, `_migrations`-ledger leeg) en markeert de historische migraties als applied, zodat de replay niet crasht (issue #18). |
+| `pb_migrations/1781000000_team_members_to_auth.js` | Converteert relation-velden naar text (data blijft behouden), dropt de oude PIN-collection en maakt `team_members` als auth-collection. Idempotent; provider wordt alleen als fast-path meegenomen als de env al aanwezig is. |
 | `pb_migrations/1781000001_tighten_api_rules.js` | Zet alle niet-systeem-collecties op `@request.auth.id != ""`. |
+| `pb_hooks/entra_oidc.pb.js` | Reconcilieert de OIDC-provider vanuit `ENTRA_*` env bij elke start + cron-tick (compare-before-save). |
+| `pb_hooks/utils.js` | Gedeelde helpers (`resolveRole`, `reconcileEntraOidc`) — per callback binnenhalen via `require()`. |
 | `pb_hooks/team_members.pb.js` | Auto-provisioning (`role`, `enrollment_status`, naam-fallback) + admin-rol re-sync. |
 | `src/lib/azureAuth.js` | Canonieke, unit-geteste helpers (`OIDC_PROVIDER`, `resolveRole`, `parseAdminEmails`, `deriveName`). |
 | `src/store/AppContext.jsx` | `loginWithAzure()` / `logout()` op basis van `pb.authStore` + `authRefresh()`. |
@@ -100,6 +107,11 @@ Ansible-variabelen (vault): `entra_tenant_id`, `entra_client_id`,
   eerste deploy.
 - Verweesde voortgangsrecords van verwijderde (pre-Azure) users zijn acceptabel:
   zonder geldig Entra-account kan niemand inloggen, dus die records zijn onbereikbaar.
-- Credential-rotatie: update env + herconfigureer de provider via de PocketBase
+- Credential-rotatie: update de `ENTRA_*` secrets (Gitea → Ansible `.env`) en
-  admin-UI (Settings → Auth providers) of ship een follow-up migratie — de
+  herstart/redeploy — `pb_hooks/entra_oidc.pb.js` reconcilieert de provider
-  create-migratie draait maar één keer.
+  automatisch bij elke start en elke cron-minuut. Geen handmatige re-apply meer.
+- **Migratiebestanden nooit hernoemen nadat ze ergens applied zijn** — de
+  `_migrations`-ledger is filename-based; hernoemen triggert een re-apply.
+- De deploy-playbooks bevatten een health-gate: als PocketBase na de deploy niet
+  healthy wordt, faalt de pipeline zichtbaar en print hij de containerlogs
+  (issue #18 bleef 2 weken onzichtbaar doordat de deploy "groen" was).

infra/development/site/deploy-playbook.yml

@@ -72,3 +72,31 @@
         state: present
         files: compose.yml
         recreate: always
+
+    # A failed migration aborts `pocketbase serve` and the container crash-loops
+    # while `docker compose up` still reports success (issue #18). Gate the
+    # deploy on PocketBase actually serving its health endpoint.
+    - name: Wait for PocketBase to become healthy
+      ansible.builtin.command: docker exec pocketbase-learning wget -q -O - http://127.0.0.1:8090/api/health
+      register: pb_health
+      until: pb_health.rc == 0
+      retries: 18
+      delay: 5
+      changed_when: false
+      ignore_errors: yes
+
+    - name: Collect PocketBase logs for diagnosis
+      ansible.builtin.command: docker logs --tail 80 pocketbase-learning
+      register: pb_logs
+      when: pb_health is failed
+      changed_when: false
+      failed_when: false
+
+    - name: Abort deploy — PocketBase is not healthy
+      ansible.builtin.fail:
+        msg: |
+          PocketBase did not become healthy after the deploy (health endpoint unreachable).
+          Recent container logs:
+          {{ pb_logs.stdout | default('') }}
+          {{ pb_logs.stderr | default('') }}
+      when: pb_health is failed

infra/production/site/deploy-playbook.yml

@@ -72,3 +72,31 @@
         state: present
         files: compose.yml
         recreate: always
+
+    # A failed migration aborts `pocketbase serve` and the container crash-loops
+    # while `docker compose up` still reports success (issue #18). Gate the
+    # deploy on PocketBase actually serving its health endpoint.
+    - name: Wait for PocketBase to become healthy
+      ansible.builtin.command: docker exec pocketbase-learning wget -q -O - http://127.0.0.1:8090/api/health
+      register: pb_health
+      until: pb_health.rc == 0
+      retries: 18
+      delay: 5
+      changed_when: false
+      ignore_errors: yes
+
+    - name: Collect PocketBase logs for diagnosis
+      ansible.builtin.command: docker logs --tail 80 pocketbase-learning
+      register: pb_logs
+      when: pb_health is failed
+      changed_when: false
+      failed_when: false
+
+    - name: Abort deploy — PocketBase is not healthy
+      ansible.builtin.fail:
+        msg: |
+          PocketBase did not become healthy after the deploy (health endpoint unreachable).
+          Recent container logs:
+          {{ pb_logs.stdout | default('') }}
+          {{ pb_logs.stderr | default('') }}
+      when: pb_health is failed

pb_hooks/entra_oidc.pb.js

@@ -0,0 +1,34 @@
+/// <reference path="../pb_data/types.d.ts" />
+//
+// Issue #18 — Reconcile the Entra (Azure) OIDC provider from the environment.
+//
+// The team_members→auth migration (pb_migrations/1781000000) is structure-only
+// and runs exactly once. Provider CONFIGURATION lives here instead, so that:
+//
+//   * an environment whose migration applied while the ENTRA_* secrets were
+//     absent gets its provider enabled on the next startup (no re-apply),
+//   * rotating ENTRA_CLIENT_SECRET / changing the tenant only requires new env
+//     values and a container restart,
+//   * a fresh database gets its provider on the first cron tick right after
+//     the migrations created the collection (onBootstrap fires BEFORE the
+//     migrations run — there is no post-migration lifecycle hook in the JSVM,
+//     hence the cron fallback).
+//
+// The reconciler compares before saving, so both hooks are cheap no-ops when
+// everything is already in sync.
+
+onBootstrap((e) => {
+  e.next();
+  const { reconcileEntraOidc } = require(`${__hooks}/utils.js`);
+  // reconcileEntraOidc logs a warn-once itself when the ENTRA_* env is absent.
+  console.log("entra_oidc reconcile (bootstrap): " + reconcileEntraOidc(e.app));
+});
+
+cronAdd("entraOidcReconcile", "* * * * *", () => {
+  const { reconcileEntraOidc } = require(`${__hooks}/utils.js`);
+  const result = reconcileEntraOidc($app);
+  // Only log state changes — this tick runs every minute.
+  if (result === "updated") {
+    console.log("entra_oidc reconcile (cron): OIDC provider (re)configured from ENTRA_* env");
+  }
+});

pb_hooks/team_members.pb.js

@@ -15,17 +15,16 @@
 // The allow-list is a comma-separated list of e-mail addresses, e.g.:
 //   ENTRA_ADMIN_EMAILS=rve@respellion.nl,admin@respellion.nl
 //
-// Keep this logic in sync with src/lib/azureAuth.js (resolveRole / parseAdminEmails),
+// resolveRole itself lives in pb_hooks/utils.js and is pulled in per-callback
-// which carries the canonical, unit-tested version for the frontend/tests.
+// via require() — PocketBase's JSVM runs each hook callback below as its own
-
+// isolated program, so a shared top-level function here would not be in scope
-function resolveRole(email) {
+// at call time. See pb_hooks/utils.js for details. Keep the logic in sync with
-  const raw = ($os.getenv("ENTRA_ADMIN_EMAILS") || "").toLowerCase();
+// src/lib/azureAuth.js (resolveRole / parseAdminEmails), which carries the
-  const allow = raw.split(",").map(function (s) { return s.trim(); }).filter(Boolean);
+// canonical, unit-tested version for the frontend/tests.
-  return allow.indexOf((email || "").toLowerCase()) !== -1 ? "admin" : "user";
-}
 
 // On creation (first login): set defaults before the record is persisted.
 onRecordCreate(function (e) {
+  const { resolveRole } = require(`${__hooks}/utils.js`);
   const email = e.record.get("email") || "";
 
   e.record.set("role", resolveRole(email));
@@ -45,6 +44,7 @@ onRecordCreate(function (e) {
 // On every successful auth (incl. OIDC): keep the admin role in sync with the
 // allow-list, so promoting/demoting an admin only requires an env change.
 onRecordAuthRequest(function (e) {
+  const { resolveRole } = require(`${__hooks}/utils.js`);
   const email = e.record.get("email") || "";
   const want = resolveRole(email);
   if (e.record.get("role") !== want) {

pb_hooks/utils.js

@@ -0,0 +1,104 @@
+/// <reference path="../pb_data/types.d.ts" />
+//
+// Shared helpers for pb_hooks/*.pb.js callbacks.
+//
+// PocketBase's JSVM serializes and runs each registered hook callback as its
+// own isolated program, so a plain top-level function declared in a *.pb.js
+// file is NOT in scope inside a callback at call time. Reusable helpers must
+// instead live in a plain (non *.pb.js, so it isn't auto-loaded as a hook)
+// module and be pulled in per-callback via require(`${__hooks}/utils.js`).
+// See: https://github.com/pocketbase/pocketbase/discussions/3599
+//
+// Loaded modules use a shared registry across callback invocations, so keep
+// this file stateless. (Deliberate exception: the warn-once latch below, which
+// exists precisely BECAUSE the registry is shared — it dedupes the env-missing
+// warning across the bootstrap hook and the per-minute cron tick.)
+//
+// Keep resolveRole in sync with src/lib/azureAuth.js's resolveRole (the
+// canonical, unit-tested version used by the frontend).
+
+let warnedEnvMissing = false;
+
+module.exports = {
+  /**
+   * Resolve a user's role from their e-mail and the ENTRA_ADMIN_EMAILS allow-list.
+   * @param {string} email
+   * @returns {"admin"|"user"}
+   */
+  resolveRole: function (email) {
+    const raw = ($os.getenv("ENTRA_ADMIN_EMAILS") || "").toLowerCase();
+    const allow = raw.split(",").map(function (s) { return s.trim(); }).filter(Boolean);
+    return allow.indexOf(String(email || "").toLowerCase()) !== -1 ? "admin" : "user";
+  },
+
+  /**
+   * Reconcile the Entra (Azure) OIDC provider on the team_members auth
+   * collection from the ENTRA_* environment variables (issue #18).
+   *
+   * Idempotent: compares the desired provider config against the stored one
+   * and only saves when something actually changed, so it is safe to call on
+   * every bootstrap and cron tick. Keeping this OUT of the one-shot migration
+   * means late or rotated secrets are picked up on the next start/tick without
+   * any manual re-apply.
+   *
+   * @param {core.App} app
+   * @returns {"collection-missing"|"not-auth-yet"|"env-missing"|"in-sync"|"updated"}
+   */
+  reconcileEntraOidc: function (app) {
+    let col;
+    try {
+      col = app.findCollectionByNameOrId("team_members");
+    } catch (_) {
+      return "collection-missing"; // migration has not created it yet
+    }
+    if (col.type !== "auth") {
+      return "not-auth-yet"; // pre-conversion PIN-era collection
+    }
+
+    const tenant       = $os.getenv("ENTRA_TENANT_ID") || "common";
+    const clientId     = $os.getenv("ENTRA_CLIENT_ID") || "";
+    const clientSecret = $os.getenv("ENTRA_CLIENT_SECRET") || "";
+    if (!clientId || !clientSecret) {
+      if (!warnedEnvMissing) {
+        warnedEnvMissing = true;
+        console.log(
+          "WARN entra_oidc: ENTRA_CLIENT_ID / ENTRA_CLIENT_SECRET not set — Microsoft login stays " +
+          "disabled until the env vars are provided (they are reconciled automatically on startup)."
+        );
+      }
+      return "env-missing";
+    }
+    warnedEnvMissing = false; // env restored — re-arm the warning for future rotations
+
+    const desired = {
+      name:         "oidc",
+      clientId:     clientId,
+      clientSecret: clientSecret,
+      authURL:      "https://login.microsoftonline.com/" + tenant + "/oauth2/v2.0/authorize",
+      tokenURL:     "https://login.microsoftonline.com/" + tenant + "/oauth2/v2.0/token",
+      userInfoURL:  "https://graph.microsoft.com/oidc/userinfo",
+      displayName:  "Microsoft",
+      pkce:         true,
+    };
+
+    const providers = (col.oauth2 && col.oauth2.providers) || [];
+    const current = providers.length === 1 ? providers[0] : null;
+    const inSync = !!current &&
+      col.oauth2.enabled === true &&
+      current.name         === desired.name &&
+      current.clientId     === desired.clientId &&
+      current.clientSecret === desired.clientSecret &&
+      current.authURL      === desired.authURL &&
+      current.tokenURL     === desired.tokenURL &&
+      current.userInfoURL  === desired.userInfoURL &&
+      current.displayName  === desired.displayName;
+    if (inSync) {
+      return "in-sync";
+    }
+
+    col.oauth2.enabled = true;
+    col.oauth2.providers = [desired];
+    app.save(col);
+    return "updated";
+  },
+};

pb_migrations/1000000000_baseline_ledger_sync.js

@@ -0,0 +1,138 @@
+/// <reference path="../pb_data/types.d.ts" />
+//
+// Issue #18 — Baseline ledger sync for out-of-band provisioned databases.
+//
+// The Labs/production PocketBase originally ran WITHOUT --migrationsDir: its
+// schema was created by scripts/setup-pb-collections.mjs / the admin UI, so its
+// _migrations ledger is empty. When the migrations dir was mounted for the
+// first time (Azure SSO, PR #17), PocketBase tried to replay the entire
+// migration history against that existing schema and crash-looped on the very
+// first file ("Collection name must be unique").
+//
+// This migration sorts before every other file. When it detects that the
+// legacy schema already exists but the ledger has never tracked it, it marks
+// the historical migrations below as applied so the replay is skipped and the
+// chain continues with the genuinely new migrations (team_members_to_auth,
+// tighten_api_rules). On fresh databases and on databases that already have a
+// populated ledger it is a no-op.
+//
+// NOTE for future agents: never rename a migration file after it has been
+// applied anywhere — the ledger is filename-based. New migrations must sort
+// after 1781100001 and be appended to environments through a normal deploy.
+const HISTORICAL_MIGRATIONS = [
+  "1778948471_created_content.js",
+  "1778948471_created_quiz_banks.js",
+  "1778948471_created_relations.js",
+  "1778948471_created_sources.js",
+  "1778948471_created_team_members.js",
+  "1778948471_created_topics.js",
+  "1778948472_created_leaderboard.js",
+  "1778948472_created_learn_progress.js",
+  "1778948472_created_quiz_cache.js",
+  "1778948472_created_quiz_results.js",
+  "1778948472_created_settings.js",
+  "1778954289_created_test_col2.js",
+  "1778954310_deleted_relations.js",
+  "1778954310_deleted_topics.js",
+  "1778954310_deleted_users.js",
+  "1778954311_deleted_content.js",
+  "1778954311_deleted_leaderboard.js",
+  "1778954311_deleted_learn_progress.js",
+  "1778954311_deleted_quiz_banks.js",
+  "1778954311_deleted_quiz_cache.js",
+  "1778954311_deleted_quiz_results.js",
+  "1778954311_deleted_settings.js",
+  "1778954311_deleted_sources.js",
+  "1778954311_deleted_team_members.js",
+  "1778954311_deleted_test_col2.js",
+  "1778954317_created_content.js",
+  "1778954317_created_leaderboard.js",
+  "1778954317_created_learn_progress.js",
+  "1778954317_created_quiz_banks.js",
+  "1778954317_created_quiz_cache.js",
+  "1778954317_created_quiz_results.js",
+  "1778954317_created_relations.js",
+  "1778954317_created_settings.js",
+  "1778954317_created_sources.js",
+  "1778954317_created_team_members.js",
+  "1778954317_created_topics.js",
+  "1779005271_created_test_col3.js",
+  "1779005289_created_test_col4.js",
+  "1779005309_deleted_content.js",
+  "1779005309_deleted_learn_progress.js",
+  "1779005309_deleted_quiz_banks.js",
+  "1779005309_deleted_quiz_cache.js",
+  "1779005309_deleted_quiz_results.js",
+  "1779005309_deleted_relations.js",
+  "1779005309_deleted_sources.js",
+  "1779005309_deleted_team_members.js",
+  "1779005309_deleted_topics.js",
+  "1779005310_deleted_leaderboard.js",
+  "1779005310_deleted_settings.js",
+  "1779005310_deleted_test_col3.js",
+  "1779005310_deleted_test_col4.js",
+  "1779005316_created_content.js",
+  "1779005316_created_quiz_banks.js",
+  "1779005316_created_relations.js",
+  "1779005316_created_sources.js",
+  "1779005316_created_topics.js",
+  "1779005317_created_leaderboard.js",
+  "1779005317_created_learn_progress.js",
+  "1779005317_created_quiz_cache.js",
+  "1779005317_created_quiz_results.js",
+  "1779005317_created_settings.js",
+  "1779005317_created_team_members.js",
+  "1779127586_created_curriculum.js",
+  "1779127759_collections_snapshot.js",
+  "1779200000_updated_sources.js",
+  "1780000001_updated_topics.js",
+  "1780500000_updated_topics_relevance_locked.js",
+  "1780500001_normalize_relation_types.js",
+  "1780500002_created_llm_calls.js",
+  "1780600000_curriculum_v2.js",
+  "1780700000_sources_progress.js",
+  "1780800000_created_micro_learnings.js",
+  "1780800001_deleted_legacy_collections.js",
+  "1780800002_update_micro_learnings_rules.js",
+  "1780900000_team_members_enrollment.js",
+  "1780900001_created_test_results.js",
+  "1781000000_created_theme_sessions.js",
+  "1781100000_created_question_bank.js",
+  "1781100001_created_on_demand_attempts.js",
+];
+
+migrate((app) => {
+  // Fresh database? (no legacy schema) -> let the normal chain run.
+  try {
+    app.findCollectionByNameOrId("content");
+  } catch (_) {
+    return;
+  }
+
+  // Ledger already tracks the history? -> normally migrated DB, nothing to do.
+  const probe = new DynamicModel({ count: 0 });
+  app.db()
+    .newQuery("SELECT COUNT(*) AS count FROM _migrations WHERE file = {:file}")
+    .bind({ file: "1778948471_created_content.js" })
+    .one(probe);
+  if (probe.count > 0) {
+    return;
+  }
+
+  // Out-of-band provisioned database: schema exists, ledger is empty.
+  // Mark the historical migrations as applied so they are not replayed.
+  const applied = Date.now();
+  for (const file of HISTORICAL_MIGRATIONS) {
+    app.db()
+      .newQuery("INSERT OR IGNORE INTO _migrations (file, applied) VALUES ({:file}, {:applied})")
+      .bind({ file: file, applied: applied })
+      .execute();
+  }
+  console.log(
+    "baseline_ledger_sync: detected out-of-band provisioned database — marked " +
+    HISTORICAL_MIGRATIONS.length + " historical migrations as applied"
+  );
+}, (_app) => {
+  // Down: intentionally a no-op. The ledger rows describe environment-specific
+  // bookkeeping; removing them would re-trigger the replay this fix prevents.
+});

pb_migrations/1781000000_team_members_to_auth.js

@@ -1,6 +1,6 @@
 /// <reference path="../pb_data/types.d.ts" />
 //
-// Issue #16 — Azure (Entra ID) login.
+// Issue #16 / #18 — Azure (Entra ID) login.
 //
 // Replaces the PIN-based `base` collection `team_members` with a PocketBase
 // `auth` collection that authenticates against Azure Entra ID via OIDC.
@@ -10,29 +10,60 @@
 // base, generated tests and micro-learnings live in OTHER collections and are
 // left completely untouched by this migration.
 //
-// OIDC provider credentials are read from the environment at apply time, so no
+// This migration is STRUCTURE ONLY and does not depend on the environment:
-// secrets are committed to git:
+// the OIDC provider credentials (ENTRA_TENANT_ID / ENTRA_CLIENT_ID /
-//   ENTRA_TENANT_ID, ENTRA_CLIENT_ID, ENTRA_CLIENT_SECRET
+// ENTRA_CLIENT_SECRET) are reconciled into the collection on every startup by
-// (set via the Ansible-rendered .env, see infra/*/site/compose.yml).
+// pb_hooks/entra_oidc.pb.js. That keeps the one-shot migration deterministic:
-//
+// applying it while the secrets are absent can no longer permanently disable
-// To rotate credentials or change the tenant later, update the env and either
+// OAuth2 (issue #18). As a fast path the provider is also attached here when
-// re-configure the provider from the PocketBase admin UI (Settings → Auth
+// the env vars are already present at apply time.
-// providers) or ship a follow-up migration.
 migrate((app) => {
-  // 1. Detach relation fields that point at team_members. PocketBase refuses to
+  // Idempotency guard: if team_members is already an auth collection (e.g. the
-  //    delete a collection that other collections relate to, so before we can
+  // conversion happened through an earlier partial rollout), leave it alone.
-  //    drop the old PIN-based team_members we convert those relations into plain
+  let existing = null;
-  //    text fields (the id is stored as text — consistent with how user_id is
+  try {
-  //    stored in test_results / on_demand_attempts). Per-user progress is reset
+    existing = app.findCollectionByNameOrId("team_members");
-  //    anyway, so dropping the FK constraint here is acceptable.
+  } catch (_) {
+    existing = null; // collection absent — nothing to convert, only to create
+  }
+  if (existing && existing.type === "auth") {
+    console.log("team_members_to_auth: team_members is already an auth collection — skipping conversion.");
+    return;
+  }
+
+  // 1. Detach relation fields that point at team_members. PocketBase refuses
+  //    to delete a collection that other collections relate to, so before we
+  //    can drop the old PIN-based team_members we convert those relations into
+  //    plain text fields (the id is stored as text — consistent with how
+  //    user_id is stored in test_results / on_demand_attempts). The column
+  //    values are preserved by the conversion.
   const deps = [
     { name: "micro_learning_completions", relId: "rel_team_member_id",  textId: "txt_mlc_team_member" },
     { name: "theme_session_completions",  relId: "rel_tsc_team_member", textId: "txt_tsc_team_member" },
   ];
   for (const dep of deps) {
-    let c;
+    let c = null;
-    try { c = app.findCollectionByNameOrId(dep.name); } catch (_) { continue; }
+    try {
-    // Drop any index that references the team_member_id column.
+      c = app.findCollectionByNameOrId(dep.name);
+    } catch (_) {
+      console.log("team_members_to_auth: " + dep.name + " does not exist yet — nothing to detach.");
+      continue;
+    }
+    if (!c.fields.getById(dep.relId)) {
+      console.log("team_members_to_auth: " + dep.name + "." + dep.relId + " already detached — skipping.");
+      continue;
+    }
+    // PocketBase diffs fields by ID, so swapping the relation field for a text
+    // field drops and recreates the underlying column. Back the values up and
+    // restore them after the schema save — the ids must survive (issue #18).
+    const backup = "_issue18_tm_" + dep.name;
+    app.db().newQuery("DROP TABLE IF EXISTS `" + backup + "`").execute();
+    app.db().newQuery(
+      "CREATE TABLE `" + backup + "` AS SELECT `id`, `team_member_id` AS `v` FROM `" + dep.name + "`"
+    ).execute();
+
+    // Drop any index that references the team_member_id column (it is rebuilt
+    // on the text column below).
     c.indexes = (c.indexes || []).filter((idx) => idx.indexOf("team_member_id") === -1);
     // Replace the relation field with a text field of the same name.
     c.fields.removeById(dep.relId);
@@ -45,34 +76,40 @@ migrate((app) => {
       max: 0,
     }));
     app.save(c);
+
+    app.db().newQuery(
+      "UPDATE `" + dep.name + "` SET `team_member_id` = COALESCE(" +
+      "(SELECT `v` FROM `" + backup + "` WHERE `" + backup + "`.`id` = `" + dep.name + "`.`id`), '')"
+    ).execute();
+    app.db().newQuery("DROP TABLE `" + backup + "`").execute();
   }
 
   // Recreate the unique (team_member_id, session_week) guard on the text column.
   try {
     const tsc = app.findCollectionByNameOrId("theme_session_completions");
-    tsc.indexes.push("CREATE UNIQUE INDEX `idx_theme_session_completions_user_week` ON `theme_session_completions` (`team_member_id`, `session_week`)");
+    const idx = "CREATE UNIQUE INDEX `idx_theme_session_completions_user_week` ON `theme_session_completions` (`team_member_id`, `session_week`)";
+    if ((tsc.indexes || []).indexOf(idx) === -1) {
+      tsc.indexes.push(idx);
       app.save(tsc);
-  } catch (_) { /* collection missing — skip */ }
+    }
-
-  // 2. Drop the old PIN-based team_members (now unreferenced).
-  try {
-    const old = app.findCollectionByNameOrId("team_members");
-    app.delete(old);
   } catch (_) {
-    // Collection did not exist — nothing to drop.
+    console.log("team_members_to_auth: theme_session_completions does not exist yet — no index to rebuild.");
   }
 
-  const tenant = $os.getenv("ENTRA_TENANT_ID") || "common";
+  // 2. Drop the old PIN-based team_members (now unreferenced). NOT wrapped in
-  const clientId = $os.getenv("ENTRA_CLIENT_ID");
+  //    a try/catch: if this fails there is an unknown relation left and the
-  const clientSecret = $os.getenv("ENTRA_CLIENT_SECRET");
+  //    migration must fail loudly instead of masking the error (issue #18).
+  if (existing) {
+    app.delete(existing);
+  }
 
-  // Only configure the OIDC provider when credentials are actually present.
+  // Fast path: attach the OIDC provider right away when the credentials are
-  // PocketBase rejects an enabled OAuth2 provider with an empty clientId /
+  // already present at apply time. When they are absent the collection is
-  // clientSecret, which would abort this migration and prevent PocketBase from
+  // created without a provider and pb_hooks/entra_oidc.pb.js configures it on
-  // starting at all (502 on every /api call). When the secrets are absent the
+  // the next startup / cron tick once the ENTRA_* env vars are supplied.
-  // collection is created without a provider; once the ENTRA_* secrets are set,
+  const tenant = $os.getenv("ENTRA_TENANT_ID") || "common";
-  // configure the provider via the PocketBase admin UI
+  const clientId = $os.getenv("ENTRA_CLIENT_ID") || "";
-  // (Settings → Auth providers → OIDC) or re-apply this migration.
+  const clientSecret = $os.getenv("ENTRA_CLIENT_SECRET") || "";
   const oidcProviders = (clientId && clientSecret) ? [
     {
       name: "oidc",
@@ -85,8 +122,15 @@ migrate((app) => {
       pkce: true,
     },
   ] : [];
+  if (oidcProviders.length === 0) {
+    console.log(
+      "team_members_to_auth: ENTRA_CLIENT_ID / ENTRA_CLIENT_SECRET not set — creating the auth " +
+      "collection without an OIDC provider. pb_hooks/entra_oidc.pb.js will configure the provider " +
+      "automatically once the env vars are present (no re-apply needed)."
+    );
+  }
 
-  // 2. Recreate `team_members` as an auth collection (same name → all existing
+  // 3. Recreate `team_members` as an auth collection (same name → all existing
   //    `user_id` references in test_results, on_demand_attempts,
   //    micro_learning_completions, leaderboard, theme_session_completions keep
   //    working unchanged).