feat: phase 5 of AI pipeline hardening — R42 retrieval & telemetry

- Add dependency-free TF-IDF retrieval (src/lib/retrieval.js) with NL+EN stopwords and a WeakMap-cached index. - Rewrite buildKbContext to ship the top-K relevant topics + verbatim- mentioned ids only, filter relations to the included set, and append a [kb_hash: <8 hex>] suffix so the ephemeral prompt cache busts when the graph changes. Returns { context, retrievedTopics, allTopics }. - Add LOOKUP_TOPIC_TOOL and drive useChat through callLLM directly with a multi-hop tool_result loop capped at 3 hops; preserve Anthropic-provided tool_use ids through callLLM so the loop can echo correct tool_use_id. - Truncate R42 history to the last 12 turns and prepend a single "(earlier conversation truncated)" assistant message. - Set R42 chat defaults: temperature 0.3, maxTokens 2048. - Add pb_migrations/1780500002_created_llm_calls.js (the best-effort logger in callLLM was already wired) and a new Admin → Diagnostics view showing the last 100 calls with token usage, cache-hit rate, and USD cost from a local Anthropic price table. - Finalize AI_PIPELINE_HARDENING_PLAN.md: mark Phases 1–5 shipped and Phase 6 (eval harness) explicitly out of scope. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
feat: phase 4 of AI pipeline hardening — quiz & content quality
2026-05-20 21:36:40 +02:00 · 2026-05-20 19:22:10 +02:00 · 2026-05-20 19:12:16 +02:00 · 2026-05-20 15:57:40 +00:00 · 2026-05-20 17:56:45 +02:00 · 2026-05-20 15:18:00 +00:00
28 changed files with 1673 additions and 180 deletions
--- a/AI_PIPELINE_HARDENING_PLAN.md
+++ b/AI_PIPELINE_HARDENING_PLAN.md
@@ -6,6 +6,8 @@

 This plan upgrades the platform's interaction with the Anthropic API: how prompts are built, how responses are parsed, how the model is retried, and how outputs are validated. It is broken into six phases that can be implemented and shipped independently. Each phase ends with verifiable acceptance criteria.

+> **Status (2026-05-20):** Phases 1–5 implemented and shipped. Phase 6 (eval harness) is intentionally **out of scope** for this initiative — the production pipeline is hardened to the level the platform needs, and a golden-set runner can be reopened later as a stand-alone task if regression risk grows. The repo no longer carries any TODO from this plan.
+
 ---

 ## 0. Operating principles
@@ -461,7 +463,9 @@ Wire `callLLM` to write to it (best-effort, never throws). Add a minimal `Admin

 ---

-## Phase 6 — Eval harness (optional, high-leverage)
+## Phase 6 — Eval harness (NOT IMPLEMENTED — out of scope)
+
+> Phase 6 was deliberately skipped. The acceptance criteria for Phases 1–5 give enough confidence in extraction, content, quiz, R42, and telemetry that a golden-set harness is not currently load-bearing. Leave this section intact as a starting point for a future, separately-scoped initiative.

 **Goal:** prompt or model changes can be measured before they ship.

--- a/pb_migrations/1780500000_updated_topics_relevance_locked.js
+++ b/pb_migrations/1780500000_updated_topics_relevance_locked.js
@@ -0,0 +1,23 @@
+/// <reference path="../pb_data/types.d.ts" />
+migrate((app) => {
+  const collection = app.findCollectionByNameOrId("pbc_2800040823")
+
+  // add field — relevance_locked is set to true whenever an admin edits
+  // learning_relevance via the UI; mergeKnowledgeGraph must never overwrite
+  // learning_relevance on a locked topic during re-extraction.
+  collection.fields.addAt(5, new Field({
+    "hidden": false,
+    "id": "bool_relevance_locked",
+    "name": "relevance_locked",
+    "presentable": false,
+    "required": false,
+    "system": false,
+    "type": "bool"
+  }))
+
+  return app.save(collection)
+}, (app) => {
+  const collection = app.findCollectionByNameOrId("pbc_2800040823")
+  collection.fields.removeById("bool_relevance_locked")
+  return app.save(collection)
+})
--- a/pb_migrations/1780500001_normalize_relation_types.js
+++ b/pb_migrations/1780500001_normalize_relation_types.js
@@ -0,0 +1,43 @@
+/// <reference path="../pb_data/types.d.ts" />
+// One-shot data migration: rewrite legacy "executes" relations to the
+// canonical "executed_by" vocabulary by swapping source and target.
+// Previously `role --executes--> process`; canonical is
+// `process --executed_by--> role`.
+migrate((app) => {
+  const records = app.findRecordsByFilter(
+    "pbc_1883724256",  // relations collection
+    'type = "executes"',
+    '',
+    0,
+    0,
+  )
+
+  for (const rec of records) {
+    const source = rec.get("source")
+    const target = rec.get("target")
+    rec.set("type", "executed_by")
+    rec.set("source", target)
+    rec.set("target", source)
+    app.save(rec)
+  }
+}, (app) => {
+  // Reverse: turn executed_by back into executes (best-effort — only those
+  // created before this migration would have been "executes"; rolling back
+  // will affect any newer executed_by rows too).
+  const records = app.findRecordsByFilter(
+    "pbc_1883724256",
+    'type = "executed_by"',
+    '',
+    0,
+    0,
+  )
+
+  for (const rec of records) {
+    const source = rec.get("source")
+    const target = rec.get("target")
+    rec.set("type", "executes")
+    rec.set("source", target)
+    rec.set("target", source)
+    app.save(rec)
+  }
+})
--- a/pb_migrations/1780500002_created_llm_calls.js
+++ b/pb_migrations/1780500002_created_llm_calls.js
@@ -0,0 +1,209 @@
+/// <reference path="../pb_data/types.d.ts" />
+migrate((app) => {
+  const collection = new Collection({
+    "createRule": "",
+    "deleteRule": "",
+    "fields": [
+      {
+        "autogeneratePattern": "[a-z0-9]{15}",
+        "help": "",
+        "hidden": false,
+        "id": "text3208210256",
+        "max": 15,
+        "min": 15,
+        "name": "id",
+        "pattern": "^[a-z0-9]+$",
+        "presentable": false,
+        "primaryKey": true,
+        "required": true,
+        "system": true,
+        "type": "text"
+      },
+      {
+        "autogeneratePattern": "",
+        "help": "",
+        "hidden": false,
+        "id": "text_llm_task",
+        "max": 0,
+        "min": 0,
+        "name": "task",
+        "pattern": "",
+        "presentable": false,
+        "primaryKey": false,
+        "required": false,
+        "system": false,
+        "type": "text"
+      },
+      {
+        "autogeneratePattern": "",
+        "help": "",
+        "hidden": false,
+        "id": "text_llm_model",
+        "max": 0,
+        "min": 0,
+        "name": "model",
+        "pattern": "",
+        "presentable": false,
+        "primaryKey": false,
+        "required": false,
+        "system": false,
+        "type": "text"
+      },
+      {
+        "autogeneratePattern": "",
+        "help": "",
+        "hidden": false,
+        "id": "text_llm_tier",
+        "max": 0,
+        "min": 0,
+        "name": "tier",
+        "pattern": "",
+        "presentable": false,
+        "primaryKey": false,
+        "required": false,
+        "system": false,
+        "type": "text"
+      },
+      {
+        "help": "",
+        "hidden": false,
+        "id": "number_llm_duration",
+        "max": null,
+        "min": null,
+        "name": "duration_ms",
+        "onlyInt": false,
+        "presentable": false,
+        "required": false,
+        "system": false,
+        "type": "number"
+      },
+      {
+        "help": "",
+        "hidden": false,
+        "id": "number_llm_input",
+        "max": null,
+        "min": null,
+        "name": "input_tokens",
+        "onlyInt": false,
+        "presentable": false,
+        "required": false,
+        "system": false,
+        "type": "number"
+      },
+      {
+        "help": "",
+        "hidden": false,
+        "id": "number_llm_output",
+        "max": null,
+        "min": null,
+        "name": "output_tokens",
+        "onlyInt": false,
+        "presentable": false,
+        "required": false,
+        "system": false,
+        "type": "number"
+      },
+      {
+        "help": "",
+        "hidden": false,
+        "id": "number_llm_cache_r",
+        "max": null,
+        "min": null,
+        "name": "cache_read_tokens",
+        "onlyInt": false,
+        "presentable": false,
+        "required": false,
+        "system": false,
+        "type": "number"
+      },
+      {
+        "help": "",
+        "hidden": false,
+        "id": "number_llm_cache_c",
+        "max": null,
+        "min": null,
+        "name": "cache_create_tokens",
+        "onlyInt": false,
+        "presentable": false,
+        "required": false,
+        "system": false,
+        "type": "number"
+      },
+      {
+        "autogeneratePattern": "",
+        "help": "",
+        "hidden": false,
+        "id": "text_llm_stop",
+        "max": 0,
+        "min": 0,
+        "name": "stop_reason",
+        "pattern": "",
+        "presentable": false,
+        "primaryKey": false,
+        "required": false,
+        "system": false,
+        "type": "text"
+      },
+      {
+        "hidden": false,
+        "id": "bool_llm_ok",
+        "name": "ok",
+        "presentable": false,
+        "required": false,
+        "system": false,
+        "type": "bool"
+      },
+      {
+        "autogeneratePattern": "",
+        "help": "",
+        "hidden": false,
+        "id": "text_llm_err",
+        "max": 0,
+        "min": 0,
+        "name": "error_msg",
+        "pattern": "",
+        "presentable": false,
+        "primaryKey": false,
+        "required": false,
+        "system": false,
+        "type": "text"
+      },
+      {
+        "hidden": false,
+        "id": "autodate_llm_created",
+        "name": "created",
+        "onCreate": true,
+        "onUpdate": false,
+        "presentable": false,
+        "system": false,
+        "type": "autodate"
+      },
+      {
+        "hidden": false,
+        "id": "autodate_llm_updated",
+        "name": "updated",
+        "onCreate": true,
+        "onUpdate": true,
+        "presentable": false,
+        "system": false,
+        "type": "autodate"
+      }
+    ],
+    "id": "pbc_llm_calls_001",
+    "indexes": [
+      "CREATE INDEX `idx_llm_calls_created` ON `llm_calls` (`created`)",
+      "CREATE INDEX `idx_llm_calls_task` ON `llm_calls` (`task`)"
+    ],
+    "listRule": "",
+    "name": "llm_calls",
+    "system": false,
+    "type": "base",
+    "updateRule": "",
+    "viewRule": ""
+  });
+
+  return app.save(collection);
+}, (app) => {
+  const collection = app.findCollectionByNameOrId("pbc_llm_calls_001");
+  return app.delete(collection);
+})
--- a/src/components/admin/Diagnostics.jsx
+++ b/src/components/admin/Diagnostics.jsx
@@ -0,0 +1,162 @@
+import { useEffect, useState } from 'react';
+import { RefreshCw, AlertCircle, CheckCircle2 } from 'lucide-react';
+import Card from '../ui/Card';
+import Button from '../ui/Button';
+import Tag from '../ui/Tag';
+import * as db from '../../lib/db';
+
+// Public Anthropic pricing per 1M tokens. Update manually when prices change.
+const PRICES = {
+  'claude-haiku-4-5-20251001':   { input: 1.0,  output: 5.0,  cache_read: 0.10, cache_write: 1.25 },
+  'claude-haiku-4-5':            { input: 1.0,  output: 5.0,  cache_read: 0.10, cache_write: 1.25 },
+  'claude-sonnet-4-6':           { input: 3.0,  output: 15.0, cache_read: 0.30, cache_write: 3.75 },
+  'claude-opus-4-7':             { input: 15.0, output: 75.0, cache_read: 1.50, cache_write: 18.75 },
+};
+
+function pricesFor(model) {
+  if (!model) return null;
+  if (PRICES[model]) return PRICES[model];
+  if (model.includes('haiku')) return PRICES['claude-haiku-4-5'];
+  if (model.includes('sonnet')) return PRICES['claude-sonnet-4-6'];
+  if (model.includes('opus')) return PRICES['claude-opus-4-7'];
+  return null;
+}
+
+function costUsd(row) {
+  const p = pricesFor(row.model);
+  if (!p) return null;
+  const inTok = (row.input_tokens || 0) - (row.cache_read_tokens || 0) - (row.cache_create_tokens || 0);
+  const out = row.output_tokens || 0;
+  const cr = row.cache_read_tokens || 0;
+  const cc = row.cache_create_tokens || 0;
+  const usd = (Math.max(inTok, 0) * p.input + out * p.output + cr * p.cache_read + cc * p.cache_write) / 1_000_000;
+  return usd;
+}
+
+function fmtUsd(n) {
+  if (n == null) return '—';
+  if (n < 0.0001) return '<$0.0001';
+  return `$${n.toFixed(4)}`;
+}
+
+function fmtMs(n) {
+  if (n == null) return '—';
+  if (n < 1000) return `${Math.round(n)}ms`;
+  return `${(n / 1000).toFixed(1)}s`;
+}
+
+const Diagnostics = () => {
+  const [rows, setRows] = useState([]);
+  const [loading, setLoading] = useState(false);
+
+  const load = async () => {
+    setLoading(true);
+    try {
+      const r = await db.getRecentLlmCalls(100);
+      setRows(r);
+    } finally {
+      setLoading(false);
+    }
+  };
+
+  useEffect(() => { load(); }, []);
+
+  const totals = rows.reduce((acc, r) => {
+    acc.input += r.input_tokens || 0;
+    acc.output += r.output_tokens || 0;
+    acc.cacheRead += r.cache_read_tokens || 0;
+    acc.cacheCreate += r.cache_create_tokens || 0;
+    const c = costUsd(r);
+    if (c != null) acc.cost += c;
+    acc.duration += r.duration_ms || 0;
+    if (r.ok) acc.ok++;
+    else acc.fail++;
+    return acc;
+  }, { input: 0, output: 0, cacheRead: 0, cacheCreate: 0, cost: 0, duration: 0, ok: 0, fail: 0 });
+
+  const cacheHitRate = totals.input > 0
+    ? Math.round((totals.cacheRead / totals.input) * 100)
+    : 0;
+
+  return (
+    <div>
+      <div className="flex items-center justify-between mb-4">
+        <p className="text-fg-muted text-sm">
+          Laatste 100 LLM-aanroepen. Kosten worden lokaal berekend met publieke Anthropic-prijzen (handmatig verversen).
+        </p>
+        <Button onClick={load} disabled={loading}>
+          <RefreshCw size={16} className={`mr-2 ${loading ? 'animate-spin' : ''}`} /> Vernieuwen
+        </Button>
+      </div>
+
+      <div className="grid grid-cols-2 md:grid-cols-4 gap-3 mb-6">
+        <Card className="p-4 border border-bg-warm">
+          <p className="text-xs text-fg-muted uppercase">Calls</p>
+          <p className="text-2xl font-bold mt-1">{rows.length}</p>
+          <p className="text-xs text-fg-muted mt-1">{totals.ok} ok · {totals.fail} fail</p>
+        </Card>
+        <Card className="p-4 border border-bg-warm">
+          <p className="text-xs text-fg-muted uppercase">Tokens (in/out)</p>
+          <p className="text-2xl font-bold mt-1">{totals.input.toLocaleString()} / {totals.output.toLocaleString()}</p>
+          <p className="text-xs text-fg-muted mt-1">cache read: {totals.cacheRead.toLocaleString()} ({cacheHitRate}%)</p>
+        </Card>
+        <Card className="p-4 border border-bg-warm">
+          <p className="text-xs text-fg-muted uppercase">Geschatte kosten</p>
+          <p className="text-2xl font-bold mt-1">{fmtUsd(totals.cost)}</p>
+          <p className="text-xs text-fg-muted mt-1">over deze 100 aanroepen</p>
+        </Card>
+        <Card className="p-4 border border-bg-warm">
+          <p className="text-xs text-fg-muted uppercase">Gem. duur</p>
+          <p className="text-2xl font-bold mt-1">{rows.length ? fmtMs(totals.duration / rows.length) : '—'}</p>
+        </Card>
+      </div>
+
+      <Card className="p-0 border border-bg-warm overflow-hidden">
+        <div className="overflow-x-auto">
+          <table className="w-full text-sm">
+            <thead className="bg-bg-warm/40 text-fg-muted text-xs uppercase">
+              <tr>
+                <th className="text-left p-3">Tijd</th>
+                <th className="text-left p-3">Task</th>
+                <th className="text-left p-3">Tier</th>
+                <th className="text-left p-3">Model</th>
+                <th className="text-right p-3">In</th>
+                <th className="text-right p-3">Out</th>
+                <th className="text-right p-3">Cache</th>
+                <th className="text-right p-3">$</th>
+                <th className="text-right p-3">Duur</th>
+                <th className="text-left p-3">Status</th>
+              </tr>
+            </thead>
+            <tbody className="divide-y divide-bg-warm">
+              {rows.length === 0 ? (
+                <tr><td colSpan={10} className="p-8 text-center text-fg-muted">Nog geen aanroepen geregistreerd.</td></tr>
+              ) : rows.map(r => (
+                <tr key={r.id} className="hover:bg-bg-warm/20">
+                  <td className="p-3 text-xs text-fg-muted whitespace-nowrap">
+                    {r.created ? new Date(r.created).toLocaleString() : '—'}
+                  </td>
+                  <td className="p-3 font-mono text-xs">{r.task || '—'}</td>
+                  <td className="p-3 text-xs">{r.tier || '—'}</td>
+                  <td className="p-3 font-mono text-xs">{r.model || '—'}</td>
+                  <td className="p-3 text-right">{(r.input_tokens || 0).toLocaleString()}</td>
+                  <td className="p-3 text-right">{(r.output_tokens || 0).toLocaleString()}</td>
+                  <td className="p-3 text-right text-fg-muted">{(r.cache_read_tokens || 0).toLocaleString()}</td>
+                  <td className="p-3 text-right">{fmtUsd(costUsd(r))}</td>
+                  <td className="p-3 text-right">{fmtMs(r.duration_ms)}</td>
+                  <td className="p-3">
+                    {r.ok
+                      ? <Tag variant="success" className="flex items-center gap-1 w-fit"><CheckCircle2 size={12}/> ok</Tag>
+                      : <Tag variant="dark" className="bg-red-100 text-red-800 flex items-center gap-1 w-fit"><AlertCircle size={12}/> {(r.error_msg || r.stop_reason || 'fail').slice(0, 24)}</Tag>}
+                  </td>
+                </tr>
+              ))}
+            </tbody>
+          </table>
+        </div>
+      </Card>
+    </div>
+  );
+};
+
+export default Diagnostics;
--- a/src/components/admin/KnowledgeGraph.jsx
+++ b/src/components/admin/KnowledgeGraph.jsx
@@ -180,10 +180,17 @@ const KnowledgeGraph = () => {
  };

  const saveEdit = async () => {
-    await db.upsertTopic({ ...selectedNode, ...editData });
-    const updated = topics.map(t => t.id === selectedNode.id ? { ...t, ...editData } : t);
+    // If the admin touched learning_relevance, lock it so re-extraction won't overwrite the choice.
+    // But an explicit relevance_locked in editData (the unlock checkbox) always wins.
+    const relevanceChanged =
+      editData.learning_relevance !== undefined &&
+      editData.learning_relevance !== selectedNode.learning_relevance;
+    const next = { ...editData };
+    if (relevanceChanged && next.relevance_locked === undefined) next.relevance_locked = true;
+    await db.upsertTopic({ ...selectedNode, ...next });
+    const updated = topics.map(t => t.id === selectedNode.id ? { ...t, ...next } : t);
    setTopics(updated);
-    setSelectedNode({ ...selectedNode, ...editData });
+    setSelectedNode({ ...selectedNode, ...next });
    setIsEditing(false);
  };

@@ -265,22 +272,11 @@ const KnowledgeGraph = () => {
          setSyncProgress(`Processing ${count} of ${filesToProcess.length}: ${file.name}...`);
          try {
            const rawContent = await getFileContent('respellion', 'employee-handbook', file.path);
+            // Pacing is handled centrally by extractionLimiter inside analyzeHandbookDelta.
            await analyzeHandbookDelta(rawContent, file.path);
            await db.updateHandbookSyncState(file.path, file.sha);
-            
-            // To respect Anthropic's 5 requests per minute rate limit on this tier, 
-            // we pause for 15 seconds before processing the next file.
-            if (count < filesToProcess.length) {
-              setSyncProgress(`Waiting 15s to avoid rate limits... (${count}/${filesToProcess.length})`);
-              await new Promise(resolve => setTimeout(resolve, 15000));
-            }
          } catch (err) {
            console.error('Failed to process file:', file.path, err);
-            // We continue processing other files even if one fails, but still wait to avoid further rate limits
-            if (count < filesToProcess.length) {
-              setSyncProgress(`Error on ${file.name}. Waiting 15s... (${count}/${filesToProcess.length})`);
-              await new Promise(resolve => setTimeout(resolve, 15000));
-            }
          }
        }
        setSyncProgress('Sync Complete! Click "Analyze & Optimize Graph" above to clean up and merge.');
@@ -369,7 +365,7 @@ ${JSON.stringify({ topics: compactTopics, relations: compactRelations })}`;
      if (actions.relevanceUpdates && Array.isArray(actions.relevanceUpdates)) {
        for (const update of actions.relevanceUpdates) {
          const topicIndex = updatedTopics.findIndex(t => t.id === update.id);
-          if (topicIndex !== -1) {
+          if (topicIndex !== -1 && !updatedTopics[topicIndex].relevance_locked) {
            updatedTopics[topicIndex] = { ...updatedTopics[topicIndex], learning_relevance: update.learning_relevance };
          }
        }
@@ -532,6 +528,17 @@ ${JSON.stringify({ topics: compactTopics, relations: compactRelations })}`;
                  <option value="peripheral">Peripheral</option>
                  <option value="exclude">Exclude</option>
                </select>
+                {selectedNode.relevance_locked && (
+                  <label className="flex items-center gap-2 text-xs text-fg-muted mt-2 cursor-pointer">
+                    <input
+                      type="checkbox"
+                      checked={editData.relevance_locked !== false}
+                      onChange={e => setEditData({...editData, relevance_locked: e.target.checked})}
+                      className="rounded bg-bg-warm border-transparent focus:ring-0 text-teal"
+                    />
+                    Locked — re-extraction will not change this
+                  </label>
+                )}
              </div>
              <div>
                <label className="text-xs text-fg-muted uppercase tracking-wider mb-1 block">Description</label>
@@ -554,9 +561,12 @@ ${JSON.stringify({ topics: compactTopics, relations: compactRelations })}`;
              </div>
              <div>
                <p className="text-xs text-fg-muted uppercase tracking-wider mb-1">Type & Relevance</p>
-                <div className="flex gap-2">
+                <div className="flex gap-2 flex-wrap">
                  <span className="inline-block px-2 py-1 bg-bg-warm rounded-[var(--r-pill)] text-xs font-mono">{selectedNode.type}</span>
                  <span className="inline-block px-2 py-1 bg-bg-warm rounded-[var(--r-pill)] text-xs font-mono opacity-80">{selectedNode.learning_relevance || 'standard'}</span>
+                  {selectedNode.relevance_locked && (
+                    <span className="inline-block px-2 py-1 bg-teal/10 text-teal rounded-[var(--r-pill)] text-xs font-mono" title="Re-extraction will not change relevance for this topic.">locked</span>
+                  )}
                </div>
              </div>
              <div>
--- a/src/components/admin/TestManager.jsx
+++ b/src/components/admin/TestManager.jsx
@@ -34,7 +34,7 @@ const TestManager = () => {
    loadData();
  }, [selectedTopic]);

-  const handleGenerate = async (topic, count = 10) => {
+  const handleGenerate = async (topic, count = 5) => {
    setLoadingTopicId(topic.id);
    setError(null);
    try {
@@ -97,8 +97,8 @@ const TestManager = () => {
          {questions.length === 0 ? (
            <Card className="text-center py-12 text-fg-muted border-dashed border-2">
              <p>No questions generated for this topic yet.</p>
-              <Button className="mt-4" onClick={() => handleGenerate(selectedTopic, 10)} disabled={loadingTopicId === selectedTopic.id}>
-                Generate 10 Questions
+              <Button className="mt-4" onClick={() => handleGenerate(selectedTopic, 5)} disabled={loadingTopicId === selectedTopic.id}>
+                Generate 5 Questions
              </Button>
            </Card>
          ) : (
--- a/src/components/admin/UploadZone.jsx
+++ b/src/components/admin/UploadZone.jsx
@@ -1,5 +1,5 @@
 import { useState, useRef } from 'react';
-import { UploadCloud, AlertCircle, CheckCircle } from 'lucide-react';
+import { UploadCloud, AlertCircle, CheckCircle, X } from 'lucide-react';
 import { processSourceText } from '../../lib/extractionPipeline';
 import Card from '../ui/Card';
 import Button from '../ui/Button';
@@ -12,24 +12,36 @@ const UploadZone = ({ onUploadComplete }) => {
  const [status, setStatus]                 = useState(null);

  const fileInputRef = useRef(null);
+  const abortRef     = useRef(null);

  // ── File upload (drag & drop / browse) ────────────────────────────────────

  const processFile = async (file) => {
    setIsProcessing(true);
    setStatus(null);
+    const controller = new AbortController();
+    abortRef.current = controller;
    try {
      const text = await file.text();
-      await processSourceText(text, file.name);
+      await processSourceText(text, file.name, { signal: controller.signal });
      setStatus({ type: 'success', msg: `Successfully processed: ${file.name}` });
      if (onUploadComplete) onUploadComplete();
    } catch (error) {
-      setStatus({ type: 'error', msg: `Error processing file: ${error.message}` });
+      if (error?.name === 'AbortError') {
+        setStatus({ type: 'error', msg: `Cancelled: ${file.name}` });
+      } else {
+        setStatus({ type: 'error', msg: `Error processing file: ${error.message}` });
+      }
    } finally {
+      abortRef.current = null;
      setIsProcessing(false);
    }
  };

+  const cancelProcessing = () => {
+    abortRef.current?.abort(new DOMException('Cancelled by user', 'AbortError'));
+  };
+
  const handleDragOver  = (e) => { e.preventDefault(); setIsDragging(true); };
  const handleDragLeave = ()  => setIsDragging(false);

@@ -80,6 +92,19 @@ const UploadZone = ({ onUploadComplete }) => {
        />
      </Card>

+      {/* Cancel sits outside the upload card so pointer-events-none doesn't disable it. */}
+      {isProcessing && (
+        <div className="flex justify-center">
+          <button
+            type="button"
+            onClick={cancelProcessing}
+            className="flex items-center gap-1 text-sm text-red-600 hover:text-red-700 underline"
+          >
+            <X size={14} /> Cancel extraction
+          </button>
+        </div>
+      )}
+
      {/* ─── Status messages ─── */}
      {status && (
        <div className={`p-4 rounded-[var(--r-sm)] flex items-start gap-3 ${
--- a/src/components/chat/prompts.js
+++ b/src/components/chat/prompts.js
@@ -68,6 +68,22 @@ export function buildSystemPrompt({ userName, isAdmin, kbContext }) {
  ];
 }

+export const LOOKUP_TOPIC_TOOL = {
+  name: 'lookup_topic',
+  description:
+    'Haal de volledige beschrijving en eventuele leerinhoud van één topic op uit de kennisgraaf. Gebruik dit wanneer de samenvattende KENNISGRAAF in de systeemprompt niet genoeg informatie bevat om de vraag te beantwoorden.',
+  input_schema: {
+    type: 'object',
+    properties: {
+      id: {
+        type: 'string',
+        description: 'Het exacte topic id (kebab-case) zoals het in de kennisgraaf staat.',
+      },
+    },
+    required: ['id'],
+  },
+};
+
 export const PROPOSE_GRAPH_DELTA_TOOL = {
  name: 'propose_graph_delta',
  description:
--- a/src/components/chat/rag.js
+++ b/src/components/chat/rag.js
@@ -1,68 +1,119 @@
 import * as db from '../../lib/db';
+import { buildIndex, retrieveTopK } from '../../lib/retrieval';
+
+const TOP_K = 10;
+
+async function sha256Hex(input) {
+  const enc = new TextEncoder().encode(input);
+  if (globalThis.crypto?.subtle?.digest) {
+    const buf = await globalThis.crypto.subtle.digest('SHA-256', enc);
+    return Array.from(new Uint8Array(buf)).map(b => b.toString(16).padStart(2, '0')).join('');
+  }
+  let h = 2166136261 >>> 0;
+  for (let i = 0; i < input.length; i++) {
+    h ^= input.charCodeAt(i);
+    h = Math.imul(h, 16777619);
+  }
+  return (h >>> 0).toString(16).padStart(8, '0');
+}

 /**
- * Build a compact knowledge-base context string to inject into the system prompt.
- * Reads topics + relations from PocketBase via db.js.
- * Topic-level content is loaded only when a topic id/label appears in the user's message.
+ * Build a retrieval-scoped KB context. Instead of dumping the whole graph,
+ * we pick the top-K topics by TF-IDF over `userMessage`, plus any topic
+ * whose id or label appears verbatim in the message. Relations are filtered
+ * to those that touch the included set.
 *
- * Returns { context: string, topics: Array } so callers can reuse the fetched topics
- * for validateDelta without a second round-trip.
+ * A `[kb_hash: …]` suffix is appended so the Anthropic ephemeral prompt
+ * cache automatically busts when topics are added/removed.
+ *
+ * Returns:
+ *   { context, retrievedTopics, allTopics }
+ * — `allTopics` is the full PocketBase list so callers can still run
+ * `validateDelta` against the entire current graph.
 */
 export async function buildKbContext(userMessage = '') {
-  const [topics, relations] = await Promise.all([
+  const [allTopics, allRelations] = await Promise.all([
    db.getTopics(),
    db.getRelations(),
  ]);

-  if (topics.length === 0) {
+  const sortedIds = allTopics.map(t => t.id).sort().join('|');
+  const fullHash = await sha256Hex(sortedIds);
+  const kbHash = fullHash.slice(0, 8);
+
+  if (allTopics.length === 0) {
    return {
-      context: 'KENNISGRAAF: (leeg — er zijn nog geen onderwerpen geëxtraheerd)',
-      topics: [],
+      context: `KENNISGRAAF: (leeg — er zijn nog geen onderwerpen geëxtraheerd)\n[kb_hash: ${kbHash}]`,
+      retrievedTopics: [],
+      allTopics: [],
    };
  }

-  const topicLines = topics.map(t => {
+  const lowered = userMessage.toLowerCase();
+  const mentionedIds = new Set();
+  for (const t of allTopics) {
+    const idHit = t.id && lowered.includes(t.id.toLowerCase());
+    const labelHit = t.label && lowered.includes(t.label.toLowerCase());
+    if (idHit || labelHit) mentionedIds.add(t.id);
+  }
+
+  const index = buildIndex(allTopics);
+  const retrieved = retrieveTopK(index, userMessage, TOP_K);
+
+  const includedById = new Map();
+  for (const id of mentionedIds) {
+    const t = allTopics.find(x => x.id === id);
+    if (t) includedById.set(id, t);
+  }
+  for (const t of retrieved) {
+    if (!includedById.has(t.id)) includedById.set(t.id, t);
+  }
+  const included = [...includedById.values()];
+
+  const topicLines = included.map(t => {
    const desc = (t.description || '').replace(/\s+/g, ' ').trim().slice(0, 200);
    return `- ${t.id} (${t.type || 'concept'}) "${t.label}": ${desc}`;
  });

-  const relLines = relations.map(r => {
+  const includedIds = new Set(included.map(t => t.id));
+  const relLines = [];
+  for (const r of allRelations) {
    const src = typeof r.source === 'object' ? r.source.id : r.source;
    const tgt = typeof r.target === 'object' ? r.target.id : r.target;
-    return `- ${src} --${r.type}--> ${tgt}`;
-  });
-
-  // Pull deep content for any topic explicitly mentioned in the user message.
-  const lowered = userMessage.toLowerCase();
-  const mentionedDeepContent = [];
-  for (const t of topics) {
-    const idHit = t.id && lowered.includes(t.id.toLowerCase());
-    const labelHit = t.label && lowered.includes(t.label.toLowerCase());
-    if (idHit || labelHit) {
-      const content = await db.getContent(t.id).catch(() => null);
-      if (content) {
-        let raw;
-        if (typeof content === 'string') raw = content;
-        else if (content.article) raw = content.article;
-        else raw = JSON.stringify(content);
-        const snippet = raw.replace(/\s+/g, ' ').trim().slice(0, 1200);
-        mentionedDeepContent.push(`### ${t.label}\n${snippet}`);
-      }
+    if (includedIds.has(src) && includedIds.has(tgt)) {
+      relLines.push(`- ${src} --${r.type}--> ${tgt}`);
    }
  }

+  const mentionedDeepContent = [];
+  for (const id of mentionedIds) {
+    const t = includedById.get(id);
+    if (!t) continue;
+    const content = await db.getContent(t.id).catch(() => null);
+    if (!content) continue;
+    let raw;
+    if (typeof content === 'string') raw = content;
+    else if (content.article) raw = typeof content.article === 'string' ? content.article : JSON.stringify(content.article);
+    else raw = JSON.stringify(content);
+    const snippet = raw.replace(/\s+/g, ' ').trim().slice(0, 1200);
+    mentionedDeepContent.push(`### ${t.label}\n${snippet}`);
+  }
+
  const context = [
-    `KENNISGRAAF — TOPICS:`,
+    `KENNISGRAAF — RELEVANTE TOPICS (top ${included.length} van ${allTopics.length}):`,
    topicLines.join('\n'),
    ``,
-    `KENNISGRAAF — RELATIES:`,
-    relLines.length ? relLines.join('\n') : '(geen relaties)',
+    `KENNISGRAAF — RELATIES (binnen deze selectie):`,
+    relLines.length ? relLines.join('\n') : '(geen relaties binnen deze selectie)',
    mentionedDeepContent.length
      ? `\n\nDIEPERE INHOUD (voor genoemde topics):\n${mentionedDeepContent.join('\n\n')}`
      : '',
+    ``,
+    `Als de relevante context hierboven te beperkt is, gebruik dan de tool "lookup_topic" om de volledige beschrijving en eventuele leerinhoud van een specifiek topic op te halen.`,
+    `[kb_hash: ${kbHash}]`,
  ].join('\n');

-  return { context, topics };
+  return { context, retrievedTopics: included, allTopics };
 }

 /**
--- a/src/components/chat/useChat.js
+++ b/src/components/chat/useChat.js
@@ -1,17 +1,59 @@
 import { useCallback, useEffect, useRef, useState } from 'react';
 import { storage } from '../../lib/storage';
-import { anthropicApi } from '../../lib/api';
+import { callLLM } from '../../lib/llm';
+import * as db from '../../lib/db';
 import { buildKbContext, validateDelta, deltaKey } from './rag';
-import { buildSystemPrompt, PROPOSE_GRAPH_DELTA_TOOL, STRINGS } from './prompts';
+import {
+  buildSystemPrompt,
+  PROPOSE_GRAPH_DELTA_TOOL,
+  LOOKUP_TOPIC_TOOL,
+  STRINGS,
+} from './prompts';

 const MAX_HISTORY = 50;
+const VERBATIM_TURNS = 12;
+const MAX_LOOKUP_HOPS = 3;
+const TRUNCATION_NOTICE = '(earlier conversation truncated)';
+
+/** Trim API history to the last N turns and prepend a truncation notice. */
+function truncateApiMessages(history) {
+  if (history.length <= VERBATIM_TURNS) return history;
+  const tail = history.slice(-VERBATIM_TURNS);
+  return [{ role: 'assistant', content: TRUNCATION_NOTICE }, ...tail];
+}
+
+async function resolveLookupTopic(id, allTopics) {
+  const topic = allTopics.find(t => t.id === id);
+  if (!topic) {
+    return { ok: false, payload: `Geen topic gevonden met id "${id}".` };
+  }
+  const content = await db.getContent(id).catch(() => null);
+  let contentSnippet = null;
+  if (content) {
+    let raw;
+    if (typeof content === 'string') raw = content;
+    else if (content.article) raw = typeof content.article === 'string' ? content.article : JSON.stringify(content.article);
+    else raw = JSON.stringify(content);
+    contentSnippet = raw.replace(/\s+/g, ' ').trim().slice(0, 2400);
+  }
+  return {
+    ok: true,
+    payload: {
+      id: topic.id,
+      label: topic.label,
+      type: topic.type || 'concept',
+      description: topic.description || '',
+      learning_content: contentSnippet,
+    },
+  };
+}

 /**
 * Conversation hook for R42.
- * Owns the message list, persists to chat:thread:{userId}, calls Anthropic,
- * and surfaces validated graph delta suggestions inline.
- *
- * Note: buildKbContext is async (reads PocketBase), so send() is fully async.
+ * Owns the message list, persists to chat:thread:{userId}, calls Anthropic
+ * via the shared `callLLM` client (with a `lookup_topic` multi-hop loop and
+ * the `propose_graph_delta` tool), and surfaces validated graph delta
+ * suggestions inline.
 */
 export function useChat({ user, isAdmin }) {
  const threadKey = user ? `chat:thread:${user.id}` : null;
@@ -20,7 +62,6 @@ export function useChat({ user, isAdmin }) {
  const [errored, setErrored] = useState(false);
  const seenDeltaKeys = useRef(new Set());

-  // Load persisted thread + seed greeting
  useEffect(() => {
    if (!user) return;
    const stored = storage.get(threadKey, null);
@@ -41,7 +82,6 @@ export function useChat({ user, isAdmin }) {
    }
  }, [user, threadKey]);

-  // Persist on change
  useEffect(() => {
    if (!threadKey) return;
    const capped = messages.slice(-MAX_HISTORY);
@@ -68,29 +108,43 @@ export function useChat({ user, isAdmin }) {
    setErrored(false);

    try {
-      const { context: kbContext, topics: kbTopics } = await buildKbContext(trimmed);
+      const { context: kbContext, allTopics } = await buildKbContext(trimmed);
      const systemPrompt = buildSystemPrompt({
        userName: user.name || 'daar',
        isAdmin,
        kbContext,
      });

-      // Strip UI-only fields before sending to Anthropic
-      const apiMessages = next
+      const historyMessages = next
        .filter(m => m.role === 'user' || m.role === 'assistant')
        .map(m => ({ role: m.role, content: m.content }));

-      const response = await anthropicApi.chat(systemPrompt, apiMessages, {
-        tools: [PROPOSE_GRAPH_DELTA_TOOL],
-      });
+      const apiMessages = truncateApiMessages(historyMessages);

      let textOut = '';
      let suggestion = null;
-      for (const block of response.content || []) {
-        if (block.type === 'text') {
-          textOut += (textOut ? '\n' : '') + (block.text || '');
-        } else if (block.type === 'tool_use' && block.name === PROPOSE_GRAPH_DELTA_TOOL.name) {
-          const validated = validateDelta(block.input, kbTopics);
+      let hops = 0;
+
+      while (true) {
+        const response = await callLLM({
+          task: 'chat.r42',
+          tier: 'standard',
+          system: systemPrompt,
+          messages: apiMessages,
+          tools: [LOOKUP_TOPIC_TOOL, PROPOSE_GRAPH_DELTA_TOOL],
+          maxTokens: 2048,
+          temperature: 0.3,
+        });
+
+        if (response.text) {
+          textOut += (textOut ? '\n' : '') + response.text;
+        }
+
+        const lookupCalls = response.toolUses.filter(tu => tu.name === LOOKUP_TOPIC_TOOL.name);
+        const deltaCall = response.toolUses.find(tu => tu.name === PROPOSE_GRAPH_DELTA_TOOL.name);
+
+        if (deltaCall && !suggestion) {
+          const validated = validateDelta(deltaCall.input, allTopics);
          if (validated) {
            const key = deltaKey(validated);
            if (!seenDeltaKeys.current.has(key)) {
@@ -99,6 +153,33 @@ export function useChat({ user, isAdmin }) {
            }
          }
        }
+
+        if (lookupCalls.length === 0 || hops >= MAX_LOOKUP_HOPS) break;
+        hops++;
+
+        const assistantBlocks = [];
+        if (response.text) assistantBlocks.push({ type: 'text', text: response.text });
+        for (const tu of response.toolUses) {
+          if (!tu.id) continue;
+          assistantBlocks.push({ type: 'tool_use', id: tu.id, name: tu.name, input: tu.input });
+        }
+        apiMessages.push({ role: 'assistant', content: assistantBlocks });
+
+        const toolResults = [];
+        for (const tu of lookupCalls) {
+          if (!tu.id) continue;
+          const topicId = String(tu.input?.id || '').trim();
+          const resolved = await resolveLookupTopic(topicId, allTopics);
+          toolResults.push({
+            type: 'tool_result',
+            tool_use_id: tu.id,
+            content: typeof resolved.payload === 'string'
+              ? resolved.payload
+              : JSON.stringify(resolved.payload),
+            ...(resolved.ok ? {} : { is_error: true }),
+          });
+        }
+        apiMessages.push({ role: 'user', content: toolResults });
      }

      const assistantMsg = {
--- a/src/lib/tests/extractionPipeline.test.js
+++ b/src/lib/tests/extractionPipeline.test.js
@@ -0,0 +1,89 @@
+import { describe, expect, it, vi, beforeEach, afterEach } from 'vitest';
+
+vi.mock('../pb', () => ({
+  pb: { collection: () => ({ create: () => ({ catch: () => {} }) }) },
+}));
+
+import { chunkText, buildKnownIdsHint, MAX_CHUNK_CHARS, OVERLAP_CHARS } from '../extractionPipeline';
+
+describe('chunkText', () => {
+  let warnSpy;
+  beforeEach(() => { warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {}); });
+  afterEach(() => { warnSpy.mockRestore(); });
+
+  it('returns the original text as a single chunk when below maxChars', () => {
+    const result = chunkText('A short paragraph.');
+    expect(result).toEqual(['A short paragraph.']);
+  });
+
+  it('returns empty array for empty/whitespace input', () => {
+    expect(chunkText('')).toEqual([]);
+    expect(chunkText('   \n  ')).toEqual([]);
+  });
+
+  it('splits along sentence boundaries with overlap between adjacent chunks', () => {
+    const sentence = 'This is a sentence with exactly a known length. ';
+    const text = sentence.repeat(100); // ~5000 chars
+    const chunks = chunkText(text, { maxChars: 600, overlapChars: 150 });
+
+    expect(chunks.length).toBeGreaterThan(1);
+    for (const c of chunks) {
+      expect(c.length).toBeLessThanOrEqual(600);
+    }
+    // Adjacent chunks share trailing text — the overlap should be non-empty.
+    for (let i = 1; i < chunks.length; i++) {
+      const tail = chunks[i - 1].slice(-150);
+      // The new chunk must begin with content that appears at the tail of the prior chunk.
+      const firstHundred = chunks[i].slice(0, 100);
+      // At least one word from the tail should appear in the head of the next chunk.
+      const words = tail.split(/\s+/).filter((w) => w.length > 3);
+      const shared = words.some((w) => firstHundred.includes(w));
+      expect(shared).toBe(true);
+    }
+  });
+
+  it('hard-splits a single sentence that exceeds maxChars and logs a warning', () => {
+    const huge = 'word '.repeat(400).trim() + '.'; // ~2000 chars, no sentence break
+    const chunks = chunkText(huge, { maxChars: 500, overlapChars: 50 });
+    expect(chunks.length).toBeGreaterThan(1);
+    expect(warnSpy).toHaveBeenCalled();
+    for (const c of chunks) expect(c.length).toBeLessThanOrEqual(500);
+  });
+
+  it('handles paragraph-only splits when no sentence punctuation is present', () => {
+    const paragraphs = Array.from({ length: 10 }, (_, i) => `paragraph ${i} content here`).join('\n\n');
+    const chunks = chunkText(paragraphs, { maxChars: 80, overlapChars: 20 });
+    expect(chunks.length).toBeGreaterThan(1);
+  });
+
+  it('uses the documented defaults', () => {
+    expect(MAX_CHUNK_CHARS).toBe(8000);
+    expect(OVERLAP_CHARS).toBe(800);
+  });
+});
+
+describe('buildKnownIdsHint', () => {
+  it('returns empty string when no IDs are known', () => {
+    expect(buildKnownIdsHint([])).toBe('');
+    expect(buildKnownIdsHint(undefined)).toBe('');
+    expect(buildKnownIdsHint(null)).toBe('');
+  });
+
+  it('formats the known IDs as a bulleted list with a leading instruction', () => {
+    const hint = buildKnownIdsHint(['software-engineer', 'onboarding-buddy']);
+    expect(hint).toContain('Already-extracted topic IDs');
+    expect(hint).toContain('- software-engineer');
+    expect(hint).toContain('- onboarding-buddy');
+    expect(hint.endsWith('\n')).toBe(true);
+  });
+
+  it('caps the hint at the 200 most recent IDs', () => {
+    const ids = Array.from({ length: 250 }, (_, i) => `topic-${i}`);
+    const hint = buildKnownIdsHint(ids);
+    // The newest IDs must appear; the oldest must not.
+    expect(hint).toContain('topic-249');
+    expect(hint).toContain('topic-50');
+    expect(hint).not.toContain('topic-49');
+    expect(hint).not.toContain('topic-0\n');
+  });
+});
--- a/src/lib/tests/llmRetry.test.js
+++ b/src/lib/tests/llmRetry.test.js
@@ -0,0 +1,72 @@
+import { describe, expect, it, vi, afterEach } from 'vitest';
+import { createLimiter, extractionLimiter } from '../llmRetry';
+
+afterEach(() => { vi.useRealTimers(); });
+
+describe('createLimiter', () => {
+  it('rejects an invalid rps', () => {
+    expect(() => createLimiter({ rps: 0 })).toThrow();
+    expect(() => createLimiter({ rps: -1 })).toThrow();
+  });
+
+  it('rejects an invalid burst', () => {
+    expect(() => createLimiter({ rps: 1, burst: 0 })).toThrow();
+  });
+
+  it('lets the first call through immediately (initial burst token)', async () => {
+    const limiter = createLimiter({ rps: 1, burst: 1 });
+    const start = Date.now();
+    await limiter.acquire();
+    expect(Date.now() - start).toBeLessThan(50);
+  });
+
+  it('queues subsequent calls to respect the spacing', async () => {
+    vi.useFakeTimers();
+    const limiter = createLimiter({ rps: 10, burst: 1 }); // 100ms spacing
+    await limiter.acquire(); // consume initial token
+
+    let resolved = false;
+    const p = limiter.acquire().then(() => { resolved = true; });
+
+    await vi.advanceTimersByTimeAsync(50);
+    expect(resolved).toBe(false);
+
+    await vi.advanceTimersByTimeAsync(100);
+    await p;
+    expect(resolved).toBe(true);
+  });
+
+  it('honours pauseUntil — no acquire returns before the pause expires', async () => {
+    vi.useFakeTimers();
+    const limiter = createLimiter({ rps: 100, burst: 5 });
+    limiter.pauseUntil(Date.now() + 1000);
+
+    let resolved = false;
+    const p = limiter.acquire().then(() => { resolved = true; });
+
+    await vi.advanceTimersByTimeAsync(500);
+    expect(resolved).toBe(false);
+
+    await vi.advanceTimersByTimeAsync(600);
+    await p;
+    expect(resolved).toBe(true);
+  });
+
+  it('aborts a queued acquire when the signal fires', async () => {
+    const limiter = createLimiter({ rps: 1, burst: 1 });
+    await limiter.acquire(); // consume
+
+    const ctl = new AbortController();
+    const p = limiter.acquire({ signal: ctl.signal });
+    ctl.abort();
+
+    await expect(p).rejects.toBeInstanceOf(DOMException);
+  });
+});
+
+describe('extractionLimiter', () => {
+  it('is exported and exposes the limiter shape', () => {
+    expect(typeof extractionLimiter.acquire).toBe('function');
+    expect(typeof extractionLimiter.pauseUntil).toBe('function');
+  });
+});
--- a/src/lib/tests/llmSchemas.test.js
+++ b/src/lib/tests/llmSchemas.test.js
@@ -108,7 +108,7 @@ describe('learning schemas', () => {
 });

 describe('quizQuestionsSchema', () => {
-  it('accepts a quiz with four options and a valid correctIndex', () => {
+  it('accepts a quiz with four options, a valid correctIndex and a difficulty', () => {
    const parsed = quizQuestionsSchema.parse({
      questions: [
        {
@@ -118,10 +118,12 @@ describe('quizQuestionsSchema', () => {
          options: ['A', 'B', 'C', 'D'],
          correctIndex: 2,
          explanation: 'C describes the buddy system best.',
+          difficulty: 'easy',
        },
      ],
    });
    expect(parsed.questions[0].options).toHaveLength(4);
+    expect(parsed.questions[0].difficulty).toBe('easy');
  });

  it('rejects three options or an out-of-range correctIndex', () => {
@@ -135,6 +137,7 @@ describe('quizQuestionsSchema', () => {
            options: ['A', 'B', 'C'],
            correctIndex: 0,
            explanation: 'e',
+            difficulty: 'medium',
          },
        ],
      }),
@@ -149,11 +152,27 @@ describe('quizQuestionsSchema', () => {
            options: ['A', 'B', 'C', 'D'],
            correctIndex: 4,
            explanation: 'e',
+            difficulty: 'medium',
          },
        ],
      }),
    ).toThrow();
  });
+
+  it('rejects a missing or unknown difficulty', () => {
+    const base = {
+      id: 'q',
+      question: 'q',
+      topicLabel: 't',
+      options: ['A', 'B', 'C', 'D'],
+      correctIndex: 0,
+      explanation: 'because',
+    };
+    expect(() => quizQuestionsSchema.parse({ questions: [base] })).toThrow();
+    expect(() =>
+      quizQuestionsSchema.parse({ questions: [{ ...base, difficulty: 'trivial' }] }),
+    ).toThrow();
+  });
 });

 describe('customTopicSchema', () => {
--- a/src/lib/tests/random.test.js
+++ b/src/lib/tests/random.test.js
@@ -0,0 +1,48 @@
+import { describe, expect, it } from 'vitest';
+import { shuffle, sample, pickInt } from '../random';
+
+describe('shuffle', () => {
+  it('returns a new array containing the same elements', () => {
+    const input = [1, 2, 3, 4, 5];
+    const out = shuffle(input);
+    expect(out).not.toBe(input);
+    expect([...out].sort()).toEqual([...input].sort());
+    expect(input).toEqual([1, 2, 3, 4, 5]);
+  });
+
+  it('handles empty and single-element arrays', () => {
+    expect(shuffle([])).toEqual([]);
+    expect(shuffle([42])).toEqual([42]);
+  });
+});
+
+describe('sample', () => {
+  it('returns up to n unique elements from the source array', () => {
+    const out = sample([1, 2, 3, 4, 5], 3);
+    expect(out).toHaveLength(3);
+    expect(new Set(out).size).toBe(3);
+    for (const v of out) expect([1, 2, 3, 4, 5]).toContain(v);
+  });
+
+  it('returns the full shuffled array when n exceeds length', () => {
+    const out = sample([1, 2, 3], 10);
+    expect(out).toHaveLength(3);
+    expect([...out].sort()).toEqual([1, 2, 3]);
+  });
+
+  it('returns an empty array when n is zero or negative', () => {
+    expect(sample([1, 2, 3], 0)).toEqual([]);
+    expect(sample([1, 2, 3], -2)).toEqual([]);
+  });
+});
+
+describe('pickInt', () => {
+  it('returns an integer in the inclusive range', () => {
+    for (let i = 0; i < 100; i++) {
+      const v = pickInt(2, 5);
+      expect(Number.isInteger(v)).toBe(true);
+      expect(v).toBeGreaterThanOrEqual(2);
+      expect(v).toBeLessThanOrEqual(5);
+    }
+  });
+});
--- a/src/lib/tests/retrieval.test.js
+++ b/src/lib/tests/retrieval.test.js
@@ -0,0 +1,58 @@
+import { describe, expect, it } from 'vitest';
+import { buildIndex, retrieveTopK, tokenize } from '../retrieval';
+
+const sampleTopics = [
+  { id: 'software-engineer', label: 'Software Engineer', description: 'Bouwt en onderhoudt applicaties; werkt in agile teams.' },
+  { id: 'onboarding-buddy', label: 'Onboarding Buddy', description: 'Begeleidt nieuwe medewerkers in hun eerste weken.' },
+  { id: 'kennisbeheer', label: 'Kennisbeheer', description: 'Het proces van het vastleggen en ontsluiten van organisatiekennis.' },
+  { id: 'wekelijkse-sessie', label: 'Wekelijkse Leersessie', description: 'Microlearning sessie waarin medewerkers wekelijks leren via AI-gegenereerde quizzen.' },
+];
+
+describe('tokenize', () => {
+  it('lowercases and splits on non-alphanumeric', () => {
+    expect(tokenize('Hello, World!')).toEqual(['hello', 'world']);
+  });
+
+  it('drops stopwords and short tokens', () => {
+    expect(tokenize('de software engineer is hier')).toEqual(['software', 'engineer', 'hier']);
+  });
+
+  it('keeps hyphenated identifiers', () => {
+    expect(tokenize('software-engineer onboarding-buddy')).toEqual(['software-engineer', 'onboarding-buddy']);
+  });
+});
+
+describe('buildIndex / retrieveTopK', () => {
+  it('returns empty for empty topics', () => {
+    const idx = buildIndex([]);
+    expect(retrieveTopK(idx, 'anything')).toEqual([]);
+  });
+
+  it('returns empty for empty query', () => {
+    const idx = buildIndex(sampleTopics);
+    expect(retrieveTopK(idx, '')).toEqual([]);
+  });
+
+  it('ranks the most relevant topic first', () => {
+    const idx = buildIndex(sampleTopics);
+    const hits = retrieveTopK(idx, 'wat doet een onboarding buddy?', 2);
+    expect(hits[0].id).toBe('onboarding-buddy');
+  });
+
+  it('matches on description when label does not contain query terms', () => {
+    const idx = buildIndex(sampleTopics);
+    const hits = retrieveTopK(idx, 'microlearning quizzen', 3);
+    expect(hits.map(h => h.id)).toContain('wekelijkse-sessie');
+  });
+
+  it('returns no hits when no terms match', () => {
+    const idx = buildIndex(sampleTopics);
+    expect(retrieveTopK(idx, 'kwantumfysica raketten')).toEqual([]);
+  });
+
+  it('caches the index per topics array reference', () => {
+    const idx1 = buildIndex(sampleTopics);
+    const idx2 = buildIndex(sampleTopics);
+    expect(idx1).toBe(idx2);
+  });
+});
--- a/src/lib/tests/testService.test.js
+++ b/src/lib/tests/testService.test.js
@@ -0,0 +1,112 @@
+import { describe, expect, it, vi, beforeEach, afterEach } from 'vitest';
+
+const bankStore = new Map();
+const callLLMMock = vi.fn();
+
+vi.mock('../pb', () => ({ pb: { collection: () => ({}) } }));
+
+vi.mock('../db', () => ({
+  getQuizBank: vi.fn(async (topicId) => bankStore.get(topicId) || []),
+  setQuizBank: vi.fn(async (topicId, qs) => { bankStore.set(topicId, qs); }),
+  getTopics: vi.fn(async () => []),
+  deleteQuestionFromBank: vi.fn(),
+  getCachedQuiz: vi.fn(),
+  setCachedQuiz: vi.fn(),
+  getQuizResult: vi.fn(),
+  saveQuizResult: vi.fn(),
+  getTeamMembers: vi.fn(async () => []),
+  upsertLeaderboardEntry: vi.fn(),
+  getCurriculum: vi.fn(),
+}));
+
+vi.mock('../llm', () => ({ callLLM: (...args) => callLLMMock(...args) }));
+vi.mock('../curriculumService', () => ({
+  getCurriculumTopic: vi.fn(async () => ({ topic: null })),
+  getQuarterForWeek: vi.fn(() => 1),
+}));
+
+import { forceGenerateTopicQuestions } from '../testService';
+
+const topic = { id: 'onboarding', label: 'Onboarding', type: 'concept', description: 'Onboarding for new joiners.' };
+
+function makeQuestion(i, overrides = {}) {
+  return {
+    id: `q-${i}`,
+    question: `Sample question ${i}?`,
+    topicLabel: 'Onboarding',
+    options: ['A) one', 'B) two', 'C) three', 'D) four'],
+    correctIndex: i % 4,
+    explanation: 'This is a substantive explanation for the correct answer.',
+    difficulty: 'medium',
+    ...overrides,
+  };
+}
+
+function llmEmits(questions) {
+  callLLMMock.mockResolvedValueOnce({ toolUses: [{ name: 'emit_quiz_questions', input: { questions } }] });
+}
+
+describe('forceGenerateTopicQuestions', () => {
+  let debugSpy, warnSpy;
+  beforeEach(() => {
+    bankStore.clear();
+    callLLMMock.mockReset();
+    debugSpy = vi.spyOn(console, 'debug').mockImplementation(() => {});
+    warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
+  });
+  afterEach(() => { debugSpy.mockRestore(); warnSpy.mockRestore(); });
+
+  it('persists a well-formed batch and assigns topic-scoped ids', async () => {
+    llmEmits([0, 1, 2, 3, 4].map((i) => makeQuestion(i)));
+    const out = await forceGenerateTopicQuestions(topic, 5);
+    expect(out).toHaveLength(5);
+    for (const q of out) expect(q.id.startsWith('onboarding-')).toBe(true);
+    expect(bankStore.get('onboarding')).toHaveLength(5);
+  });
+
+  it('re-rolls when one correctIndex dominates the batch, then accepts on the third try', async () => {
+    const allZero = [0, 1, 2, 3, 4].map((i) => makeQuestion(i, { correctIndex: 0 }));
+    llmEmits(allZero);
+    llmEmits(allZero);
+    llmEmits(allZero);
+    const out = await forceGenerateTopicQuestions(topic, 5);
+    expect(callLLMMock).toHaveBeenCalledTimes(3);
+    expect(out).toHaveLength(5);
+    expect(warnSpy).toHaveBeenCalledWith(expect.stringContaining('correctIndex dominated'));
+  });
+
+  it('rejects a batch containing a banned "all of the above" option', async () => {
+    const bad = [0, 1, 2, 3, 4].map((i) =>
+      makeQuestion(i, { options: ['A) x', 'B) y', 'C) z', 'D) All of the above'] }),
+    );
+    llmEmits(bad);
+    llmEmits(bad);
+    llmEmits(bad);
+    await expect(forceGenerateTopicQuestions(topic, 5)).rejects.toThrow(/banned filler|rejected/i);
+  });
+
+  it('rejects a batch where an explanation is too short', async () => {
+    const bad = [0, 1, 2, 3, 4].map((i) => makeQuestion(i, { explanation: 'Because.' }));
+    llmEmits(bad);
+    llmEmits(bad);
+    llmEmits(bad);
+    await expect(forceGenerateTopicQuestions(topic, 5)).rejects.toThrow(/too short|rejected/i);
+  });
+
+  it('drops duplicates whose normalized text matches an existing bank entry', async () => {
+    bankStore.set('onboarding', [
+      { ...makeQuestion(99), id: 'old-1', question: 'What is the BUDDY system???' },
+    ]);
+    llmEmits([
+      makeQuestion(0, { question: 'what is the buddy system!' }),
+      makeQuestion(1, { question: 'Brand new question one?' }),
+      makeQuestion(2, { question: 'Brand new question two?' }),
+      makeQuestion(3, { question: 'Brand new question three?' }),
+      makeQuestion(4, { question: 'Brand new question four?' }),
+    ]);
+    const out = await forceGenerateTopicQuestions(topic, 5);
+    expect(out).toHaveLength(4);
+    expect(out.find((q) => q.question.toLowerCase().includes('buddy'))).toBeUndefined();
+    expect(debugSpy).toHaveBeenCalledWith(expect.stringContaining('dropped duplicate'), expect.any(String));
+  });
+});
--- a/src/lib/db.js
+++ b/src/lib/db.js
@@ -18,6 +18,7 @@ export async function saveTopics(topics) {
      type: t.type,
      description: t.description,
      learning_relevance: t.learning_relevance || 'standard',
+      relevance_locked: t.relevance_locked === true,
    }, { requestKey: null });
  }
 }
@@ -89,10 +90,15 @@ export async function deleteContent(topicId) {

 // ── Quiz Banks ───────────────────────────────────────────────────────────────

+function normalizeQuizQuestion(q) {
+  if (!q || typeof q !== 'object') return q;
+  return q.difficulty ? q : { ...q, difficulty: 'medium' };
+}
+
 export async function getQuizBank(topicId) {
  try {
    const r = await pb.collection('quiz_banks').getFirstListItem(`topic_id="${topicId}"`);
-    return r.questions || [];
+    return (r.questions || []).map(normalizeQuizQuestion);
  } catch { return []; }
 }

@@ -321,6 +327,17 @@ export async function bulkSetCurriculum(year, weeks) {
  );
 }

+// ── LLM Call Telemetry ───────────────────────────────────────────────────────
+
+export async function getRecentLlmCalls(limit = 100) {
+  try {
+    const r = await pb.collection('llm_calls').getList(1, limit, { sort: '-created' });
+    return r.items;
+  } catch {
+    return [];
+  }
+}
+
 // ── Handbook Sync State ───────────────────────────────────────────────────────

 export async function getHandbookSyncStates() {
--- a/src/lib/extractionPipeline.js
+++ b/src/lib/extractionPipeline.js
@@ -1,8 +1,28 @@
 import * as db from './db';
 import { callLLM } from './llm';
+import { extractionLimiter } from './llmRetry';
 import { EMIT_KNOWLEDGE_GRAPH_TOOL, EMIT_HANDBOOK_DELTA_TOOL } from './llmTools';
 import { normalizeHandbookResult } from './llmSchemas';

+const MAX_KNOWN_IDS_HINT = 200;
+
+/**
+ * Build the "already-extracted topic IDs" hint that prepends every chunk
+ * after the first. Capped at the most-recent `MAX_KNOWN_IDS_HINT` IDs so
+ * the prompt stays a bounded size; the model uses this list to reuse IDs
+ * rather than invent variants like `software-developer` for
+ * `software-engineer`.
+ */
+export function buildKnownIdsHint(ids) {
+  if (!ids || !ids.length) return '';
+  const recent = ids.slice(-MAX_KNOWN_IDS_HINT);
+  return [
+    'Already-extracted topic IDs (do NOT create new IDs for these — reuse them if the same concept appears here):',
+    ...recent.map((id) => `- ${id}`),
+    '',
+  ].join('\n');
+}
+
 const EXTRACTION_SYSTEM_PROMPT = `You are an AI knowledge extractor for Respellion, an IT company built on radical transparency.
 You receive a source text. Extract every distinct concept, role, and process from it and emit them through the emit_knowledge_graph tool.

@@ -28,17 +48,17 @@ Relation types: related_to | depends_on | part_of | executed_by.
 const HANDBOOK_SYSTEM_PROMPT = `You are analysing an update to the Respellion Employee Handbook. Emit the extracted topics and relations through the emit_handbook_delta tool.

 CRITICAL INSTRUCTIONS:
- Every process must have a role attached (the role that executes it).
+- Every process must have a role attached. Express this as: process --executed_by--> role.
 - Every concept must connect to a process or role.
 - Mark handbook topics with metadata.source = "github_handbook".
 - Assign learning_relevance using the same scale as extraction: core | standard | peripheral | exclude.

-Relation types: related_to | depends_on | part_of | executed_by. (Legacy "executes" relations are normalised by the client into executed_by with source/target swapped.)
+Relation types: related_to | depends_on | part_of | executed_by.
 `;

 const cachedSystem = (text) => [{ type: 'text', text, cache_control: { type: 'ephemeral' } }];

-export async function analyzeHandbookDelta(fileContent, filePath) {
+export async function analyzeHandbookDelta(fileContent, filePath, { signal } = {}) {
  const result = await callLLM({
    task: 'extract.handbook',
    tier: 'standard',
@@ -47,6 +67,8 @@ export async function analyzeHandbookDelta(fileContent, filePath) {
    tools: [EMIT_HANDBOOK_DELTA_TOOL],
    toolChoice: { type: 'tool', name: EMIT_HANDBOOK_DELTA_TOOL.name },
    maxTokens: 8192,
+    limiter: extractionLimiter,
+    signal,
  });

  const raw = result.toolUses[0]?.input;
@@ -57,24 +79,79 @@ export async function analyzeHandbookDelta(fileContent, filePath) {
  return { success: true, data: extractedData };
 }

-function chunkText(text, maxChunkSize = 4000) {
-  const paragraphs = text.split(/\n+/);
-  const chunks = [];
-  let currentChunk = '';
+/**
+ * Sentence-aware chunker with overlap.
+ *
+ * Targets ~2000 input tokens per chunk (`MAX_CHUNK_CHARS / 4`). Splits on
+ * sentence boundaries first, then falls back to paragraph boundaries, and
+ * hard-splits inside an oversized sentence as a last resort. Adjacent chunks
+ * share `overlapChars` of trailing text to preserve cross-boundary context
+ * for the model.
+ *
+ * Exported for unit tests; callers in this module use it directly.
+ *
+ * @param {string} text
+ * @param {{ maxChars?: number, overlapChars?: number }} [opts]
+ * @returns {string[]}
+ */
+export const MAX_CHUNK_CHARS = 8000;
+export const OVERLAP_CHARS = 800;

-  for (const para of paragraphs) {
-    if ((currentChunk + '\n' + para).length > maxChunkSize) {
-      if (currentChunk) chunks.push(currentChunk.trim());
-      currentChunk = para;
-    } else {
-      currentChunk = currentChunk ? currentChunk + '\n' + para : para;
+export function chunkText(text, { maxChars = MAX_CHUNK_CHARS, overlapChars = OVERLAP_CHARS } = {}) {
+  if (typeof text !== 'string' || !text.trim()) return [];
+  const trimmed = text.trim();
+  if (trimmed.length <= maxChars) return [trimmed];
+
+  const units = splitIntoChunkableUnits(trimmed, maxChars);
+  if (units.length === 0) return [];
+
+  const chunks = [];
+  let buf = '';
+  let bufLen = 0; // length of new (non-overlap) content added since last flush
+
+  for (const unit of units) {
+    const wouldOverflow = (buf ? buf.length + 1 + unit.length : unit.length) > maxChars;
+    if (wouldOverflow && bufLen > 0) {
+      chunks.push(buf.trim());
+      const overlap = buf.length > overlapChars ? buf.slice(-overlapChars) : '';
+      buf = overlap;
+      bufLen = 0;
    }
+    // If the overlap + unit still won't fit, drop the overlap so the unit fits cleanly.
+    if (buf && (buf.length + 1 + unit.length) > maxChars) {
+      buf = '';
+    }
+    buf = buf ? buf + ' ' + unit : unit;
+    bufLen += unit.length + (bufLen > 0 ? 1 : 0);
  }
-  if (currentChunk) chunks.push(currentChunk.trim());
+  if (bufLen > 0 && buf.trim()) chunks.push(buf.trim());
  return chunks;
 }

-export async function processSourceText(textContent, sourceName) {
+function splitIntoChunkableUnits(text, maxChars) {
+  const paragraphs = text.split(/\n\s*\n+/);
+  const units = [];
+  for (const para of paragraphs) {
+    const trimmedPara = para.trim();
+    if (!trimmedPara) continue;
+    const sentences = trimmedPara.split(/(?<=[.!?])\s+/);
+    for (const s of sentences) {
+      const sentence = s.trim();
+      if (!sentence) continue;
+      if (sentence.length <= maxChars) {
+        units.push(sentence);
+      } else {
+        for (let i = 0; i < sentence.length; i += maxChars) {
+          units.push(sentence.slice(i, i + maxChars));
+        }
+        console.warn(`[chunkText] Hard-split a sentence of ${sentence.length} chars (exceeds maxChars=${maxChars}).`);
+      }
+    }
+  }
+  return units;
+}
+
+export async function processSourceText(textContent, sourceName, { signal } = {}) {
  const existing = await db.getSources();
  const alreadyDone = existing.find(
    s => s.name === sourceName && s.status === 'completed'
@@ -87,36 +164,42 @@ export async function processSourceText(textContent, sourceName) {
  const sourceId = rec.id;

  try {
-    const chunks = chunkText(textContent, 4000);
+    const chunks = chunkText(textContent);
    console.log(`[Pipeline] Split "${sourceName}" into ${chunks.length} chunks for processing.`);

-    let allExtractedTopics = [];
-    let allExtractedRelations = [];
+    const existingTopics = await db.getTopics();
+    const knownIds = existingTopics.map((t) => t.id);
+
+    const allExtractedTopics = [];
+    const allExtractedRelations = [];

    for (let i = 0; i < chunks.length; i++) {
-      if (i > 0) {
-        console.log(`[Pipeline] Pacing delay (12s) to prevent rate limits before chunk ${i + 1}/${chunks.length}...`);
-        await new Promise(r => setTimeout(r, 12000));
-      }
+      if (signal?.aborted) throw signal.reason ?? new DOMException('Aborted', 'AbortError');

      console.log(`[Pipeline] Processing chunk ${i + 1}/${chunks.length} (${chunks[i].length} chars)...`);
+      const hint = i > 0 ? buildKnownIdsHint(knownIds) : '';
      const result = await callLLM({
        task: 'extract.source',
        tier: 'standard',
        system: cachedSystem(EXTRACTION_SYSTEM_PROMPT),
-        user: `Analyze this part of the document (${i + 1}/${chunks.length}):\n\n${chunks[i]}`,
+        user: `${hint}Analyze this part of the document (${i + 1}/${chunks.length}):\n\n${chunks[i]}`,
        tools: [EMIT_KNOWLEDGE_GRAPH_TOOL],
        toolChoice: { type: 'tool', name: EMIT_KNOWLEDGE_GRAPH_TOOL.name },
        maxTokens: 8192,
+        limiter: extractionLimiter,
+        signal,
      });

      const extractedData = result.toolUses[0]?.input;
      if (!extractedData) throw new Error(`Extraction did not emit a tool result for chunk ${i + 1}.`);

-      if (extractedData.topics && Array.isArray(extractedData.topics)) {
+      if (Array.isArray(extractedData.topics)) {
        allExtractedTopics.push(...extractedData.topics);
+        for (const t of extractedData.topics) {
+          if (t?.id && !knownIds.includes(t.id)) knownIds.push(t.id);
+        }
      }
-      if (extractedData.relations && Array.isArray(extractedData.relations)) {
+      if (Array.isArray(extractedData.relations)) {
        allExtractedRelations.push(...extractedData.relations);
      }
    }
@@ -127,7 +210,8 @@ export async function processSourceText(textContent, sourceName) {
    return { success: true, data: { topics: allExtractedTopics, relations: allExtractedRelations } };

  } catch (error) {
-    await db.updateSourceStatus(sourceId, 'failed', error.message);
+    const isAbort = error?.name === 'AbortError';
+    await db.updateSourceStatus(sourceId, isAbort ? 'cancelled' : 'failed', isAbort ? 'cancelled by user' : error.message);
    throw error;
  }
 }
@@ -142,11 +226,16 @@ async function mergeKnowledgeGraph(newData) {
    for (const t of newData.topics) {
      if (topicsMap.has(t.id)) {
        const existing = topicsMap.get(t.id);
-        topicsMap.set(t.id, {
+        const merged = {
          ...existing,
          ...t,
          description: t.description || existing.description,
-        });
+        };
+        if (existing.relevance_locked) {
+          merged.learning_relevance = existing.learning_relevance;
+          merged.relevance_locked = true;
+        }
+        topicsMap.set(t.id, merged);
      } else {
        topicsMap.set(t.id, t);
      }
--- a/src/lib/learningService.js
+++ b/src/lib/learningService.js
@@ -154,6 +154,28 @@ export async function deleteCachedContent(topicId) {
  return db.deleteContent(topicId);
 }

+function slugify(label) {
+  const base = String(label || '')
+    .toLowerCase()
+    .normalize('NFKD')
+    .replace(/\p{Diacritic}/gu, '')
+    .replace(/[^a-z0-9]+/g, '-')
+    .replace(/^-+|-+$/g, '');
+  return base || 'topic';
+}
+
+async function pickUniqueTopicId(label) {
+  const existing = await db.getTopics();
+  const used = new Set(existing.map((t) => t.id));
+  const base = slugify(label);
+  if (!used.has(base)) return base;
+  for (let i = 2; i < 1000; i++) {
+    const candidate = `${base}-${i}`;
+    if (!used.has(candidate)) return candidate;
+  }
+  return `${base}-${Date.now().toString(36)}`;
+}
+
 export async function generateCustomTopic(label) {
  const result = await callLLM({
    task: 'topic.custom',
@@ -168,7 +190,12 @@ export async function generateCustomTopic(label) {
  const emitted = result.toolUses[0]?.input;
  if (!emitted) throw new Error('Could not process custom topic. Please try again.');

-  const newTopic = { ...emitted, id: 'custom_' + Date.now().toString(36) };
+  const id = await pickUniqueTopicId(emitted.label);
+  const newTopic = {
+    ...emitted,
+    id,
+    learning_relevance: emitted.learning_relevance || 'standard',
+  };
  await db.upsertTopic(newTopic);
  return newTopic;
 }
--- a/src/lib/llm.js
+++ b/src/lib/llm.js
@@ -236,7 +236,7 @@ function extractToolUses(content) {
  if (!Array.isArray(content)) return [];
  return content
    .filter((b) => b?.type === 'tool_use')
-    .map((b) => ({ name: b.name, input: b.input }));
+    .map((b) => ({ id: b.id, name: b.name, input: b.input }));
 }

 function extractText(content) {
@@ -272,6 +272,7 @@ function validateToolInputs(toolUses, task, toolSchemas) {
 * @property {number} [maxTokens=4096]
 * @property {number} [temperature=0]
 * @property {AbortSignal} [signal]
+ * @property {{ acquire: (opts?:{signal?:AbortSignal}) => Promise<void>, pauseUntil: (untilMs:number) => void }} [limiter]
 */

 /**
@@ -291,6 +292,7 @@ export async function callLLM(options) {
    maxTokens = 4096,
    temperature = 0,
    signal,
+    limiter,
  } = options;
  if (!task) throw new Error('callLLM requires a `task` label.');

@@ -318,6 +320,7 @@ export async function callLLM(options) {
  try {
    result = await withRetry(
      async () => {
+        if (limiter) await limiter.acquire({ signal });
        const timeoutCtl = signal ? null : new AbortController();
        const timer = timeoutCtl ? setTimeout(() => timeoutCtl.abort(new DOMException('Timeout', 'AbortError')), DEFAULT_TIMEOUT_MS) : null;
        const fetchSignal = linkSignals(signal, timeoutCtl?.signal);
@@ -337,6 +340,9 @@ export async function callLLM(options) {
            const errBody = await response.json().catch(() => ({}));
            if (isRetryableStatus(response.status)) {
              const retryAfterMs = parseRetryAfter(response.headers.get('Retry-After'));
+              if (response.status === 429 && retryAfterMs != null && limiter) {
+                limiter.pauseUntil(Date.now() + retryAfterMs);
+              }
              throw new RetryableError(response.status, retryAfterMs, `HTTP ${response.status}`);
            }
            throw new LLMHttpError(response.status, response.statusText, errBody);
--- a/src/lib/llmRetry.js
+++ b/src/lib/llmRetry.js
@@ -62,6 +62,109 @@ function sleep(ms, signal) {
  });
 }

+/**
+ * Token-bucket rate limiter shared by callers that hit the same upstream
+ * quota (e.g. handbook extraction loops). Replaces the static
+ * `setTimeout(..., 12000)` / 15s sleeps that Phase 1 relied on. The bucket
+ * refills continuously; `acquire` resolves either immediately (token
+ * available) or after the next refill tick.
+ *
+ * `rps` is "requests per second" (use fractional values for per-minute
+ * limits: `5/60` for 5 req/min). `burst` is the maximum number of tokens
+ * the bucket can hold; default 1 means strict spacing.
+ *
+ * Call `pauseUntil(timestampMs)` after a 429 with a `Retry-After` hint —
+ * no acquire returns before that timestamp.
+ *
+ * @param {{ rps?: number, burst?: number }} [opts]
+ */
+export function createLimiter({ rps = 1, burst = 1 } = {}) {
+  if (rps <= 0) throw new Error('createLimiter: rps must be > 0');
+  if (burst < 1) throw new Error('createLimiter: burst must be >= 1');
+  const intervalMs = 1000 / rps;
+  let tokens = burst;
+  let lastRefill = Date.now();
+  let pausedUntil = 0;
+  const waiters = [];
+
+  function refill(now) {
+    const elapsed = now - lastRefill;
+    if (elapsed <= 0) return;
+    const earned = elapsed / intervalMs;
+    if (earned >= 1) {
+      tokens = Math.min(burst, tokens + Math.floor(earned));
+      lastRefill = now;
+    }
+  }
+
+  function drain() {
+    while (waiters.length) {
+      const now = Date.now();
+      if (now < pausedUntil) {
+        scheduleWake(pausedUntil - now);
+        return;
+      }
+      refill(now);
+      if (tokens >= 1) {
+        tokens -= 1;
+        const w = waiters.shift();
+        w.resolve();
+      } else {
+        const wait = Math.max(intervalMs - (now - lastRefill), 0);
+        scheduleWake(wait);
+        return;
+      }
+    }
+  }
+
+  let wakeTimer = null;
+  function scheduleWake(ms) {
+    if (wakeTimer) return;
+    wakeTimer = setTimeout(() => {
+      wakeTimer = null;
+      drain();
+    }, ms);
+  }
+
+  return {
+    /** @param {{signal?:AbortSignal}} [opts] */
+    async acquire({ signal } = {}) {
+      if (signal?.aborted) throw signal.reason ?? new DOMException('Aborted', 'AbortError');
+      return new Promise((resolve, reject) => {
+        const entry = { resolve, reject };
+        const onAbort = () => {
+          const i = waiters.indexOf(entry);
+          if (i !== -1) waiters.splice(i, 1);
+          reject(signal.reason ?? new DOMException('Aborted', 'AbortError'));
+        };
+        if (signal) signal.addEventListener('abort', onAbort, { once: true });
+        const wrapped = {
+          resolve: () => { if (signal) signal.removeEventListener('abort', onAbort); resolve(); },
+          reject,
+        };
+        waiters.push(wrapped);
+        drain();
+      });
+    },
+    /** Block all `acquire`s until `untilMs` (epoch milliseconds). */
+    pauseUntil(untilMs) {
+      if (untilMs > pausedUntil) pausedUntil = untilMs;
+      drain();
+    },
+    /** Inspect state — primarily for tests. */
+    _state() {
+      return { tokens, pausedUntil, waiters: waiters.length };
+    },
+  };
+}
+
+/**
+ * Shared limiter for the multi-call extraction loops (source chunks,
+ * handbook file sync). 5 requests/minute matches the lowest published
+ * Anthropic tier so we stay well clear of 429.
+ */
+export const extractionLimiter = createLimiter({ rps: 5 / 60, burst: 1 });
+
 /**
 * Run `fn(attempt)` with retry. `fn` may throw a `RetryableError` to request
 * a retry, or any other error to fail immediately.
--- a/src/lib/llmSchemas.js
+++ b/src/lib/llmSchemas.js
@@ -122,6 +122,8 @@ export const learningAllSchema = z.object({
  infographic: infographicBodySchema,
 });

+const quizDifficultyEnum = z.enum(['easy', 'medium', 'hard']);
+
 const quizQuestionSchema = z.object({
  id: z.string().min(1),
  question: z.string().min(1),
@@ -129,6 +131,7 @@ const quizQuestionSchema = z.object({
  options: z.array(z.string().min(1)).length(4),
  correctIndex: z.number().int().min(0).max(3),
  explanation: z.string().min(1),
+  difficulty: quizDifficultyEnum,
 });

 export const quizQuestionsSchema = z.object({
--- a/src/lib/llmTools.js
+++ b/src/lib/llmTools.js
@@ -205,6 +205,8 @@ export const EMIT_CUSTOM_TOPIC_TOOL = {
  },
 };

+const QUIZ_DIFFICULTIES = ['easy', 'medium', 'hard'];
+
 const quizQuestionSchema = {
  type: 'object',
  properties: {
@@ -214,8 +216,9 @@ const quizQuestionSchema = {
    options: { type: 'array', items: { type: 'string' }, minItems: 4, maxItems: 4 },
    correctIndex: { type: 'integer', minimum: 0, maximum: 3 },
    explanation: { type: 'string', description: 'Why the correct answer is correct (1–2 sentences).' },
+    difficulty: { type: 'string', enum: QUIZ_DIFFICULTIES, description: 'Per-question difficulty tag.' },
  },
-  required: ['id', 'question', 'topicLabel', 'options', 'correctIndex', 'explanation'],
+  required: ['id', 'question', 'topicLabel', 'options', 'correctIndex', 'explanation', 'difficulty'],
 };

 export const EMIT_QUIZ_QUESTIONS_TOOL = {
--- a/src/lib/random.js
+++ b/src/lib/random.js
@@ -0,0 +1,29 @@
+/**
+ * Shared randomness helpers.
+ *
+ * `Array.prototype.sort(() => 0.5 - Math.random())` is biased — modern V8
+ * sorts use Timsort, which compares each element more than once and skews
+ * the resulting permutation. Use `shuffle` for anything user-visible
+ * (quiz options, review topic selection, leaderboards).
+ */
+
+export function shuffle(arr) {
+  const out = [...arr];
+  for (let i = out.length - 1; i > 0; i--) {
+    const j = Math.floor(Math.random() * (i + 1));
+    [out[i], out[j]] = [out[j], out[i]];
+  }
+  return out;
+}
+
+export function sample(arr, n) {
+  if (n <= 0) return [];
+  if (n >= arr.length) return shuffle(arr);
+  return shuffle(arr).slice(0, n);
+}
+
+export function pickInt(min, maxInclusive) {
+  const lo = Math.ceil(min);
+  const hi = Math.floor(maxInclusive);
+  return lo + Math.floor(Math.random() * (hi - lo + 1));
+}
--- a/src/lib/retrieval.js
+++ b/src/lib/retrieval.js
@@ -0,0 +1,95 @@
+/**
+ * Lightweight, dependency-free TF-IDF retrieval over the knowledge graph.
+ *
+ * `buildIndex(topics)` tokenises the `label + description` of each topic and
+ * computes document-frequency stats so queries can be scored with TF-IDF in
+ * `retrieveTopK`. The index is cached against the `topics` array reference,
+ * so repeated calls with the same array don't rebuild.
+ *
+ * Tokeniser: lowercase, split on `[^a-zA-Z0-9-]`, drop short tokens and a
+ * small Dutch/English stopword list.
+ */
+
+const STOPWORDS = new Set([
+  // English
+  'a', 'an', 'and', 'are', 'as', 'at', 'be', 'by', 'for', 'from', 'has', 'have',
+  'how', 'i', 'in', 'is', 'it', 'its', 'of', 'on', 'or', 'than', 'that', 'the',
+  'this', 'to', 'was', 'were', 'what', 'when', 'where', 'which', 'who', 'why',
+  'with', 'you', 'your', 'do', 'does', 'did',
+  // Dutch
+  'de', 'het', 'een', 'en', 'of', 'in', 'op', 'aan', 'bij', 'voor', 'naar',
+  'met', 'uit', 'om', 'door', 'over', 'tegen', 'ook', 'er', 'is', 'zijn',
+  'was', 'waren', 'wat', 'wie', 'hoe', 'waar', 'wanneer', 'welke', 'die',
+  'dat', 'deze', 'dit', 'ik', 'jij', 'hij', 'zij', 'we', 'wij', 'jullie',
+  'als', 'dan', 'maar', 'want', 'omdat', 'niet', 'wel', 'heeft', 'hebben',
+  'word', 'wordt', 'worden', 'kan', 'kunnen', 'mag', 'moet', 'moeten',
+  'zal', 'zou', 'zouden', 'al', 'ook', 'nog', 'naar',
+]);
+
+export function tokenize(text) {
+  if (!text) return [];
+  return String(text)
+    .toLowerCase()
+    .split(/[^a-z0-9-]+/i)
+    .filter(t => t.length >= 2 && !STOPWORDS.has(t));
+}
+
+const indexCache = new WeakMap();
+
+export function buildIndex(topics) {
+  if (!Array.isArray(topics) || topics.length === 0) {
+    return { topics: [], docFreq: new Map(), termsByDoc: [], N: 0 };
+  }
+  const cached = indexCache.get(topics);
+  if (cached) return cached;
+
+  const termsByDoc = topics.map(t => {
+    const text = `${t.label || ''} ${t.description || ''}`;
+    const tokens = tokenize(text);
+    const tf = new Map();
+    for (const tk of tokens) tf.set(tk, (tf.get(tk) || 0) + 1);
+    return tf;
+  });
+
+  const docFreq = new Map();
+  for (const tf of termsByDoc) {
+    for (const term of tf.keys()) {
+      docFreq.set(term, (docFreq.get(term) || 0) + 1);
+    }
+  }
+
+  const index = { topics, docFreq, termsByDoc, N: topics.length };
+  indexCache.set(topics, index);
+  return index;
+}
+
+export function retrieveTopK(index, query, k = 10) {
+  if (!index || !index.N || !query) return [];
+  const qTokens = tokenize(query);
+  if (qTokens.length === 0) return [];
+
+  const idf = (term) => {
+    const df = index.docFreq.get(term) || 0;
+    if (df === 0) return 0;
+    return Math.log((index.N + 1) / (df + 1)) + 1;
+  };
+
+  const scores = new Array(index.N);
+  for (let i = 0; i < index.N; i++) {
+    const tf = index.termsByDoc[i];
+    let s = 0;
+    for (const t of qTokens) {
+      const f = tf.get(t);
+      if (!f) continue;
+      s += (1 + Math.log(f)) * idf(t);
+    }
+    scores[i] = s;
+  }
+
+  const ranked = [];
+  for (let i = 0; i < index.N; i++) {
+    if (scores[i] > 0) ranked.push({ i, s: scores[i] });
+  }
+  ranked.sort((a, b) => b.s - a.s);
+  return ranked.slice(0, k).map(r => index.topics[r.i]);
+}
--- a/src/lib/testService.js
+++ b/src/lib/testService.js
@@ -2,81 +2,73 @@ import * as db from './db';
 import { callLLM } from './llm';
 import { EMIT_QUIZ_QUESTIONS_TOOL } from './llmTools';
 import { getCurriculumTopic, getQuarterForWeek } from './curriculumService';
+import { shuffle, sample } from './random';

 const QUIZ_SYSTEM = `You are a quiz generator for Respellion, an internal IT company learning platform.
 You generate multiple-choice questions to test employee knowledge on specific topics.
 Always write in clear, professional English.

-Emit questions through the emit_quiz_questions tool. Each question has exactly four options; correctIndex is 0-based; mix difficulty roughly 4 easy / 4 medium / 2 hard.`;
+Emit questions through the emit_quiz_questions tool. Each question has exactly four options; correctIndex is 0-based. Tag every question with difficulty ('easy', 'medium' or 'hard'). For a typical 5-question batch, mix difficulty roughly 2 easy / 2 medium / 1 hard; scale the ratio proportionally for larger batches.
+
+Distribute correctIndex roughly evenly across 0, 1, 2, and 3. Do not place the correct answer at the same position more than 4 out of 10 times.
+
+Never use filler options such as "all of the above", "none of the above", or "both A and B". Every explanation must be a substantive sentence (≥ 20 characters) describing why the correct answer is correct.`;
+
+const BANNED_OPTION_PATTERNS = [
+  /all of the above/i,
+  /none of the above/i,
+  /both a and b/i,
+  /both b and c/i,
+  /both c and d/i,
+  /both a and c/i,
+  /both b and d/i,
+  /both a and d/i,
+];

 const cachedSystem = (text) => [{ type: 'text', text, cache_control: { type: 'ephemeral' } }];

-async function selectTestTopics(userId, weekNumber) {
-  const allTopics = await db.getTopics();
-  const topics = allTopics.filter(t => t.type !== 'fact' && t.learning_relevance !== 'exclude');
-  if (!topics || topics.length === 0) return { primaryTopic: null, reviewTopics: [], isReviewWeek: false };
-
-  // Try curriculum-based selection first
-  try {
-    const { topic, curriculumEntry } = await getCurriculumTopic(weekNumber);
-
-    if (curriculumEntry?.is_review_week) {
-      // Review week: pull topics from the whole quarter
-      const quarter = getQuarterForWeek(weekNumber);
-      const curriculum = await db.getCurriculum(new Date().getFullYear());
-      const quarterTopicIds = curriculum
-        .filter(w => w.quarter === quarter && w.topic_id && !w.is_review_week)
-        .map(w => w.topic_id);
-      const quarterTopics = topics.filter(t => quarterTopicIds.includes(t.id));
-      // Use all quarter topics as review topics (no single primary)
-      return {
-        primaryTopic: quarterTopics[0] || topics[0],
-        reviewTopics: quarterTopics.slice(1),
-        isReviewWeek: true,
-      };
-    }
-
-    if (topic) {
-      const others = topics.filter(t => t.id !== topic.id);
-      const shuffled = others.sort(() => 0.5 - Math.random());
-      const reviewTopics = shuffled.slice(0, Math.min(5, shuffled.length));
-      return { primaryTopic: topic, reviewTopics, isReviewWeek: false };
-    }
-  } catch (e) {
-    console.warn('[Test] Curriculum lookup failed, falling back to hash:', e.message);
-  }
-
-  // Fallback: hash-based selection
-  const str = `${userId}:${weekNumber}`;
-  let hash = 0;
-  for (let i = 0; i < str.length; i++) {
-    hash = (hash << 5) - hash + str.charCodeAt(i);
-    hash |= 0;
-  }
-  const primaryIndex = Math.abs(hash) % topics.length;
-  const primaryTopic = topics[primaryIndex];
-
-  const others = topics.filter((_, i) => i !== primaryIndex);
-  const shuffled = others.sort(() => 0.5 - Math.random());
-  const reviewTopics = shuffled.slice(0, Math.min(5, shuffled.length));
-
-  return { primaryTopic, reviewTopics, isReviewWeek: false };
+function normalizeQuestionText(text) {
+  return String(text || '')
+    .toLowerCase()
+    .replace(/[\p{P}\p{S}]/gu, ' ')
+    .replace(/\s+/g, ' ')
+    .trim();
 }

-export async function getCachedQuiz(userId, weekNumber) {
-  return db.getCachedQuiz(userId, weekNumber);
+function dominantCorrectIndex(questions) {
+  if (!questions.length) return null;
+  const counts = [0, 0, 0, 0];
+  for (const q of questions) counts[q.correctIndex] = (counts[q.correctIndex] || 0) + 1;
+  const max = Math.max(...counts);
+  return max / questions.length > 0.5 ? { index: counts.indexOf(max), ratio: max / questions.length } : null;
 }

-export async function forceGenerateTopicQuestions(topic, count = 10) {
-  let bank = await db.getQuizBank(topic.id);
+function validateBatchQuality(questions) {
+  for (const q of questions) {
+    const distinct = new Set(q.options.map((o) => o.trim().toLowerCase()));
+    if (distinct.size < 4) {
+      return `Question "${q.question}" has duplicate options.`;
+    }
+    for (const opt of q.options) {
+      if (BANNED_OPTION_PATTERNS.some((re) => re.test(opt))) {
+        return `Question "${q.question}" uses a banned filler option ("${opt}").`;
+      }
+    }
+    if (!q.explanation || q.explanation.trim().length < 20) {
+      return `Question "${q.question}" has an explanation that is too short.`;
+    }
+  }
+  return null;
+}

+async function callQuizModel(topic, count) {
  const prompt = `Generate exactly ${count} multiple-choice quiz questions for this knowledge topic and emit them via the emit_quiz_questions tool:

 Topic: ${topic.label}
 Type: ${topic.type}
 Description: ${topic.description}

-Options must be prefixed "A) ", "B) ", "C) ", "D) ". Make questions specific and practical, not trivial.`;
+Options must be prefixed "A) ", "B) ", "C) ", "D) ". Make questions specific and practical, not trivial. Example: a question whose correct answer is option C uses "correctIndex": 2.`;

  const result = await callLLM({
    task: 'quiz.generate',
@@ -89,28 +81,125 @@ Options must be prefixed "A) ", "B) ", "C) ", "D) ". Make questions specific and
  });

  const emitted = result.toolUses[0]?.input;
-  if (!emitted) throw new Error(`Could not generate questions for ${topic.label}`);
+  if (!emitted?.questions?.length) {
+    throw new Error(`Could not generate questions for ${topic.label}`);
+  }
+  return emitted.questions;
+}

-  const newQuestions = (emitted.questions || []).map(q => ({
-    ...q,
-    id: `${topic.id}-${Math.random().toString(36).slice(2, 11)}`,
-  }));
+async function selectTestTopics(userId, weekNumber) {
+  const allTopics = await db.getTopics();
+  const topics = allTopics.filter(t => t.type !== 'fact' && t.learning_relevance !== 'exclude');
+  if (!topics || topics.length === 0) return { primaryTopic: null, reviewTopics: [], isReviewWeek: false };

-  bank = [...bank, ...newQuestions];
-  await db.setQuizBank(topic.id, bank);
-  return newQuestions;
+  try {
+    const { topic, curriculumEntry } = await getCurriculumTopic(weekNumber);
+
+    if (curriculumEntry?.is_review_week) {
+      const quarter = getQuarterForWeek(weekNumber);
+      const curriculum = await db.getCurriculum(new Date().getFullYear());
+      const quarterTopicIds = curriculum
+        .filter(w => w.quarter === quarter && w.topic_id && !w.is_review_week)
+        .map(w => w.topic_id);
+      const quarterTopics = topics.filter(t => quarterTopicIds.includes(t.id));
+      return {
+        primaryTopic: quarterTopics[0] || topics[0],
+        reviewTopics: quarterTopics.slice(1),
+        isReviewWeek: true,
+      };
+    }
+
+    if (topic) {
+      const others = topics.filter(t => t.id !== topic.id);
+      const reviewTopics = sample(others, Math.min(5, others.length));
+      return { primaryTopic: topic, reviewTopics, isReviewWeek: false };
+    }
+  } catch (e) {
+    console.warn('[Test] Curriculum lookup failed, falling back to hash:', e.message);
+  }
+
+  const str = `${userId}:${weekNumber}`;
+  let hash = 0;
+  for (let i = 0; i < str.length; i++) {
+    hash = (hash << 5) - hash + str.charCodeAt(i);
+    hash |= 0;
+  }
+  const primaryIndex = Math.abs(hash) % topics.length;
+  const primaryTopic = topics[primaryIndex];
+
+  const others = topics.filter((_, i) => i !== primaryIndex);
+  const reviewTopics = sample(others, Math.min(5, others.length));
+
+  return { primaryTopic, reviewTopics, isReviewWeek: false };
+}
+
+export async function getCachedQuiz(userId, weekNumber) {
+  return db.getCachedQuiz(userId, weekNumber);
+}
+
+export async function forceGenerateTopicQuestions(topic, count = 5) {
+  const existingBank = await db.getQuizBank(topic.id);
+  const existingKeys = new Set(existingBank.map((q) => normalizeQuestionText(q.question)));
+
+  let lastQualityError = null;
+  let candidates = null;
+
+  for (let attempt = 0; attempt < 3; attempt++) {
+    const questions = await callQuizModel(topic, count);
+
+    const qualityError = validateBatchQuality(questions);
+    if (qualityError) {
+      lastQualityError = qualityError;
+      console.warn(`[quiz] batch rejected (attempt ${attempt + 1}): ${qualityError}`);
+      continue;
+    }
+
+    const dominant = dominantCorrectIndex(questions);
+    if (dominant && attempt < 2) {
+      console.warn(`[quiz] correctIndex dominated by ${dominant.index} (${Math.round(dominant.ratio * 100)}%) — re-rolling`);
+      continue;
+    }
+
+    candidates = questions;
+    break;
+  }
+
+  if (!candidates) {
+    throw new Error(`Quality gate rejected the generated batch for ${topic.label}: ${lastQualityError || 'unbalanced answer distribution'}. Click "Generate" to try again.`);
+  }
+
+  const accepted = [];
+  for (const q of candidates) {
+    const key = normalizeQuestionText(q.question);
+    if (existingKeys.has(key)) {
+      console.debug('[quiz] dropped duplicate:', q.question);
+      continue;
+    }
+    existingKeys.add(key);
+    accepted.push({
+      ...q,
+      id: `${topic.id}-${Math.random().toString(36).slice(2, 11)}`,
+    });
+  }
+
+  if (!accepted.length) {
+    throw new Error(`All generated questions for ${topic.label} were duplicates of existing ones.`);
+  }
+
+  const merged = [...existingBank, ...accepted];
+  await db.setQuizBank(topic.id, merged);
+  return accepted;
 }

 async function getOrGenerateTopicQuestions(topic, count) {
  let bank = await db.getQuizBank(topic.id);

  if (bank.length < count) {
-    await forceGenerateTopicQuestions(topic, 10);
+    await forceGenerateTopicQuestions(topic, 5);
    bank = await db.getQuizBank(topic.id);
  }

-  const shuffled = [...bank].sort(() => 0.5 - Math.random());
-  return shuffled.slice(0, Math.min(count, shuffled.length));
+  return sample(bank, Math.min(count, bank.length));
 }

 export async function getTopicQuestionBank(topicId) {
@@ -151,10 +240,10 @@ export async function generateWeeklyQuiz(userId, weekNumber, force = false) {
    }
  }

-  questions.sort(() => 0.5 - Math.random());
+  const shuffled = shuffle(questions);

  const quiz = {
-    questions,
+    questions: shuffled,
    meta: {
      userId,
      weekNumber,
--- a/src/pages/Admin/index.jsx
+++ b/src/pages/Admin/index.jsx
@@ -1,5 +1,5 @@
 import { useState, useEffect } from 'react';
-import { Database, FileText, Settings, Users, Network, Clock, CheckCircle2, AlertCircle, Save, Info, Layers, CheckSquare, CalendarDays } from 'lucide-react';
+import { Database, FileText, Settings, Users, Network, Clock, CheckCircle2, AlertCircle, Save, Info, Layers, CheckSquare, CalendarDays, Activity } from 'lucide-react';
 import Card from '../../components/ui/Card';
 import Tag from '../../components/ui/Tag';
 import Button from '../../components/ui/Button';
@@ -12,6 +12,7 @@ import ContentManager from '../../components/admin/ContentManager';
 import TestManager from '../../components/admin/TestManager';
 import TeamManager from '../../components/admin/TeamManager';
 import CurriculumManager from '../../components/admin/CurriculumManager';
+import Diagnostics from '../../components/admin/Diagnostics';
 import { Trash2 } from 'lucide-react';

 const TIER_PLACEHOLDERS = {
@@ -71,6 +72,7 @@ const Admin = () => {
    { key: 'curriculum', icon: CalendarDays, label: 'Curriculum' },
    { key: 'graph', icon: Network, label: 'Graph' },
    { key: 'team', icon: Users, label: 'Team' },
+    { key: 'diagnostics', icon: Activity, label: 'Diagnostics' },
    { key: 'settings', icon: Settings, label: 'Settings', bottom: true },
  ];

@@ -181,6 +183,14 @@ const Admin = () => {
          </div>
        )}

+        {activeTab === 'diagnostics' && (
+          <div className="animate-in fade-in duration-300 max-w-6xl mx-auto">
+            <h1 className="text-3xl text-teal mb-2">Diagnostics</h1>
+            <p className="text-fg-muted mb-8">LLM-aanroepen, tokenverbruik en geschatte kosten.</p>
+            <Diagnostics />
+          </div>
+        )}
+
        {activeTab === 'settings' && (
          <div className="animate-in fade-in duration-300 max-w-2xl">
            <h1 className="text-3xl text-teal mb-2">Settings</h1>
Author	SHA1	Message	Date
RaymondVerhoef	229246f7b6	feat: phase 5 of AI pipeline hardening — R42 retrieval & telemetry All checks were successful On Pull Request to Main / test (pull_request) Successful in 32s Details On Pull Request to Main / publish (pull_request) Successful in 57s Details On Pull Request to Main / deploy-dev (pull_request) Successful in 1m34s Details - Add dependency-free TF-IDF retrieval (src/lib/retrieval.js) with NL+EN stopwords and a WeakMap-cached index. - Rewrite buildKbContext to ship the top-K relevant topics + verbatim- mentioned ids only, filter relations to the included set, and append a [kb_hash: <8 hex>] suffix so the ephemeral prompt cache busts when the graph changes. Returns { context, retrievedTopics, allTopics }. - Add LOOKUP_TOPIC_TOOL and drive useChat through callLLM directly with a multi-hop tool_result loop capped at 3 hops; preserve Anthropic-provided tool_use ids through callLLM so the loop can echo correct tool_use_id. - Truncate R42 history to the last 12 turns and prepend a single "(earlier conversation truncated)" assistant message. - Set R42 chat defaults: temperature 0.3, maxTokens 2048. - Add pb_migrations/1780500002_created_llm_calls.js (the best-effort logger in callLLM was already wired) and a new Admin → Diagnostics view showing the last 100 calls with token usage, cache-hit rate, and USD cost from a local Anthropic price table. - Finalize AI_PIPELINE_HARDENING_PLAN.md: mark Phases 1–5 shipped and Phase 6 (eval harness) explicitly out of scope. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-20 21:36:40 +02:00
RaymondVerhoef	66e0c275da	feat: phase 4 of AI pipeline hardening — quiz & content quality All checks were successful On Pull Request to Main / test (pull_request) Successful in 31s Details On Pull Request to Main / publish (pull_request) Successful in 1m1s Details On Pull Request to Main / deploy-dev (pull_request) Successful in 1m31s Details - src/lib/random.js: Fisher–Yates shuffle/sample/pickInt; replace every biased .sort(() => 0.5 - Math.random()) site in testService. - testService: debias correctIndex via prompt + runtime re-roll (up to 2x when one position holds >50%); quality gate rejecting <4 distinct options, banned filler ("all of the above" etc) and explanations shorter than 20 chars; dedup new questions against the existing bank via normalised question text. - Quiz schema/tool/prompt require difficulty ('easy'\|'medium'\|'hard'); db.getQuizBank defaults legacy records to 'medium' on read. - learningService.generateCustomTopic: kebab-case slug ID from the polished label with collision suffixes; default learning_relevance 'standard' when the model omits it. - Tests for random helpers, dedup/quality-gate behaviour and the extended quiz schema. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-20 19:22:10 +02:00
RaymondVerhoef	c82e4fc3a1	feat: reduce initial question batch size for a topic to 5 All checks were successful On Push to Main / test (push) Successful in 32s Details On Push to Main / publish (push) Successful in 1m0s Details On Push to Main / deploy-dev (push) Successful in 1m33s Details When a topic's quiz bank is empty (or below the requested count), we previously seeded it with a fresh batch of 10 questions. That meant the first weekly quiz for any new topic triggered a 10-question LLM call — heavy for what's ultimately a 1-question sample for review topics, and overkill for the typical 5-question primary topic. - forceGenerateTopicQuestions default count: 10 → 5 - getOrGenerateTopicQuestions seed amount: 10 → 5 - TestManager "Generate" defaults + empty-state button copy: 10 → 5 - QUIZ_SYSTEM difficulty hint: rewritten for a 5-question batch (2 easy / 2 medium / 1 hard) with explicit "scale proportionally for larger batches" so admins can still generate 10+ via TestManager when they want more depth. Tests 61/61 pass, lint clean (0 errors), build clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-20 19:12:16 +02:00
Raymond Verhoef	fd3b849c19	Merge pull request 'feat: phase 3 of AI pipeline hardening — extraction quality' (#5 ) from feat/ai-pipeline-hardening-phase-3 into main All checks were successful On Push to Main / test (push) Successful in 25s Details On Push to Main / publish (push) Successful in 1m10s Details On Push to Main / deploy-dev (push) Successful in 1m31s Details Reviewed-on: #5	2026-05-20 15:57:40 +00:00
RaymondVerhoef	aeb197d5f4	feat: phase 3 of AI pipeline hardening — extraction quality All checks were successful On Pull Request to Main / test (pull_request) Successful in 31s Details On Pull Request to Main / publish (pull_request) Successful in 1m1s Details On Pull Request to Main / deploy-dev (pull_request) Successful in 1m32s Details Replace stateless one-shot extraction with a stateful, paced, cancellable pipeline. Six subtasks: - 3.1 Sentence-aware chunking with 800-char overlap (was paragraph-only at 4000 chars). Hard-split fallback for runaway sentences. - 3.2 Stateful extraction: chunks 2+ receive an "already-extracted topic IDs" hint capped at 200 IDs, so the model reuses IDs instead of inventing variants like software-developer vs software-engineer. - 3.3 Token-bucket limiter in llmRetry.js (extractionLimiter, 5 req/min). callLLM awaits the limiter before fetch; 429+Retry-After calls pauseUntil. Replaces hard setTimeout(12000) and setTimeout(15000). - 3.4 relevance_locked column on topics — admin edits to relevance are sticky across re-extraction. Migration + merge respects the flag + unlock checkbox in KnowledgeGraph edit form. - 3.5 Unify relation vocabulary — handbook prompt no longer mentions legacy "executes"; one-shot migration rewrites existing executes rows to executed_by with source/target swapped. - 3.6 Cancellation — Cancel button on UploadZone wired to an AbortController threaded into callLLM; aborted runs persist status = "cancelled" rather than "failed". Tests: 16 new unit tests for chunkText, buildKnownIdsHint, and createLimiter. All 61 tests pass, 0 lint errors, build clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-20 17:56:45 +02:00
Raymond Verhoef	9771928926	Merge pull request 'Fix: exclude temperature parameter for reasoning-tier models' (#4 ) from feat/ai-pipeline-hardening-phase-2 into main All checks were successful On Push to Main / test (push) Successful in 25s Details On Push to Main / publish (push) Successful in 57s Details On Push to Main / deploy-dev (push) Successful in 1m33s Details Reviewed-on: #4	2026-05-20 15:18:00 +00:00
Raymond Verhoef	33529dfb2b	Merge pull request 'feat: phase 2 of AI pipeline hardening — tool-based structured outputs + prompt caching' (#3 ) from feat/ai-pipeline-hardening-phase-2 into main All checks were successful On Push to Main / test (push) Successful in 25s Details On Push to Main / publish (push) Successful in 58s Details On Push to Main / deploy-dev (push) Successful in 1m32s Details Reviewed-on: #3	2026-05-20 13:48:08 +00:00