Handbook sync ran files sequentially under a 5 req/min limiter with a
hardcoded 60s LLM timeout, causing long syncs and AbortError timeouts on
large files. Now: limiter at 20 req/min, files processed with concurrency
4, handbook extraction timeout raised to 180s, and near-empty files skip
the LLM call.
callLLM gains a timeoutMs option; passing a signal no longer silently
disables the per-request timeout.
llm_calls telemetry self-disables after the first 404 so deploys without
the migration applied don't spam the console.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace stateless one-shot extraction with a stateful, paced, cancellable
pipeline. Six subtasks:
- 3.1 Sentence-aware chunking with 800-char overlap (was paragraph-only
at 4000 chars). Hard-split fallback for runaway sentences.
- 3.2 Stateful extraction: chunks 2+ receive an "already-extracted topic
IDs" hint capped at 200 IDs, so the model reuses IDs instead of
inventing variants like software-developer vs software-engineer.
- 3.3 Token-bucket limiter in llmRetry.js (extractionLimiter, 5 req/min).
callLLM awaits the limiter before fetch; 429+Retry-After calls
pauseUntil. Replaces hard setTimeout(12000) and setTimeout(15000).
- 3.4 relevance_locked column on topics — admin edits to relevance are
sticky across re-extraction. Migration + merge respects the flag +
unlock checkbox in KnowledgeGraph edit form.
- 3.5 Unify relation vocabulary — handbook prompt no longer mentions
legacy "executes"; one-shot migration rewrites existing executes rows
to executed_by with source/target swapped.
- 3.6 Cancellation — Cancel button on UploadZone wired to an
AbortController threaded into callLLM; aborted runs persist status =
"cancelled" rather than "failed".
Tests: 16 new unit tests for chunkText, buildKnownIdsHint, and
createLimiter. All 61 tests pass, 0 lint errors, build clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Every structured-output call now uses an Anthropic tool instead of
parsing JSON out of free-form prose, and stable system prompts are
sent as cacheable blocks. Behaviour-equivalent to phase 1 from the
caller's point of view; the savings show up in token usage and in the
absence of "AI returned non-JSON response" failure modes.
* src/lib/llmTools.js — single source of truth for tool definitions:
emit_knowledge_graph, emit_handbook_delta, emit_learning_article /
_slides / _infographic / _all, emit_custom_topic, emit_quiz_questions,
emit_graph_actions, plus five article-patch tools (set_intro,
set_section, add_section, remove_section, replace_takeaways).
* src/lib/articlePatches.js — pure applyArticlePatches +
applyAndValidate; rebuilds the article from a sequence of patch tool
calls and re-validates against learningArticleSchema. set_section
falls back to appending when no matching heading exists so the
model's intent is preserved rather than silently dropped.
* src/lib/llmSchemas.js — Zod schemas for the five patch ops,
registered in toolSchemaRegistry so callLLM validates them
automatically.
* src/lib/llm.js — simulation mode now returns a tool_use stub matching
toolChoice.name, so the UI keeps working with Simulation Mode on
after the structured-output migration.
* src/lib/extractionPipeline.js — processSourceText and
analyzeHandbookDelta migrated to callLLM + tool use. System prompts
sent as { cache_control: ephemeral } blocks. Handbook results pass
through normalizeHandbookResult to collapse legacy "executes"
relations into executed_by with swapped source/target.
* src/lib/learningService.js — generateLearningContent picks the right
tool per selectedType; generateCustomTopic uses emit_custom_topic;
refineLearningContent now drives the five patch tools with
toolChoice 'any' and rejects the whole turn if the patched article
fails validation. Article-only refinement is intentional for phase 2;
refining a topic without an article surfaces a clear error.
* src/lib/testService.js — quiz generation via emit_quiz_questions.
* src/components/admin/KnowledgeGraph.jsx — analyzeGraph routed through
the reasoning tier (Opus) since graph-wide consolidation benefits
from a stronger reasoner.
* src/components/chat/prompts.js — buildSystemPrompt now returns three
text blocks: stable preamble (cached), KB context (cached, hash-bust
deferred to phase 5), per-turn user/admin tail (uncached).
* src/lib/__tests__/ — 13 new tests covering each patch op, multi-op
sequencing, post-patch validation failure, and tool/registry shape.
Acceptance: lint and 45/45 tests green; build succeeds; no
`match(/\{[\s\S]*\}/)` JSON extraction left in src/. Live verification
of cache hits on a second extraction within 5 minutes is deferred to
manual smoke testing — needs real `/api/anthropic` traffic.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implements phase 1 of AI_PIPELINE_HARDENING_PLAN.md. Every Anthropic call
now goes through one module that owns retry, timeout, abort, structured-
output parsing, schema validation, and best-effort call telemetry.
* src/lib/llm.js — single callLLM entry point. Resolves model per tier
(fast / standard / reasoning) with admin:model legacy fallback for the
standard tier; 60s default timeout via AbortController; balanced-brace
JSON extraction; LLMHttpError, LLMTruncatedError, LLMOutputError, and
LLMValidationError surface clearly distinct failure modes.
* src/lib/llmRetry.js — exponential backoff with full jitter, retries
only on transient HTTP statuses, honours Retry-After up to 60s, never
retries on AbortError.
* src/lib/llmSchemas.js — Zod schemas for every structured task plus
normalizeHandbookResult (collapses legacy "executes" relations into
the canonical "executed_by" vocabulary).
* src/lib/api.js — thin shim over callLLM so existing callers (extraction
pipeline, learning, quiz, R42, knowledge graph) keep working unchanged.
* src/lib/__tests__/ — 32 Vitest cases covering parse paths, error
surfaces, simulation mode, model resolution, and schema validation.
* src/pages/Admin/index.jsx — three model inputs (fast / standard /
reasoning) replacing the single legacy field; legacy value falls back
for the standard tier so existing overrides survive.
Adds Zod and Vitest, plus an "npm run test" script.
Also cleans up the pre-existing repo-wide ESLint failures so phase 1's
"npm run lint passes" acceptance criterion can be checked: drops unused
React imports across the JSX tree (React 19 JSX runtime auto-imports),
attaches cause to rethrown errors in the service modules, ignores
pb_migrations in the ESLint config (PocketBase JSVM globals), and
removes one dead handleCreateCustom function in Leren.jsx. A real
behaviour bug surfaced in Testen.jsx — the quiz timer captured a stale
finishQuiz via setInterval closure; now updated via finishQuizRef so the
timer always invokes the latest callback.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds AI_PIPELINE_HARDENING_PLAN.md — a phased, self-contained plan an AI
agent can execute to harden the Anthropic integration (central LLM
client, tool-based structured outputs, prompt caching, retrieval-based
R42 context, eval harness).
Renames src/lib/giteaService.js to src/lib/githubService.js. The module
calls api.github.com and raw.githubusercontent.com; the previous name
was misleading. No behaviour change. Updates the single import site in
src/components/admin/KnowledgeGraph.jsx.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
