feat: phase 2 of AI pipeline hardening — tool-based structured outputs + prompt caching
All checks were successful
On Pull Request to Main / test (pull_request) Successful in 31s
On Pull Request to Main / publish (pull_request) Successful in 1m1s
On Pull Request to Main / deploy-dev (pull_request) Successful in 1m31s

Every structured-output call now uses an Anthropic tool instead of
parsing JSON out of free-form prose, and stable system prompts are
sent as cacheable blocks. Behaviour-equivalent to phase 1 from the
caller's point of view; the savings show up in token usage and in the
absence of "AI returned non-JSON response" failure modes.

* src/lib/llmTools.js — single source of truth for tool definitions:
  emit_knowledge_graph, emit_handbook_delta, emit_learning_article /
  _slides / _infographic / _all, emit_custom_topic, emit_quiz_questions,
  emit_graph_actions, plus five article-patch tools (set_intro,
  set_section, add_section, remove_section, replace_takeaways).
* src/lib/articlePatches.js — pure applyArticlePatches +
  applyAndValidate; rebuilds the article from a sequence of patch tool
  calls and re-validates against learningArticleSchema. set_section
  falls back to appending when no matching heading exists so the
  model's intent is preserved rather than silently dropped.
* src/lib/llmSchemas.js — Zod schemas for the five patch ops,
  registered in toolSchemaRegistry so callLLM validates them
  automatically.
* src/lib/llm.js — simulation mode now returns a tool_use stub matching
  toolChoice.name, so the UI keeps working with Simulation Mode on
  after the structured-output migration.
* src/lib/extractionPipeline.js — processSourceText and
  analyzeHandbookDelta migrated to callLLM + tool use. System prompts
  sent as { cache_control: ephemeral } blocks. Handbook results pass
  through normalizeHandbookResult to collapse legacy "executes"
  relations into executed_by with swapped source/target.
* src/lib/learningService.js — generateLearningContent picks the right
  tool per selectedType; generateCustomTopic uses emit_custom_topic;
  refineLearningContent now drives the five patch tools with
  toolChoice 'any' and rejects the whole turn if the patched article
  fails validation. Article-only refinement is intentional for phase 2;
  refining a topic without an article surfaces a clear error.
* src/lib/testService.js — quiz generation via emit_quiz_questions.
* src/components/admin/KnowledgeGraph.jsx — analyzeGraph routed through
  the reasoning tier (Opus) since graph-wide consolidation benefits
  from a stronger reasoner.
* src/components/chat/prompts.js — buildSystemPrompt now returns three
  text blocks: stable preamble (cached), KB context (cached, hash-bust
  deferred to phase 5), per-turn user/admin tail (uncached).
* src/lib/__tests__/ — 13 new tests covering each patch op, multi-op
  sequencing, post-patch validation failure, and tool/registry shape.

Acceptance: lint and 45/45 tests green; build succeeds; no
`match(/\{[\s\S]*\}/)` JSON extraction left in src/. Live verification
of cache hits on a second extraction within 5 minutes is deferred to
manual smoke testing — needs real `/api/anthropic` traffic.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
RaymondVerhoef
2026-05-20 15:47:20 +02:00
parent 8a8745fad2
commit f838755991
11 changed files with 872 additions and 291 deletions

80
src/lib/articlePatches.js Normal file
View File

@@ -0,0 +1,80 @@
/**
* Apply a sequence of patch operations (the tool_use calls returned by
* `refineLearningContent`) to an article object, in order. The returned
* article is a fresh object — the input is not mutated.
*
* Recognised tool names mirror `llmTools.js`:
* set_intro, set_section, add_section, remove_section, replace_takeaways.
*
* Unknown tool names are ignored on purpose; the caller validates the
* result against `learningArticleSchema` and rejects the whole turn if
* the patches produced an invalid article.
*/
import { learningArticleSchema } from './llmSchemas';
function matchesHeading(section, heading) {
return (section.heading ?? '').trim().toLowerCase() === heading.trim().toLowerCase();
}
function cloneArticle(article) {
return {
...article,
sections: article.sections.map((s) => ({ ...s })),
keyTakeaways: [...article.keyTakeaways],
};
}
export function applyArticlePatches(article, toolUses) {
let next = cloneArticle(article);
for (const tu of toolUses) {
switch (tu.name) {
case 'set_intro':
next = { ...next, intro: tu.input.intro };
break;
case 'set_section': {
const idx = next.sections.findIndex((s) => matchesHeading(s, tu.input.heading));
if (idx === -1) {
// No matching section — fall back to appending so the model's
// intent (provide that body) is preserved rather than lost.
next.sections = [...next.sections, { heading: tu.input.heading, body: tu.input.body }];
} else {
next.sections = next.sections.map((s, i) => (i === idx ? { ...s, body: tu.input.body } : s));
}
break;
}
case 'add_section': {
const newSection = { heading: tu.input.heading, body: tu.input.body };
next.sections = tu.input.position === 'start'
? [newSection, ...next.sections]
: [...next.sections, newSection];
break;
}
case 'remove_section':
next.sections = next.sections.filter((s) => !matchesHeading(s, tu.input.heading));
break;
case 'replace_takeaways':
next = { ...next, keyTakeaways: [...tu.input.items] };
break;
default:
// Unknown patch op — ignore.
break;
}
}
return next;
}
/**
* Apply the patches and re-validate against the article schema. Throws
* a clear error if the result is invalid.
*/
export function applyAndValidate(article, toolUses) {
const updated = applyArticlePatches(article, toolUses);
const parsed = learningArticleSchema.safeParse({ article: updated });
if (!parsed.success) {
const err = new Error(`Refinement produced an invalid article: ${parsed.error.message}`);
err.cause = parsed.error;
throw err;
}
return parsed.data.article;
}