fix: bescherm excluded en locked topics tegen AI-deletion/merge
All checks were successful
On Pull Request to Main / test (pull_request) Successful in 46s
On Pull Request to Main / publish (pull_request) Successful in 1m26s
On Pull Request to Main / deploy-dev (pull_request) Successful in 1m49s

Topics met learning_relevance="exclude" of relevance_locked=true werden
door de Full Analysis stilletjes verwijderd: het model zag exclude als
"irrelevant" en stelde ze voor in actions.deletions / actions.merges,
en analyzeGraph paste die zonder filter toe. Het locked-vlaggetje werd
in full-scope payload niet eens meegestuurd, dus zelfs een nieuwe prompt
kon niet helpen.

Defense-in-depth fixes:

1. Pure graphGuard.filterAiActions strips elke deletion/merge waarvan de
   target excluded of locked is, vóór bulkSave. Merges waarin een
   protected topic juist de keepId is (canonical survivor) blijven door.

2. SYSTEM_PROMPTS.full krijgt een expliciete "PROTECTED TOPICS" sectie
   die exclude en locked topics als nooit-te-verwijderen markeert.

3. relevance_locked wordt nu ook in de full-scope compactTopics payload
   meegestuurd, zodat het model de vlag überhaupt ziet.

4. UI-feedback: GraphControls toont een ShieldCheck banner met aantal
   geblokkeerde acties na de analyze, en het excluded-aantal naast de
   "Show Excluded Nodes" toggle (visible/hidden).

5. 13 nieuwe Vitest cases dekken protected detection en alle drop/keep
   paden van filterAiActions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
RaymondVerhoef
2026-06-04 08:22:12 +02:00
parent e310b6d85b
commit 9395ea11fe
4 changed files with 267 additions and 6 deletions

View File

@@ -5,6 +5,7 @@ import { callLLM } from '../../lib/llm';
import { EMIT_GRAPH_ACTIONS_TOOL } from '../../lib/llmTools';
import { Network, Table2 } from 'lucide-react';
import { useGraphData } from '../../hooks/useGraphData';
import { filterAiActions } from '../../lib/graphGuard';
import GraphControls from './graph/GraphControls';
import NodeDetailPanel from './graph/NodeDetailPanel';
import GraphTable from './graph/GraphTable';
@@ -25,11 +26,21 @@ const SYSTEM_PROMPTS = {
full: `You are a strict Data Quality AI maintaining a Knowledge Graph for Respellion.
Evaluate the provided topics and relations and emit the actions to take via the emit_graph_actions tool.
PROTECTED TOPICS — NEVER include these in merges (as deleteId) or in deletions:
• Topics with learning_relevance="exclude" are REFERENCE MATERIAL, kept on purpose
so R42 can still answer questions about them. They are intentionally outside
the theme-learning flow. Do not propose deleting or merging them away.
• Topics with relevance_locked=true are admin-pinned. Do not change their
learning_relevance and do not delete or merge them.
These topics may appear as the keepId in a merge (others fold into them), and
may appear as source/target of newRelations, but their own row must survive.
Rules:
1. Identify topics that mean exactly the same thing. Choose one to keep, one to delete (merges).
2. Identify topics that are too vague, irrelevant, or malformed (deletions).
3. Identify missing logical relations (depends_on, part_of, related_to, executed_by) between conceptually linked topics (newRelations).
4. Evaluate learning_relevance. Mark purely operational topics (printer guides, etc.) as "exclude"; low-priority as "peripheral" (relevanceUpdates).
Skip topics where relevance_locked=true — omit them from relevanceUpdates entirely.
Do not return the entire graph — only the actions to take.`,
@@ -73,6 +84,7 @@ const KnowledgeGraph = () => {
const [showExcludeNodes, setShowExcludeNodes] = useState(false);
const [isAnalyzing, setIsAnalyzing] = useState(false);
const [analyzeError, setAnalyzeError] = useState(null);
const [analyzeNotice, setAnalyzeNotice] = useState(null); // info-level message from last analyze
const [isRestoring, setIsRestoring] = useState(false);
const [viewMode, setViewMode] = useState('graph'); // 'graph' | 'table'
@@ -305,6 +317,7 @@ const KnowledgeGraph = () => {
setIsAnalyzing(true);
setAnalyzeError(null);
setAnalyzeNotice(null);
try {
const [currentTopics, currentRelations] = await Promise.all([
@@ -314,13 +327,16 @@ const KnowledgeGraph = () => {
const tier = scope === 'full' ? 'reasoning' : 'standard';
// For relevance scope, include relevance_locked so the model can skip those.
// For full + relevance scopes the model needs to see relevance_locked so
// it can honor the "do not touch protected topics" rule. (relations-scope
// only emits newRelations, so the flag is irrelevant there.)
const sendLocked = scope === 'full' || scope === 'relevance';
const compactTopics = currentTopics.map(t => ({
id: t.id,
label: t.label,
type: t.type,
learning_relevance: t.learning_relevance,
...(scope === 'relevance' && t.relevance_locked ? { relevance_locked: true } : {}),
...(sendLocked && t.relevance_locked ? { relevance_locked: true } : {}),
}));
const compactRelations = currentRelations.map(({ source, target, type }) => ({
source, target, type,
@@ -336,8 +352,26 @@ const KnowledgeGraph = () => {
maxTokens: scope === 'full' ? 4096 : 2048,
});
const actions = llmResult.toolUses[0]?.input;
if (!actions) throw new Error('Graph analysis did not emit a tool result.');
const rawActions = llmResult.toolUses[0]?.input;
if (!rawActions) throw new Error('Graph analysis did not emit a tool result.');
// ── Safety guard ─────────────────────────────────────────────────────
// Strip merges/deletions that target excluded or locked topics. Excluded
// topics are reference material kept on purpose for R42; locked topics
// are admin-pinned. The AI may suggest removing them anyway — we drop
// those suggestions before they reach bulkSave.
const { filtered: actions, dropped } = filterAiActions(currentTopics, rawActions);
const droppedCount = dropped.deletions.length + dropped.merges.length;
if (droppedCount > 0) {
// Visible feedback so the admin knows the AI tried to touch protected rows.
console.warn(
`[graph guard] Blocked ${droppedCount} destructive action(s) on protected topics`,
dropped,
);
setAnalyzeNotice(
`Guard blocked ${droppedCount} destructive AI action${droppedCount === 1 ? '' : 's'} on excluded or locked topics. ${dropped.deletions.length} deletion${dropped.deletions.length === 1 ? '' : 's'}, ${dropped.merges.length} merge${dropped.merges.length === 1 ? '' : 's'}.`,
);
}
let updatedTopics = [...currentTopics];
let updatedRelations = [...currentRelations];
@@ -460,9 +494,11 @@ const KnowledgeGraph = () => {
<GraphControls
showExcludeNodes={showExcludeNodes}
onShowExcludeChange={setShowExcludeNodes}
excludedCount={topics.filter(t => t.learning_relevance === 'exclude').length}
onAnalyze={analyzeGraph}
isAnalyzing={isAnalyzing}
analyzeError={analyzeError}
analyzeNotice={analyzeNotice}
disabled={topics.length === 0}
onApplied={reload}
snapshotMeta={snapshotMeta}