docs(test): generated living behaviour spec + FE/BE seam drift check (WP-71)
Gherkin/Cucumber was considered and rejected for business-readable BDD scenarios: step-binding by runtime string match undoes the compile-time guarantees WP-70 just added, and needs two frameworks for .NET+TS with no non-technical co-author in view. Instead scripts/gen-behaviour-spec.mjs (modeled on the existing gen-snippets.mjs) extracts every describe/it and [Fact]/[Theory] name straight from the real suites into libs/shared/docs/behaviour-spec.mdx, gated for drift in CI exactly like gen-snippets/gen-api — the page can never diverge from the tests because it's generated from them, and test names stay the single source of truth. scripts/check-seam.sh guards the one FE/BE rule duplication most likely to silently diverge: IntakePolicy.cs's ScholingThreshold vs intake.machine.ts's SCHOLING_THRESHOLD_DEFAULT, two unlinked literals pinned separately in each side's own tests but never against each other. package.json/CI wiring for both (gen:behaviour-spec, check:seam) shipped in the prior commit alongside the typecheck gate, since all three touch the same few config files. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Claude Sonnet 5
@@ -0,0 +1,218 @@
|
||||
#!/usr/bin/env node
|
||||
// Generate a business-readable "behaviour spec" page FROM real test names (WP-71, Track D).
|
||||
// The team considered Cucumber/Gherkin for BDD scenarios and rejected it (runtime string
|
||||
// matching undoes the compile-time guarantees WP-70 just bought, and needs two frameworks for
|
||||
// .NET+TS). Instead: test names ARE the spec — this script only extracts and formats them, so
|
||||
// the page can never drift from the suite. Mirrors the gen-snippets.mjs pattern (pure Node,
|
||||
// reads real source files, writes ONE generated file, checked for drift in CI the same way).
|
||||
// Run: `npm run gen:behaviour-spec`.
|
||||
import { readFileSync, writeFileSync, readdirSync, statSync } from 'node:fs';
|
||||
import { join, relative, sep } from 'node:path';
|
||||
|
||||
const EXCLUDED_DIRS = new Set(['node_modules', 'dist', 'coverage', 'bin', 'obj', '.git']);
|
||||
|
||||
/** Recursively collect files under `dir` matching `pattern`, skipping excluded directories. */
|
||||
function walk(dir, pattern) {
|
||||
const out = [];
|
||||
for (const entry of readdirSync(dir)) {
|
||||
if (EXCLUDED_DIRS.has(entry)) continue;
|
||||
const full = join(dir, entry);
|
||||
const st = statSync(full);
|
||||
if (st.isDirectory()) out.push(...walk(full, pattern));
|
||||
else if (pattern.test(entry)) out.push(full);
|
||||
}
|
||||
return out.sort();
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Frontend: apps/**/*.spec.ts + libs/**/*.spec.ts — describe()/it() pairs.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const QUOTED = `(?:'([^']*)'|"([^"]*)"|` + '`([^`]*)`)';
|
||||
const DESCRIBE_RE = new RegExp(`\\bdescribe(?:\\.\\w+)?\\(\\s*${QUOTED}`);
|
||||
const IT_RE = new RegExp(`\\bit(?:\\.\\w+)?\\(\\s*${QUOTED}`);
|
||||
|
||||
/** Which app/context folder a spec file belongs to, for grouping (registratie, brief, …). */
|
||||
function feContextFor(path) {
|
||||
const norm = path.split(sep).join('/');
|
||||
const appMatch = norm.match(/^apps\/(?:ssp|behandelportal)\/src\/app\/([^/]+)\//);
|
||||
if (appMatch) return appMatch[1];
|
||||
const libMatch = norm.match(/^libs\/([^/]+)\/src\//);
|
||||
if (libMatch) return libMatch[1];
|
||||
return 'other';
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract { describePath: string[], text: string } for every `it(...)` in a spec file, using
|
||||
* a brace-depth stack to track nested `describe(...)` blocks (a line-scan, not a TS parser —
|
||||
* this repo's spec files are one describe/it call per line, same precedent as gen-snippets.mjs).
|
||||
*/
|
||||
function extractSpecBehaviours(source) {
|
||||
const lines = source.split('\n');
|
||||
let depth = 0;
|
||||
const stack = []; // { name, depth }
|
||||
const results = [];
|
||||
for (const line of lines) {
|
||||
if (/^\s*\/\//.test(line)) continue; // skip commented-out lines
|
||||
const dm = line.match(DESCRIBE_RE);
|
||||
const im = !dm && line.match(IT_RE);
|
||||
if (dm) {
|
||||
stack.push({ name: dm[1] ?? dm[2] ?? dm[3], depth });
|
||||
} else if (im) {
|
||||
results.push({ describePath: stack.map((s) => s.name), text: im[1] ?? im[2] ?? im[3] });
|
||||
}
|
||||
const open = (line.match(/{/g) || []).length;
|
||||
const close = (line.match(/}/g) || []).length;
|
||||
depth += open - close;
|
||||
while (stack.length && depth <= stack[stack.length - 1].depth) stack.pop();
|
||||
}
|
||||
return results;
|
||||
}
|
||||
|
||||
const feSpecFiles = [...walk('apps', /\.spec\.ts$/), ...walk('libs', /\.spec\.ts$/)];
|
||||
|
||||
/** @type {Map<string, Map<string, string[]>>} context -> describe-block label -> it() texts */
|
||||
const feBehaviour = new Map();
|
||||
for (const file of feSpecFiles) {
|
||||
const context = feContextFor(file);
|
||||
const relPath = relative('.', file).split(sep).join('/');
|
||||
const behaviours = extractSpecBehaviours(readFileSync(file, 'utf8'));
|
||||
for (const { describePath, text } of behaviours) {
|
||||
const label = describePath.length ? describePath.join(' › ') : `(${relPath})`;
|
||||
if (!feBehaviour.has(context)) feBehaviour.set(context, new Map());
|
||||
const byLabel = feBehaviour.get(context);
|
||||
if (!byLabel.has(label)) byLabel.set(label, []);
|
||||
byLabel.get(label).push(text);
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Backend: backend/tests/BigRegister.Tests/**/*.cs — [Fact]/[Theory] methods.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const CLASS_RE = /^\s*(?:public|internal)\s+(?:sealed\s+)?class\s+(\w+)/;
|
||||
const FACT_OR_THEORY_RE = /^\s*\[(?:Fact|Theory)\b/;
|
||||
const METHOD_RE = /\b(?:void|Task(?:<[^>]*>)?)\s+(\w+)\s*\(/;
|
||||
|
||||
/** PascalCase_snake_sentence method name -> readable sentence (just spaces for underscores). */
|
||||
function toSentence(methodName) {
|
||||
return methodName.replace(/_/g, ' ');
|
||||
}
|
||||
|
||||
/** Extract { className, sentence } for every [Fact]/[Theory]-attributed method in a .cs file. */
|
||||
function extractCsBehaviours(source) {
|
||||
const lines = source.split('\n');
|
||||
let currentClass = null;
|
||||
const results = [];
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
const cm = lines[i].match(CLASS_RE);
|
||||
if (cm) {
|
||||
currentClass = cm[1];
|
||||
continue;
|
||||
}
|
||||
if (!FACT_OR_THEORY_RE.test(lines[i])) continue;
|
||||
// Skip any further attribute lines (e.g. [InlineData(...)] rows on a [Theory]) and blank
|
||||
// lines to reach the method declaration itself.
|
||||
let j = i + 1;
|
||||
while (j < lines.length && (/^\s*\[/.test(lines[j]) || /^\s*$/.test(lines[j]))) j++;
|
||||
const mm = lines[j] && lines[j].match(METHOD_RE);
|
||||
if (mm && currentClass) results.push({ className: currentClass, sentence: toSentence(mm[1]) });
|
||||
}
|
||||
return results;
|
||||
}
|
||||
|
||||
const csFiles = walk('backend/tests/BigRegister.Tests', /\.cs$/);
|
||||
|
||||
/** @type {Map<string, string[]>} class name -> sentences */
|
||||
const beBehaviour = new Map();
|
||||
for (const file of csFiles) {
|
||||
for (const { className, sentence } of extractCsBehaviours(readFileSync(file, 'utf8'))) {
|
||||
if (!beBehaviour.has(className)) beBehaviour.set(className, []);
|
||||
beBehaviour.get(className).push(sentence);
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Emit libs/shared/docs/behaviour-spec.mdx
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// MDX parses markdown as JSX-in-Markdown: a bare `<tag>`/`{expr}` in test-name text (e.g.
|
||||
// "renders each field group as its own grey <fieldset>") would otherwise be read as JSX and
|
||||
// fail the build. Test names are data, not markup — escape them before embedding.
|
||||
function mdxEscape(text) {
|
||||
return text
|
||||
.replace(/&/g, '&')
|
||||
.replace(/</g, '<')
|
||||
.replace(/>/g, '>')
|
||||
.replace(/\{/g, '{')
|
||||
.replace(/\}/g, '}');
|
||||
}
|
||||
|
||||
function renderFeSection(context) {
|
||||
const byLabel = feBehaviour.get(context);
|
||||
const labels = [...byLabel.keys()].sort();
|
||||
const blocks = labels.map((label) => {
|
||||
const items = byLabel
|
||||
.get(label)
|
||||
.map((t) => `- ${mdxEscape(t)}`)
|
||||
.join('\n');
|
||||
return `#### ${mdxEscape(label)}\n\n${items}`;
|
||||
});
|
||||
return `### ${mdxEscape(context)}\n\n${blocks.join('\n\n')}`;
|
||||
}
|
||||
|
||||
function renderBeSection(className) {
|
||||
const items = beBehaviour
|
||||
.get(className)
|
||||
.map((t) => `- ${mdxEscape(t)}`)
|
||||
.join('\n');
|
||||
return `### ${mdxEscape(className)}\n\n${items}`;
|
||||
}
|
||||
|
||||
const feContexts = [...feBehaviour.keys()].sort();
|
||||
const feCount = feContexts.reduce((n, c) => n + [...feBehaviour.get(c).values()].flat().length, 0);
|
||||
const beClasses = [...beBehaviour.keys()].sort();
|
||||
const beCount = beClasses.reduce((n, c) => n + beBehaviour.get(c).length, 0);
|
||||
|
||||
const feSections = feContexts.map(renderFeSection).join('\n\n');
|
||||
const beSections = beClasses.map(renderBeSection).join('\n\n');
|
||||
|
||||
const mdx = `{/* GENERATED by \`npm run gen:behaviour-spec\` (scripts/gen-behaviour-spec.mjs) — do not
|
||||
edit. Every bullet below is a real \`it()\` title or backend test method name, extracted
|
||||
verbatim from the suite. The team rejected Cucumber/Gherkin for BDD scenarios (runtime string
|
||||
matching undoes the compile-time guarantees WP-70 bought, and needs two frameworks for
|
||||
.NET+TS) — this page is the replacement: business-readable documentation generated FROM test
|
||||
names, so it can never drift from what the suite actually asserts. A test name changing (or a
|
||||
test being added/removed) is the only way this page changes; hand-editing it is pointless,
|
||||
the next \`npm run gen:behaviour-spec\` overwrites it. */}
|
||||
|
||||
import { Meta } from '@storybook/addon-docs/blocks';
|
||||
|
||||
<Meta title="Foundations/Behaviour spec" />
|
||||
|
||||
# Behaviour spec
|
||||
|
||||
_Generated by \`npm run gen:behaviour-spec\` — do not hand-edit; the next generation
|
||||
overwrites this page. See [BDD](?path=/docs/foundations-bdd--docs) for how these names are
|
||||
written, and [Testing strategy](?path=/docs/foundations-testing-strategy--docs) for what gets
|
||||
tested where._
|
||||
|
||||
Every bullet below is a real test name from the suite — an \`it()\` title (frontend) or a test
|
||||
method name (backend), read as a sentence. Nothing here is hand-written prose: this page
|
||||
**is** the suite, reshaped for a business reader. ${feCount} frontend behaviours across
|
||||
${feContexts.length} contexts; ${beCount} backend behaviours across ${beClasses.length} test
|
||||
classes.
|
||||
|
||||
## Frontend (by context)
|
||||
|
||||
${feSections}
|
||||
|
||||
## Backend (by test class)
|
||||
|
||||
${beSections}
|
||||
`;
|
||||
|
||||
writeFileSync('libs/shared/docs/behaviour-spec.mdx', mdx);
|
||||
console.log(
|
||||
`wrote libs/shared/docs/behaviour-spec.mdx (${feCount} frontend behaviours in ${feContexts.length} contexts, ${beCount} backend behaviours in ${beClasses.length} classes)`,
|
||||
);
|
||||
Reference in New Issue
Block a user