Add comprehensive documentation for key organizational aspects
- Introduced "Pension Scheme & Benefits" detailing secondary employment benefits and pension specifics. - Created "Roles & Accountabilities" outlining the Holacracy role structure and responsibilities within Respellion. - Added "Security" section covering GDPR compliance and workplace safety protocols. - Established "Spending and Contracting" policy detailing expense categories and submission processes. - Documented "Who We Are" to define Respellion's identity, services, and operational model under Holacracy and ISO 9001.
This commit is contained in:
RaymondVerhoef
@@ -1,583 +1,87 @@
|
||||
# Generation service spec
|
||||
# Generation spec: learning content & micro-learnings
|
||||
|
||||
## Responsibility
|
||||
|
||||
Accepts a Theme ID from the admin app (on batch approval) and generates all 10
|
||||
micro learning types for every published Topic in that Theme. One Claude Sonnet 4
|
||||
call per type per topic. All outputs validated through Zod schemas before write.
|
||||
|
||||
This service runs entirely server-side. The admin app calls it via REST. All AI
|
||||
calls go through the Anthropic API. No generation logic lives in the frontend.
|
||||
Two generators turn a topic into learner-facing material. Both go through
|
||||
`callLLM` with forced tool use and Zod-validated output. All content is cached in
|
||||
PocketBase so it is generated once per topic/type.
|
||||
|
||||
---
|
||||
|
||||
## Service location
|
||||
## A. Long-form content — `src/lib/learningService.js`
|
||||
|
||||
```
|
||||
app/services/generation/
|
||||
├── src/
|
||||
│ ├── index.ts entry point, Fastify server
|
||||
│ ├── routes/
|
||||
│ │ ├── generate.ts POST /generate, GET /status/:jobId
|
||||
│ │ └── publish.ts PATCH /micro-learnings/:id
|
||||
│ ├── pipeline/
|
||||
│ │ └── generate.ts per-type generation logic
|
||||
│ ├── jobs/
|
||||
│ │ └── queue.ts async job queue (in-memory)
|
||||
│ ├── lib/
|
||||
│ │ ├── pocketbase.ts PocketBase client
|
||||
│ │ └── anthropic.ts Anthropic client
|
||||
│ └── types.ts shared TypeScript types + Zod schemas
|
||||
├── package.json
|
||||
├── tsconfig.json
|
||||
├── .env.example
|
||||
└── .gitignore
|
||||
```
|
||||
Stored in the `content` collection (one record per topic, `data` is a merged
|
||||
object). Three types, generated **on demand**:
|
||||
|
||||
| Type | Tool | Min requirements |
|
||||
|---|---|---|
|
||||
| `article` | `emit_learning_article` | ≥3 sections, ≥2 takeaways |
|
||||
| `slides` | `emit_learning_slides` | ≥4 slides |
|
||||
| `infographic` | `emit_learning_infographic` | ≥3 stats, ≥3 steps |
|
||||
|
||||
`generateLearningContent(topic, force, selectedType)`:
|
||||
- tier `standard`, `maxTokens: 8192`
|
||||
- `selectedType` is one of the three, or `'all'` (`emit_learning_all`) for admin regeneration
|
||||
- cache check looks at `content[selectedType]`; on generation the new payload is
|
||||
**shallow-merged** into the cached object so other types survive
|
||||
- there is **no podcast type**
|
||||
|
||||
**Article refinement** (`refineLearningContent`): the admin describes a change and
|
||||
the model edits via targeted patch tools — `set_intro`, `set_section`,
|
||||
`add_section`, `remove_section`, `replace_takeaways` — so only the affected parts
|
||||
change. Patches are applied and re-validated in `src/lib/articlePatches.js`.
|
||||
|
||||
---
|
||||
|
||||
## API surface
|
||||
## B. Micro-learnings — `src/lib/microLearningService.js`
|
||||
|
||||
### POST /generate
|
||||
Stored in the `micro_learnings` collection (one record per topic per type,
|
||||
`status='published'`). Three types:
|
||||
|
||||
Triggered by admin app when a Theme batch is approved.
|
||||
| Type | Tool | Tier | Shape |
|
||||
|---|---|---|---|
|
||||
| `concept_explainer` | `emit_concept_explainer` | standard | `{ sections: [{ title, content (HTML) }] }`, ≥3 sections |
|
||||
| `scenario_quiz` | `emit_scenario_quiz` | standard | `{ scenario, options: [{ text, isCorrect, explanation }] }`, 3–4 options, exactly 1 correct |
|
||||
| `flashcard_set` | `emit_flashcard_set` | fast (Haiku) | `{ cards: [{ front, back }] }`, 5–10 cards |
|
||||
|
||||
Request:
|
||||
```json
|
||||
{
|
||||
"themeId": "string"
|
||||
}
|
||||
```
|
||||
`getOrGenerateMicroLearning(topicId, type)`:
|
||||
- returns the cached published record if one exists (`findExisting`)
|
||||
- otherwise loads the topic, calls `callLLM` with forced tool choice, and creates a
|
||||
`micro_learnings` record with the validated `content`
|
||||
|
||||
Response (202 Accepted):
|
||||
```json
|
||||
{
|
||||
"jobId": "string",
|
||||
"status": "queued",
|
||||
"topicsFound": 5,
|
||||
"totalItems": 50
|
||||
}
|
||||
```
|
||||
> A former `reflection_prompt` type was dropped. Do not re-add it.
|
||||
|
||||
Processing is async. The admin app polls job status.
|
||||
|
||||
Behaviour:
|
||||
- Fetches all published Topics for the given themeId
|
||||
- Creates one micro_learnings record per topic per type with status `queued`
|
||||
- Generates each item sequentially; updates status to `generated` on success
|
||||
- On failure: sets individual item status to `failed`, continues remaining items
|
||||
- Job completes when all items are either `generated` or `failed`
|
||||
Completion is recorded (append-only) by `useMicroLearningCompletions` into
|
||||
`micro_learning_completions` with `{ team_member_id, micro_learning_id, topic_id,
|
||||
type, session_week }`.
|
||||
|
||||
---
|
||||
|
||||
### GET /status/:jobId
|
||||
## C. Weekly quiz — `src/lib/testService.js`
|
||||
|
||||
Returns current job progress.
|
||||
Generates a 5-question multiple-choice test for the user's current week.
|
||||
|
||||
Response:
|
||||
```json
|
||||
{
|
||||
"jobId": "string",
|
||||
"status": "queued" | "running" | "done" | "failed",
|
||||
"progress": {
|
||||
"topicsTotal": 5,
|
||||
"topicsProcessed": 3,
|
||||
"itemsTotal": 50,
|
||||
"itemsGenerated": 28,
|
||||
"itemsFailed": 2
|
||||
},
|
||||
"error": "string | null"
|
||||
}
|
||||
```
|
||||
- **Topic selection** (`selectTestTopics`): primary topic from the active
|
||||
curriculum week (else hash fallback) + a few review topics for breadth.
|
||||
- **Batch generation** (`callQuizBatchModel`): a single `fast`-tier call
|
||||
(`emit_quiz_questions`, `maxTokens: 4096`, 25s timeout) returns all 5 questions.
|
||||
- **Quality gates** (`validateBatchQuality`): no duplicate options; no banned
|
||||
fillers ("all/none of the above", "both A and B"); explanations ≥20 chars; reject
|
||||
if `correctIndex` is dominated by one position (>80%) and re-roll.
|
||||
- **Scoring** (`saveTestResult`): `pointsEarned = score * 2`, written to
|
||||
`leaderboard` via `db.upsertLeaderboardEntry`.
|
||||
|
||||
Question shape: `{ id, question, topicLabel, options[4], correctIndex (0–3),
|
||||
explanation, difficulty }`.
|
||||
|
||||
---
|
||||
|
||||
### PATCH /micro-learnings/:id
|
||||
|
||||
Admin publishes or rejects an individual micro learning.
|
||||
|
||||
Request:
|
||||
```json
|
||||
{
|
||||
"status": "published" | "rejected"
|
||||
}
|
||||
```
|
||||
|
||||
Response (200 OK):
|
||||
```json
|
||||
{
|
||||
"id": "string",
|
||||
"status": "published" | "rejected",
|
||||
"published_at": "datetime | null"
|
||||
}
|
||||
```
|
||||
|
||||
Rules:
|
||||
- Only `generated` records can be published or rejected
|
||||
- `published_at` set on publish, left null on reject
|
||||
- Returns 400 if record is not in `generated` status
|
||||
- Returns 404 if record not found
|
||||
|
||||
---
|
||||
|
||||
## Generation pipeline
|
||||
|
||||
### Input
|
||||
|
||||
For each Topic in the approved Theme:
|
||||
```
|
||||
topic.title: string
|
||||
topic.body: string
|
||||
topic.key_terms: string[]
|
||||
topic.difficulty: 'introductory' | 'intermediate' | 'advanced'
|
||||
```
|
||||
|
||||
### Output
|
||||
|
||||
10 micro_learnings records per topic, one per type.
|
||||
|
||||
---
|
||||
|
||||
## AI call configuration
|
||||
|
||||
```typescript
|
||||
{
|
||||
model: 'claude-sonnet-4-20250514',
|
||||
max_tokens: 2000,
|
||||
temperature: 0 // deterministic structured output
|
||||
}
|
||||
```
|
||||
|
||||
One call per type per topic. Do not batch multiple types into one call — isolated
|
||||
calls are easier to retry and validate independently.
|
||||
|
||||
---
|
||||
|
||||
## Prompt strategy
|
||||
|
||||
### System prompt (all types)
|
||||
|
||||
```
|
||||
You are a learning content designer. Your task is to generate structured learning
|
||||
content for a specific topic in an employee learning platform.
|
||||
|
||||
Output ONLY valid JSON matching the schema provided. No preamble, no explanation,
|
||||
no markdown fences.
|
||||
|
||||
The content should be accurate, practical, and appropriate for the stated
|
||||
difficulty level. Tone: professional but accessible.
|
||||
```
|
||||
|
||||
### User prompt template (all types)
|
||||
|
||||
```
|
||||
Topic: {topic.title}
|
||||
Difficulty: {topic.difficulty}
|
||||
Body:
|
||||
{topic.body}
|
||||
|
||||
Key terms: {topic.key_terms.join(', ')}
|
||||
|
||||
Generate a {type_label} for this topic.
|
||||
|
||||
Output schema:
|
||||
{JSON.stringify(schemaDescription)}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Type-specific prompts and schemas
|
||||
|
||||
### concept_explainer
|
||||
|
||||
Type label: `Concept Explainer`
|
||||
|
||||
Schema description:
|
||||
```json
|
||||
{
|
||||
"paragraphs": ["2 to 3 paragraphs explaining the concept in plain language"],
|
||||
"example": "one concrete real-world example"
|
||||
}
|
||||
```
|
||||
|
||||
Zod schema:
|
||||
```typescript
|
||||
z.object({
|
||||
paragraphs: z.array(z.string()).min(2).max(3),
|
||||
example: z.string().min(20)
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### scenario_quiz
|
||||
|
||||
Type label: `Scenario Quiz`
|
||||
|
||||
Schema description:
|
||||
```json
|
||||
{
|
||||
"scenario": "a realistic workplace scenario",
|
||||
"options": [
|
||||
{ "label": "A", "text": "answer text", "correct": false, "explanation": "why" },
|
||||
{ "label": "B", "text": "answer text", "correct": true, "explanation": "why" },
|
||||
{ "label": "C", "text": "answer text", "correct": false, "explanation": "why" },
|
||||
{ "label": "D", "text": "answer text", "correct": false, "explanation": "why" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Rules: exactly 4 options, exactly 1 correct.
|
||||
|
||||
Zod schema:
|
||||
```typescript
|
||||
z.object({
|
||||
scenario: z.string().min(30),
|
||||
options: z.array(z.object({
|
||||
label: z.enum(['A', 'B', 'C', 'D']),
|
||||
text: z.string().min(5),
|
||||
correct: z.boolean(),
|
||||
explanation: z.string().min(10)
|
||||
})).length(4).refine(
|
||||
opts => opts.filter(o => o.correct).length === 1,
|
||||
{ message: 'exactly one correct option required' }
|
||||
)
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### misconceptions
|
||||
|
||||
Type label: `Misconceptions`
|
||||
|
||||
Schema description:
|
||||
```json
|
||||
{
|
||||
"items": [
|
||||
{ "misconception": "common wrong belief", "correction": "accurate explanation" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Rules: 3 to 5 items.
|
||||
|
||||
Zod schema:
|
||||
```typescript
|
||||
z.object({
|
||||
items: z.array(z.object({
|
||||
misconception: z.string().min(10),
|
||||
correction: z.string().min(10)
|
||||
})).min(3).max(5)
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### how_to
|
||||
|
||||
Type label: `How-To Guide`
|
||||
|
||||
Schema description:
|
||||
```json
|
||||
{
|
||||
"steps": [
|
||||
{ "number": 1, "instruction": "what to do" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Rules: 3 to 8 steps.
|
||||
|
||||
Zod schema:
|
||||
```typescript
|
||||
z.object({
|
||||
steps: z.array(z.object({
|
||||
number: z.number().int().positive(),
|
||||
instruction: z.string().min(10)
|
||||
})).min(3).max(8)
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### comparison_card
|
||||
|
||||
Type label: `Comparison Card`
|
||||
|
||||
Schema description:
|
||||
```json
|
||||
{
|
||||
"subject_a": "first concept or approach",
|
||||
"subject_b": "second concept or approach",
|
||||
"dimensions": [
|
||||
{ "label": "dimension name", "a": "how A differs", "b": "how B differs" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Rules: 3 to 6 dimensions.
|
||||
|
||||
Zod schema:
|
||||
```typescript
|
||||
z.object({
|
||||
subject_a: z.string().min(2),
|
||||
subject_b: z.string().min(2),
|
||||
dimensions: z.array(z.object({
|
||||
label: z.string().min(2),
|
||||
a: z.string().min(5),
|
||||
b: z.string().min(5)
|
||||
})).min(3).max(6)
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### reflection_prompt
|
||||
|
||||
Type label: `Reflection Prompt`
|
||||
|
||||
Schema description:
|
||||
```json
|
||||
{
|
||||
"prompt": "open-ended question for the employee to reflect on",
|
||||
"model_answer": "a thoughtful example answer the employee can compare against"
|
||||
}
|
||||
```
|
||||
|
||||
Zod schema:
|
||||
```typescript
|
||||
z.object({
|
||||
prompt: z.string().min(20),
|
||||
model_answer: z.string().min(50)
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### flashcard_set
|
||||
|
||||
Type label: `Flashcard Set`
|
||||
|
||||
Schema description:
|
||||
```json
|
||||
{
|
||||
"cards": [
|
||||
{ "question": "question text", "answer": "answer text" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Rules: 5 to 10 cards.
|
||||
|
||||
Zod schema:
|
||||
```typescript
|
||||
z.object({
|
||||
cards: z.array(z.object({
|
||||
question: z.string().min(5),
|
||||
answer: z.string().min(5)
|
||||
})).min(5).max(10)
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### case_study
|
||||
|
||||
Type label: `Case Study`
|
||||
|
||||
Schema description:
|
||||
```json
|
||||
{
|
||||
"scenario": "a detailed real-world scenario (150+ words)",
|
||||
"questions": ["discussion or reflection question 1", "discussion or reflection question 2"]
|
||||
}
|
||||
```
|
||||
|
||||
Rules: 2 to 4 questions.
|
||||
|
||||
Zod schema:
|
||||
```typescript
|
||||
z.object({
|
||||
scenario: z.string().min(150),
|
||||
questions: z.array(z.string().min(10)).min(2).max(4)
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### glossary_anchor
|
||||
|
||||
Type label: `Glossary Anchor`
|
||||
|
||||
Schema description:
|
||||
```json
|
||||
{
|
||||
"term": "the key term",
|
||||
"definition": "precise definition",
|
||||
"correct_use": "example sentence showing correct use",
|
||||
"misuse": "common incorrect usage to avoid"
|
||||
}
|
||||
```
|
||||
|
||||
Prompt addition: use the first key term from `topic.key_terms` as the anchor term.
|
||||
|
||||
Zod schema:
|
||||
```typescript
|
||||
z.object({
|
||||
term: z.string().min(2),
|
||||
definition: z.string().min(20),
|
||||
correct_use: z.string().min(20),
|
||||
misuse: z.string().min(20)
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### myth_vs_evidence
|
||||
|
||||
Type label: `Myth vs Evidence`
|
||||
|
||||
Schema description:
|
||||
```json
|
||||
{
|
||||
"myth": "a commonly held misconception about this topic",
|
||||
"evidence": "the evidence-based counterpoint",
|
||||
"sources": ["source or reference if applicable — leave empty array if none"]
|
||||
}
|
||||
```
|
||||
|
||||
Zod schema:
|
||||
```typescript
|
||||
z.object({
|
||||
myth: z.string().min(20),
|
||||
evidence: z.string().min(30),
|
||||
sources: z.array(z.string())
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Error handling
|
||||
|
||||
**Per item:**
|
||||
- JSON parse failure → retry once with stricter prompt ("respond with valid JSON only, no other text")
|
||||
- Second failure → set micro_learning status to `failed`, log raw response, continue to next item
|
||||
- Zod validation failure → same as parse failure: retry once, then `failed`
|
||||
- Anthropic API error (rate limit / timeout) → exponential backoff, 3 retries, then `failed`
|
||||
|
||||
**Per job:**
|
||||
- If all items for a topic fail → log, continue to next topic
|
||||
- Job status becomes `done` when all items processed, regardless of individual failures
|
||||
- Job status becomes `failed` only if the initial topic fetch fails (PocketBase error before generation starts)
|
||||
|
||||
---
|
||||
|
||||
## PocketBase write
|
||||
|
||||
For each generated item:
|
||||
```typescript
|
||||
{
|
||||
topic: topicId,
|
||||
type: type, // one of the 10 type enum values
|
||||
content: validatedContent, // JSON, validated by Zod
|
||||
status: 'generated',
|
||||
generation_model: 'claude-sonnet-4-20250514',
|
||||
generated_at: new Date().toISOString()
|
||||
}
|
||||
```
|
||||
|
||||
Create record with status `queued` before generation starts.
|
||||
Update to `generated` (with content) or `failed` after attempt.
|
||||
|
||||
---
|
||||
|
||||
## Job lifecycle
|
||||
|
||||
```
|
||||
POST /generate received
|
||||
↓
|
||||
Fetch published Topics for Theme
|
||||
↓
|
||||
Create micro_learning records: status = queued
|
||||
↓
|
||||
Job created → status: running
|
||||
↓
|
||||
For each topic:
|
||||
For each of 10 types:
|
||||
Claude call → validate → write content → status = generated
|
||||
↓
|
||||
All items processed
|
||||
↓
|
||||
Job status: done
|
||||
```
|
||||
|
||||
On topic fetch failure:
|
||||
```
|
||||
status: failed
|
||||
error: { reason: 'topic_fetch_failed', detail: ... }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Environment variables required
|
||||
|
||||
```
|
||||
ANTHROPIC_API_KEY=
|
||||
POCKETBASE_URL=
|
||||
POCKETBASE_ADMIN_EMAIL=
|
||||
POCKETBASE_ADMIN_PASSWORD=
|
||||
GENERATION_PORT=3002
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Dependencies
|
||||
|
||||
```json
|
||||
{
|
||||
"dependencies": {
|
||||
"fastify": "^4",
|
||||
"@anthropic-ai/sdk": "^0.24",
|
||||
"pocketbase": "^0.21",
|
||||
"uuid": "^9",
|
||||
"zod": "^3"
|
||||
},
|
||||
"devDependencies": {
|
||||
"typescript": "^5",
|
||||
"@types/node": "^20",
|
||||
"tsx": "^4"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## TypeScript strict mode requirements
|
||||
|
||||
- No `any` types
|
||||
- All Claude response parsing through Zod schema validation before PocketBase write
|
||||
- All PocketBase writes typed against micro_learnings schema from data-model.md
|
||||
- Content type is `unknown` after JSON.parse — always validate through Zod before use
|
||||
|
||||
---
|
||||
|
||||
## What this service does NOT do
|
||||
|
||||
- Does not extract or chunk source documents → ingestion service
|
||||
- Does not build or schedule the curriculum → curriculum service
|
||||
- Does not handle admin auth → PocketBase + admin app
|
||||
- Does not embed content into Qdrant → ingestion service handles all embeddings
|
||||
- Does not serve R42 queries → chat service
|
||||
|
||||
---
|
||||
|
||||
## Testing checkpoints
|
||||
|
||||
1. Call POST /generate with a themeId that has 2 published topics → verify 20 micro_learning records created
|
||||
2. All 10 types generated for each topic → verify content JSON parses correctly
|
||||
3. All Zod schemas pass for each of the 10 types
|
||||
4. PATCH /micro-learnings/:id with `published` → verify status + published_at updated
|
||||
5. PATCH /micro-learnings/:id with `rejected` → verify status updated, published_at null
|
||||
6. Force a JSON parse error (mock) → verify retry logic fires once, then sets status to `failed`
|
||||
7. GET /status/:jobId during processing → verify progress counters increment correctly
|
||||
## Shared infrastructure (`src/lib/llm.js`)
|
||||
|
||||
- **Tiers:** `fast` (Haiku 4.5), `standard` (Sonnet 4.6), `reasoning` (Opus 4.7);
|
||||
per-tier admin overrides via `admin:model:{tier}`.
|
||||
- **Structured output:** prefer tool use with forced `toolChoice`; inputs validated
|
||||
by `toolSchemaRegistry`. Text responses go through `parseStructuredText`.
|
||||
- **Caching:** wrap stable system text with `cachedSystem(...)`.
|
||||
- **Retry/limits:** `src/lib/llmRetry.js` — backoff + jitter on 408/425/429/5xx/529,
|
||||
honors `Retry-After`, rate limiters for bulk work.
|
||||
- **Telemetry:** every call logged to `llm_calls`.
|
||||
- **Simulation:** with `admin:use_simulation`, calls return stub output (no API hit).
|
||||
|
||||
Reference in New Issue
Block a user