rve/learning-platform
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add comprehensive documentation for key organizational aspects
All checks were successful
On Push to Main / test (push) Successful in 1m33s
Details
On Push to Main / publish (push) Successful in 1m31s
Details
On Push to Main / deploy-dev (push) Successful in 2m3s
Details

Browse Source 
- Introduced "Pension Scheme & Benefits" detailing secondary employment benefits and pension specifics.
- Created "Roles & Accountabilities" outlining the Holacracy role structure and responsibilities within Respellion.
- Added "Security" section covering GDPR compliance and workplace safety protocols.
- Established "Spending and Contracting" policy detailing expense categories and submission processes.
- Documented "Who We Are" to define Respellion's identity, services, and operational model under Holacracy and ISO 9001.
This commit is contained in:
RaymondVerhoef
2026-05-27 08:24:56 +02:00
parent 7066f881f9
commit 07af2783dc
26 changed files with 2631 additions and 4598 deletions
Download Patch File Download Diff File Expand all files Collapse all files

326
docs/architecture.md
View File

@@ -1,279 +1,179 @@
# Architecture: employee learning platform
# Architecture: Respellion Learning Platform
## Overview
A mobile-first progressive web application that provides employees with a structured knowledge library, a 26-week perpetual learning curriculum, and an AI-powered assistant (R42). The knowledge base is the single source of truth for all content, micro learnings, curriculum scheduling, and chat retrieval.
A mobile-first single-page web application that gives employees a structured
knowledge library, a 26-week per-user learning curriculum, and an AI assistant
(R42). The **knowledge graph** stored in PocketBase is the single source of truth
for all content, micro-learnings, curriculum scheduling, and chat retrieval.
Unlike the original design (a Next.js multi-service system with Qdrant), the
shipped platform is a **React/Vite SPA talking directly to PocketBase**, with the
Anthropic API reached through a thin reverse proxy. All application logic — AI
orchestration, retrieval, generation, scoring — runs in the browser.
```
Browser (React SPA)
├── PocketBase SDK ───────────────► PocketBase (SQLite): all structured data + auth + files
└── callLLM ──► /api/anthropic ──► Anthropic API (key injected by the proxy)
(Caddy in prod / Vite proxy in dev)
```
---
## System domains
### Admin app
Browser-based interface for content administrators.
Responsibilities:
- Upload source documents (PDF, MD, TXT)
- Review and approve AI-generated Theme batches
- Edit and finetune AI-generated curriculum
- Confirm curriculum regeneration after KB updates
- Monitor ingestion and generation job status
### Employee app
Mobile-first PWA accessible on all devices.
Responsibilities:
- Weekly session delivery (Theme + Topics + micro learning type selection)
- Knowledge library (browse all published Topics)
- Gamification profile (heatmap, badges, streak, leaderboard)
- R42 chatbot (available on every screen)
### Backend services
Six discrete services, each with a single responsibility.
| Service | Responsibility |
|---|---|
| Ingestion service | Document upload → chunk → extract KB structure |
| Generation service | Topics → 10 micro learning types (structured JSON) |
| Curriculum service | KB graph → 26-week schedule, versioning, regeneration |
| Embedding service | Chunks + topic summaries → Qdrant |
| Chat service (R42) | Query → vector retrieval → grounded response |
| Progress service | Completions → XP → badges → streak |
---
## Deployment topology
## Runtime topology
```
repo/
├── .github/workflows/ ← pipeline (frozen)
├── docker-compose.yml ← infrastructure (frozen)
├── Dockerfile ← updated once to point at /app
├── ansible/ ← provisioning (frozen)
├── legacy/ ← original prototype (read-only reference)
└── app/
├── frontend/ ← Next.js PWA (admin + employee)
└── services/
├── ingestion/
├── generation/
├── curriculum/
├── embedding/
├── chat/
└── progress/
┌─────────────────────────────┐ ┌──────────────────────────┐
│ Caddy container │ │ PocketBase container │
│ - serves built SPA (/srv) │ /api/*│ - SQLite data │
│ - /api/anthropic/* → Claude │───────►│ - auth (team_members) │
│ - /api/*, /_/* → PocketBase │ /_/* │ - file storage │
│ - injects ANTHROPIC_API_KEY │ │ - migrations (pb_migrations)
└─────────────────────────────┘ └──────────────────────────┘
```
In local dev, `vite.config.js` replaces Caddy: it proxies `/api/anthropic` to
`https://api.anthropic.com` and injects `ANTHROPIC_API_KEY`. PocketBase runs
directly (`./pocketbase.exe serve`).
---
## Tech stack
| Layer | Technology | Rationale |
|---|---|---|
| Frontend | Next.js 14, TypeScript, Tailwind CSS | PWA support, single codebase for admin + employee |
| Backend state | PocketBase | Auth, file storage, admin UI, SQLite — no infra overhead |
| Vector store | Qdrant (Docker) | RAG retrieval, runs as single container |
| AI generation | Claude Sonnet 4 via Anthropic API | Structured JSON output, long-form drafting, graph reasoning |
| AI chat (R42) | Claude Haiku 4.5 via Anthropic API | Low latency, cost-effective, grounded by retrieval layer |
| Embeddings | OpenAI text-embedding-3-small | Cost-effective, high quality at this scale |
| Auth | PocketBase built-in | Role-based: admin / employee |
| Frontend | React 19 + Vite 8, React Router 7 | Fast SPA, single codebase for admin + employee |
| Styling | CSS variables + Tailwind v4 | Premium design system; Tailwind mapped to variables |
| Backend state | PocketBase (SQLite) | Auth, file storage, admin UI — no infra overhead |
| Retrieval | Local TF-IDF (`src/lib/retrieval.js`) | Grounds R42 with zero external vector infra |
| AI | Claude via Anthropic API (proxied) | Structured tool output, long-form drafting, chat |
| Auth | PocketBase `team_members` + PIN | Lightweight internal auth, role = admin / (employee) |
| Infra | Docker + Caddy, Ansible (`infra/`) | Containerized deploy to dev/prod |
There is **no Qdrant, no OpenAI/embeddings service, and no separate Node backend.**
---
## AI model responsibilities
| Task | Model |
|---|---|
| Document → KB structure extraction | Claude Sonnet 4 |
| Topic body drafting | Claude Sonnet 4 |
| Micro learning generation (all 10 types) | Claude Sonnet 4 |
| Curriculum generation + versioning | Claude Sonnet 4 |
| R42 chat responses | Claude Haiku 4.5 |
| Embeddings | text-embedding-3-small |
`callLLM` (`src/lib/llm.js`) selects a Claude model by **tier**:
| Tier | Model | Used for |
|---|---|---|
| `fast` | `claude-haiku-4-5-20251001` | R42 chat, weekly quiz batch, flashcard sets |
| `standard` | `claude-sonnet-4-6` | KB extraction, article/slides/infographic, micro-learnings, curriculum generation |
| `reasoning` | `claude-opus-4-7` | reserved for heavier reasoning tasks |
Admins can override the model string per tier from the Settings tab.
---
## Document ingestion pipeline
## Knowledge ingestion pipeline
```
Admin uploads file (PDF / MD / TXT)
↓
Format detection → text extraction
MD: split on headings → preserve hierarchy
PDF: pdfplumber → page + paragraph detection
TXT: sliding window chunking with overlap
↓
Chunk cleaning (strip headers/footers/artefacts)
↓
Claude Sonnet 4 reads chunks → extracts:
- candidate Themes
- candidate Topics per Theme
- Topic→Topic relationships (related, prerequisite, contrast)
- key terms for glossary
↓
Draft KB written to PocketBase (status: draft)
↓
Embedding service: embed source chunks → write to Qdrant
↓
Admin reviews Theme batch → approves / edits / rejects
↓
On approval: Topics published, micro learning generation queued
↓
Curriculum regeneration notification queued for admin
Admin uploads .txt / .md (≤5 MB) in the Sources tab
↓
extractionPipeline.js chunks the text (~8000 chars, 800 overlap)
↓
Per chunk: callLLM (standard tier) with the emit_knowledge_graph tool
→ topics (id, label, type, description, learning_relevance)
→ relations (source, target, type)
↓
Results merged into the `topics` and `relations` collections
(topic id de-dup; relevance_locked topics keep their relevance)
↓
Source status tracked in `sources` (processing → completed / failed / cancelled)
```
Note: embeddings are generated from **source chunks**, not only from AI-generated topic summaries. R42 retrieves from grounded source material.
There are no embeddings. Retrieval for R42 is computed on the fly with TF-IDF
over `topics` (`label + description`).
MD source files are the preferred format for admins — heading structure maps directly to Theme → Topic hierarchy and improves extraction quality.
See `docs/ingestion-spec.md`.
---
## Curriculum lifecycle
## Content generation
Two generators, both via `callLLM` with forced tool use and Zod-validated output:
- **Long-form content** (`learningService.js`) → `content` collection. Three types
generated on demand and shallow-merged: `article`, `slides`, `infographic`.
- **Micro-learnings** (`microLearningService.js`) → `micro_learnings` collection.
Three types: `concept_explainer`, `scenario_quiz`, `flashcard_set`.
See `docs/generation-spec.md`.
---
## Curriculum lifecycle (per-user)
### Generation
Input: all published Themes, Topics, relationship graph, complexity weights
Process: cluster by Theme → sequence pedagogically (prerequisites first, complexity gradient) → distribute across 26 weeks → ensure full KB coverage
Output: versioned 26-week draft schedule
Input: published topics grouped by `theme`, ordered by `complexity_weight`.
`curriculumService.generateCurriculumDraft()` asks Claude for a 26-week schedule
via `emit_curriculum_schedule`, validates it, and stores a `curriculum_versions`
row (`status='draft'`). Admin previews and confirms → `active`. Only one active
version exists; the prior active becomes `superseded`.
### Perpetual cycling
The curriculum runs continuously. After week 26, the employee begins cycle 2 on the latest curriculum version.
### Per-user cycling
The curriculum is **not** anchored to the calendar. Each employee enrolls when
they choose (first-login onboarding), which sets `team_members.curriculum_started_at`.
Their position is derived:
Second and subsequent cycles are not identical to cycle 1:
- Theme sequence is varied
- Recommended micro learning types surface types the employee has not yet used
- Topics with low engagement in prior cycles receive increased coverage
```
personalWeek = floor(daysSinceStart / 7) + 1 // absolute counter, ≥1
curriculumWeek = ((personalWeek - 1) % 26) + 1 // 1..26 slot
cycle = floor((personalWeek - 1) / 26) + 1 // 1, 2, 3, ...
```
### Versioning rules
After week 26 the cycle restarts at week 1 with the **same** content.
| Event | Action |
|---|---|
| New source doc published to KB | Regenerate curriculum from week N+1 for all active employees |
| Topic body edited | Micro learnings regenerated; curriculum unaffected |
| Theme batch approved | Regeneration queued; admin confirms before it applies |
Completed weeks are immutable. Regeneration only affects future unstarted weeks.
### Admin regeneration flow
Admin receives notification: "N new topics added. Regenerate curriculum? This will update unstarted weeks for all active employees."
Admin can preview the proposed new schedule before confirming.
See `docs/curriculum-spec.md`.
---
## Weekly session flow (employee)
```
Week N opens
Enroll (first login) → curriculum_started_at set
↓
Employee sees assigned Theme + Topics for the week
Dashboard shows current cycle / week / assigned topic
↓
Per Topic: employee selects micro learning type
(all published types for that topic are available)
Learning Station: complete ≥1 micro-learning for the week's topic(s)
↓
Employee completes one or more types per topic
Weekly Test: 5 AI-generated questions → +2 points per correct answer
↓
Completion recorded → XP awarded → badges evaluated
↓
Progress visible on public leaderboard and activity feed
Leaderboard updates; badges evaluated at render time
```
Sessions support multiple micro learning types per topic in a single session.
---
## Micro learning types
All 10 types are generated by Claude Sonnet 4 as structured JSON, stored in PocketBase, and rendered by the frontend. One or more types may be published per topic.
| # | Type | Format |
|---|---|---|
| 1 | Concept explainer | 2–3 paragraphs + example |
| 2 | Scenario quiz | situation + 3–4 MCQ options + explained answers |
| 3 | Common misconceptions | 3–5 false beliefs + corrections |
| 4 | Step-by-step how-to | numbered procedure |
| 5 | Comparison card | side-by-side on 4–6 dimensions |
| 6 | Reflection prompt | open question + model answer |
| 7 | Spaced repetition flashcards | 5–10 Q&A pairs |
| 8 | Case study mini-analysis | 150–200 word scenario + guiding questions |
| 9 | Glossary anchor | term + definition + correct use + misuse |
| 10 | Myth vs. evidence | false claim + evidence-based rebuttal |
---
## R42 — chat service design
R42 is a functional KB-grounded assistant available on every screen in the employee app.
R42 is a KB-grounded assistant on every screen (`src/components/chat/`).
Behaviour:
- Stateless per session (no memory between conversations)
- Retrieves relevant chunks from Qdrant using the employee's query
- Knows the employee's current curriculum week → retrieval is context-weighted
- Cites source topic in every response ("based on the **Holacratic roles** topic")
- Explicitly refuses to answer outside KB scope rather than hallucinating
- Scope: internal KB only
- Persists the conversation per user in `localStorage` (`chat:thread:{userId}`, cap 50; ~12 turns sent to the API).
- Builds context with the TF-IDF index (top-K topics + verbatim mentions), injects related relations and limited deep content.
- Can propose a `propose_graph_delta` (≤3 topics, ≤5 relations). Admins apply directly; non-admins queue a suggestion for admin approval.
- Hidden during quizzes (the `quiz:active:{userId}` integrity rule).
Implementation:
- Employee query → embed → Qdrant nearest-neighbour retrieval → top-K chunks
- Chunks + employee context injected into Haiku 4.5 prompt
- Response streamed to frontend
UI: floating button bottom-right, unobtrusive on mobile.
See `docs/r42-spec.md`.
---
## Gamification system
## Gamification
Inspired by the visual language of GitHub, Stack Overflow, and Duolingo. Mechanics use developer-native terminology.
- Points: +2 per correct quiz answer, stored in `leaderboard`.
- Badges (render-time): First Steps (1 test), Veteran (5 tests), Perfectionist (100% score).
- Leaderboard excludes admins.
### XP unit: commits
Every completed topic earns commits. Quantity varies by micro learning type complexity.
### Levels
`Intern → Junior → Medior → Senior → Staff → Principal`
Based on cumulative commits across all cycles.
### Streak
Counted in consecutive weeks, not days. Resets if a week is skipped entirely.
### Activity heatmap
GitHub-style contribution graph spanning the full 26-week cycle. Cell darkness = number of types completed that week.
### Badges
| Tier | Condition |
|---|---|
| Bronze | Complete any session |
| Silver | 5 sessions completed, 5 different types used |
| Gold | 13 sessions without skipping a week |
| Legendary | All 26 sessions, all 10 types used at least once |
Named content badges (examples):
- `governance nerd` — all holacratic structure topics completed
- `process architect` — all internal process topics completed
- `deep reader` — case study type used 5+ times
### Milestone cards (public)
At weeks 13 and 26, a public card is posted to the shared activity feed:
```
🚀 [Name] shipped the full curriculum
26 weeks · [N] commits · [badges]
Longest streak: [N] weeks
```
Language: shipping vocabulary, not school vocabulary.
### Leaderboard
Not ranked 1–N by score. Displays multiple dimensions:
| Employee | Commits | Streak | Types used | Badges |
|---|---|---|---|---|
Multiple paths to visibility. No single metric determines standing.
See `docs/gamification-spec.md`.
---
## Security and privacy
- Auth: PocketBase role-based (admin / employee)
- Gamification data (commits, badges, streak) is public to all employees
- Session completion data (which topic, which type, when) is public
- Source documents are admin-only
- No PII beyond display name stored in gamification context
- R42 is stateless — no chat history persisted
- Auth: PocketBase `team_members` with PIN; role `admin` unlocks the Admin panel.
- The Anthropic API key never reaches the client — it is injected by Caddy / the Vite proxy.
- R42 conversations are stored client-side per user; no server-side chat history.
- Source documents and the knowledge graph are managed by admins.