# Data model: employee learning platform

## Overview

Two storage systems:
- **PocketBase** — all structured relational data (SQLite under the hood)
- **Qdrant** — all vector embeddings for RAG retrieval

---

## PocketBase collections

### `source_documents`
Uploaded source files. Parent of all generated KB content.

| Field | Type | Notes |
|---|---|---|
| id | string | PocketBase auto |
| filename | string | original filename |
| file | file | PocketBase file storage |
| format | select | `pdf` `md` `txt` |
| status | select | `processing` `processed` `failed` |
| ingested_at | datetime | |
| chunk_count | number | total chunks extracted |
| created_by | relation → `users` | admin who uploaded |

---

### `themes`
Top-level content groupings. One Theme = one weekly session.

| Field | Type | Notes |
|---|---|---|
| id | string | PocketBase auto |
| title | string | |
| description | text | AI drafted, admin editable |
| status | select | `draft` `published` |
| source_documents | relation[] → `source_documents` | which docs contributed |
| approved_by | relation → `users` | admin who approved batch |
| approved_at | datetime | |
| created_at | datetime | |
| updated_at | datetime | |

---

### `topics`
Atomic knowledge units. Always belong to a Theme.

| Field | Type | Notes |
|---|---|---|
| id | string | PocketBase auto |
| theme | relation → `themes` | parent theme |
| title | string | |
| body | text (rich) | AI drafted, admin editable |
| difficulty | select | `introductory` `intermediate` `advanced` |
| complexity_weight | number | 1–5, used by curriculum generator |
| status | select | `draft` `published` |
| related_topics | relation[] → `topics` | lateral relationships |
| prerequisite_topics | relation[] → `topics` | must-complete-first |
| contrast_topics | relation[] → `topics` | deliberate opposites |
| key_terms | json | string[] — feeds glossary |
| qdrant_chunk_ids | json | string[] — references to embedded chunks |
| created_at | datetime | |
| updated_at | datetime | |

Relationship types (related / prerequisite / contrast) are stored via the three explicit relation fields rather than a generic relationship table. This keeps queries simple at this scale.

---

### `micro_learnings`
Generated content artifacts. One record per topic per type.

| Field | Type | Notes |
|---|---|---|
| id | string | PocketBase auto |
| topic | relation → `topics` | |
| type | select | see type enum below |
| content | json | structured output — schema varies per type |
| status | select | `queued` `generated` `published` `rejected` |
| generation_model | string | model version used |
| generated_at | datetime | |
| published_at | datetime | |
| updated_at | datetime | |

**Type enum:**
`concept_explainer` `scenario_quiz` `misconceptions` `how_to` `comparison_card` `reflection_prompt` `flashcard_set` `case_study` `glossary_anchor` `myth_vs_evidence`

**Content JSON schemas per type:**

```json
// concept_explainer
{
  "paragraphs": ["string", "string"],
  "example": "string"
}

// scenario_quiz
{
  "scenario": "string",
  "options": [
    { "label": "A", "text": "string", "correct": true, "explanation": "string" }
  ]
}

// misconceptions
{
  "items": [
    { "misconception": "string", "correction": "string" }
  ]
}

// how_to
{
  "steps": [
    { "number": 1, "instruction": "string" }
  ]
}

// comparison_card
{
  "subject_a": "string",
  "subject_b": "string",
  "dimensions": [
    { "label": "string", "a": "string", "b": "string" }
  ]
}

// reflection_prompt
{
  "prompt": "string",
  "model_answer": "string"
}

// flashcard_set
{
  "cards": [
    { "question": "string", "answer": "string" }
  ]
}

// case_study
{
  "scenario": "string",
  "questions": ["string"]
}

// glossary_anchor
{
  "term": "string",
  "definition": "string",
  "correct_use": "string",
  "misuse": "string"
}

// myth_vs_evidence
{
  "myth": "string",
  "evidence": "string",
  "sources": ["string"]
}
```

---

### `curriculum_versions`
Versioned 26-week schedule. New version created on each regeneration.

| Field | Type | Notes |
|---|---|---|
| id | string | PocketBase auto |
| version | number | increments on each regeneration |
| status | select | `draft` `active` `superseded` |
| generated_at | datetime | |
| approved_by | relation → `users` | admin who confirmed regeneration |
| approved_at | datetime | |
| generation_notes | text | why this version was created |

Only one version has status `active` at any time.

---

### `curriculum_weeks`
Individual week slots. Child of a curriculum version.

| Field | Type | Notes |
|---|---|---|
| id | string | PocketBase auto |
| curriculum_version | relation → `curriculum_versions` | |
| week_number | number | 1–26 |
| theme | relation → `themes` | |
| topics | relation[] → `topics` | ordered subset of theme topics |
| topic_order | json | number[] — explicit ordering |
| estimated_duration_minutes | number | AI estimate |
| admin_notes | text | freeform admin annotation |

---

### `employee_curriculum_state`
Tracks each employee's position in the curriculum. One record per employee.

| Field | Type | Notes |
|---|---|---|
| id | string | PocketBase auto |
| user | relation → `users` | |
| current_cycle | number | starts at 1, increments on loop |
| current_week | number | 1–26 |
| start_date | datetime | rolling start |
| active_version | relation → `curriculum_versions` | version employee is on |
| updated_at | datetime | |

When curriculum regenerates: `active_version` updates for all employees whose `current_week` is less than the first regenerated week.

---

### `session_completions`
Immutable completion records. One record per employee per topic per type.

| Field | Type | Notes |
|---|---|---|
| id | string | PocketBase auto |
| user | relation → `users` | |
| topic | relation → `topics` | |
| micro_learning | relation → `micro_learnings` | specific type completed |
| week_number | number | curriculum week at time of completion |
| cycle | number | which cycle |
| completed_at | datetime | |

Records are never updated or deleted. This is the canonical history.

---

### `gamification_profiles`
One record per employee. Updated by progress service on each completion.

| Field | Type | Notes |
|---|---|---|
| id | string | PocketBase auto |
| user | relation → `users` | |
| total_commits | number | cumulative XP |
| current_level | select | `intern` `junior` `medior` `senior` `staff` `principal` |
| current_streak_weeks | number | consecutive weeks with ≥1 completion |
| longest_streak_weeks | number | all-time high |
| types_used | json | string[] — which of 10 types used at least once |
| last_active_week | number | used to detect streak breaks |
| updated_at | datetime | |

---

### `badges`
Badge definitions. Seeded at startup, not user-generated.

| Field | Type | Notes |
|---|---|---|
| id | string | PocketBase auto |
| key | string | unique slug e.g. `governance_nerd` |
| tier | select | `bronze` `silver` `gold` `legendary` `content` |
| label | string | display name |
| description | string | award condition description |
| icon | string | emoji or icon key |

---

### `employee_badges`
Junction: which employees have earned which badges.

| Field | Type | Notes |
|---|---|---|
| id | string | PocketBase auto |
| user | relation → `users` | |
| badge | relation → `badges` | |
| earned_at | datetime | |
| cycle | number | which cycle it was earned in |

---

### `milestone_cards`
Public milestone events at weeks 13 and 26.

| Field | Type | Notes |
|---|---|---|
| id | string | PocketBase auto |
| user | relation → `users` | |
| cycle | number | |
| week | number | 13 or 26 |
| total_commits | number | snapshot at time of milestone |
| streak_weeks | number | snapshot |
| badge_keys | json | string[] — badges held at milestone |
| created_at | datetime | public feed ordered by this |

---

## PocketBase users collection (extended)

Standard PocketBase `users` collection with additional fields:

| Field | Type | Notes |
|---|---|---|
| role | select | `admin` `employee` |
| display_name | string | used in gamification feed |
| avatar | file | optional |

---

## Qdrant collections

### `source_chunks`
Embeddings of raw source document chunks. Primary retrieval target for R42.

| Field | Type | Notes |
|---|---|---|
| id | string | UUID |
| vector | float[] | 1536 dimensions (text-embedding-3-small) |
| source_document_id | string | reference to PocketBase |
| chunk_index | number | position within document |
| text | string | raw chunk text |
| theme_id | string | assigned theme (post-extraction) |
| topic_id | string | assigned topic (post-extraction, nullable) |
| format | string | pdf / md / txt |

### `topic_summaries`
Embeddings of AI-generated topic body text. Secondary retrieval target.

| Field | Type | Notes |
|---|---|---|
| id | string | UUID |
| vector | float[] | 1536 dimensions |
| topic_id | string | reference to PocketBase |
| theme_id | string | |
| title | string | for display in R42 citation |
| text | string | full topic body |

---

## Retrieval strategy for R42

R42 queries both Qdrant collections and merges results:

```
Employee query
  ↓
Embed query → text-embedding-3-small
  ↓
Qdrant search: source_chunks (top 5) + topic_summaries (top 3)
  ↓
Filter: boost chunks from employee's current week theme
  ↓
Merge + deduplicate by topic_id
  ↓
Top-K context injected into Haiku 4.5 prompt
  ↓
Response includes: answer + cited topic title(s)
```

Source chunks are weighted higher than topic summaries to keep R42 grounded in original source material rather than AI-generated abstractions.

---

## Indexes and query patterns

Critical query patterns the data model must support efficiently:

| Query | Collection | Index |
|---|---|---|
| All published topics for a theme | topics | theme + status |
| All micro learnings for a topic | micro_learnings | topic + status |
| Employee's current week | employee_curriculum_state | user |
| Weeks for a curriculum version | curriculum_weeks | curriculum_version + week_number |
| Employee completion history | session_completions | user + cycle |
| Public leaderboard | gamification_profiles | total_commits + streak |
| Milestone feed | milestone_cards | created_at DESC |
| Badges earned by employee | employee_badges | user |

PocketBase creates indexes on relation fields by default. Composite indexes on `status` fields should be added manually where query frequency warrants it.

---

## Data flow summary

```
source_documents
  └── (ingestion service)
        └── qdrant: source_chunks
        └── themes (draft)
              └── topics (draft)
                    └── (approval)
                          └── topics (published)
                                └── qdrant: topic_summaries
                                └── micro_learnings (queued → published)
                                └── (curriculum service)
                                      └── curriculum_versions
                                            └── curriculum_weeks
                                                  └── (employee progress)
                                                        └── session_completions
                                                        └── gamification_profiles
                                                        └── employee_badges
                                                        └── milestone_cards
```