43a71e2110
Adds a /topic-test route where learners can take a 5-question test on any eligible topic at any time. Questions come from a shared, admin-curated question_bank — no LLM calls on the user path. Points feed the existing leaderboard with a 10pt/topic/week cap (per ISO week) so the bank can't be farmed, and repeats from a thin bank yield 0 points. - New PB collections: question_bank, on_demand_attempts (+ migrations and setup-pb-collections.mjs entries). - db.js: un-deprecates getQuizBank/setQuizBank against question_bank so the existing admin TestManager panel becomes functional again; adds saveOnDemandAttempt, getOnDemandPointsThisWeek, getUserSeenQuestionIds, getAllOnDemandAttempts, isoWeekKey. - testService.js: getEligibleTopicsForOnDemand, startOnDemandTest (unseen- first draw), finishOnDemandTest (cap enforcement + leaderboard upsert). - New TopicTest page, route, and nav entry; weekly test flow untouched. - Leaderboard folds on-demand 100%s into perfect-score count and merges on-demand attempts into the recent activity feed. - Spec: docs/on-demand-tests-spec.md with ADRs and extension points. - Tests: 5 new vitest cases covering eligibility, draw fallback, scoring, partial cap, and exhausted cap (85/85 total). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
140 lines
6.1 KiB
Markdown
140 lines
6.1 KiB
Markdown
# On-demand topic tests
|
|
|
|
Lets a learner take a 5-question multiple-choice test on any topic they pick,
|
|
at any time. Questions come from a shared, admin-curated bank — the Anthropic
|
|
API is **not** called from the user path. Points flow to the same leaderboard
|
|
as the weekly test, with a per-(user, topic) weekly cap to prevent farming.
|
|
|
|
The weekly test (`/test`, `test_results`, `generateWeeklyQuiz`,
|
|
`curriculumService.getCurrentWeekContent`) is unchanged. This module sits
|
|
alongside it.
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
### Collections
|
|
|
|
- **`question_bank`** — one record per topic.
|
|
- `topic_id: text` (required), `questions: json[]` (array of question objects).
|
|
- Each question object: `{ id, question, topicLabel, options[4], correctIndex, explanation, difficulty }`.
|
|
- Admin-curated only — no automatic top-up from the user path.
|
|
|
|
- **`on_demand_attempts`** — one record per completed on-demand test.
|
|
- Keys: `user_id`, `topic_id`, plus `iso_week` (the "YYYY-Www" of completion)
|
|
for fast weekly-cap aggregation.
|
|
- Stores: `question_ids[]` (the bank IDs served — used to compute "unseen"
|
|
for future attempts), `score`, `total`, `percentage`, `time_used`,
|
|
`unseen_correct`, `points_earned`, `capped`, `breakdown`, `completed_at`.
|
|
|
|
### Service entry points (`src/lib/testService.js`)
|
|
|
|
| Function | Caller | Notes |
|
|
|---|---|---|
|
|
| `getEligibleTopicsForOnDemand()` | TopicTest picker | Filters `type !== 'fact'`, `learning_relevance !== 'exclude'`, **and** `bank.length >= 5`. |
|
|
| `startOnDemandTest(userId, topicId)` | TopicTest | Picks 5 questions, prefers unseen, tags each with `isUnseen: bool`. |
|
|
| `finishOnDemandTest(userId, topicId, topicLabel, quiz, answers, timeUsed)` | TopicTest | Computes score, enforces cap, writes `on_demand_attempts`, updates leaderboard. |
|
|
| `ON_DEMAND_TEST_SIZE` / `WEEKLY_POINT_CAP_PER_TOPIC` | constants | 5 and 10 respectively. |
|
|
|
|
### Draw strategy
|
|
|
|
```text
|
|
bank = db.getQuestionBank(topic_id)
|
|
seen = db.getUserSeenQuestionIds(user_id, topic_id) # union of all this user's
|
|
# prior question_ids for topic
|
|
picked = shuffle(bank - seen)[:5]
|
|
if len(picked) < 5:
|
|
picked += shuffle(bank ∩ seen)[:5 - len(picked)] # repeats, flagged isUnseen=false
|
|
```
|
|
|
|
A repeated question is shown to the learner with a `Practice (no points)` tag
|
|
and yields 0 points regardless of correctness.
|
|
|
|
### Anti-farm cap
|
|
|
|
```text
|
|
raw_points = unseen_correct * 2
|
|
already_earned = sum(points_earned) for (user, topic, current_iso_week)
|
|
points_earned = min(raw_points, max(0, 10 - already_earned))
|
|
capped = raw_points > points_earned
|
|
```
|
|
|
|
Cap is enforced at write-time inside `finishOnDemandTest`. The user can keep
|
|
playing the topic, but excess attempts within the same ISO week show a
|
|
"cap reached" notice and earn 0 points.
|
|
|
|
Points flow to the leaderboard via the existing
|
|
`db.upsertLeaderboardEntry(userId, name, +pointsEarned, +1)` — same path as
|
|
the weekly test, so the points column, badges, and Contributions feed pick up
|
|
on-demand activity without extra wiring.
|
|
|
|
### Heatmap policy
|
|
|
|
The Leaderboard's 26-week heatmap stays **weekly-test-only**. On-demand attempts
|
|
have no curriculum-week slot, so adding them would muddle the visual.
|
|
|
|
On-demand contributions surface in:
|
|
- The leaderboard's "points" column (commits) via `upsertLeaderboardEntry`.
|
|
- The "perfect" stat (any 100% counts, weekly *or* on-demand).
|
|
- The Recent Activity feed (merged with weekly-test entries, sorted by `created`).
|
|
|
|
---
|
|
|
|
## ADR-style decisions
|
|
|
|
- **Blob-per-topic over row-per-question.** Existing `TestManager.jsx` consumes
|
|
`bank.length` and operates on an array. Row-per-question would double the
|
|
migration/CRUD surface without measurable benefit for banks < 100 questions.
|
|
|
|
- **Un-deprecation over rename.** The legacy `getQuizBank` / `setQuizBank` /
|
|
`deleteQuestionFromBank` stubs in `db.js` were aliased to the new
|
|
`question_bank` collection rather than renamed. This keeps
|
|
`forceGenerateTopicQuestions` and the admin panel working with zero
|
|
UI changes.
|
|
|
|
- **Cap = 10 pt/topic/week, equal to a perfect weekly run.** Keeps weekly and
|
|
on-demand value-symmetric. Tunable via the `WEEKLY_POINT_CAP_PER_TOPIC`
|
|
export.
|
|
|
|
- **Practice-allowed over block-on-thin-bank.** When the bank is exhausted for
|
|
a user, the test still runs with repeats but yields 0 points for repeated
|
|
questions. Confirmed product preference: "oefenen mag altijd."
|
|
|
|
- **No `quiz_cache` for on-demand.** Each on-demand test draws fresh from the
|
|
bank — there's nothing to cache and per-user cache keys would only confuse
|
|
the seen/unseen logic.
|
|
|
|
---
|
|
|
|
## Aanroepconventies (für eine nachfolgende Agent)
|
|
|
|
Wat aanroepen waarvoor:
|
|
|
|
- **Een topic bevragen voor on-demand picker**: `getEligibleTopicsForOnDemand()`
|
|
→ `[{ id, label, description, theme, bankSize, ... }]`.
|
|
- **Een test starten**: `startOnDemandTest(userId, topicId)`
|
|
→ `{ questions: [{...q, isUnseen}], meta }`. Houd `isUnseen` per vraag bij
|
|
in de UI; geef `q` ongewijzigd door aan `finishOnDemandTest`.
|
|
- **Een test afsluiten**: `finishOnDemandTest(userId, topicId, topicLabel, quiz, answers, timeUsed)`
|
|
→ `{ score, total, percentage, pointsEarned, capped, rawPoints, breakdown, ... }`.
|
|
De service handelt cap, PB-write en leaderboard-update zelf af; geen extra
|
|
calls vanuit de UI.
|
|
- **Bank-CRUD vanuit admin**: blijft op de bestaande
|
|
`forceGenerateTopicQuestions` / `getTopicQuestionBank` / `deleteQuestion`
|
|
exports — die routeren intern naar de `question_bank` collectie.
|
|
|
|
---
|
|
|
|
## Uitbreidingspunten
|
|
|
|
1. **Cap-tuning via settings**: lift `WEEKLY_POINT_CAP_PER_TOPIC` naar de
|
|
`settings`-collectie en lees in `finishOnDemandTest`.
|
|
2. **Per-vraag stats**: voeg `times_served`, `times_correct` toe aan elk
|
|
bank-question object voor toekomstige difficulty-tuning / IRT.
|
|
3. **Dashboard surfacing**: een "Topic Test" tile op het Dashboard die de
|
|
topics met de meeste ongeziene vragen voor de huidige user toont
|
|
(`getEligibleTopicsForOnDemand` + `getUserSeenQuestionIds`).
|
|
4. **Heatmap fold-in**: als de heatmap ooit niet meer week-gebonden hoeft te
|
|
zijn, kunnen on-demand attempts erbij worden gemixt — vergt een redesign
|
|
van de visuele as.
|