learning-platform/docs/on-demand-tests-spec.md

On-demand topic tests

Lets a learner take a 5-question multiple-choice test on any topic they pick, at any time. Questions come from a shared, admin-curated bank — the Anthropic API is not called from the user path. Points flow to the same leaderboard as the weekly test, with a per-(user, topic) weekly cap to prevent farming.

The weekly test (/test, test_results, generateWeeklyQuiz, curriculumService.getCurrentWeekContent) is unchanged. This module sits alongside it.

---

Architecture

Collections

• question_bank — one record per topic.

  • topic_id: text (required), questions: json[] (array of question objects).
  • Each question object: { id, question, topicLabel, options[4], correctIndex, explanation, difficulty }.
  • Admin-curated only — no automatic top-up from the user path.

• on_demand_attempts — one record per completed on-demand test.

  • Keys: user_id, topic_id, plus iso_week (the "YYYY-Www" of completion) for fast weekly-cap aggregation.
  • Stores: question_ids[] (the bank IDs served — used to compute "unseen" for future attempts), score, total, percentage, time_used, unseen_correct, points_earned, capped, breakdown, completed_at.

Service entry points (src/lib/testService.js)

Function	Caller	Notes
getEligibleTopicsForOnDemand()	TopicTest picker	Filters type !== 'fact', learning_relevance !== 'exclude', and bank.length >= 5.
startOnDemandTest(userId, topicId)	TopicTest	Picks 5 questions, prefers unseen, tags each with isUnseen: bool.
finishOnDemandTest(userId, topicId, topicLabel, quiz, answers, timeUsed)	TopicTest	Computes score, enforces cap, writes on_demand_attempts, updates leaderboard.
ON_DEMAND_TEST_SIZE / WEEKLY_POINT_CAP_PER_TOPIC	constants	5 and 10 respectively.

Draw strategy

bank = db.getQuestionBank(topic_id)
seen = db.getUserSeenQuestionIds(user_id, topic_id)   # union of all this user's
                                                      # prior question_ids for topic
picked = shuffle(bank - seen)[:5]
if len(picked) < 5:
    picked += shuffle(bank ∩ seen)[:5 - len(picked)]  # repeats, flagged isUnseen=false

A repeated question is shown to the learner with a Practice (no points) tag and yields 0 points regardless of correctness.

Anti-farm cap

raw_points       = unseen_correct * 2
already_earned   = sum(points_earned) for (user, topic, current_iso_week)
points_earned    = min(raw_points, max(0, 10 - already_earned))
capped           = raw_points > points_earned

Cap is enforced at write-time inside finishOnDemandTest. The user can keep playing the topic, but excess attempts within the same ISO week show a "cap reached" notice and earn 0 points.

Points flow to the leaderboard via the existing db.upsertLeaderboardEntry(userId, name, +pointsEarned, +1) — same path as the weekly test, so the points column, badges, and Contributions feed pick up on-demand activity without extra wiring.

Heatmap policy

The Leaderboard's 26-week heatmap stays weekly-test-only. On-demand attempts have no curriculum-week slot, so adding them would muddle the visual.

On-demand contributions surface in:

• The leaderboard's "points" column (commits) via upsertLeaderboardEntry.
• The "perfect" stat (any 100% counts, weekly or on-demand).
• The Recent Activity feed (merged with weekly-test entries, sorted by created).

---

ADR-style decisions

• Blob-per-topic over row-per-question. Existing TestManager.jsx consumes bank.length and operates on an array. Row-per-question would double the migration/CRUD surface without measurable benefit for banks < 100 questions.

• Un-deprecation over rename. The legacy getQuizBank / setQuizBank / deleteQuestionFromBank stubs in db.js were aliased to the new question_bank collection rather than renamed. This keeps forceGenerateTopicQuestions and the admin panel working with zero UI changes.

• Cap = 10 pt/topic/week, equal to a perfect weekly run. Keeps weekly and on-demand value-symmetric. Tunable via the WEEKLY_POINT_CAP_PER_TOPIC export.

• Practice-allowed over block-on-thin-bank. When the bank is exhausted for a user, the test still runs with repeats but yields 0 points for repeated questions. Confirmed product preference: "oefenen mag altijd."

• No quiz_cache for on-demand. Each on-demand test draws fresh from the bank — there's nothing to cache and per-user cache keys would only confuse the seen/unseen logic.

---

Aanroepconventies (für eine nachfolgende Agent)

Wat aanroepen waarvoor:

• Een topic bevragen voor on-demand picker: getEligibleTopicsForOnDemand() → [{ id, label, description, theme, bankSize, ... }].
• Een test starten: startOnDemandTest(userId, topicId) → { questions: [{...q, isUnseen}], meta }. Houd isUnseen per vraag bij in de UI; geef q ongewijzigd door aan finishOnDemandTest.
• Een test afsluiten: finishOnDemandTest(userId, topicId, topicLabel, quiz, answers, timeUsed) → { score, total, percentage, pointsEarned, capped, rawPoints, breakdown, ... }. De service handelt cap, PB-write en leaderboard-update zelf af; geen extra calls vanuit de UI.
• Bank-CRUD vanuit admin: blijft op de bestaande forceGenerateTopicQuestions / getTopicQuestionBank / deleteQuestion exports — die routeren intern naar de question_bank collectie.

---

Uitbreidingspunten

1. Cap-tuning via settings: lift WEEKLY_POINT_CAP_PER_TOPIC naar de settings-collectie en lees in finishOnDemandTest.
2. Per-vraag stats: voeg times_served, times_correct toe aan elk bank-question object voor toekomstige difficulty-tuning / IRT.
3. Dashboard surfacing: een "Topic Test" tile op het Dashboard die de topics met de meeste ongeziene vragen voor de huidige user toont (getEligibleTopicsForOnDemand + getUserSeenQuestionIds).
4. Heatmap fold-in: als de heatmap ooit niet meer week-gebonden hoeft te zijn, kunnen on-demand attempts erbij worden gemixt — vergt een redesign van de visuele as.