Appearance
CS Fundamentals Pipeline
Generates and validates the theory bank — OS, CN, DBMS, OOP, System Design — with AI-graded free-text answers at runtime. It's deliberately separate from qgen: no Judge0, no test cases; the "verifier" here is a grading contract implemented twice and kept identical.
The two halves
rendering diagram…
drag to pan · ctrl / ⌘ + scroll to zoom
The same grading contract — prompt shape + JSON verdict schema — is implemented once in Python (tools/cs_eval/cs_eval.py, for offline/batch/CI grading) and once in JS (functions/grade-concept.js, for the live browser flow). A question grades the same in both places; if the contract changes, it changes in both or the tests fail.
The AIService swap point
Provider choice is isolated behind one abstraction on each side:
| Layer | File | Swap point |
|---|---|---|
| Python (authoring) | cs_eval/ai_service.py | get_ai_service() → _PROVIDERS map |
| JS (runtime) | grade-concept.js via _ai-providers.js | the nine-provider chain |
| JS (client) | src/lib/aiService.js | provider-agnostic — just calls the Function |
Adding a provider is one subclass + one map line (Python) or one entry in the shared chain (JS). Nothing else moves.
Question format
Frontmatter (id, subject, topic, difficulty, evaluationMethod: ai | mcq) plus <!--STATEMENT-->, <!--HINTS-->, <!--EXPECTED--> (model answer), <!--RUBRIC--> (what earns/loses marks). MCQ-typed questions skip AI entirely — graded deterministically client-side.
Quality history worth knowing
The bank didn't grow by bulk-dumping generations:
- A gap analysis across 241 questions mapped coverage holes per subject before any generation, so the 56-question OS/CN/DBMS batch filled measured gaps rather than duplicating strengths.
- A sweep found 33 files missing
<!--HINTS-->— fixed in bulk, and the pipeline validation now rejects the omission. - MCQ content bugs (wrong keys, ambiguous options) were corrected and turned into validation rules where checkable.
- Full authored sets to date: OS 41, CN 40, DBMS 22, OOP/System-Design 22 — on top of the pre-existing bank, totalling ~350 CS questions live.
The recurring pattern: every content bug becomes a pipeline check, so the class of bug dies rather than the instance.
Grading rules encoded in the prompt
The grader accepts correct answers worded differently from the model answer (it grades understanding, not string match), penalizes factual errors against the rubric, never quotes the full model answer verbatim in feedback (that would leak answers into the UI), and must respond in strict JSON — enforced by coerceJson on the way out (AI Pipeline).