Aptitude Pipeline

The aptitude system's defining idea: correctness never depends on a model. The right answer is known exactly (an option index or a number), so grading is deterministic on both surfaces, and AI is layered on top purely as coaching — the "why," not the verdict. Grading stays correct and free even when every provider is down.

The two halves

diagram100%

rendering diagram…

drag to pan · ctrl / ⌘ + scroll to zoom

A deliberate sibling of the CS Pipeline: it reuses site_config.py, content_utils.py, and the AIService abstraction, so the two can never drift on "where is the website" or "how do we call a model." Same twice-implemented contract discipline — the coaching prompt + feedback JSON schema exist in Python and JS and must agree.

Running it

python3 tools/aptitude_pipeline.py            # process all drafts/aptitude/*.md
python3 tools/aptitude_pipeline.py --dry-run  # validate only, ship nothing
python3 tools/aptitude_pipeline.py --no-lint  # skip the final lint gate

Approved drafts are moved into the website's src/content/aptitude/, then the website's own lint-content.mjs runs as the final gate — the same gate npm run build uses.

What validation enforces

• kebab-case filename (matching the lint rules exactly)
• frontmatter required fields: title, difficulty (∈ Easy/Medium/Hard), topic, companies, tags
• questionType ∈ {mcq, numerical} cross-checked against evaluationMethod:
  • mcq → valid <!--MCQ--> block (≥2 options, in-range answer index)
  • numerical → valid <!--ANSWER--> block (finite value)
• <!--SOLUTION--> present and non-empty; <!--TESTS--> forbidden (no code cases in aptitude)
• duplicate-title and slug-collision checks against live website content (the shared plumbing that prevents a repeat of the 34-duplicate incident)

Unknown topic values are a warning, not an error — new topics auto-create a filter on the site, so the pipeline scales to new aptitude categories without a code change.

Runtime flow

AptitudeRunner.jsx grades instantly and locally via aptitudeEval.js, shows the verdict, then asynchronously fetches coaching from functions/evaluate-aptitude.js — which explains the known result rather than deciding it. The Function shares the nine-provider chain; this pipeline's rollout is actually what forced the chain's hardening, after cascading free-tier failures (deprecated models, rate-limit storms, invalid model IDs) surfaced during aptitude's launch.