AI Pipeline

Everything AI on Ediky flows through one module: functions/_ai-providers.js. Grading, tutoring, optimization review, aptitude coaching, SQL review, and this docs assistant all share the same provider resolution, the same JSON coercion, and the same failure posture. That's deliberate — reliability work done once benefits every feature.

The nine-provider fallback chain

diagram100%

rendering diagram…

drag to pan · ctrl / ⌘ + scroll to zoom

Every provider in the chain has a free tier; the chain exists because free tiers fail constantly — rate limits, deprecated model IDs, regional outages. A student mid-exam should never see that. A request walks the chain until one provider answers; only providers with a configured key participate.

What the shared module exports

Export	Role
DEFAULT_MODELS	per-provider default model id
KEY_FOR / MODEL_ENV_FOR	maps provider → its key/model env var names
providerHasKey(env, p)	participation check
REAL_PROVIDERS	the chain, in order
resolveProviderChain(env, preferred?)	final ordered chain for this deployment
coerceJson / firstBalanced	extract strict JSON from chatty model output
json(status, body)	uniform response helper

Configuration

Env var	Effect
<PROVIDER>_API_KEY (e.g. GEMINI_API_KEY, GROQ_API_KEY)	enables that provider
<PROVIDER>_MODEL	overrides its default model id
AI_PROVIDER	pins the first provider to try
AI_FALLBACKS	comma-list overriding chain order

No keys at all → the echo stub answers, prefixed [echo provider] No real model was called…. It keeps dev environments working and makes the degraded state impossible to mistake for a real answer.

Hard-won lessons encoded here

The chain wasn't speculative armor — it was built after cascading production failures: a deprecated Gemini model id, Groq rate-limit storms, and an invalid model string each took down grading at different times. Diagnosis got its own tooling: every AI route's GET is a health check reporting the resolved chain and picked model names (never key values).

coerceJson exists because free-tier models decorate JSON with markdown fences and commentary; firstBalanced pulls the first balanced {…} out of whatever came back. Prompts additionally demand JSON-only — belt and suspenders, and both have caught real failures.

Who calls it

Caller	Route	Mode
AskAiPopover (tutor)	ai-code-review	ask
AI Help on a problem	ai-code-review	help / chat
Optimization review	ai-code-review	optimize
CS concept grading	grade-concept	rubric grading
Aptitude coaching	evaluate-aptitude	explanation only
SQL review	grade-sql-practice	quality review
OA "Regrade with AI"	grade-concept	re-run after the fact
This docs site	api/ask-docs	retrieval-grounded Q&A

The Python authoring pipelines mirror the same abstraction (cs_eval/ai_service.py) so authoring-time and runtime grading agree on one contract — see CS Pipeline.

The docs assistant

The widget on this site is the same architecture with one addition: retrieval grounding.

1. scripts/build-ai-index.mjs runs before every build, indexing every docs page into /ai/docs-map.json (titles, headings, excerpts) + per-page text files.
2. POST /api/ask-docs keyword-ranks pages against the question (title > path > headings > excerpt, small boost for the page the reader is on), stuffs the top 4 into the prompt, and demands JSON { reply, sources }.
3. Returned sources are filtered against the real index — the assistant can cite only pages that exist.
4. Same chain, same echo stub, same health check on GET.

Grounding means the assistant answers from the documentation instead of hallucinating an architecture — and tells you where it read the answer.