Code Execution & DSA Engine: Judge0 & Code Execution

Part 1 of 5 in the Code Execution & DSA Engine series.

Task 1 documented the infrastructure underneath Ediky, and Task 2 documented the UI layer students touch every day. This series covers what happens the moment a student presses Run or Submit — how a block of source code becomes a verdict. Code Execution (Judge0) is the canonical page for why Judge0 is self-hosted and stays authoritative on that; this page goes one layer deeper, into the client code that actually talks to it.

One judge, two entry points

src/lib/judge0.js exposes a single runCode() for one program against one stdin, and two batch wrappers built on top of it: runTests() (sequential) and runTestsParallel() (a small worker pool). Both the "Run" button (visible sample cases) and a graded Submission funnel through the same runCode() — the only difference is which wrapper calls it and with what flags.

// Simplified real shape of runTestsParallel (judge0.js)
async function runTestsParallel({ language, source, tests, concurrency = 1, failFast = false }) {
  let next = 0, stop = false;
  async function worker() {
    while (!stop) {
      const i = next++;
      if (i >= tests.length) return;
      const result = await runCode({ language, source, stdin: tests[i].stdin });
      const judged = judgeCase(result, tests[i].expected);
      if (failFast && !judged.pass) stop = true; // submissions only
    }
  }
  await Promise.all(Array.from({ length: concurrency }, worker));
}

failFast is what separates the two callers: "Run" always reports every visible case so a student can see all of them fail or pass together; a Submission stops scheduling new cases after the first failure, since the shared Oracle VM is a single free-tier box, not an elastic cluster. Benchmarking showed the box is effectively single-core, so production runs at concurrency = 1 regardless — parallel dispatch just added contention and spurious timeouts.

The submission path

diagram100%

rendering diagram…

drag to pan · ctrl / ⌘ + scroll to zoom

Graded submissions add a queue in front of this: the write lands in Firestore, a cron-driven judging Worker claims it, and only then does this same runCode() path run — see Code Execution (Judge0) for that outer loop. This page's diagram is the inner loop shared by both paths.

Turning a Judge0 status into a verdict

Judge0 CE reports a numeric status id, not a verdict Ediky's UI understands directly. verdictFromStatusId() maps it, and a second pass reclassifies memory-adjacent runtime errors as MLE — Judge0 has no dedicated "Memory Limit Exceeded" status of its own for every runtime, so a crash whose peak memory lands within 3% of the enforced limit is treated as MLE instead of a generic runtime error.

Judge0 status id	Meaning	Ediky verdict
3	Accepted	ok (pending output comparison)
4	Wrong Answer	wrong_answer
5	Time Limit Exceeded	tle
6	Compilation Error	compile_error
7–12	Runtime Error (SIGSEGV, SIGXFSZ, SIGFPE, abort, NZEC, other)	runtime_error (or mle if peak memory ≥ 97% of the enforced limit)
13–14	Internal / Exec Format Error	internal_error

// Simplified real shape of shapeResult's MLE reclassification (judge0.js)
function shapeResult(data, memoryLimitMb) {
  let verdict = verdictFromStatusId(data.status.id);
  const memMb = toMb(data.memory);
  if (verdict === "runtime_error" && memMb >= memoryLimitMb * 0.97) {
    verdict = "mle";
  }
  return { verdict, timeMs: toMs(data.time), memoryMb: memMb };
}

Note that status id 3 is not itself a pass — it means the program ran and produced output. Whether that output is correct is a separate step, judgeCase(), which compares normalized stdout against the stored expected string. That separation matters: a program can be "Accepted" by Judge0 (ran cleanly, no crash, no timeout) and still be Wrong Answer by Ediky's own comparison.

One comparison function, not two

An earlier version of the app normalized output differently on the Run path than on the Submit path — Run additionally stripped leading whitespace, Submit didn't — so a program with a stray leading blank line could pass locally and fail on grading. normalizeOutput() is now the single function both paths call: it strips trailing whitespace per line and drops leading/trailing blank lines, but leaves internal spacing and case untouched, matching standard competitive-judge behavior.

The request deadline

requestTimeoutMs() derives a hard client-side abort deadline from the problem's actually enforced limits rather than a flat constant, so a legitimately slow-but-progressing case is never preempted early: CPU time is clamped to 15s server-side, wall time is cpu + 5s capped at 30s, compilation gets its own 15s budget, and the worst case (compile once + run + one TLE retry) sets the deadline — clamped between a 25s floor and a 90s ceiling. Without this, a hung request to the shared Judge0 box could leave a submission stuck in "Running" forever, since the judging Worker's own heartbeat keeps writing on its own schedule regardless of whether the case itself is making progress.

Self-assessment

Word count for this page: 887 words.

---

→ Next: Test Case Generation & Verification