Code Execution (Judge0)

Why self-host

Every managed code-execution API evaluated either rate-limited too aggressively for the OA use case (many students submitting to the same problem within a tight exam window) or billed per execution, which doesn't work for a free student-facing product with no revenue. Judge0 CE is open-source and self-hostable, so it runs on an Oracle Cloud Free Tier VM (x86_64 Ubuntu) — always-free compute, no per-run cost, no external rate limit.

One networking lesson came free with the setup: Cloudflare Pages Functions refused to fetch the judge by raw IP (egress policy). The fix was a DNS A record pointing a hostname at the VM and calling that — documented in Troubleshooting because it looks like a Judge0 outage when you first hit it.

What "self-hosting a judge" actually involves

"Integrated Judge0" undersells the work. The judging layer (src/lib/judgeRunner.js + judge0.js + the run-code Function) handles:

• Language support beyond the defaults — PyPy3 required building a custom image on top of Judge0's base, since it isn't a stock runtime. The language registry spans C, C++17/20/23, Python, PyPy3, Java, JavaScript, TypeScript, Go, Rust, Kotlin.
• Per-language judging fairness — compiled C++ vs interpreted Python vs JVM startup are wildly different baselines. The layer applies per-language time multipliers and additive memory overhead for JVM/Node/Python so a correct O(n) Python solution isn't flagged TLE next to an equivalent C++ one. Limits are also clamped per language so a question's raw limits can't produce absurd effective budgets.
• Fail-fast evaluation — a submission stops at the first failing hidden test rather than running all of them, saving compute on obviously-wrong submissions (the VM is a shared free-tier resource, not an elastic cluster).
• Byte-safe I/O — source, stdin, and expected output round-trip through base64 to survive unicode and trailing-newline edge cases; a unified output normalizer (trim trailing whitespace per line, normalize final newline) is applied to both sides before comparison so verdicts don't hinge on invisible characters.
• Honest MLE detection — an earlier version mis-reported some runtime failures as Memory Limit Exceeded because of how Judge0 encodes statuses; verdict mapping now cross-checks status + signals before claiming MLE.
• Bounded retry on transient faults — a judge call that fails with a connection-level error retries a small fixed number of times with backoff, then surfaces an explicit "judge unavailable" verdict instead of hanging.
• Complexity estimation — complexityEstimate.js runs a log-log regression over per-test-case timing across increasing input sizes to estimate time complexity from empirical runtime, rather than static code analysis.

Concurrency: measured, not guessed

The Oracle free-tier VM is effectively single-core. Benchmarking batch submissions at concurrency 1/2/4 showed that parallel dispatch just made every run slower and noisier (timings smeared by contention, occasional spurious TLEs). The runner therefore dispatches strictly sequentially (concurrency = 1) — the rare case where the right pool size is one, and it was determined empirically rather than assumed.

The submission flow

diagram100%

rendering diagram…

drag to pan · ctrl / ⌘ + scroll to zoom

The browser never talks to Judge0 directly, the client never polls, and interactive runs ("Run" on visible cases) go through the same Function so there is exactly one judging implementation.

A ban with a story

Reference solutions may not include <bits/stdc++.h>. A production incident traced compile timeouts to that single header (it drags the entire standard library through the compiler on a slow VM). lint-content.mjs now fails the build if it appears — the include list is part of the judged artifact, not a style preference.

SQL is deliberately different

SQL Practice does not go through Judge0 at all. It runs sql.js (SQLite compiled to WASM) entirely client-side: no server-side query execution risk, no round-trip for a query that should feel instant, no sandboxing arbitrary SQL against a real database. The tradeoff — grading is "does the result set match" rather than a full judge verdict — is the right scope for a practice tool. See SQL Pipeline.