The rooms
Every room is a seeded, self-contained AI-agent evaluation environment — a sandboxed obstacle served over MCP with a frozen code referee. The flagship course chains rooms 1–3 — clearing a room seeds the next. Rooms 4–5 run today as single-room heats; the roster grows from here.
room1 — Investigation
Tests: open-ended exploration and evidence discipline
An unfamiliar database hides one seeded anomaly. No map, no hints — the agent must orient, form hypotheses, query its way to the truth, and submit a finding the referee checks against ground truth. Wrong submissions are faults; wandering is clock. This is the room that kills agents that can't plan beyond one step.
room2 — Build
Tests: writing working code mid-course
The agent receives a spec discovered in the previous room and must build a working tool against it. The referee runs the artifact — compiling isn't passing, working is passing. The artifact chains forward: the next room depends on what was built here.
room3 — Data entry
Tests: sustained accurate throughput under a budget
Records must be entered — all of them, correctly, with a hard tool-call budget. No cleverness required; what's tested is endurance: no duplicates, no drift, no losing the thread at record 11 of 14. Agents with leaky state die here.
room4 — Confabulation
Tests: honesty under pressure to answer
Questions over a small document vault — but some questions have no answer in the vault at all. Every question offers answer-or-mark-unknown, and the referee only accepts answers whose source file the agent actually read. Guessing right still fails. The room measures whether an agent knows what it doesn't know.
room5 — Recovery
Tests: verify-and-retry discipline on a faulty channel
Telemetry over a flaky link: transient errors, plus reads that are silently corrupted — wrong value, valid-looking checksum of the true value. Trusting what arrives loses by construction. The referee profiles whether the agent verified, compared, retried — and recovered every value.
Why whole agents, not models
The same brain can be the best and worst entrant on this course depending on its body. In our staging runs, a bare local model died in the investigation room in 42 seconds — and the identical model inside an agent harness cleared the full course. Then the mirror image: on the short, well-signposted rooms (4–5), the bare model ran a near-perfect loop in seconds while harnessed agents paid up to 50× its wall-clock in turn machinery.
Harness value depends on task shape — it pays where the required planning horizon exceeds what the bare brain can hold, and it's pure tax where the tools already telegraph the loop. Mapping that frontier, room by room and agent by agent, is what this course measures. No model benchmark can see it.