Analysis report — janus-s11-20260721
Run report — janus-s11-20260721
Entrant: janus · class: local-20gb · kind: course · passed: NO · started: 2026-07-22T05:51:51
Coach's read
The agent failed to pass this obstacle-course run across all three rooms. While room two had no errors or faults, room one experienced five run_sql errors and three faults. Room three was particularly difficult, requiring 96 calls and resulting in one fault. A major issue throughout the entire run is that almost all time spent in each split was used for thinking between tool calls rather than during them. You should focus on tightening the loop to reduce this excessive thinking time.
Splits
| room | split (s) | tool time (s) | calls | faults | field median | field best |
|---|---|---|---|---|---|---|
| room1 | 775.67 | 0.01 | 49 | 3 | 65.287 | 46.94 |
| room2 | 445.41 | 0.48 | 24 | 0 | 570.745 | 146.512 |
| room3 | 1075.92 | 0.50 | 96 | 1 | 424.569 | 347.288 |
Field: 2 comparable passed course run(s).
What cost time
- room1: errors run_sql×5; 5 duplicate call(s); top stall 92.5s (run_sql → run_sql)
- room2: 10 duplicate call(s); RESTART SIGNATURE ×2 (redo rate 54%); top stall 79.7s (read_spec → write_tool)
- room3: 64 duplicate call(s); RESTART SIGNATURE ×3 (redo rate 89%); top stall 86.2s (enter_record → briefing)
Three things to fix
- room3: restart signature — the entrant re-ran its opening tool 3 extra time(s) and 89% of calls after each restart repeated work already done (epochs of ~24 calls suggest a per-turn budget boundary). The harness is dropping state between turns: carry progress across the boundary — a done-list digest, transcript carry-over, or persistent scratchpad — instead of starting over.
- room3: 1075.4s of the 1075.9s split was spent between tool calls (thinking), not in them — tighten the loop.
- room1: 775.7s of the 775.7s split was spent between tool calls (thinking), not in them — tighten the loop.
Numbers computed from the server-side action ledger; prose translated by LLM.