MNIST A100 · difficulty 5 · October 12, 2026
The 2.58 s result is supported; warmups kept #119 off the leaderboard
Sung Jae Bae’s original three official records pass their numerical audit. The PR remains open because the additional maintainer reproduction completed only one of three runs: two later untimed warmups exceeded 90 seconds. The PR itself adds no leaderboard row.
The recorded speed is corroborated
Swipe the table to see all measurements.
| Evidence set | Passes | Ranked time | MNIST error |
|---|---|---|---|
| Original author records | 3 / 3 | Median 2.580846 s | 1.848182% pooled |
| Fresh maintainer reproduction | 1 / 3 | Passing run 2.571362 s | Passing run 1.788182% |
Each author run passes D5’s 1.90% mean-error gate. Their 3/3 median remains separate from the reviewer set, which has no qualifying three-run median. These records use different draws and devices.
Reviewer runs 2 and 3 each saved seven timed calls before the next worker’s foreign warmup timed out: KMNIST in run 2, Fashion-MNIST in run 3. Those incomplete runs cannot receive an accuracy qualification. All original outcomes are retained.
Every worker starts cold; the later budget is half as long
Every scoring call creates a fresh process and private compiler caches. Warmup covers first-call setup, compilation, graph capture, fitting and readout. The ranked fit follows warmup in the same worker, reloading inputs and resetting learned state while reusing its compiled code and graphs.
The saved failure logs contain no phase timings or stack trace. Compilation or host variation is plausible, but the exact stalled phase—and whether the cause lies in the learner, compiler or infrastructure—is unresolved.
An additional review requirement held up the row
The maintainer applied an additional requirement that a separate fresh reviewer batch pass 3/3 before merging. The published instructions specify three official Modal runs and their median; they do not explicitly require another maintainer-generated 3/3 batch. This review hold explains the absent row.
Current status: #119 remains open. The recorded 2.58 s performance is supported, and the two reproduction warmup failures remain documented. No learner bytes, watchdog limits, merge status or leaderboard entry were changed by this explanation.