MNIST A100 · difficulty 5 · October 12, 2026

The 2.58 s result is supported; warmups kept #119 off the leaderboard

Sung Jae Bae’s original three official records pass their numerical audit. The PR remains open because the additional maintainer reproduction completed only one of three runs: two later untimed warmups exceeded 90 seconds. The PR itself adds no leaderboard row.

2.581 sAuthor’s median · three complete passes
2.571 sOne complete fresh reviewer pass
1 / 3Fresh reviewer runs completed and passed

The recorded speed is corroborated

Swipe the table to see all measurements.

Evidence setPassesRanked timeMNIST error
Original author records3 / 3Median 2.580846 s1.848182% pooled
Fresh maintainer reproduction1 / 3Passing run 2.571362 sPassing run 1.788182%

Each author run passes D5’s 1.90% mean-error gate. Their 3/3 median remains separate from the reviewer set, which has no qualifying three-run median. These records use different draws and devices.

Reviewer runs 2 and 3 each saved seven timed calls before the next worker’s foreign warmup timed out: KMNIST in run 2, Fashion-MNIST in run 3. Those incomplete runs cannot receive an accuracy qualification. All original outcomes are retained.

Every worker starts cold; the later budget is half as long

First worker · fresh caches
180 s
Later workers · fresh caches
90 s
The public docstring says each warmup allows 180 seconds; the implementation gives only the first scoring warmup that budget.

Every scoring call creates a fresh process and private compiler caches. Warmup covers first-call setup, compilation, graph capture, fitting and readout. The ranked fit follows warmup in the same worker, reloading inputs and resetting learned state while reusing its compiled code and graphs.

The saved failure logs contain no phase timings or stack trace. Compilation or host variation is plausible, but the exact stalled phase—and whether the cause lies in the learner, compiler or infrastructure—is unresolved.

An additional review requirement held up the row

The maintainer applied an additional requirement that a separate fresh reviewer batch pass 3/3 before merging. The published instructions specify three official Modal runs and their median; they do not explicitly require another maintainer-generated 3/3 batch. This review hold explains the absent row.

Current status: #119 remains open. The recorded 2.58 s performance is supported, and the two reproduction warmup failures remain documented. No learner bytes, watchdog limits, merge status or leaderboard entry were changed by this explanation.