The Verification Layer That Wasn't
OpenAI's central argument for skipping peer review is that Lean formalization - machine-checkable proof code - makes manual verification 'more practical' than traditional refereeing [1]. In practice that substitute is incomplete and, in at least one case, misleading. As of October 7, only about 42% of the 719 surviving top-line results - 300 manuscripts - had a machine-checked Lean proof attached, meaning the majority of claims rest on OpenAI's own account of its internal model's reasoning rather than independent confirmation [2]. Worse, a companion analysis cross-checking one of the Lean-formalized headline results - the catalogue's Navier-Stokes manuscript - found the certificate didn't actually match the written argument: one estimate in the formalization required four derivatives where the natural-language proof claimed five, and a pressure-flux bound was proved via a different argument than the one described in prose. A Lean file that compiles cleanly, in other words, doesn't guarantee the English proof it's supposed to certify is the proof that was actually checked - a distinction the release's framing elides. Gary Marcus's critique lands on the same gap from a different direction: without disclosure of the model's architecture, failure rate, or generation procedure, there is no way to judge whether the 42% that did verify is representative of the 58% that didn't. 'This would never pass peer review,' he wrote, arguing that transparency about method, not volume of output, is what separates a scientific claim from a demonstration [3].


