OpenAI unveils Jalapeño inference chip, claims it beats Nvidia

The headline numbers come from SemiAnalysis's public InferenceX suite, but the fine print matters. As the firm itself notes: "All numbers are provided to us by OpenAI. We verified the InferenceX runs in person in the lab, but we did not run the full suite of InferenceX benchmarks" ^[1]. The comparison also excludes Nvidia's newest Vera Rubin systems, which recently began shipping ^[2]- meaning Jalapeño was measured against the outgoing GB200/GB300 generation rather than Nvidia's current answer. No multi-turn or long-context (AgentX) benchmarks were run, only single-turn 8k/1k tests ^[1]. Online technical discussion has also raised a further nuance worth flagging as unverified: that the Jalapeño runs reportedly skipped speculative decoding, a technique commonly used in Rubin-class comparisons, which some argue could flatter Jalapeño's relative numbers - a claim from community discussion, not confirmed in the underlying reporting, so it should be read as an open question rather than a settled fact.

OpenAI's head of hardware, Richard Ho, frames the project simply: "Jalapeño can serve more AI work per unit of power, while also returning responses more quickly" ^[3]. The architecture behind that claim is deliberate - OpenAI says it built Jalapeño "to minimize data movement and communication delays" ^[4], keeping model state and KV cache close via a large SRAM cache paired with 216 GB of HBM4 memory running at 15.4 TB/s per chip ^[5]. That design targets the prefill and communication phases that typically bottleneck inference at scale - the layer OpenAI can only optimize by controlling silicon, networking, and models together, rather than buying commodity GPU systems off the shelf. The project also moved unusually fast: from architecture concept in late 2024 to RTL freeze and tapeout in 2025, with Codex and then ChatGPT running on the chip in early 2026 ^[5], using OpenAI's own AI models paired with the XLS hardware description language to accelerate development ^[5]- a roughly nine-month RTL-to-tapeout cycle that is fast even by custom-silicon standards ^[6]- a pace that drew attention beyond OpenAI's own materials, with online commentary singling it out as the single most striking detail in the whole announcement. In a Bloomberg Tech interview, Ho put the stakes in blunter terms: achieving high throughput and low latency at the same time, on the same chip, is an industry first, he said, and it's the kind of gain that only comes from controlling the full stack - models, software, and silicon together - rather than assembling systems from off-the-shelf Nvidia parts. He also said OpenAI deliberately chose SemiAnalysis's InferenceX suite because it is open-source and neutral rather than a benchmark OpenAI itself controls.

Jalapeño is an inference-only chip - it doesn't train models, the workload where Nvidia's hardware remains unchallenged ^[2]. That distinction caps how much independence OpenAI can realistically claim from a single announcement. Even under an aggressive rollout - limited volume by the end of 2026, a more significant deployment in 2027 ^[7]- OpenAI's training stack, and the billions of dollars in GPU commitments behind it, stays tied to Nvidia. Jalapeño is a claim about the economics of serving models cheaply at scale, not about building them.

CNBC frames Jalapeño as a fresh threat to Nvidia's margins as custom silicon from model makers and hyperscalers takes share of the fast-growing inference market ^[8]. Not everyone buys the framing, though: analyst Daniel Newman credited Broadcom for Jalapeño's engineering while calling 'Nvidia killer' talk exhausting hype ^[9]. Reaction online split along similar lines - some treated the results skeptically, as a vendor-run demo to be trusted only once independently reproduced, while others pointed to SemiAnalysis's in-lab verification as reason to take the numbers seriously. The disagreement is less about the raw figures than about how much weight a self-reported, first-generation chip announcement should carry against years of Nvidia's production track record. The public framing has also shifted markedly over time: when Broadcom CEO Hock Tan first showed Jalapeño to Sam Altman and Greg Brockman back in June 2026, he told Reuters its performance was 'apparently the same' as Nvidia's latest Blackwell chips and Google's TPUs - a far more modest claim than the outperformance numbers OpenAI presented at Hot Chips two months later. That gap between the original, cautious framing and August's benchmark-backed claims is itself part of the story: the narrative around Jalapeño escalated considerably between unveiling and proof.