OpenAI's Jalapeño custom inference chip debut

OpenAI's chip effort moved from initial design to tapeout in an estimated nine to sixteen months ^[5], an unusually fast timeline that one industry analysis flagged as a possible record for chip design ^[1]- compressed largely because OpenAI turned its own models loose on the hardware itself. AI-written kernel implementations for Jalapeño outperformed human expert kernels by 1.5 to 1.8 times ^[2], with AI also used to accelerate verification loops and low-level kernel optimization throughout the design process ^[2]. A technical teardown found the AI-tuned matrix units delivered roughly a 56% gain on BF16 multiply operations while shrinking matrix-unit die area by about 10% ^[3]. In a Bloomberg Technology interview, Richard Ho pointed to a different, faster-moving marker of that speed: three very different AI models were brought up and running on the new hardware within a couple of months of OpenAI receiving it, which he cited as evidence that the programming model itself had matured fast enough to keep pace with the silicon. The resulting single chip packs 13.4 PFLOPS of MXFP4 compute, 216 GiB of HBM4 memory, and 15.4 TB/s of bandwidth in a 700W envelope ^[3], scaling to 27 EFLOPS and 432 TiB of memory across a 2,048-chip pod ^[3]. The notable part isn't just that a lab built a chip - it's that the chip's own creation leaned on the AI it exists to run, folding model-assisted engineering into a discipline that historically took human teams years.

OpenAI's headline numbers - 1.5x-1.9x higher throughput per watt and 1.7x-3.6x lower end-to-end latency than Nvidia's GB200/GB300 systems ^[4]- sound decisive, but the fine print complicates the story. SemiAnalysis, which independently reviewed the InferenceX comparisons, called the results 'industry-leading' for a first-generation chip ^[5], while also cautioning that measuring Jalapeño against Blackwell is somewhat incomplete and unfair, since Nvidia's upcoming HBM4-based Vera Rubin platform is the more contemporaneous rival, and OpenAI's tests used single-token prediction where Rubin comparisons typically use speculative decoding ^[5]. Richard Ho, OpenAI's head of hardware, framed the results as 'a very, very significant performance advance over state of the art' ^[6]. Separately, in a Bloomberg Technology interview, Ho noted the benchmarks were run on InferenceX, an open-source benchmarking tool chosen specifically so the comparison wouldn't rest on OpenAI's own scoring. Even so, no independent full technical report has been published yet, and that gap between OpenAI's framing and the benchmark's actual scope is exactly where the loudest skepticism lives: the comparison is against a two-year-old architecture, not the chip Nvidia is about to ship. Discussion across r/OpenAI and r/tomshardware read as impressed but cautious, and the same Rubin-skepticism surfaced independently there too - a widely upvoted r/tomshardware thread (224 upvotes) made a point of noting Vera Rubin's absence from the comparison, arriving at the same conclusion as SemiAnalysis without citing it. A separate, heavily upvoted Reddit comment pushed the critique further, arguing Jalapeño's real edge is concentrated in low-latency scenarios rather than raw total throughput - a distinction OpenAI's headline multiples tend to blur.

Broadcom CEO Hock Tan pegs the accelerator's cost advantage at roughly 50% savings versus typical AI GPUs ^[7], and OpenAI designed Jalapeño specifically to minimize data movement and communication delays across the prefill and decode phases of LLM serving ^[6]- the parts of inference that dominate cost at OpenAI's scale. But the economics only matter once the chip ships in volume, and that is still a while off: deployment in OpenAI's own infrastructure starts in very small volumes by the end of 2026, with more substantial production not expected until 2027 ^[6]. That timeline means Jalapeño's near-term effect on OpenAI's Nvidia spend, and on Nvidia's roughly 92% share of the GPU market ^[8], is limited - the announcement reads as a statement of intent and a proof of technical capability more than an immediate supply-chain shift. Reddit commentary converged on the same point from a different angle: several commenters noted Jalapeño is still 'in early production' while Nvidia's Rubin cards are already being produced in the hundreds of thousands to millions, a volume gap that will matter more to near-term economics than any benchmark multiple.

Nvidia CEO Jensen Huang's public response leaned on dismissal rather than direct rebuttal: 'Lots of projects get started. Lots of projects get canceled,' he said, framing custom-silicon efforts like Jalapeño as a common and often unsuccessful pattern among Nvidia's largest customers ^[8]. Reporting also framed the chip as a new pressure point on Nvidia's margins specifically in the inference segment, where purpose-built ASICs are best positioned to compete ^[9]. Huang's response is notable less for what it disputes and more for what it doesn't: it doesn't contest OpenAI's benchmark numbers, it bets instead that Nvidia's platform breadth, and the sheer difficulty of taking a first-generation chip to reliable volume production, will outlast one strong showing at Hot Chips. On X, OpenAI's own two announcement and benchmark posts drove far more engagement than any independent commentary - 22,544 and 14,127 likes respectively - underscoring that this is a story the company is actively driving rather than one that emerged organically from outside reaction. Independent voices did weigh in: a finance-focused account, @RealNickMugalli, reacted to Huang's remarks with 'SHOTS FIRED,' framing the response as combative bravado rather than a substantive rebuttal - a read that lines up with how little Huang's comments actually engaged the benchmark numbers themselves.