OpenAI's Jalapeño inference chip challenges Nvidia

OpenAI's headline claim is stark: Jalapeño delivers 1.5x-1.9x more AI work per watt and 1.7x-3.6x lower latency than Nvidia's Blackwell-generation GB300 systems, running GPT-OSS 120B, DeepSeek's R1, and Moonshot AI's Kimi K2.5 ^[1]. On DeepSeek R1 specifically, OpenAI reported over 700 tokens per second per user at low concurrency, and close to 1,400 tokens per second per user on GPT-OSS - notably without using multi-token prediction, a common speed trick that rival systems reportedly did use in their own comparison numbers ^[2].

But the source of those numbers is also the source of the loudest caveat. SemiAnalysis, whose InferenceX benchmark suite generated the results, wrote in its own newsletter that comparing Jalapeño to Blackwell is 'somewhat incomplete and unfair,' since Nvidia's actual current-generation successor, Rubin, not Blackwell, would be the more applicable rival ^[2]. Community benchmarking discussion pushed the point further, noting that Jalapeño's own tests reportedly excluded speculative decoding - a technique that predicts several tokens ahead to save time - while OpenAI's Rubin comparisons apparently did include it, an asymmetry that, if real, would understate Jalapeño's true advantage once methodology is normalized. It also means nobody outside OpenAI has actually run that normalization. SemiAnalysis itself noted it hadn't independently executed the full InferenceX suite, nor seen results from AgentX, the benchmark meant to reflect real production agentic workloads ^[2]. Until someone outside OpenAI publishes numbers on the same models under the same conditions, 'beats Nvidia' is a claim resting entirely on the claimant's own test harness.

Custom AI silicon typically takes years to go from concept to working hardware. Jalapeño reportedly moved from early hiring and schematics to a fabrication-ready, reticle-sized ASIC in roughly nine months ^[3], or about sixteen months counting from initial hiring to tape-out by a more conservative account ^[2]. Either way, it is an unusually compressed cycle for the industry.

The speed came from an aggressive form of software-hardware co-design that used OpenAI's own models during development, including generating kernels through its coding tools ^[4]. Architecturally, the chip skips a common industry pattern - splitting prefill (processing the incoming prompt) and decode (generating the response token by token) across separate hardware pools - and instead keeps both on unified resources ^[2], specifically to cut the data-movement and communication delays that usually bottleneck inference at scale ^[1]. That design choice is a bet that avoiding hardware specialization overhead matters more than optimizing each phase separately, and it's a bet OpenAI could only make because it controls both the model workloads and the chip design simultaneously - something a merchant silicon vendor selling to many customers with different workloads cannot easily do.

The plainest explanation for why Jalapeño exists came from OpenAI's own hardware lead: 'We have so much need for compute' ^[5]. That shortage, not a strategic desire to displace Nvidia outright, is the stated driver - OpenAI's compute demand has outstripped what the general-purpose GPU market can supply on its own terms. Greg Brockman framed the payoff in economic terms, calling it 'a real performance improvement...on performance per watt and performance per dollar' ^[4], and multiple outlets reported the efficiency gains translate to roughly a 50 percent cut in inference costs ^[4].

But the scope is narrower than the framing sometimes suggests. Jalapeño only handles inference, not training, and OpenAI's overall compute stack stays reliant on Nvidia GPUs (and other suppliers) for training and a large share of inference workloads well into 2027 ^[1]. Coverage has described the chip as putting 'Nvidia's pricing power on notice' ^[6], which is a more accurate read than 'Nvidia killer': Jalapeño gives OpenAI a credible alternative to point to in negotiations and a hedge against GPU supply constraints, without actually replacing the bulk of what it buys from Nvidia. It's optionality, not independence.

Even by OpenAI's own account, Jalapeño's near-term footprint is small. The company describes an initial rollout at very small volumes by the end of 2026, with more substantial deployment not expected until 2027 ^[1], and a second-generation chip is already in advanced development with tape-out expected soon ^[5]- suggesting OpenAI is iterating faster than it is shipping at scale.

Skepticism online has centered on whether that 2027 volume target is realistic at all, given the ongoing HBM memory shortage and Nvidia's entrenched grip on advanced packaging and fab capacity that any competing chip program also has to draw on. That skepticism sits alongside a broader, more structural question raised in the same discussions: whether an ASIC with better raw performance-per-watt can meaningfully dent Nvidia's position when Nvidia's actual moat is arguably its software and networking ecosystem - CUDA, interconnects, tooling - rather than silicon efficiency alone. Jalapeño's benchmark numbers, however they shake out under independent scrutiny, don't resolve that question; they just raise the stakes for whether OpenAI can actually manufacture and deploy at the volume it's promising.