OpenAI's Jalapeño inference chip benchmarks vs Nvidia Blackwell

OpenAI published Jalapeño's first public benchmark results at Hot Chips 2026 on August 25, 2026, running the chip through SemiAnalysis's InferenceX suite rather than a benchmark of its own design ^[1]. The chip itself is a 700-watt-TDP ASIC fabricated on TSMC's N3P process, packing 13.4 petaFLOPS of MXFP4 matrix compute per chip with 216 GB of HBM4 memory delivering 15.4 TB/s of bandwidth; a single rack holds 128 chips, and OpenAI's full reference system scales to 2,048 chips for 27 exaFLOP/s and 432 TiB of pooled memory ^[2]. On the two flagship workloads OpenAI published, GPT-OSS 120B and DeepSeek R1 670B, Jalapeño posted 1.9x and 1.7x higher peak mixed tokens-per-second-per-kilowatt than Nvidia's GB200, alongside 1.7x and 3.6x lower end-to-end latency ^[2].

The efficiency gap traces to a specific architectural choice: OpenAI designed the chip to minimize data movement so that model state, including the KV cache generated while a response streams out, can be explicitly placed and kept local rather than shuttled across the system ^[3]. That is what let OpenAI claim something GPUs traditionally struggle to deliver at once - high throughput and low latency on the same hardware - a result OpenAI's head of hardware, Richard Ho, described as a very significant performance advance over the state of the art ^[1]. A later B0 silicon stepping is reported to add roughly another 25% performance-per-watt over the earlier A0 silicon ^[4].

Jalapeño's own disclosed timeline is almost as notable as its benchmark scores: architecture concept work began in late 2024, an RTL freeze followed in 2025, tapeout landed in late 2025, and OpenAI's Codex model was reportedly running on early silicon by early 2026 ^[2]. That is an unusually compressed cycle for a from-scratch inference ASIC, and it fits the broader positioning OpenAI staked out when it first unveiled the Broadcom partnership: that OpenAI is no longer just a model company but is building the infrastructure underneath its products, down to chip architecture, kernels, memory systems, networking, and scheduling ^[5].

Social commentary around the Hot Chips talk framed the speed itself as part of the story, with engineers describing an AI-assisted co-design loop and AI-generated kernel code that reportedly beat hand-tuned implementations on some critical blocks. Whether or not that fully explains the compressed schedule, it lines up with OpenAI's stated rationale for building Jalapeño at all: designing for one company's own well-understood, repeatable inference workload lets a much smaller team move faster than a general-purpose GPU roadmap built to serve every customer at once.

The headline beats-Nvidia framing comes with real caveats that SemiAnalysis - the firm whose own InferenceX benchmark was used - was careful to flag. All of the published numbers came from OpenAI; SemiAnalysis says it verified the InferenceX runs in person in OpenAI's lab but did not independently execute the full benchmark suite itself ^[6]. More pointedly, SemiAnalysis noted the comparison to Blackwell is somewhat incomplete and unfair, since Jalapeño's real competitor generation is Nvidia's upcoming Rubin platform ^[6]. SemiAnalysis's own read, however, was still that Jalapeño beats Blackwell on performance-per-watt across almost all scenarios without being tuned for any specific point ^[6].

That same tension - real efficiency gains, measured against a benchmark that skips the actual next-generation rival - showed up in community reaction, where the chip was treated like a vendor demo until someone independent reproduces it, with recurring skepticism that excluding Rubin stacks the comparison in OpenAI's favor. A separate thread questioned whether an ASIC tuned for today's transformer-based inference risks aging out as model architectures shift, a concern OpenAI's own framing implicitly answers by targeting general transformer-style inference as a stable workload rather than betting on one model family.

The strategic logic behind Jalapeño predates the benchmarks by two months: OpenAI and Broadcom first revealed the partnership on June 24, 2026, targeting roughly 10 gigawatts of custom chip capacity for gigawatt-scale data center deployment ^[7]. The underlying motive is cost and leverage - chipmakers like Nvidia have historically commanded profit margins as high as 75% on AI hardware, and even a modest volume of self-designed inference chips strengthens OpenAI's negotiating position with its suppliers ^[8]. Inference is also simply a better fit for custom silicon than training: the workload is large, stable, and repeatable at OpenAI's scale, which is why Jalapeño is explicitly not designed to train models at all ^[9].

Analysts have largely read the results as significant for Nvidia's economics; one Yole Group analyst argued the results show a hyperscaler-designed chip can now match or beat Nvidia's Blackwell-class GPUs on inference efficiency, calling it a real threat to Nvidia's fast-growing inference margins even though Nvidia retains most AI compute share and CUDA lock-in ^[8]. Nvidia CEO Jensen Huang publicly waved off the threat, noting that lots of projects get started and lots of projects get canceled, while pointing to Nvidia's own accelerating market share and revenue as proof its position isn't at risk ^[10]. For now, the practical impact is limited: initial Jalapeño production is planned in small volumes by the end of 2026, with the bulk of OpenAI's inference traffic staying on Nvidia hardware for roughly the next 18 months ^[8].