NVIDIA Vera Rubin NVL72's MLPerf Inference v6.1 Debut

Vera Rubin NVL72's throughput lead over GB300 NVL72 traces to a combination of hardware and software changes rather than any single trick. NVIDIA describes the system's enhanced Tensor Cores and Transformer Engine as accelerating both the prefill and decode stages of inference, while NVFP4 precision shrinks the memory footprint of model weights, attention, and KV cache - raising throughput with what the company calls minimal loss of output quality ^[1]. Those software gains ride on a rack that packs 72 Rubin GPUs and 36 Vera CPUs with 20.7 TB of HBM4 GPU memory, 1,400 TB/s of GPU memory bandwidth, and 216 TB/s of NVLink bandwidth, rated at 3,600 PFLOPS of NVFP4 inference compute ^[2]. The DeepSeek-R1 result (2.5x) ran on TensorRT-LLM across offline, server, and interactive scenarios, while the larger Qwen3-VL gain (3.7x) used vLLM paired with NVIDIA Dynamo - a reminder that the advertised leap is really two separate stacks tuned for different workload shapes, not one universal multiplier.

MLPerf's preview category exists for platforms expected to reach commercial availability by the next submission round, and NVIDIA used it to put Vera Rubin NVL72 numbers on the record while GB300 NVL72 remains the only Rubin-era product customers can actually buy today ^[3]. That is a deliberate sequencing: publish third-party-verified proof of the next platform's performance ceiling before general availability, so cloud buyers can plan capital allocation around it rather than around marketing slides alone. Nebius is the clearest evidence the strategy is landing with real customers - the cloud provider ran its own Vera Rubin NVL72 preview submission and has committed to offering the platform commercially in the US and Europe from the second half of 2026 ^[4]. It is also a contrast play: NVIDIA measures Vera Rubin's preview numbers directly against GB300 NVL72's own MLPerf v6.0 record - a 288-GPU, four-rack run that pushed 2.5 million tokens per second on DeepSeek-R1 and was, at the time, the largest submission scale in the benchmark's five-year history ^[5]. Vera Rubin's preview debut is framed to make that record look like the previous generation almost immediately.

NVIDIA's preview numbers are the headline, but the same v6.1 round quietly shows how much closer AMD and Intel have gotten to NVIDIA's shipping hardware. AMD posted its first MLPerf results for Instinct MI350P and Ryzen AI Max+ 395, and partnered with Crusoe to run 512 Instinct MI355X GPUs across 64 nodes on standard RoCE Ethernet - the largest system ever submitted to MLPerf Inference, delivering 5.75 million tokens per second offline and 5.39 million tokens per second in the server scenario on gpt-oss-120b ^[6]. Intel, meanwhile, made its own MLPerf debut with Arc Pro B70 GPUs, posting generational gains across Llama 3.1 8B, Llama 2 70B, gpt-oss-120B, Whisper, and the new End-to-end RAG benchmark. The round drew a record 30 submitting organizations and 486 datacenter and edge results in total ^[7], and MLCommons added the End-to-end RAG test to this cycle specifically because deployment patterns have moved past single-turn query-answering, which is exactly the terrain Vera Rubin NVL72 is architected for ^[6]. NVIDIA is still ahead, but the field it is racing is visibly closing distance at the same moment it previews its next platform.

The most attention-grabbing figure from this round is not a MLPerf metric at all: Vera Rubin NVL72 posted a 30x performance gain over GB300 NVL72 on the SemiAnalysis AgentX benchmark, a third-party test designed specifically to measure agentic workloads ^[8]. Numbers at that scale invite exactly the kind of skepticism that surfaced in investor-focused discussion of the results, where a thread summarizing a related SemiAnalysis performance-per-dollar analysis cited far larger multiples - tens of times more tokens per dollar of total cost of ownership than GB300 at high-interactivity service levels. A commenter in that same thread pushed back that those headline multiples 'only apply at high interactivity where the base is very low,' meaning the eye-catching ratios compress dramatically once you look at typical, moderate-throughput deployment rather than the narrow edge case that produces the biggest number. That tension - real, verified architectural gains on one side, and best-case marketing multiples on the other - is the honest way to read every number in this launch: the 2.5x and 3.7x MLPerf figures are peer-reviewed and reproducible, but the more dramatic double- and triple-digit multiples circulating around Vera Rubin depend heavily on which slice of the workload curve gets measured.