NVIDIA Vera Rubin's Agentic AI Infrastructure Rollout

NVIDIA's central pitch for Vera Rubin is a single multiple: 10x agent throughput at scale versus the previous-generation Grace Blackwell platform^[1], extending to a projected 35x higher inference throughput per megawatt for trillion-plus-parameter models running long-context, low-latency workloads once Groq 3 LPX is paired with Vera Rubin NVL72^[2]. NVIDIA is positioning Vera Rubin as an answer to the defining bottleneck of the next AI build-out cycle, energy and latency per token, with the platform positioned to unlock an estimated $200 billion addressable market^[3].

The catch is that the number is not really an apples-to-apples silicon comparison, and the online reaction has split into two camps. On Reddit, the top comment in the r/nvidia thread on the announcement dismisses NVIDIA's figure as a 'fluff piece,' arguing the headline gain does not reflect a uniform leap in the underlying silicon and attributing a meaningful share of it to 'a dedicated LPU that's now onboard, developed by the collaboration between Groq and Nvidia,' bolted onto the rack rather than built into the Rubin GPU die itself. That reading lines up with how the comparison is actually built: the 10x figure sets an entire heterogeneous rack, Rubin GPUs plus a newly acquired specialist inference chip bought for roughly $20 billion in NVIDIA's largest deal on record^[4], against last generation's GPU-only system, which is a different exercise than a straight process-node comparison.

But the skepticism is not the whole picture. CoreWeave, an independent cloud operator running Vera Rubin NVL72 in production, has cited what it describes as its own first measured result, not a projection: roughly a 10x improvement in tokens-per-second-per-megawatt on DeepSeek-R1 compared with the prior Blackwell generation. That is a materially different kind of evidence than NVIDIA's own marketing claim, a real deployment reporting a real number, and it sits in direct tension with the Reddit thread's 'fluff' verdict. Whether the truth lands closer to the vendor's framing or the community's discount likely will not be settled until more operators, beyond NVIDIA's own launch partners, publish their own measured numbers.

The most counterintuitive detail in this rollout is that SpaceXAI's headline deployment for Grok is not Rubin GPUs at all, it is standalone Vera CPUs^[5]. Vera packs 88 custom Olympus (Armv9.2-compatible) cores, 176 threads, 164MB of unified L3 cache, and up to 1.2 TB/s of LPDDR5X memory bandwidth, which NVIDIA pitches as up to 50% higher IPC than the outgoing Grace chip^[6]. SpaceXAI president Mike Nicolls explained the logic directly: "Vera gives us the CPU performance and memory bandwidth to run enormous amounts of orchestration, code and data processing while keeping GPUs doing what they do best."^[5]NVIDIA's own VP, Ian Buck, frames it the same way from the vendor side: "Vera gives AI agents the CPU performance to act in real time, executing code, processing data and coordinating complex tasks."^[5]

The framing matters because it reveals what NVIDIA now believes the actual bottleneck in agentic AI is. Jensen Huang describes a single agent prompt as triggering "a thousand-step journey of reasoning, retrieval, tool use and response generation"^[1], and the overwhelming majority of those thousand steps are control-flow and orchestration work, the kind of latency-sensitive, branch-heavy computation GPUs handle poorly and CPUs are purpose-built for. By decoupling Vera CPUs from Rubin GPUs and selling them as a standalone orchestration layer, NVIDIA is effectively arguing that scaling agentic AI requires buying more CPU capacity, not simply more GPU capacity, a meaningful break from the GPU-centric spending narrative that has defined AI infrastructure since the generative-AI boom began.

NVIDIA's own technical materials make the same division of labor explicit at the hardware level. Groq 3 LPX packs 256 Groq LPUs across 16 trays and roughly 40 petabytes per second of SRAM bandwidth, purpose-built for ultra-low latency, while Vera Rubin NVL72 is tuned for raw throughput: NVIDIA describes the pairing as 'NVL72 generates tokens at the highest throughput, Groq LPX generates them at the lowest latency,' with the two engines meant to run side by side rather than compete for the same workload. At Hot Chips 2026, NVIDIA framed the full lineup, Vera CPU, Rubin GPU, Groq 3 LPX, Spectrum-X Multiplane networking and BlueField-4 scale-in fabric, as one integrated AI stack rather than a set of separate chips, reinforcing that orchestration and networking, not any single chip, are what NVIDIA is actually selling. Microsoft has already stood up the first operational Vera Rubin NVL72 deployment, with NVIDIA citing fully automated manufacturing, compute trays assembled in about a minute, as evidence the platform is shipping at real volume rather than staying in pilot mode.

NVIDIA's acquisition of Groq is not, technically, an acquisition. "NVIDIA is not acquiring Groq as a company, but rather structuring it as an asset purchase,"^[4]paying roughly $20 billion in cash for the startup's key inference-chip assets, its largest deal on record, surpassing the roughly $7 billion Mellanox acquisition in 2019^[4]. Jensen Huang has confirmed the intent is to fold Groq's low-latency processors directly into NVIDIA's AI factory architecture: "We plan to integrate Groq's low-latency processors into the NVIDIA AI factory architecture."^[7]The result, branded Groq 3 LPX, is now the low-latency half of a dual-engine inference architecture running alongside Vera Rubin NVL72.

That asset-purchase-plus-acquihire structure, leaving Groq's legal entity behind while NVIDIA absorbs the technology, is precisely what drew scrutiny from Washington. US Senators Elizabeth Warren and Richard Blumenthal publicly questioned whether the deal was designed to evade antitrust review, citing NVIDIA's roughly 90% share of the GPU market: "By further consolidating NVIDIA's control over the AI chip industry, the Groq deal limits consumer choice and innovation, which will ultimately raise prices and threaten domestic firms' ability to compete with China."^[8]The tension is structural, not incidental: the same acquihire mechanics that let NVIDIA move fast and integrate Groq's technology into a shipping product within months are the mechanics regulators view as a way to sidestep the merger review a full corporate acquisition would trigger.

Beyond terrestrial clouds, NVIDIA and SpaceXAI have committed to something more unusual: launching a space-optimized Vera Rubin NVL72, ordinarily a 72-Rubin-GPU, 36-Vera-CPU, fully liquid-cooled rack^[5], into orbit aboard SpaceX's first-generation Starmind satellite. Reported coverage citing Elon Musk puts the first modified unit in orbit in the fourth quarter of 2027, with a much larger rollout planned for 2028^[9]. But that timeline is notably absent from NVIDIA and SpaceXAI's own announcement materials, which gave no official launch date, timeline, or capacity figure for the orbital deployment^[5], meaning the most concrete date attached to this plan currently traces back to a public statement rather than a company commitment.

The gap matters because the engineering lift is substantial: a rack designed to run in a climate-controlled data center with facility water loops has to be reworked for radiation exposure, heat rejection via radiators instead of liquid-cooling infrastructure, and launch vibration tolerance^[9]. It also arrives against a backdrop of real execution risk on the ground: NVIDIA's next-generation server rack system reportedly hit manufacturing delays tied to a specialized connector circuit board, even as Huang maintained Vera Rubin production volumes were on track to be 'giant'^[10]. Set against that, India's AM Intelligence placing an order for 9,000 Vera Rubin systems for delivery next year^[11]shows the terrestrial rollout is real and large regardless of how the orbital bet plays out, but it also underscores how much manufacturing bandwidth NVIDIA is already committing before an orbital variant has a confirmed launch date.