Nvidia's Vera Rubin and Groq Inference Buildout for Agentic AI

Nvidia's new stack treats an AI agent's workflow as three separate jobs instead of one. The Vera Rubin NVL72 rack, built around the Rubin GPU, handles the heavy reasoning and retrieval steps agentic tasks require, while a separate Vera CPU rack manages orchestration - tracking tool calls and coordinating the thousand-step journeys a single prompt can now trigger. The Groq 3 LPX rack, inherited from Nvidia's Groq acquisition, is dedicated purely to generation: turning that reasoning into a fast, readable response. Nvidia says the full seven-chip Vera Rubin platform is now ramping into full production, with shipments to customers beginning this fall ^[1].

That specialization shows up directly in the benchmarks. On Gemma 4 31B with a 100,000-token context, Groq 3 LPX produced 3,400 output tokens per second - about four times faster than the nearest alternative platform for the same long-context workload ^[2]. The number matters less as a raw speed record than as a signal of what Nvidia thinks agentic AI actually needs: not one chip doing everything, but a pipeline where the slowest, most user-facing step gets its own purpose-built hardware.

The strategy extends past Nvidia's own silicon. NVLink Fusion opens the same rack infrastructure - NVLink components, MGX rack designs, manufacturing partners, and cluster software - to outside chipmakers building custom XPUs and CPUs, with Intel contributing x86 CPUs and Samsung Foundry handling design-to-manufacturing for the resulting third-party silicon ^[3]. Read together, the message is that Nvidia wants to own the agentic AI factory's plumbing even for hardware it didn't build.

Nvidia's Groq purchase is being described everywhere as an acquisition - its largest ever, dwarfing the roughly $7 billion Mellanox deal that had previously been the company's biggest bet ^[4]. But the deal itself isn't structured as one. It's a licensing agreement paired with the hiring of Groq's key personnel rather than a formal merger, and Groq's separate Cloud business was carved out and continues operating independently. Nvidia says the Groq racks tied to the deal will be online before the end of the year ^[5].

That structure has drawn real scrutiny. Community discussion has flagged the licensing-plus-hiring shape as a plausible way to sidestep the antitrust review a straightforward $20 billion acquisition would invite, noting that Groq employees who weren't individually hired were left out of the arrangement while shareholders were compensated through licensing fees rather than a standard M&A payout - though that framing has also been directly disputed elsewhere in the same discussion, citing reporting that shareholders were still compensated. A separate strand of that discussion goes further, tying Groq's cap table to politically connected investors and alleging Nvidia structured the deal to avoid scrutiny under the current administration - a claim that remains community allegation rather than confirmed fact, but one that's shaping how the deal is being read.

Industry watchers see a familiar playbook here: acquire a promising chip startup, absorb its IP and talent, and let the standalone hardware business fade into the parent company's roadmap - the same pattern Nvidia is seen to have followed after Mellanox. Whether Groq's inference technology ends up mainly as an internal upgrade to Nvidia's own rack architecture, rather than surviving as independent competition, is the open question the deal's unusual structure was arguably designed to make moot.

What separates a chip announcement from a real product is who is actually running it, and Vera Rubin's answer arrived unusually fast. CoreWeave completed the industry's first bring-up and validation of Vera Rubin NVL72 in production on June 1, 2026, using Dell PowerEdge XE9812 servers and its own liquid-cooling and rack-control tooling ^[6]. Nebius followed by committing to deploy Vera Rubin NVL72 across its US and European data centers starting in the second half of 2026, and became the first AI cloud to offer Groq 3 LPX through its Token Factory service ^[7].

SpaceX and SpaceXAI are building their AI infrastructure - from Earth-based data centers to orbital compute - specifically around Vera Rubin's rack-scale design, adopting Vera CPUs for the orchestration, tool use, and simulation work agentic systems require ^[2]. Engineering-stage racks are also already running inside Microsoft's data centers, according to customer accounts of the rollout, putting three of the largest cloud and compute buyers in the market on the same new architecture within weeks of each other.

That speed cuts both ways. It's a genuine vote of confidence from operators who don't take unproven infrastructure into production lightly. It's also a competitive risk for exactly those operators: analysts have warned that SpaceX's exclusive commitment to Nvidia chips could push neoclouds like CoreWeave and Nebius further back in the queue for GPU allocation, even though both received billion-dollar Nvidia equity investments that might otherwise imply preferential access ^[8].

In the same week it announced Groq 3 LPX and Vera Rubin hitting full production, Nvidia disclosed a minority equity investment - several hundred million dollars - in Cloverleaf Infrastructure, a land developer that has already sold more than 7 gigawatts of powered land to data-center operators ^[9]. That came just days after Nvidia took a roughly $1.5 billion equity stake in SB Energy, SoftBank's power subsidiary, tied to a 20-year Ohio campus already leased to OpenAI ^[9]. A separate $2 billion investment in power developer Lancium - whose Abilene, Texas campus hosts Oracle infrastructure that OpenAI rents - rounds out a pattern of Nvidia putting capital directly into the electricity and land supply chain rather than just the chips that sit on top of it ^[10].

The logic is straightforward: Nvidia can build all the GPUs it wants, but if there's nowhere to plug them in, the chips don't ship revenue. Equity stakes in land and power developers are a way of guaranteeing that data-center capacity keeps pace with chip demand, instead of leaving that bottleneck for customers to solve on their own timeline ^[11].

It also makes Nvidia a direct participant in a market it used to only supply into. When the same company holds equity in the power developer, in the campus operator, and in the chip itself, the usual arm's-length relationship between hardware vendor and infrastructure operator starts to blur - and it isn't yet clear how that concentration plays out for smaller buyers competing for the same land and power.

Nvidia's official comparisons - 10x the agent throughput of Grace Blackwell, up to 10x better inference-per-watt - are the headline figures behind the Vera Rubin launch ^[1]. Wall Street has largely taken the bait: Jefferies expects Vera Rubin revenue to climb from roughly 12% of Nvidia's GPU revenue in the third fiscal quarter of 2027 to more than 40% by the fourth, with rack shipments rising from about 13,000 by the end of 2026 to over 120,000 during 2027 ^[12].

The technical community is less convinced by the round numbers. Independent analysts have pointed out that a reported 35x throughput-per-megawatt figure circulating around the Groq integration still needs third-party validation before it should be taken at face value, and community discussion of the "10x" efficiency claim has openly called it marketing pending independent confirmation. Nvidia has a mixed track record here - the previous-generation GB300 NVL72 was projected at roughly 30x the performance of H100 and reportedly ended up delivering closer to 100x once deployed, which cuts against reflexive skepticism even as it doesn't resolve it.

A related question getting real traction in developer circles: why Groq and not Cerebras. The architectural answer floated there is that Groq's LPU trades memory capacity for a deterministic, compiler-scheduled pipeline with no stalls, and its multi-chip rack design slots more cleanly into Nvidia's existing CUDA and NVLink stack than Cerebras's single giant wafer-scale chip would - even as some in that same discussion note Cerebras remains better suited to training, meaning the two chips were never fully substitutable to begin with.