Splitting Inference in Two: Why Agentic AI Needed a GPU-Plus-LPU Architecture
NVIDIA's core bet with Vera Rubin is that agentic AI is not just a bigger version of chatbot inference - it is a structurally different workload[1]. A chat model answers a prompt once; an agent observes, reasons, plans, calls tools, manages sprawling context, and can spin up sub-agents on demand, all of which spends far more time on orchestration than on next-token prediction[5]. NVIDIA's response was to stop asking a single chip to do everything. The Rubin GPU handles raw throughput and long-context processing, the new Groq 3 LPX handles low-latency decoding for interactive sessions, and the Vera CPU - built with 88 NVIDIA Olympus cores and up to 1.2TB/s of memory bandwidth - absorbs the code execution, data processing, and task coordination that agents generate between model calls[4]. On a 100,000-token agentic benchmark, Groq 3 LPX posted 3,400 output tokens per second, about four times faster than the nearest alternative[3]. The seven-chip platform ties GPU, LPU, CPU, and networking silicon together into what NVIDIA calls a single AI factory engine[2]. The architectural wager is that agentic workloads reward this kind of specialization more than they reward simply adding more GPUs.



