Local AI inference hardware: Nvidia Spark vs AMD

The headline shock of this matchup is that a Strix Halo mini PC starting around $1,499 generates LLM tokens at roughly the same speed as the $3,999 DGX Spark ^[6]. That is not a fluke. Single-batch LLM token generation (the decode phase, where the model emits one token at a time) is memory-bandwidth-bound, not compute-bound: every generated token requires streaming the model's weights through memory, so throughput is gated by how fast you can move bytes, not how many FLOPS you can do. The two boxes have nearly identical bandwidth, about 273 GB/s on the Spark versus 256 GB/s on the Strix Halo HP Z2 Mini G1a ^[2], so they 'churn out tokens at a similar pace' ^[1]. The benchmarks bear this out: on gpt-oss 120B the Spark hit ~38.6 tok/s versus ~34.1 on Strix Halo, and on a Llama 3.3 70B run the Strix Halo actually edged ahead at 4.97 versus 4.67 tok/s ^[4]. LMSYS measured the mechanism directly, calling the limited bandwidth 'the key bottleneck in AI inference' ^[2]. Nvidia's far larger compute budget (6,144 CUDA cores against the Radeon 8060S's 40 CUs / 2,560 stream processors) simply has nothing to do during decode ^[1].

Decode is only one of three workloads, and Nvidia dominates the other two. Prompt processing (prefill, where the model ingests your context before answering) is compute-bound, and here the Spark is in a different class: roughly 1,723 tok/s versus the Strix Halo's ~340 tok/s, a 2-3x lead on shorter sequences ^[1][4]. The gap widens further for creation and training. On FLUX.1 image generation the Spark runs about 2.5x faster, and it finished a QLoRA fine-tune of Llama 3.1 70B in roughly 20 minutes against ~50 minutes on the AMD box ^[1]. That maps cleanly onto its BF16 tensor performance of 56 teraFLOPS versus the HP Z2 Mini G1a's ~46 ^[1]. So the buying decision is genuinely workload-shaped: if you mostly chat with a local 70B model, AMD's value is unbeatable; if you process long documents, generate images and video, or fine-tune, the Spark earns its premium. AMD does win the CPU side outright, with Zen 5 delivering 'between 10 and 15 percent higher performance' than the Spark's Arm cores in Sysbench, 7zip, and HandBrake ^[1].

The social conversation has crystallized around a price-disruptor narrative (one widely shared post framed it as AMD's CEO killing Nvidia's $4,000 box with a $1,499 lunchbox running a 235B model live), and on raw token throughput that framing has real merit. But benchmark obsession glosses over three things. First, software friction: getting vLLM, BitsandBytes, and Flash Attention 2 running on AMD often meant source compilation or AMD-specific builds, because 'a lot of software can be made to work on AMD's consumer hardware, but it's not always as simple as running something like pip install xyz-package' ^[1], against Nvidia's drop-in CUDA experience. Second, scalability: the most thoughtful defenders of the Spark on Reddit argue its real magic is ConnectX networking and near-lossless scaling, where two Sparks combine to serve a much larger model with high prefill, rather than raw single-box bandwidth. Third, the lived experience cuts both ways. The same threads that defend the Spark on scalability also include an owner of both machines admitting they still reach for the Strix Halo more often. The honest read is that 'AMD killed Nvidia' is half-true: AMD won the value crown for interactive chat, but Nvidia kept the crown for generation, fine-tuning, and multi-box production.

These boxes exist because 128GB of unified memory crosses a threshold that consumer GPU VRAM never could: a 70B-plus model loads directly into memory, turning local exploration of frontier-class models from impossible into merely slow ^[2][5]. The demand driver is the wish to run large models, agents, and fine-tuning locally without cloud cost or data exposure, where nothing leaves your network. Community math frames the economics starkly: AMD's break-even against renting cloud GPUs lands around six months, with a roughly $4,500 three-year cost versus $25K-plus in the cloud. But the same physics that enables the category also caps it. Large-model interactivity is slow on both platforms (Llama 3.1 70B FP8 delivered only ~2.7 tok/s decode on the Spark, and real-world 70B INT4 runs land around 3-5 tok/s) ^[2][4], and Apple's M4 Max already offers ~546 GB/s, roughly double the Spark, hinting at how much headroom the next generation has on the one axis that matters most for chat ^[6].