Google DeepMind releases DiffusionGemma text-diffusion model

DiffusionGemma abandons the autoregressive loop that every mainstream LLM uses. Instead of predicting the next token conditioned on everything before it, the model starts each block with a canvas of random placeholder tokens and iteratively locks in confident tokens until the whole block snaps into focus, 256 tokens per forward pass ^[1]. It denoises up to 256 tokens per step rather than emitting one at a time ^[2], locking roughly 15-20 tokens per forward pass and refining the rest across iterations ^[3].

The architecture is a hybrid: diffusion within each block, autoregressive across blocks. The decisive property is bidirectional attention. Because the model sees the whole canvas at once, it generates entire paragraphs rather than individual, next-token guesses, ensuring global logical consistency ^[3], and it can self-correct, revising tokens it placed earlier in the same block. That same global view is what makes it natively suited to non-linear tasks like in-line editing and code infilling, where an autoregressive model would have to regenerate from the edit point forward. The model card frames the shift as moving from token-by-token autoregression to block-autoregressive multi-canvas sampling ^[4].

The speedup is not free lunch from a better algorithm, it comes from moving work to where consumer hardware has spare capacity. On local GPUs the main bottleneck for autoregressive models is memory bandwidth: each token requires streaming the full weight set, and the GPU's compute units sit largely idle waiting on memory ^[5]. DiffusionGemma's parallel block decoding inverts this. Pulling a full 256-token block through the transformer in parallel is a compute-bound workload, exactly what NVIDIA GPUs are built for ^[2], so the model trades memory-bandwidth pressure for compute it can actually use ^[6].

The numbers bear this out: 700+ tokens/sec on an RTX 5090 ^[5], over 1,000 tok/s on a single H100, roughly 5x the autoregressive baseline ^[2], and 1,288 generation tok/s on an H200, around 6x autoregressive and 3x multi-token prediction ^[6]. Combined with the MoE design, 26B total parameters but only 3.8B active ^[1], the quantized model fits in about 18GB of VRAM, landing it on a single RTX 5090 or 4090 ^[5]. Google's bet, made explicit at launch, is that this can upend the cost economics of local AI: fast, low-VRAM, no cloud dependency. The local-inference community agreed loudly, with Unsloth, llama.cpp, and SGLang shipping same-day support and comparing the throughput to Groq- and Cerebras-class hardware.

The dominant tension across the technical community is speed versus accuracy, and the skeptics have receipts. A widely shared factual-recall benchmark pitted standard Gemma 4 against DiffusionGemma, and the diffusion model came out roughly 4x faster but several times more error-prone on grounded facts, fabricating names and details that an autoregressive model got right. That tracks with the official guidance: DiffusionGemma scores below standard Gemma 4 on general benchmarks including MMLU and coding evals, and Google explicitly recommends standard Gemma 4 for maximum-quality production use ^[5].

The speed advantage is also conditional. Parallel decoding wins specifically where the GPU has spare compute and memory bandwidth is the bottleneck: single-user, low-concurrency, local workloads. Under high-concurrency server batching, autoregressive models already saturate compute, so the diffusion gains diminish ^[3]. There are ecosystem gaps too: the model needs a specialized drafter module that is not yet in some mainstream runtimes like mlx-lm and LM Studio, and dLLMs needed a custom serving path outside the standard autoregressive stack ^[7]. The community's emerging consensus reframes the model accordingly, not as a drop-in Gemma 4 replacement, but as a fast explorer or sub-agent for constraint-heavy, verifiable tasks like code infilling, where bidirectional attention shines and a downstream check can catch errors.