The Silicon Trick That's 74x Faster - and Maybe Already Behind the Curve

Taalas's core innovation is physical, not architectural cleverness bolted onto existing chips: its HC1 part etches roughly 53 billion transistors across an 815 square-millimeter die on TSMC's 6nm process, with the weights of Meta's Llama 3.1 8B model embedded directly into the CMOS silicon rather than stored in HBM memory and fetched during every forward pass [1]. That single design choice removes the memory-bandwidth wall that caps GPU inference throughput, and the result isn't incremental: HC1 has demonstrated up to 17,000 tokens per second per user versus roughly 230 tokens per second for the same model on an Nvidia H200 - about a 74x gap - with independent testing from Cambrian AI's Karl Freund landing in the 14,357-16,960 tokens/second range, closely corroborating AMD and Taalas's own figures [2].
The catch is that the speed comes from permanence. Because the model's weights are physically part of the circuit, upgrading to a newer or larger model means replacing the chip, not pushing a software update - a rigidity flexible GPU accelerators don't have. That trade-off is also where the sharpest skepticism about the deal lives: on Reddit, one detailed technical critique argued the "bake the model into silicon" premise has weakened significantly since Taalas was founded in 2023, because production context windows have since grown from under 4,000 tokens to 128,000-1,000,000 tokens - and a growing KV cache can't be etched into fixed circuitry, it has to live in large, flexible on-die SRAM, which is closer to what Cerebras' wafer-scale approach already does. In other words, AMD may be acquiring a chip optimized for the AI deployment patterns of two years ago rather than the long-context, agentic workloads now shaping demand - a tension neither AMD's nor Taalas's own announcements address, though other commenters proposed narrower use cases where the trade-off still pays off, like an always-fast front-end model for real-time translation or low-latency robotics control loops where speed and power matter more than frontier intelligence.


