Neural Computers: A New Computing Paradigm Using Video Generation Architecture

The Neural Computers paper represents the culmination of a research trajectory that began over a decade ago with Neural Turing Machines in 2014. Those early systems coupled neural networks with external memory banks — the network learned to read and write from a separate, structured memory using attention mechanisms. Differentiable Neural Computers in 2016 refined this approach with more sophisticated memory addressing. But both designs preserved the fundamental Von Neumann separation: the neural network was the processor, and memory remained an external resource.

Neural Computers break with this lineage in a radical way. Rather than giving a neural network access to separate memory and I/O systems, the entire computing stack collapses into a single learned latent state. As ArXivIQ described it, this is 'a fundamental shift from the traditional Von Neumann hardware/software stack to a unified neural latent stack.' The system does not execute instructions fetched from memory in the classical sense. Instead, built on a diffusion transformer architecture (Wan2.1), it generates successive screen frames — rolling out visual computation step by step. The implications are architectural: there is no operating system, no file system, no instruction set architecture. The model learns what all of those things should do from data.

This shift also reframes the relationship between agents and computers. 36Kr's analysis maps three eras: conventional Human-to-Computer interaction, the current Human-to-Agent-to-Computer paradigm where AI mediates between users and traditional software, and the proposed Human-to-Neural Computer relationship where the intermediary disappears because the AI is the machine itself. The paper identifies three converging trends making this plausible now: agents improving at real work (citing MetaGPT, Cursor, Claude Code), world models advancing in environment simulation (GameNGen, Genie 2/3, Waymo), and the structural friction of conventional computers when handling open-ended, long-horizon tasks.

One of the most striking empirical findings in the Neural Computers paper is the dramatic gap between intentional and random training data. The researchers found that 110 hours of goal-directed data outperformed 1,400 hours of random interaction data. This is not a marginal improvement — it means roughly one-thirteenth the data volume produced superior results, simply because the data captured purposeful behavior rather than aimless clicking and typing.

This finding has implications well beyond Neural Computers. The broader AI training paradigm has long emphasized scale: more data, more compute, more parameters. The NC results suggest that for systems learning to simulate interactive environments, the intentionality and structure of the data matters far more than its volume. The GUIWorld prototype was trained on approximately 1,510 hours of Ubuntu desktop recordings, while CLIGen used roughly 1,100 hours of terminal interaction recordings. But the quality signal — goal-directed behavior with clear cause-and-effect relationships between user actions and system responses — proved to be the decisive factor. This echoes a growing body of evidence across AI research that curated, high-quality datasets can outperform massive but noisy ones, but the 12x efficiency gap reported here is particularly dramatic and offers a concrete benchmark for future work in interactive world models.

The paper's authors and outside analysts are refreshingly candid about the current limitations. ArXivIQ's analysis describes the prototypes as having a 'severe limitation in native symbolic reasoning, making current video-based instantiations exceptional renderers but fragile reasoners.' The system can convincingly generate what a terminal or desktop screen should look like in response to user input, but it struggles with the computational substance behind those visuals. Current prototypes cannot reliably perform two-digit arithmetic. They suffer from behavioral drift on multi-step tasks, where small errors compound across generated frames. And they experience catastrophic forgetting when trained on new capabilities — learning a new skill can degrade previously learned ones.

These are not minor engineering bugs to be patched; they represent fundamental challenges in making a generative model behave like a deterministic computing system. The paper defines the Completely Neural Computer (CNC) as the mature realization of this paradigm, requiring four properties: Turing completeness, universal programmability, behavioral consistency, and machine-native semantics. The gap between current prototypes and these requirements is vast. Turing completeness demands that the system can, in principle, compute anything a traditional computer can — but a system that fails at two-digit addition is nowhere near this bar. Behavioral consistency requires that identical inputs produce identical outputs, which is inherently at odds with the stochastic nature of diffusion models.

Pebblous AI's analysis frames the transformation as one where programming shifts from writing code to teaching behavior. But the current inability to maintain stable behavior across even modest task horizons suggests this teaching paradigm needs fundamental breakthroughs, not just incremental improvements. The lead author's estimate of approximately three years to a functional Neural Computer reflects awareness of these challenges. This is a position paper staking out a research direction, not a product roadmap — and that honesty about the gap between vision and reality is arguably one of its strengths.