Original Brew.
Original blogs on AI systems, agents, news deep dives, technology teardowns.
The World's Largest Chip: Why Cerebras Built It, and How
The first time I ran into the Cerebras WSE-3, I had to look twice. It’s a single chip about the size of a dinner plate: 21.5 cm on a side, comfortably bigger than my face, and it holds the record for the largest chip ever made. Why would anyone build a chip this big? How do you even manufacture something that large without it falling apart? And what actually breaks when you try? The why, the how, and the what-breaks: those three questions are what this post is about. !wse3wafer.jpghttps://jjxxiuscechxakmindos.supabase.co/storage/v1/object/public/newsletterimages/originals/original-2/01-1e4d76499f543a7b.jpg?v=1e4d76499f54 The Cerebras WSE-3, one wafer, uncut, held like a serving tray. Part One: The chip itself Cerebras’s flagship is the WSE-3 Wafer-Scale Engine 3, and the simplest way to describe it is a chip that never got cut up. A normal fab prints dozens of identical dies onto a 12-inch wafer and then slices them apart into separate chips. Cerebras does the opposite: it leaves the wafer whole and wires the dies together, turning the scribe lines the thin dead zones between dies into data highways. Every row and column gets connected, and the whole wafer becomes one continuous 2D mesh: 84 dies, roughly 10,000 cores each, with every core split half-and-half between SRAM 48 KB and logic. That “don’t cut, connect” move is the genuinely novel part. The patent lists five ways to pull it off; the animation below walks through the first. If that didn’t quite land, it helps to back up to how ordinary chip-printing works. The round silicon disc is the wafer. Each finished block of circuitry on it is a die. During lithography, a scanner projects the circuit pattern off a photomask onto the wafer, lining the mask up over one die at a time. !Video: LithoBasicsEN.mp4https://jjxxiuscechxakmindos.supabase.co/storage/v1/object/public/newsletterimages/originals/original-2/02-29b0c5200e89bf25.mp4?v=29b0c5200e89 Cerebras’s trick is almost cheeky: nudge that photomask a hair along the X or Y axis an offset exposure so the pattern straddles the gap and prints wiring between two neighboring dies. Nothing about the lithography process changes; only where the mask lands. And that’s the whole point: talking across dies this way is far cheaper than stitching separate chips together on a board. !Video: OffsetExposureEN.mp4https://jjxxiuscechxakmindos.supabase.co/storage/v1/object/public/newsletterimages/originals/original-2/03-c659b3ef0793464b.mp4?v=c659b3ef0793 The second break from convention is where the memory lives. A GPU keeps most of its data in HBM stacks parked off to the side of the processor, reached across a silicon interposer and several layers of solder. The WSE-3 keeps compute and memory on the same die, joined by plain copper, and that single decision sends bandwidth through the roof: roughly 21,000 TB/s of main-memory bandwidth, against 8 TB/s on a B200 and 3.35 TB/s on an H100. Not a little more. Thousands of times more. !Video: MemoryEN.mp4https://jjxxiuscechxakmindos.supabase.co/storage/v1/object/public/newsletterimages/originals/original-2/04-74b7b8b963f7abf6.mp4?v=74b7b8b963f7 Part Two: Why bandwidth is the whole game Why is more bandwidth so important in AI inference? it does solve one very specific, very expensive problem: it stops the compute cores from sitting idle, waiting for data. When a large model generates text, producing each token means reading a mountain of weights, and the bottleneck usually isn’t “the math is too slow.” It’s “the data can’t arrive fast enough.” That’s the memory wall, and it’s the thing everyone in this business is fighting. The cleanest way to reason about it is the roofline model, which ties together three numbers: how fast you can compute FLOP/s, how fast you can move data bandwidth, and arithmetic intensity how many FLOPs you do per byte you move. Your real-world performance is whichever ceiling you hit first: raw compute, or bandwidth times intensity. Achievable performance = min peak FLOPs, bandwidth × arithmetic intensity !Video: RooflineEN.mp4https://jjxxiuscechxakmindos.supabase.co/storage/v1/object/public/newsletterimages/originals/original-2/01-ed6e8893c2d57cd3.mp4?v=ed6e8893c2d5 The roofline: the flat compute ceiling, the bandwidth ramp, and the ridge between them. Everything to the left of that ridge is memory-bound: the cores idle, starved for data. And here’s the kicker: nearly all AI inference lives on the left side. To generate token N, the chip has to read the entire model’s weights plus the KV cache just to compute that one token: trivial math, enormous data movement. In some cases you’re using well under one percent of the chip’s peak compute. Which is exactly why Groq’s LPU, Google’s inference TPUs, and the WSE-3 are all obsessed with the same thing: feeding the cores faster. Part Three: Had anyone even tried this? The dream of one giant chip isn’t new. Someone chased it decades ago, and it’s a genuinely strange story. Back in the 1980s, Gene Amdahl, the lead architect behind IBM’s System mainframes, set out to build a near-wafer-scale “super chip” that kept most of its data on the processor itself. His company, Trilogy, raised a then-record $230 million. Then everything that could go wrong did: a storm knocked out the fab’s air conditioning and let dust into the cleanroom, a senior executive died of a brain tumor, and Amdahl himself got tangled up in a car accident and a lawsuit. The packaging, cooling, and testing of the era weren’t close to ready either. The whole thing collapsed. Decades later, the failed dream finally met an era that could support it, but the engineering was still brutal. Cerebras’s own IPO prospectus put it about as dramatically as a legal filing can: Nobody knew how to yield a chip 58 times larger than the leading GPU. Nobody knew how to deliver power to a chip the size of a dinner plate without melting the motherboard. Nobody knew how to package such a big chip without cracking it. Nobody knew how to cool a chip of this size, with air or water, without the coolant getting warm before it reached the other side. Nobody knew, and we didn’t know either. Cerebras S-1 prospectus Four walls, then. Let’s take them one at a time. Wall One: Yield, a chip 58× bigger than a GPU Yield is just the fraction of chips that come off the line actually working, and the intuition for why it’s a nightmare here is easy to build: Think of Minesweeper. The bigger the board, the more mines it hides. A tiny 3×3 grid might have none, and you clear it without thinking. Blow the board up to fill your screen and mines are everywhere. Chips are the same. The bigger the wafer, the more defects: a speck of dust, a chemical non-uniformity, a lithography slip. Normally you just toss the bad dies and sell the good ones. But a chip that’s one giant die fails the instant a single spot goes bad. Out of ten thousand wafers, not one would come out flawless. Cerebras beat this with two moves. First, it makes each core absurdly small, just 0.05 mm², so a single defect only takes out a sliver of the chip. On a GPU, one bad spot can force you to disable a whole 6.2 mm² unit, over a hundred times larger. Second, it holds about 7% of the cores in reserve; when testing turns up dead ones, the on-chip network quietly reroutes around them and swaps in the spares. Net result: 93% of cores active, and yield close to 100%. Tiny cores plus redundant routing: how a “zero-yield” chip becomes shippable. Wall Two: Power, without melting the board Cerebras built its chip a custom home, the CS-3 server, and it draws 25 kW, on the order of 25,000 amps. Put it in household terms: it’s the current of every main breaker in about 625 homes, all maxed out at once, feeding a single server. Current that big gets delivered the usual way, by stepping the voltage down in stages: high-voltage/low-current far from the chip, dropping to around 1 V right at the die. But the flat, sideways layout most boards use has two nasty failure modes once you’re pushing 25,000 A. Problem one: voltage drop. From Vdrop = I × R, at 25,000 A even a hundred-thousandth of an ohm produces a 0.25 V drop, and against a 1 V core, that’s a 25% loss before the power even arrives. Problem two: heat. From P = I × V, that same 0.25 V becomes 0.25 × 25,000 = 6,250 watts of heat dumped into one small spot. Cerebras’s answer, built on Vicor’s hardware, is to stop sending the current sideways at all. In place of the lateral layout it uses vertical power delivery: the final voltage converters sit directly underneath the processor, so the enormous current travels the shortest path it possibly can. Vertical delivery: converters tucked right under the chip, path cut shorter. !vicormodule.jpeghttps://jjxxiuscechxakmindos.supabase.co/storage/v1/object/public/newsletterimages/originals/original-2/02-4c75ceebbc38d33b.jpg?v=4c75ceebbc38 Wall Three: Packaging and cooling A 25 kW machine has a serious heat problem to begin with, and the WSE-3’s sheer size adds a twist. Almost everything expands when it heats up. A small GPU die barely budges. Scale the area up, though, and even a mild temperature swing produces a very real amount of movement, and inside a package, a few tens of microns is enough to tear micron-scale solder joints apart. Worse, heat the wafer unevenly and it expands unevenly, which means mechanical stress and warping. The real villain is the CTE mismatch: the fact that different materials expand at different rates. Silicon barely moves ~2–3 ppm/°C; copper and the PCB move a lot ~15–17 ppm/°C. Heat both by 60 °C and the difference in expansion reaches about 160 μm, roughly the width of a human hair, more than enough to rip solder joints and pry a package open. A 3 cm GPU die, over the same temperature rise, differs by only a few microns, which the solder’s own springiness can soak up. That pushed Cerebras into a genuinely unusual package, the heart of the CS-3, which they call the Engine Block: a cold plate on top, the chip in the middle, the PCB underneath, and special connection layers in between. The guiding idea is easy to state and hard to build: let the vertical direction do the real work carry heat or current straight through while letting the horizontal direction slide freely to absorb all that mismatched expansion. Take the sliding thermal layer as an example. It’s indium on the cold-plate side which conducts heat well pressed against an ultra-thin PTFE Teflon film on the wafer side which is slippery. The two are pressed together but never soldered, so they can slide against each other. Between the wafer and the PCB sits a silicone sheet studded with conductive particles that touch vertically to carry current while still sliding horizontally. A normal GPU just solders its die straight to copper, and it’s small enough that the solder’s elasticity handles the rest. !Video: EngineBlockEN.mp4https://jjxxiuscechxakmindos.supabase.co/storage/v1/object/public/newsletterimages/originals/original-2/01-1457089c08d75cd6.mp4?v=1457089c08d7 The catch: a lot of things are custom Add it all up and the list of things Cerebras had to reinvent is long: the chip’s size, scribe-lines-turned-wiring, high yield on a monster die, tiny cores, on-die SRAM, the solder-free Engine Block, vertical power delivery. And it pays off: in published benchmarks, the WSE-3 puts out tokens per second tens of times faster than a GPU. But every one of those wins comes with the same string attached: it’s all bespoke. The cross-die process is an exclusive partnership with TSMC “not easily portable to another foundry,” as the prospectus admits, so if TSMC’s capacity gets tight, there’s no plan B. The vertical power comes from Vicor, which has already said it’s running at capacity. And the CS-3 runs so hot per square inch that it needs colder inlet water than a standard machine room provides: bigger pumps, fatter pipes, higher-flow fittings, none of it off-the-shelf. It all points the same way. The custom route is exactly what makes the WSE-3 possible, and exactly what caps how fast Cerebras can grow: single-source suppliers at every critical node, no fit into a commodity data center, power and cooling both built to order. Whether all of that can turn into a dependable supply chain is, in the end, the question that decides how steady the company’s path turns out to be.
The Butterfly Effect Behind Falcon 1's Fatal Failures
Before Falcon 1 ever reached orbit, it failed three times — and each failure traced back to a detail almost too small to notice. A corroded nut. A half-empty tank. A few drops of leftover fuel. !cover.pnghttps://jjxxiuscechxakmindos.supabase.co/storage/v1/object/public/newsletterimages/originals/draft-mqo49o3f/01-cd0f588840965242.png?v=cd0f58884096 SpaceX's Falcon 1 reached orbit on its fourth launch — the very first attempt that actually worked. But Falcon 1's road to orbit was anything but smooth. Before that moment, it had failed three times, and those three failures pushed both Elon Musk and SpaceX to the brink. You may have heard that all three failures were caused by tiny details. In this post, I want to walk through the root cause behind each one, and how each small detail triggered a butterfly effect that snowballed into catastrophe. When I started researching, I was honestly a little worried the topic would be too technical. But the chain reactions hiding inside these tiny details turned out to be genuinely fascinating — and they gave me a much deeper appreciation for the old saying: the devil is in the details. Failure One — A Single Corroded Nut On March 24, 2006, the highly anticipated Falcon 1 made its historic first flight. But only about 33 seconds after liftoff, the first-stage engine suddenly lost thrust. The vehicle went out of control, and the launch failed. A detailed debris analysis led by DARPA — the Defense Advanced Research Projects Agency — later revealed a deeply hidden engineering blind spot. The root cause was this tiny component: the B-nut. It's a connecting nut used on tube fittings in fields like aviation and automotive engineering. On the outside it has a hex surface for a wrench; on the inside, a round threaded hole. Its job is to join a tube to a fitting, provide clamping force, create a seal, and keep fuel from leaking. On that first flight, the inside of this nut had been badly corroded — corroded enough to crack. Fuel that should have stayed sealed began leaking out and ran down the outer wall of the thrust chamber. The high-pressure fuel was quickly ignited by the exhaust plume. The fire burned through the control lines almost instantly, and 34 seconds after liftoff the engine shut down completely. That was the end of Falcon 1's first flight. This is where the butterfly effect becomes impossible to miss: the corrosion of one nut shut down an entire engine. So why did it corrode in the first place? !Video: CorrosionEN.mp4https://jjxxiuscechxakmindos.supabase.co/storage/v1/object/public/newsletterimages/originals/draft-mqo49o3f/02-c7a58ae4a6843fbf.mp4?v=c7a58ae4a684 The nut itself was made of aluminum, while the tube it connected to was stainless steel. Put two different metals together, then set them at a launch site on an island near the equator in the Pacific — warm, humid, the air full of salty mist — and you have a serious problem. What you've built, basically, is a battery. Saltwater conducts electricity. Stainless steel has a higher electrical potential and is more stable, so think of it as the positive electrode. Aluminum has a lower potential and gives up electrons more easily — the negative electrode. And just like a battery slowly discharging, that aluminum nut corroded away. The fix was almost insultingly simple. To eliminate the risk, SpaceX swapped every aluminum nut in later designs for a stainless steel one. Now the tube and the nut were the same metal, there was no potential difference between them, and the deadly "battery effect" simply couldn't happen again. Failure Two — The Death Wobble One year later, on March 21, 2007, Falcon 1 made its second attempt. This time the first-stage flight looked good. But during stage separation, the first stage bounced back and lightly tapped the niobium-alloy engine nozzle of the second stage. The impact was slight — but it introduced a tiny deviation into the second stage. And that tiny deviation was enough to set the gears of fate turning again. To correct that deviation, the second stage's control system stepped in. Imagine the flight path drifting a little to the left; the control system would gimbal the engine nozzle slightly, nudging the path back to the right. But by now the rocket was in its second-stage phase, and the oxidizer was already half spent — part of the tank was empty. During the correction, the liquid oxygen inside began to slosh. Inertia slammed it toward the far wall of the tank, shifting the center of gravity. The controller detected that shift, decided it now needed to correct the other way — and the liquid sloshed back. According to the records, this "death wobble" between the control system and the fluid first appeared about 4 minutes and 20 seconds after liftoff, and continued for more than three minutes. It got worse. The frequency at which the engine was gimbaling to correct lined up almost perfectly with the rhythm of the liquid oxygen sloshing in the half-empty tank. !Video: DeathWobbleEN.mp4https://jjxxiuscechxakmindos.supabase.co/storage/v1/object/public/newsletterimages/originals/draft-mqo49o3f/03-983f734760059c3a.mp4?v=983f73476005 What does that mean? Picture the liquid oxygen as someone on a swing. Every time it was just about to swing, the engine happened to give it another push in the same direction. The sloshing grew larger and larger. Under the strong centrifugal force, the liquid oxygen was flung toward the tank's inner walls while a huge vortex opened up in the center. At that point the second-stage engine could no longer draw in liquid oxygen — instead, it started sucking in the gas used to pressurize the tank. In aerospace engineering this is extremely dangerous: the engine loses its oxidizer and begins "running dry" on ingested gas. Eventually the protection system forced a shutdown, and the rocket lost power and dropped out of the sky. The physics here is almost identical to a story I heard as a kid: British soldiers marching across a bridge and bringing it down. In 1831, on England's Broughton Suspension Bridge, soldiers crossed in perfect, synchronized step — and their marching frequency happened to match the bridge's natural vibration frequency. The bridge collapsed. Sound familiar? One frequency lining up with another, producing harmonic oscillation, ending in disaster. To cure it, engineers added physical ring baffles inside the second stage's liquid-oxygen tank. The baffles increased fluid damping and reduced how much the liquid could slosh. They also rewrote the control logic in the flight software, deliberately shifting the control frequency so the system's frequency band stayed a safe distance from the fluid's natural frequency. That broke the feedback loop — the runaway cycle where every wobble feeds the next one and the sloshing just keeps growing. And so, once again, the butterfly effect of a "half-full bottle of water sloshing around" destroyed a launch vehicle worth tens of millions of dollars. Failure Three — A Few Drops of Leftover Fuel The third failure came from an upgrade to the engine's cooling system. Most metals melt below 2,000°C, but when a rocket engine ignites, the combustion chamber can easily blow past 3,000°C. Without proper cooling, the engine wall simply can't survive that heat. The fascinating part: the coolant they used was the fuel itself — kerosene. Using the fuel as the coolant. Here's how it works. Kerosene from the fuel tank, pressurized by the turbopump, flows into cooling channels built into the engine wall — channels made from hundreds of extremely fine, thin-walled metal tubes welded tightly together. The fuel winds around the nozzle and combustion chamber, soaking up heat as it goes. So it cools the chamber and arrives pre-heated, ready to burn. This trick is called regenerative cooling. So where did it go wrong? Once again, at stage separation. The first stage had finished its job and stopped feeding fresh fuel to the nozzle. But — the devil is in the details — inside the cooling channels along the nozzle wall, there was still some kerosene left over. The surrounding hot metal kept heating that residual kerosene, which rapidly vaporized and expanded. Even with the main valves and turbopump shut, the expanding leftover fuel was pushed through the channels into the combustion chamber, where it kept burning, on a small scale, with the remaining liquid oxygen. The result: for several seconds after the shutdown command, the engine didn't fully switch off. It kept producing a tiny, weak push — residual thrust. And because of that thrust, when the two stages separated, the first stage was still creeping forward — and it rear-ended the second stage. Launch failed. What makes this one especially cruel is that it was invisible on the ground. At sea level, atmospheric pressure is higher than the chamber pressure this residual thrust could produce, so the weak internal push was simply suppressed by the outside air — the rocket looked completely shut down. Only in the near-vacuum of space, with no atmosphere to push back against the chamber, did that residual pressure finally express itself as real forward thrust. Reflecting on this failure, Musk later admitted, with real regret, that if they'd added just one more second of waiting time in the software, the tragedy could have been avoided. And the eventual fix involved no hardware changes at all. They changed a few lines of code, adding roughly a 3.5-second countdown to wait for the thrust to fall to zero. That was it. The Darkest Hour After three failures, Musk and SpaceX entered their darkest period. The three Falcon 1 attempts had burned through 100 million dollars. Tesla was facing a severe supply-chain crisis. The global financial crisis was drying up investment everywhere — never mind for a high-risk, hardcore venture like SpaceX. Musk himself was going through a divorce, and was so short on cash that he couldn't make his own rent and had to borrow money from friends. But heroes tend to have something extraordinary about them. It was exactly here that Musk delivered one of the most powerful displays of leadership in his career. Former employees recall the day of the third failure: when the video froze on the moment the rocket broke apart, the entire mission control room fell into a deathly silence. Musk — who had been working more than 20 hours straight and was utterly exhausted — walked to the front of the team and addressed everyone. He acknowledged the failure, and then said: "For my part, I will never give up. And I mean never. As long as you stand with me, we will win." — Elon Musk, to the SpaceX team, after the third Falcon 1 failure And then the story took a dramatic turn. Falcon 1's fourth launch finally succeeded. From there, SpaceX went on to win NASA contracts and to develop Falcon 9, Starship, Dragon, and everything that followed. Hopefully, under Elon's leadership, I might one day get the chance to see Mars for myself. Even if it's a one-way trip — that would be fine too.