Data centres and what's next

The full story

This is the episode's narration, word for word. Headings jump to that point in the video.

The question 0:00

Last time, we filled a building with chips. So where does all this go next, and what's stopping it? The short answer is heat. Let's walk into one of those buildings, then look at the industry's tricks for keeping going: light from exploding tin, chips cut into pieces, new materials, and wires made of light.

Inside the building 0:23

A data centre is a building full of racks, and each rack is a stack of servers about the size of pizza boxes. Around them are three other systems. Power, with batteries to ride out short blackouts and generators for long ones. Cooling, with air conditioning, or liquid pumped straight to the processors. And networking: switches and fibre, so thousands of servers can work together.

How much electricity? 0:50

So how much power do they use? The International Energy Agency estimates that in 2024, data centres used about four hundred and fifteen terawatt-hours, around one and a half per cent of the world's electricity. That's a steady forty-seven gigawatts, around the clock. In 2025 it grew another seventeen per cent. The agency's main scenario, from April 2025, has it more than doubling by 2030, with AI the biggest driver. That's a projection, not a promise.

A heater that computes 1:27

Where does all that energy go? In episode seven, every switch filled a tiny capacitor and dumped it, and all of that energy became heat. So a data centre is, in a sense, a giant heater that does arithmetic on the side. The servers take around sixty per cent of the power. Cooling takes anywhere from seven to over thirty per cent, depending on how efficient the building is. And an AI rack can need a hundred and twenty kilowatts of cooling, about thirteen times a typical rack, so it's cooled with liquid.

The free lunch 2:03

For about thirty years, shrinking hid that problem. Episode five's Dennard scaling said: shrink a transistor and lower its voltage in step, and the heat per square millimetre stays the same. Switching power goes with the square of the voltage, so turning the voltage down paid for everything. In principle, each generation could run faster without getting any hotter per square millimetre.

Off isn't quite off 2:30

Then the voltage almost stopped falling. Here's why. A transistor that's switched off still leaks. Heat gives some electrons enough energy to get over the barrier the gate holds up, the same thermal jostling from episode one. Below the switching point, the current only falls ten times for every sixty millivolts, at best. That sixty millivolts is set by temperature, not by engineering. So if you lower the supply voltage, you have to lower that switching point too, and every small step down multiplies the leak.

The power wall 3:08

So by about 2006, Dennard scaling had broken. Smaller transistors kept coming, but the voltage barely fell, so every shrink packed in more heat. In 2004, Intel cancelled its next Pentium 4, a move analysts tied to how hot its chips were getting, and later that year shelved its four gigahertz chip. Top clock speeds have sat at a few gigahertz ever since. Instead, chips got more cores, running slower and cooler, side by side. That's the power wall, and GPUs are where it leads.

Still shrinking 3:47

But transistors still shrink, and for that you need finer light. Episode six left us at one hundred and ninety-three nanometres, with a promise. The next step is extreme ultraviolet, at thirteen and a half nanometres, about fourteen times shorter. The catch is making it, and then bending it.

Shooting tin 4:09

There's no lamp that makes this light. Instead, a generator fires drops of molten tin, each about twenty-five micrometres across, at seventy metres a second. A laser pulse flattens each drop into a pancake. A second, more powerful laser pulse blasts it into a plasma, hotter than the surface of the Sun, which gives off light around thirteen and a half nanometres. It does this fifty thousand times a second.

Mirrors in a vacuum 4:40

No lenses, either. Extreme ultraviolet is absorbed by everything, even air. Each photon carries about ninety electron volts, several times what it takes to knock an electron off any atom at all, so every material soaks it up. So the whole path is in a vacuum, and the light is steered with mirrors made of up to a hundred layers of molybdenum and silicon, each a few atoms thick. Even then, each mirror reflects at most seventy per cent. After six of them, only about an eighth of the light is left.

One company 5:17

Only one company makes these machines: ASML, in the Netherlands, after more than two decades of development. Chips made with them reached phones in 2019. The newest version cost about three hundred and fifty million euros, weighs about as much as two airliners, and arrived in two hundred and fifty crates.

Three nanometres of what? 5:41

So, are today's chips really "three nanometre" chips? Not in any single feature you could point to. Since the mid-1990s, the node names have drifted away from the real sizes. In the industry roadmap, a "three nanometre" chip has gates forty-eight nanometres apart and its finest wires twenty-four apart. The name is a generation label, not a ruler.

Wrapping the gate 6:08

What has changed is the shape. In a flat transistor, the gate sits on top, and leakage can sneak along paths deeper down, where the gate's grip is weak. So in 2011, Intel announced it would stand the channel up as a thin fin, with the gate on three sides. Now the gate wraps all the way around stacked sheets: Samsung started in 2022, and TSMC in late 2025. The goal is the same: a switch that's really off.

Why break a chip into pieces? 6:41

Next trick. Remember from episode six: one speck of dust can kill a whole die, so big dies are expensive. AMD worked out that one of its server processors, built as a single die, would need seven hundred and seventy-seven square millimetres. Instead, it used four smaller chiplets. That took about ten per cent more silicon, but cost only about sixty per cent as much.

Mix and match 7:11

Chiplets let you mix processes, too. AMD put its processor cores on a newer process and the input and output on an older, cheaper one. There's also a size limit, because a lithography machine can only print an area so big in one shot. So Nvidia's Blackwell GPU is two dies at that limit, joined by a link carrying ten terabytes a second.

Wider gaps 7:37

Some chips just switch power, and here silicon has rivals: silicon carbide and gallium nitride. Their band gaps are about three times silicon's. Remember episode one: the gap is the energy needed to free an electron. A strong electric field can fling an electron hard enough to knock others loose, in an avalanche, and a wider gap takes a stronger field to start one. So these materials can stand about ten times the field, and they stay in control when hot, because heat frees far fewer electrons across a wide gap.

Chargers and cars 8:16

That makes small, efficient power switches possible. Gallium nitride went into small phone chargers from 2018. Silicon carbide went into electric-car inverters, which turn battery power into motor power, including Tesla's Model 3.

Wires are hot too 8:35

Now, moving data. At high speeds, the current in a copper wire crowds into a thin skin at its surface, so the wire acts more resistive. At two hundred gigabits a second per lane, plain copper reaches less than a metre. Inside a rack, copper still wins: Nvidia's AI rack uses more than five thousand copper cables. Between racks, data goes by light. But Nvidia says a fast optical transceiver can burn about thirty watts, more than half of it in the chip that cleans up the electrical signal.

Wires of light 9:12

Silicon photonics builds the optical parts, the guides, switches and detectors, right into silicon chips. And it works because of the band gap. Infrared light at fifteen hundred and fifty nanometres carries only point eight electron volts, less than silicon's gap, so it passes straight through. To that light, silicon is glass. But silicon still can't make light well, for the reason we met in episode three, so the lasers are made of other materials, either bonded on or plugged in alongside. In 2025, Nvidia announced switches with the optics built into the package, and claimed three and a half times better power efficiency.

What the roadmaps say 9:58

So what's next? Here's what the companies say, with dates. TSMC said in 2024 that its next step, with power wiring moved to the back of the chip, would go into production in 2026. In 2023, the research centre imec laid out names down to "two ångström" by 2036, which are names, not sizes. Even the industry roadmap has the voltage creeping down only from about three-quarters of a volt to just over half a volt by 2034, nothing like the old free lunch.

From a warm crystal 10:36

The whole season, in thirty seconds. In 1833, Michael Faraday warmed a crystal and watched it conduct better. In 1940, a cracked silicon rod made a voltage in a flashlight beam. In 1947, holes in germanium gave us the transistor. In 1958 and 1959, whole circuits went onto one chip. In 1965, Gordon Moore saw the count doubling. Then switches shrank, and multiplied into the billions, until heat put a ceiling on the clock. Faraday's mystery was what heat does to a crystal, and nearly two centuries later, it's still the question that matters most.

What should we explain next? 11:21

We started with sand, and ended with buildings that think. Now it's your turn. What don't you understand, and want to? Tell us in the comments, and the best question might just become season two.

Sources

Every factual claim in the episode is tied to one of these. Spotted an error? Tell us.

1. IBM Think, "What is a data center?" — ibm.com
2. IEA, Energy and AI (10 April 2025), Executive summary — iea.org
3. IEA news, "Data centre electricity use surged in 2025, even with tightening… — iea.org
4. IEA, Energy and AI (10 April 2025), "Energy demand from AI" — iea.org
5. Uptime Institute, Global Data Center Survey 2025 (Report 180, July 2025) — datacenter.uptimeinstitute.com
6. NVIDIA Technical Blog, A. Elmeleegy, "NVIDIA Contributes NVIDIA GB200 NVL72 Designs to… — developer.nvidia.com
7. M. Bohr, "A 30 Year Retrospective on Dennard's MOSFET Scaling Paper", IEEE SSCS News… — sscs.ieee.org
8. R. H. Dennard et al., "Design of Ion-Implanted MOSFET's with Very Small Physical… — eng.auburn.edu
9. S.-W. Jun, UC Irvine CS 250B (Spring 2022), lecture 1 "End of Scaling" (Dennard ended… — ics.uci.edu
10. C. C. Hu, Modern Semiconductor Devices for Integrated Circuits, ch. 7 "MOSFETs in… — chu.berkeley.edu
11. H. Esmaeilzadeh et al., "Dark Silicon and the End of Multicore Scaling", ISCA 2011 — pages.cs.wisc.edu
12. Reuters via NBC News, "Intel scraps new desktop, server chips" (7 May 2004) — nbcnews.com
13. The Register, "Intel decides speed matters less these days, kills 4GHz Pentium" (14… — theregister.com
14. K. Rupp, "42 Years of Microprocessor Trend Data" (15 Feb 2018) and data repository (CC… — karlrupp.net
15. ASML, "Light & lasers – Lithography principles" — asml.com
16. ASML, "EUV lithography systems" — asml.com
17. ASML, "The hottest thing about our machines" — asml.com
18. ZEISS SMT, "EUV lithography" — zeiss.com
19. J. Wilson, Introduction to Physical Electronics (LibreTexts), §1.12 "Light Emitting… — eng.libretexts.org
20. NIST Physical Measurement Laboratory, Handbook of Basic Atomic Spectroscopic Data,… — physics.nist.gov
21. ASML, S. Hofman, "Making EUV: from lab to fab" (30 Mar 2022) — asml.com
22. TSMC press release, N7+ EUV in high volume (7 Oct 2019) — pr.tsmc.com
23. Data Center Knowledge (Bloomberg), "ASML Shows Off $380M, 165-Ton Machine Behind AI… — datacenterknowledge.com
24. S. K. Moore, "A Better Way to Measure Progress in Semiconductors", IEEE Spectrum (21… — spectrum.ieee.org
25. IEEE IRDS, International Roadmap for Devices and Systems 2021 Update: More Moore,… — irds.ieee.org
26. Intel news release, "Intel Reinvents Transistors Using New 3-D Structure" (4 May 2011;… — rmsystem.cz
27. TechCrunch, "Samsung Electronics starts 3-nanometer chip production ahead of TSMC" (30… — techcrunch.com
28. TSMC, "2nm Technology" — tsmc.com
29. Intel Foundry, "Intel 18A" — intel.com
30. The Next Platform, N. Hemsoth Prickett, "AMD on Why Chiplets—And Why Now" (9 Jun 2021;… — nextplatform.com
31. Tech Design Forums, "Scaling costs tip balance toward chiplets for AMD server… — techdesignforums.com
32. NVIDIA, "Blackwell Architecture" — nvidia.com
33. US DOE Advanced Manufacturing Office, "Wide Bandgap Semiconductors: Pursuing the… — www1.eere.energy.gov
34. Ioffe NSM Archive, Silicon (band structure, electrical, thermal, basic) — ioffe.ru
35. Ioffe NSM Archive, Silicon carbide (band structure, electrical basics, thermal) — ioffe.ru
36. Ioffe NSM Archive, Gallium nitride (band structure, electrical basics, thermal) — ioffe.ru
37. O. Slobodyan et al., "Analysis of the dependence of critical electric field on… — osti.gov
38. T. Ayalew, SiC Semiconductor Devices Technology, Modeling, and Simulation, PhD… — iue.tuwien.ac.at
39. ChargerLAB, "Anker PowerPort Atom PD 1 30W GaN charger teardown review" (3 Jun 2019) — chargerlab.com
40. Yole / System Plus via GlobeNewswire, "Complete Teardown Report of… — globenewswire.com
41. Wikipedia, "Skin effect" (secondary; copper skin depth 2.06 µm at 1 GHz) — en.wikipedia.org
42. Semtech, "CopperEdge" (active copper reach) — semtech.com
43. NVIDIA Blog, "NVIDIA Blackwell: Born for Extreme-Scale AI Inference" (18 Sep 2025) — blogs.nvidia.com
44. NVIDIA Technical Blog, B. Smith, "A New Era in Data Center Networking with NVIDIA… — developer.nvidia.com
45. Wikipedia, "Silicon photonics" (secondary; transparency above ~1.1 µm, 1.3/1.55 µm… — en.wikipedia.org
46. Intel / UCSB, "First Electrically Pumped Hybrid Silicon Laser" (18 Sep 2006) — intel.com
47. NVIDIA press release, "NVIDIA Announces Spectrum-X Photonics, Co-Packaged Optics… — investor.nvidia.com
48. TSMC press release, 2024 North America Technology Symposium (A16) (24 Apr 2024) — pr.tsmc.com
49. imec, "Smaller, better, faster: imec presents chip scaling roadmap" (2 Feb 2023) — imec-int.com
50. Computer History Museum, Silicon Engine: "1833: First Semiconductor Effect is Recorded" — computerhistory.org
51. Computer History Museum, Silicon Engine: "1940: Discovery of the p-n Junction" — computerhistory.org
52. PBS / ScienCentral & American Institute of Physics, Transistorized! (1999): "The P-N… — pbs.org
53. Computer History Museum, Silicon Engine: "1947: Invention of the Point-Contact Transistor" — computerhistory.org
54. J. Bardeen, "Semiconductor research leading to the point contact transistor", Nobel… — nobelprize.org
55. IEEE / ETHW: "Milestones: First Semiconductor Integrated Circuit (IC), 1958" — ethw.org
56. Computer History Museum, Silicon Engine: "1959: Practical Monolithic Integrated… — computerhistory.org
57. Computer History Museum, Silicon Engine: "1965: 'Moore's Law' Predicts the Future of… — computerhistory.org
58. H. Sutter, "The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in… — gotw.ca

Researched and scripted with AI assistance, fact-checked claim by claim, with synthetic narration and diagrams drawn in code. How we make episodes.