What is a neural network, and how is it not like a brain?

The full story

The episode's narration, word for word. Numbers link to the sources below; timestamps play that moment.

The question 0:00

Last time, Arthur Samuel pointed to a road he didn't take: a network of simple switches inspired by nerve cells ¹. Today's chatbots took that road, with billions of numbers, in what's called a neural network ^2,3,4. The name sounds like a brain. So what is one, and how much is it really like the thing inside your head?

A runaway and a neurophysiologist 0:24

In Chicago in the early 1940s, Walter Pitts was a teenage runaway who had taught himself logic, and Warren McCulloch was a neurophysiologist more than twenty years older, who took him into his family's home ⁵. In 1943 they published "A Logical Calculus of the Ideas Immanent in Nervous Activity" ⁶. Because a nerve cell either fires or doesn't, a brain's activity could be written down as logic ⁶.

A neuron as a switch 0:53

A nerve cell gets signals from many others at junctions called synapses ⁷. Some push it towards firing, others hold it back, and they add up ⁷. If the total crosses a threshold, the cell fires a pulse, all or nothing ⁷. McCulloch and Pitts turned that into a switch: count the inputs that are on, and fire if the count reaches the threshold ⁶. With two inputs, a threshold of two gives you "and", and a threshold of one gives you "or" ⁶. But they assumed the wiring never changes, so their networks couldn't learn ⁶.

The machine that sorted cards 1:30

Learning arrived with Frank Rosenblatt, a psychologist at the Cornell Aeronautical Laboratory in Buffalo, New York ⁸. In 1958, with US Navy research money, he described a network he called the perceptron ⁹. That July, the Navy showed it running on an IBM 704, a room-sized computer weighing about five tons ⁸. It was fed punch cards marked on the left or the right, and after fifty tries it had taught itself to tell them apart ⁸.

One artificial neuron 2:02

Let's build Rosenblatt's kind of neuron, still the heart of today's networks ^9,10. Say it decides whether you go for a swim, from four inputs, each one for yes or zero for no: is it sunny, is the water warm, are you between the flags, and have bluebottles washed up? Each input has a weight, a number for how much it matters ^9,11. Multiply each input by its weight, add them up, and if the total reaches the threshold, the neuron fires ⁹. Sunny, cold water, between the flags: two plus four makes six ¹². The threshold is five, so it fires, and in you go ¹².

A vote against 2:45

Now the bluebottles turn up. Their weight is minus six, so the total drops to zero, and the neuron stays quiet ¹². A negative weight is the artificial version of a signal that holds a nerve cell back ^9,7. That's the whole neuron: multiply, add, compare ⁹. I picked these weights myself to make the point. The real question is how a machine finds them itself.

Dials turned by motors 3:12

Rosenblatt built the weights out of hardware. In his Mark I Perceptron, each weight was a potentiometer, the same kind of part as a volume knob, turned by an electric motor ¹³. From here on, picture every weight as a dial. A big patch panel of cables wired more than five hundred units together ¹³. When it got an answer wrong in training, motors turned the dials ^9,13,11. It learned simple shapes and block letters, and the whole alphabet was about its limit ¹³.

Conscious of its existence 3:45

The New York Times headline in July 1958 was "New Navy Device Learns by Doing" ^8,14. It reported that the Navy expected the perceptron to walk, talk, see, write, reproduce itself and be conscious of its existence ¹¹. The New Yorker called it the first serious rival to the human brain ever devised ⁸. What it had actually done was sort cards by which side the mark was on ⁸. Remember that gap when an AI headline sounds too good.

One straight cut 4:17

Give one neuron two inputs, each zero or one, and there are four possible cases, like the corners of a square. Firing when the weighted sum reaches a threshold is the same as drawing one straight line across that square: fire on one side, stay quiet on the other ^11,15. For "and", one line cuts off the corner where both inputs are on ¹². For "or", one line cuts off the corner where both are off ¹².

The puzzle one neuron can't solve 4:48

Now try "one or the other, but not both", which computer scientists call exclusive or ¹⁵. Its two yes answers sit in opposite corners, and no straight line can fence them off from the other two ¹⁵. We tried thirteen hundred and seventy-seven combinations of weights and thresholds ¹². Twenty do "and", sixty do "or", and not one does exclusive or ¹². Searching harder won't help, because a few lines of algebra prove that no single neuron can ever do it ^15,12.

Perceptrons, 1969 5:21

In 1969, Marvin Minsky and Seymour Papert at MIT published a book called Perceptrons ¹⁵. It proved hard limits on simple perceptrons: one neuron can't do exclusive or, and a layer of small, local detectors can't tell whether a shape is all one connected piece ¹⁵. They also wrote that there was no reason to suppose the perceptron's virtues carried over to the many-layered version ¹⁶. They called that an intuitive judgment worth testing, but it was widely read as a verdict ^16,17.

Add a layer 5:56

Yet adding a layer fixes it ¹². Put two neurons first: one fires for "or", the other for "and" ¹². Then a third neuron listens to both, and fires when "or" is on but "and" is off ¹². This is what layers do: each one builds new features out of the last, and stacked up, they can compute far more ^11,15. The catch, in the 1960s, was that there was no practical, widely used way to train those middle layers: it was hard to tell which hidden dial was to blame for a wrong answer ^18,19.

The first AI winter 6:31

What followed is often called the first AI winter ¹⁸. In Britain, the Lighthill Report of 1973 was highly critical of basic AI research ²⁰. Minsky and Papert's book is often blamed for freezing neural networks: Cornell's own history says it essentially sealed the perceptron's fate ⁸. But the sociologist Mikel Olazaran argues that story was shaped by the rival, symbol-based school of AI, and the authors later said progress had already stalled ^16,17. Rosenblatt never saw the revival: he drowned in a sailing accident on his forty-third birthday, in 1971 ⁸.

From nineteen dials to billions 7:13

A tiny network with four inputs, three middle neurons and one output has nineteen dials ¹². As their makers described them in 2023 and 2024, the models behind ChatGPT, Gemini, Claude, Llama and DeepSeek are all built from layers of the same kind of unit ^{21,3,4,22,23,24}. One change: modern neurons pass on a number, not a hard yes or no: in the textbook version, just the total if it's above zero, and big chatbots use smoother variants of that ^3,4,10. GPT-3 had a hundred and seventy-five billion dials in 2020, and DeepSeek-V3 had six hundred and seventy-one billion in late 2024 ^2,4. Many no longer say: OpenAI's 2023 GPT-4 report left it out, and so did the reports on Google's larger Gemini models and Anthropic's Claude 3 ^21,22,25.

What came from the brain 8:13

So what was borrowed from brains? Adding up signals that push and hold back, firing past a threshold, and learning by strengthening or weakening connections ^6,9,7. But the resemblance breaks down in at least four big ways ^26,19,27,28.

Break one: spikes, not numbers 8:29

First, a real neuron talks in brief electrical spikes, each rising in about a thousandth of a second ^7,29. Neuroscientists still argue over whether the message is in how often spikes come or in their exact timing ²⁸. An artificial neuron passes along a single number, and the network works in neat steps, layer by layer ¹⁰.

Break two: one cell, a whole network 8:52

Second, one real neuron is far more than one unit ²⁶. In 2021, researchers at the Hebrew University of Jerusalem tried to copy a detailed computer model of a single cortical neuron with an artificial network ²⁶. To match it millisecond by millisecond, they needed a network five to eight layers deep ²⁶. Much of that complexity came from the cell's branching inputs, its dendrites ²⁶.

Break three: scale 9:21

Third, scale. Your brain has about eighty-six billion neurons ³⁰. Its outer layer, the neocortex, alone has around a hundred and sixty-four trillion synapses ³¹. That's hundreds of times more connections than GPT-3 or DeepSeek-V3 have dials ^31,2,4,12. But one synapse isn't one weight, and one neuron isn't one unit, so counting isn't comparing ²⁶.

Break four: learning and energy 9:49

Fourth, learning and energy. Artificial networks learn with a method called backpropagation, which is next episode's story ¹⁹. Whether the brain does anything like it is an open question ^19,27. In 1989, Francis Crick wrote that most neural networks of the day were unrealistic in important respects ²⁷. In 2020, a team including Geoffrey Hinton argued the brain might approximate it ¹⁹. Your brain runs on about twenty watts ³². Training GPT-3 took an estimated thirteen hundred megawatt hours, more than seven thousand years of a brain's energy use ^33,12.

Useful model, or misleading metaphor? 10:30

Is the brain comparison useful at all? Scientists disagree ^34,35. In 2014, a team including James DiCarlo found that a network trained only to recognise objects predicted the responses of neurons in a monkey's visual cortex ³⁴. In 2022, Jeffrey Bowers and colleagues argued that such networks explain almost none of the findings of psychology experiments, so a good prediction doesn't prove a shared mechanism ³⁵.

Who turns the dials? 11:02

So a neural network is a stack of very simple units, each multiplying, adding and comparing ^9,10. As one standard textbook puts it, they're loosely inspired by neuroscience, and the goal is not to perfectly model the brain ¹⁰. Everything a network knows lives in its weights ⁹. A modern one has billions of them in many layers, and nothing in the training data says which dial caused a mistake ¹⁹. How does a machine get better by being wrong?

Sources

Every factual claim in the episode is tied to one of these. Spotted an error? Tell us.

1. Samuel, A. L. (1959), "Some Studies in Machine Learning Using the Game of Checkers",… — doi.org
2. Brown, T. B. et al. (OpenAI, May 2020), "Language Models are Few-Shot Learners" (GPT-3) — arxiv.org
3. Llama Team, AI @ Meta (July 2024), "The Llama 3 Herd of Models" — arxiv.org
4. DeepSeek-AI (Dec 2024), "DeepSeek-V3 Technical Report" — arxiv.org
5. Gefter, A. (29 Jan 2015), "The Man Who Tried to Redeem the World with Logic", Nautilus — nautil.us
6. McCulloch, W. S. & Pitts, W. (1943), "A Logical Calculus of the Ideas Immanent in… — doi.org
7. OpenStax, Biology 2e, §35.2 "How Neurons Communicate" (CC BY 4.0) — openstax.org
8. Lefkowitz, M. (25 Sep 2019), "Professor's perceptron paved the way for AI – 60 years… — news.cornell.edu
9. Rosenblatt, F. (1958), "The perceptron: a probabilistic model for information storage… — doi.org
10. Goodfellow, I., Bengio, Y. & Courville, A. (2016), Deep Learning, MIT Press, ch. 6… — deeplearningbook.org
11. Wikipedia, "Perceptron" (Mark I hardware, citing Bishop 2006; NYT quote; linear… — en.wikipedia.org
12. This episode's own worked examples, calc.py (asserted at render time; arithmetic box…
13. Rosenblatt, F. (1961), Principles of Neurodynamics: Perceptrons and the Theory of… — papers.baulab.info
14. The New York Times (8 July 1958), "New Navy Device Learns by Doing: Psychologist Shows… — nytimes.com
15. Minsky, M. & Papert, S. (1969), Perceptrons: An Introduction to Computational… — en.wikipedia.org
16. Quote Investigator (24 Jul 2026), "Our Intuitive Judgment That the Extension Is… — quoteinvestigator.com
17. Olazaran, M. (1996), "A Sociological Study of the Official History of the Perceptrons… — doi.org
18. Wikipedia, "AI winter", accessed 2026-10-15 — en.wikipedia.org
19. Lillicrap, T. P., Santoro, A., Marris, L., Akerman, C. J. & Hinton, G. (2020),… — doi.org
20. Howe, J. (1994, rev. 2007), "Artificial Intelligence at Edinburgh University: a… — inf.ed.ac.uk
21. OpenAI (March 2023), "GPT-4 Technical Report" — arxiv.org
22. Gemini Team, Google (Dec 2023), "Gemini: A Family of Highly Capable Multimodal Models" — arxiv.org
23. Anthropic (21 May 2024), "Mapping the Mind of a Large Language Model" — anthropic.com
24. Vaswani, A. et al. (2017), "Attention Is All You Need" — arxiv.org
25. Anthropic (March 2024), "The Claude 3 Model Family: Opus, Sonnet, Haiku" (model card;… — www-cdn.anthropic.com
26. Beniaguev, D., Segev, I. & London, M. (2021), "Single cortical neurons as deep… — doi.org
27. Crick, F. (1989), "The recent excitement about neural networks", Nature 337, 129–132 — doi.org
28. Brette, R. (2015), "Philosophy of the Spike: Rate-Based vs. Spike-Based Theories of… — doi.org
29. HyperPhysics (R. Nave, Georgia State Univ.), "Action Potential" — hyperphysics.gsu.edu
30. Azevedo, F. A. C. et al. (2009), "Equal numbers of neuronal and nonneuronal cells make… — doi.org
31. Tang, Y., Nyengaard, J. R., De Groot, D. M. & Gundersen, H. J. (2001), "Total regional… — doi.org
32. Levy, W. B. & Calvert, V. G. (2021), "Communication consumes 35 times more energy than… — doi.org
33. Patterson, D. et al. (2021), "Carbon Emissions and Large Neural Network Training" — arxiv.org
34. Yamins, D. L. K. et al. (2014), "Performance-optimized hierarchical models predict… — doi.org
35. Bowers, J. S. et al. (2022), "Deep problems with neural network models of human… — doi.org

Researched and scripted with AI assistance, fact-checked claim by claim, with synthetic narration and diagrams drawn in code. How we make episodes.