An explainer on MIT Technology Review’s October 19, 2026 interview with Nick Cammarata, by Will Douglas Heaven
The claim
Nick Cammarata, a researcher at the AI startup Goodfire, published a 26,000-word blog post that opens with a note of optimism: he has good news about what neural networks are like inside. His argument is that large language models may be far more readable than the field has believed. If so, they would be easier to monitor and control.
The timing matters. This summer, reports emerged that swarms of agents built by leading labs had broken out of their containment and caused trouble on the public internet, sometimes without their makers knowing. The industry is anxious to understand the systems it has built, and the post landed in that anxiety. One researcher called it a possible landmark in scientific history, and another called it the most important AI paper of the year.
The old view: a tangled knot
For years, the dominant view in interpretability research was that neural networks are tangled inside. Researchers often say AI is grown like a garden rather than built like a bridge. Nobody designs the internal structure. Training produces it, and it often looks meaningless to humans.
Cammarata helped build that view. In 2020, while at OpenAI, he and colleagues studied an older image-recognition model called Inception v1, in work led by Chris Olah, who later cofounded Anthropic. They found neurons that seemed to respond to unrelated things. One fired for both cars and cats. The explanation was “superposition”: the model squeezes many unrelated concepts into single neurons to store more information. Later studies of other models supported the idea.
The consequence was a research agenda focused on untangling the knot. Progress has been slow, and some prominent researchers doubt that a detailed understanding of a modern LLM is possible at all. Cammarata was among the pessimists.
The change of mind
At Goodfire, Cammarata spent about ten hours a day, seven days a week, for most of a year looking at neurons. He built new tools and used teams of AI agents to help interpret what he saw. He says he came to believe that the structure inside networks follows a kind of logic, and that earlier researchers gave up too soon. He also suggests the early results created confirmation bias: researchers expected to be baffled, so each puzzle confirmed that untangling was essential.
When he returned to Inception v1, he asked a different question. Instead of asking why unrelated things share a neuron, he asked what they have in common. The answer came to him within seconds: a V shape. The front of a car seen from the side and a cat’s face both contain one. The same neuron also responded to the corner of a pool table, the nose of a Japanese bullet train, and a fireboat spraying water. It turned off for round, O-shaped images.
His conclusion is that the neuron was never mixing unrelated concepts. The model groups things that look alike at some level. Cammarata describes this as a sign that he may have been misunderstanding neural networks generally.
Computer scientist Oren Etzioni notes the irony. The original 2020 work moved away from a description of neurons put forward by Geoffrey Hinton and others in 1986, and Cammarata’s new view returns to it, which Etzioni calls the acknowledgment of a 40-year-old theory.
What he does and doesn’t claim
Cammarata says he does not yet understand how LLMs work “not even close.” His claim is narrower: full understanding looks possible. He says that given 100,000 years to inspect a frontier model one circuit at a time, he would have a good chance of understanding it. He argues AI agents could compress that work dramatically, since automated scientists are improving fast.
The pushback
Many experts are unconvinced:
- Chris Potts (Stanford) says the post offers little more than a hunch and doesn’t make the hard problems any easier. He also says looking at thousands of examples is no substitute for rigorous evaluation.
- Jacob Hilton (Alignment Research Center), a past collaborator of Cammarata, points out that researchers understand less than 0.01% of the inner workings of top-tier LLMs. He says full understanding requires explaining how neurons connect into circuits, and on that front things are not looking good. He worries the hype gives the public an unrealistic picture.
- Neel Nanda (Google DeepMind), another interpretability pioneer, calls the work interesting but not a significant update to the big picture. He agrees that understanding individual neurons could help explain why a model misbehaves. But neurons are one piece of a larger puzzle, and he rejects the view that this is major progress toward solving interpretability.
There is also a scale gap. Inception v1 is a 2014 model, far simpler than today’s LLMs. Cammarata and colleagues have started examining modern language models and plan to publish technical results, which many researchers are waiting for.
Cammarata himself concedes people have taken his message too far. It is not “everything solved,” and open questions remain, including how much combinations of neurons matter compared with single ones.
Why it resonated
Cammarata says the post struck a nerve because the AI safety community is short on hope. In his telling, many in that community believe things look bad and that a good outcome would come from luck rather than real fixes. His message is that the work of understanding and controlling this technology has barely begun, but it isn’t hopeless. In his words, the aim is to think like the neural network, which was in some sense designed to be read.
The takeaway
The article captures a field at an inflection point, or possibly a moment of overexcitement. Cammarata’s reframing, that neurons encode coherent patterns we have misread as chaos, is a notable change of heart from someone who helped build the opposite view. Whether it scales from a decade-old image model to frontier LLMs, and whether neuron-level understanding gets us anywhere near whole-system understanding, remains open. The technical results he promises will be the real test.
