CONTEXT JAMMING

Field notes from inside the context window.

Share

FounderFiles ·N°062·Interpretability

Filed 10.18.26

Nick Cammarata — FounderFile portrait
Fig. · InterpretabilityPortrait

Subject ·Nick Cammarata·researcher · instruments · revision

Nick Cammarata.

Researcher · Goodfire · Mechanistic interpretability

Nick Cammarata helped build the instruments that made neural networks legible; his latest experiment asks whether those instruments also taught researchers to see the wrong thing.

The car fed into the dogs. Nick Cammarata helped turn that strange circuit into an example of superposition; in October 2026, he returned to it and argued that he had misread his own discovery. The builder of the microscope was revising the picture.

FROM
Newburyport, Massachusetts
AT
Goodfire
FILE
N°062
§ 01 · THE FIRST INSTRUMENT

Build the thing that teaches

In May 2011, Nick Cammarata was an eighteen-year-old high-school senior in Newburyport, Massachusetts, with a place in the inaugural Thiel Fellowship. His local newspaper described a student whose attention followed computer-programming books more reliably than classroom lectures. It also reported that he would forgo an accepted place in Carnegie Mellon’s computer science program [single-source]. He and David Merfield were working on OPEN, software for making and sharing lessons. That early ambition was to change how people learned.

There is a tempting shortcut through this biography: draw a straight line from the teenager bypassing college to the researcher bypassing consensus. The record does not supply that line. His route into machine learning has gaps, and a fellowship is neither a technical result nor a personality test. What the public work eventually makes visible is a practical habit. Cammarata builds something that lets a person inspect a problem, then uses what becomes visible to ask a better question.

The interface comes before the explanation. That sequence matters because interpretability studies systems whose internal operations are difficult to describe even when their behavior is familiar. A model recognizes an animal; a researcher wants to know how. An image, a slider, a graph of connections can become an instrument for discovering the answer. It can also suggest an answer before the researcher has earned it. This file turns on that second possibility.

§ 02 · MAKING VISION VISIBLE

The microscope was the work

By the Circuits program’s public arrival in 2020, Cammarata was helping turn neural-network internals into things a reader could examine. The program treated a vision model as a collection of interacting mechanisms. Individual neurons responded to edges, curves, textures or more elaborate patterns; their connections offered a way to follow how one computation became another. The ambition was larger than naming neurons. It was to explain a model through the smaller operations that composed its behavior.

Cammarata was an author of the program’s introduction, Zoom In, and first author of Curve Detectors. The latter developed ways to investigate neurons sensitive to curvature, including radial tuning curves and infrastructure for extracting figures. He also contributed to OpenAI Microscope, which assembled visualizations across eight vision models. This was research conducted through instruments, with the instruments published alongside the argument. A compelling picture was useful because it opened a mechanism to inspection.

In 2021, Multimodal Neurons carried that approach into a model connecting images and language. Cammarata was an equal-contribution author. The paper explored neurons responding across representations of the same concept, bringing a suggestive connection to neuroscience into view. The work made him a participant in the construction of interpretability’s early visual vocabulary. That is the source of his later reversal’s force: he knew how the examples had been found, how the pictures had been made, and how an interpretation could harden around them.

“The interface comes before the explanation. Sometimes it writes the explanation, too.”
The thesis, compressed
§ 03 · THE CANONICAL CIRCUIT

A car inside the dogs

The origin story comes from Cammarata’s own 2026 retrospective. In 2019, while building his first Circuits interface, he found connections that seemed to send a car representation into neurons whose feature visualizations looked like dogs. His interface showed generated images and the weights connecting neurons. He took the circuit to Chris Olah. In Cammarata’s account, they interpreted it as a kind of representational folding and called it superposition. The circuit appeared in Zoom In the following year.

The idea addressed a serious puzzle. A neuron sometimes appeared to respond to several unrelated things. If networks needed more concepts than they had available dimensions, sharing representational space offered an explanation. A single neuron’s confusing behavior could be a shadow of a cleaner organization elsewhere. Recovering that organization would require more than reading the neuron directly. The car-and-dogs picture gave an otherwise abstract problem a memorable physical form.

In October 2026, Cammarata returned to the example and proposed a different mechanism. Cars and dogs could meet through shared pose; the generated pictures could overemphasize dogs because those images were an easy route to activating the neuron. The circuit, he argued, had been ordinary composition seen through an incomplete instrument. That is a revision of a particular piece of evidence, supported by his new reading of its connections. It does not settle every mathematical or empirical claim that has traveled under the same name.

§ 04 · THE EXPLAINER

Ask a model to look

In 2023, the instrument changed again. OpenAI’s Language models can explain neurons in language models used GPT-4 to propose explanations for neurons, then tested those explanations by asking a model to predict activations. The paper credits Cammarata with the initial idea and with helping establish the proof of concept, codebase and neuron viewer. William Saunders and Jeff Wu led the project. The project gave his work a new direction: language models could become instruments for investigating other language models.

The results were limited. The researchers obtained more than a thousand explanations scoring at least 0.8 under their measure, while most explanations scored poorly. An automated explanation was useful only insofar as it captured what the neuron actually did. The experiment exposed a knot in the measurement itself: a weak explanation could reflect a complicated neuron, an inadequate explainer, a deficient scoring method, or some combination of the three. A disappointing score did not tell the researcher which part had failed.

His 2026 essay revisits that problem from inside the earlier project. Cammarata now argues that weak tools helped make neurons look harder to understand than they were. He questions the old simulation scores and reports better explanations using newer methods. These are claims in the new essay, rather than a retrospective correction already established by the 2023 paper. The continuity is the useful part. He keeps returning to the apparatus that decides what counts as understanding.

§ 05 · GOODFIRE

The tools become a company

The Goodfire team posed on a sofa in front of a brick wall, with the Goodfire logo
Fig. · GoodfireTeam

By Goodfire’s August 2024 emergence from stealth, its announcement described Cammarata as formerly at OpenAI. By April 2025, Menlo Ventures named him as a member of the Goodfire team. The exact date of the transition is not established in the available record. The role relevant here is researcher at Goodfire, a company trying to turn an ability to inspect neural networks into an ability to work on them. Better explanations would become engineering tools: ways to investigate behavior and make deliberate changes.

Goodfire’s founders were Eric Ho, Dan Balsam and Tom McGrath. Cammarata’s place in this story is the research bench. The company announced a $150 million Series B at a $1.25 billion valuation in February 2026, bringing its reported backing above $200 million. Anthropic had participated in the previous year’s Series A. Interpretability had acquired a company, customers to pursue and investors with an interest in whether reading a network could become a repeatable engineering practice. A funding round measures that wager. It does not measure how much of a model the researchers understand.

Cammarata’s own work makes that connection concrete. In May 2025, he was first author of Painting with concepts using diffusion model latents. The project offered an image-editing canvas built around concepts extracted from a diffusion model, using a sparse autoencoder and further factorization. A sparse autoencoder learns a larger collection of selectively active features from a model’s internal activity. It supplies a dictionary with which researchers can try to describe—and sometimes steer—the system.

The October essay questions where such dictionaries belong in the investigative sequence. Cammarata argues for understanding the native neurons first, then adding higher-level tools when they earn their place. That creates a productive tension within his own published record. The researcher who helped build a concept-editing canvas is asking whether decomposition sometimes conceals an organization already present. Goodfire is publishing the argument from inside the business of interpretability. The commercial stakes sharpen the question; they do not answer it.

“A dictionary can clarify a model. It can also split apart the thing you were trying to understand.”
The thesis, compressed
§ 06 · THE OCTOBER REVISION

One shape, several familiar things

Neural Networks Are Already Clean appeared on October 15, 2026. Its most accessible exhibit is neuron 4e:55 in InceptionV1, famous for responding to cat faces and cars. Cammarata proposes that the shared feature is a V shape. Once he asks what one thing the examples might have in common, the mixture begins to resolve. His essay examines the neuron’s other activation pole and its inputs as evidence for that interpretation. The claim is about a mechanism, rather than a resemblance alone.

He calls the broader process recursive generalization: a neuron’s function can develop related branches, with later extensions growing from earlier ones. Several recognizable subjects in an activation grid can therefore belong to one learned structure. Architecture and visualization methods also matter. The essay reports that better tools make more InceptionV1 neurons understandable, while modernized training and a different architecture improve the result further. Its roughly 99% figure concerns last-layer neurons in a comparable-size ConvNeXt, not a demonstrated property of every network.

The language-model evidence is preliminary. His Qwen3-32B audit estimates that about 90% of neuron poles have one main job, about 7% have two related jobs, and roughly 2–4% remain unexplained. A pole is one side of a neuron’s activity, so the denominator matters. These are the essay’s reported findings; no independent replication was located as of October 18. The essay describes a technical companion paper as forthcoming. The arresting image is ready for public inspection; the larger empirical case still needs to be tested.

§ 07 · WHAT THE REVISION EARNS

A better question for the microscope

Cammarata’s target is arbitrary packing: unrelated concepts sharing a representation because they seldom interfere. He distinguishes that claim from the broader mathematical sense in which a system represents more features than it has dimensions. He explicitly respects the experiments in Toy Models of Superposition. Treating his essay as proof that superposition never happens would erase the distinction on which his argument depends. Treating the title as a measured census of all neural networks would do the same.

The limits are part of the story. The essay acknowledges unreliable tests for weak activations, sensitivity to thresholds, and speculation in its broadest extrapolation. Its sparse-autoencoder comparisons concern the vision models studied there. A cleaner account of a famous neuron is evidence about that neuron; moving from there to frontier language models requires a wider chain of evidence. Independent researchers need methods they can reproduce, examples that resist the explanation, and tests that separate a persuasive story from a causal account.

What Cammarata has already put on the table is a demanding act of revision. He returns to a circuit he helped establish, identifies the instrument that shaped its interpretation, and offers another mechanism to inspect. His career gives that move continuity: classroom software, visualization systems, automated explainers, tools for working on models. Each promises to make something easier to see. The October essay asks the harder question those instruments bring with them. When the picture becomes familiar, will its maker still look again?

“The strongest part of the reversal is the return to an example he helped make canonical.”
The thesis, compressed
Timeline

The Work, in Sequence

  1. 2011Thiel Fellowship · Named in the inaugural class; works with David Merfield on classroom software.
  2. 2019Circuits · Finds the car-and-dogs circuit, according to his 2026 retrospective.
  3. Mar 2020Distill · Co-authors Zoom In: An Introduction to Circuits.
  4. Jun 2020Distill · First author of Curve Detectors.
  5. Mar 2021Distill · Equal-contribution author of Multimodal Neurons.
  6. May 2023OpenAI · Core contributor to the GPT-4 neuron-explainer project; credited with its initial idea.
  7. Apr 2025Goodfire · Named as a team member in Menlo Ventures’ Series A announcement.
  8. May 2025Goodfire · First author of Painting with concepts using diffusion model latents.
  9. Oct 15, 2026Goodfire · Publishes Neural Networks Are Already Clean.
Index

Numbers With Conditions

Inaugural Thiel Fellowship class
2011
The InceptionV1 cat/car neuron revisited in the essay
4e:55
Neuron poles with one main job in the essay’s preliminary Qwen3-32B audit; no independent replication located as of Oct. 18
~90%
Publication date of the 2026 revision
15 Oct
Dossier

Nick Cammarata

Nick Cammarata · Researcher at Goodfire · Previously at OpenAI · Newburyport, Massachusetts.

Work in this file: Circuits, neural-network visualization, automated neuron explanation, concept-based image editing.

Collaborators represented in this record include Christopher Olah, Gabriel Goh and Curt Tigges.

Goodfire’s founding team: Eric Ho, Dan Balsam and Tom McGrath. Cammarata is profiled here for his research and tool-building contributions.

Provenance: the 2019 discovery narrative is Cammarata’s retrospective. The 2026 results are claims in his essay. No independent replication was located as of October 18, 2026; the essay describes a technical companion paper as forthcoming.

FounderFiles N°062 · Editorial profile · Record reviewed through October 18, 2026.

Filed October 18, 2026. The preliminary findings and publication status above are current to that date.

Career Shape
tree-shaped — depth across many branches of one field

Tree-Canopy Autodidact

Editorial inference from his public work: builds tools to make neural mechanisms visible, then revisits the explanation when the instrument reveals a simpler account. The shape and research horizon are interpretations; his personal capital posture is not established.

Credential Path
Autodidact
Abstraction
Bottom-up / algorithmic
Exit Horizon
Deferred
Moat Instinct
Interpretability
Capital Posture
Not Established
Role-Model Reference Class
  • Circuits / Distill visual-explanation tradition
Share
FounderFiles N°062 · Nick Cammarata
Filed by Bret Kerr · ACRA Insight LLC · Franklin, MA
contextjamming.com · @bretkerr
← back to Context Jamming