Bigram
[A] → [B]The direct embedding-to-unembedding path stores local statistics.
CONTEXT JAMMING / MECHANISTIC INTERPRETABILITY / 22 DEC 2021
TRANSFORMER CIRCUITS
& INDUCTION HEADS
How two layers of attention invent in-context learning
Attention is not merely a blur of token-to-token links. Expand the residual stream into end-to-end paths and a small transformer becomes legible: QK circuits choose where to look; OV circuits decide what to write. In two layers, those parts compose into a reusable copying algorithm.
PAPER-DERIVED ARCHITECTURE · CONCEPTUAL RECONSTRUCTION
ZERO-LAYERbigram tableONE-LAYERskip-trigram ensembleTWO-LAYERinduction via K-composition
01 · The old mental modelCONCEPTUAL JOB · separate visible attention from a causal explanationPAPER ANCHOR · Transformer Overview; Attention Heads as Information Movement
“Attention moves information” is true—and radically incomplete.
A head does two different jobs. Its QK circuit decides which source positions receive attention. Its OV circuit decides how information from an attended source changes the destination and, ultimately, the next-token logits.
Once “where” and “what” are separated, the sudden advantage of a second layer becomes a tractable anomaly rather than a vague scaling story.
02 · The anomalyCONCEPTUAL JOB · show why an extra layer changes the available algorithmPAPER ANCHOR · Summary of Results; One- and Two-Layer Attention-Only Transformers
One layer can copy. Two layers can discover a rule.
Skip-trigram
[B] … [A] → [B]A head can copy a plausible earlier token, conditioned on learned token statistics.
Induction
[A][B] … [A] → [B]One head shifts a key; another matches the prefix and copies its continuation.
ESTABLISHED RESULTThe paper reverse-engineers these algorithms in small attention-only models.AUTHOR INTERPRETATIONThis constitutes a transition to a more general form of in-context learning.CONTEXT JAMMING EXTENSIONLater, we compare the circuit’s shifted lookup to pointer arithmetic without claiming identity.
To see the difference in the weights, we first need the paper’s decisive coordinate change: replace a stack of layers with a sum of paths.
03 · The coordinate changeCONCEPTUAL JOB · turn layer products into additive end-to-end circuitsPAPER ANCHOR · The Path Expansion Trick; Path Expansion of Logits
Expand the product. Read the paths.
Residual connections make each layer an identity path plus a collection of attention-head writes. Multiplying layers out produces a sum: one term for every end-to-end route from input token to output logit.
PAPER-DERIVED VALUES · TERM-IMPORTANCE TABLE · TWO-LAYER MODELORDER 0-1.8 nats1 term · -1.8 nats/term ORDER 1-5.2 nats24 terms · -0.2 nats/term ORDER 2-0.3 nats144 terms · -0.002 nats/term
The embedding-to-unembedding route; a readable bigram table.
Each head contributes an end-to-end token-to-logit map; together these behave like skip-trigram tables.
V-composed virtual heads are elegant, but have little marginal effect in the studied two-layer model.
The nats are the paper's aggregate ablation values, not a token-specific decomposition for “ Dursley.” The token selector changes the conceptual route being inspected. Higher-order virtual-head terms are small in this model.
At order zero and one, the abstract paths collapse into something almost embarrassingly concrete: tables of token relationships.
04 · Zero- and one-layer baselinesCONCEPTUAL JOB · read bigrams and skip-trigrams directly from expanded weightsPAPER ANCHOR · Zero-Layer Transformers; Interpretation as Skip-Trigrams
A tiny transformer is a compressed room full of index cards.
The zero-layer matrix maps the present token directly to next-token logits: a bigram table. Add one layer and each head contributes a factored skip-trigram table. QK selects a source given the destination; OV changes output logits given that source.
PAPER-DERIVED LARGE-ENTRY EXAMPLES · NORMALIZED CONCEPTUAL RECONSTRUCTION
keep and open def github Ralph
keep
and
open
def
github
Ralph
The paper publishes enormous expanded matrices (roughly 2.5 billion entries at a 50k vocabulary) and curated large-entry tables. This compact viewer preserves published example families and sign logic; its normalized cell intensities are not a model checkpoint dump.
A second layer adds more than another pile of cards. It lets later heads read what earlier heads wrote.
05 · The three operatorsCONCEPTUAL JOB · separate the three ways attention heads can composePAPER ANCHOR · Three Kinds of Composition; corrected composition diagram
Q changes the question. K changes the address. V chains the payload.
A later head can read an earlier head’s write through its query, key, or value projection. Q- and K-composition alter the attention pattern. V-composition alters what is moved and creates an implicit virtual head.
PAPER-DERIVED QUALITATIVE RESULT · CORRECTED COMPOSITION DIAGRAM
Bar lengths are schematic, not Frobenius-norm measurements. The original paper figure contained a library bug; the correction preserves the previous-token K-composition result while revealing additional composition for two induction heads.
With the operators separated, the induction algorithm can be stated as a three-step program rather than a mysterious diagonal in an attention map.
06 · Induction heads appearCONCEPTUAL JOB · recompute the minimal previous-token + match + copy algorithmPAPER ANCHOR · Induction Heads; How Induction Heads Work
Shift the key. Match the prefix. Copy what came next.
01
A previous-token head writes information about token j − 1 at position j.
02K-composition lets the induction head compare the current query with that shifted key.
03A copying OV circuit raises the logit for the token found immediately after the earlier match.
PAPER-DERIVED MECHANISM · ILLUSTRATIVE DETERMINISTIC RECOMPUTATION
01 SHIFT KEY02 MATCH SAME TOKEN03 COPY NEXT TOKEN
CURRENT RESULTsmall → son
The current token matches an earlier token; the shifted key lands attention on the token that followed it.
The attention rule is a deterministic reconstruction of the paper's mechanism, not a running transformer. Positive QK and OV eigenvalues are a working formalization of matching and copying, not a complete definition.
Natural language can hide a lookup-table trick inside familiar statistics. Random repeated tokens remove that escape hatch.
07 · The out-of-distribution testCONCEPTUAL JOB · remove every distributional cue except repetitionPAPER ANCHOR · Function of Induction Heads; random repeated-sequence experiment
The algorithm survives when the language disappears.
Uniformly sampled vocabulary tokens do not form plausible phrases. Yet when a random sequence repeats, the induction pattern continues: find the earlier copy of the current token and attend to what followed it.
ILLUSTRATIVE MODEL · SYNTHETIC TOKENS · PAPER-OBSERVED BEHAVIOR
SEED<39811>SEED<7598>SEED<1764>SEED<8873>SEED<5186>SEED<40593>COPY 1<39811>← predicts <7598>COPY 1<7598>← predicts <1764>COPY 1<1764>← predicts <8873>COPY 1<8873>← predicts <5186>COPY 1<5186>← predicts <40593>COPY 1<40593>← predicts <39811>COPY 2<39811>← predicts <7598>COPY 2<7598>← predicts <1764>COPY 2<1764>← predicts <8873>COPY 2<8873>← predicts <5186>COPY 2<5186>← predicts <40593>COPY 2<40593>← predicts <39811>
SEED 1741INDUCTION SIGNAL PRESENT
The tokens have no linguistic relation. Repetition alone creates the address structure an induction head needs.
A clean mechanism still needs an importance test: elegant terms can exist algebraically without doing much useful work.
08 · Ablations and boundariesCONCEPTUAL JOB · measure path-order importance and expose the framework’s limitsPAPER ANCHOR · Term Importance Analysis; Virtual Attention Heads; Where Does This Leave Us?
Not every path earns its keep.
PAPER-DERIVED VALUES · TERM IMPORTANCE ANALYSIS
REMAINING MARGINAL LOSS REDUCTION7.3 nats
Order 2 contributes only 0.3 nats in aggregate—about 0.002 nats per virtual-head term.
This ablation isolates V-composition by freezing ground-truth attention patterns. It does not imply that Q- or K-composition is unimportant; K-composition is essential to the induction circuit.
Virtual heads
Order-2 V-composed paths contribute only −0.3 nats in aggregate in the studied model, versus −5.2 nats for individual heads.
MLP layers
The paper deliberately removes MLP blocks. It offers little traction on the feature superposition and nonlinear computation they add.
Eigenvalue positivity
The authors use it as a working summary for matching and copying matrices, while explicitly declining to call it a complete definition.
Scale
This paper does not causally establish induction heads as the primary driver of in-context learning in large, modern transformers.
Those limits make the structural analogy useful only if it is fenced, testable, and kept separate from the paper’s findings.
09 · Context Jamming extensionCONCEPTUAL JOB · map shifted lookup to addressable agent memory without asserting identityPAPER ANCHOR · Additional Intuition: pointer arithmetic; Context Jamming extension
A circuit that behaves like pointer arithmetic.
CONTEXT JAMMING EXTENSION / STRUCTURAL ANALOGY, NOT IDENTITY
An induction head converts token content plus a one-step shift into an address: “find this earlier symbol, advance, retrieve.” That resembles pointer arithmetic in a memory system. It is a structural rhyme, not a claim that today’s agents store or retrieve memory through this exact circuit.
CONTEXT JAMMING EXTENSION · STRUCTURAL ANALOGY, NOT IDENTITY
INDUCTION-STYLE RETRIEVALred → blue
Treat the current symbol as an address: find its earlier occurrence, advance one position, and retrieve what came next.
PAPER OBJECTPrevious-token shift+PAPER OBJECTSame-token match+PAPER OBJECTCopying OV↔STRUCTURAL ANALOGYAddress + offset + retrieval
Where the analogy breaks
- The paper studies pure attention-only toy models; real agents contain MLPs, many more layers, tools, retrieval systems, and external state.
- The paper discovers induction heads by weight inspection in two-layer models; an agent may implement similar behavior through entirely different circuits or software.
- The paper provides no causal evidence that induction heads are the primary driver of in-context learning at scale; that question is explicitly deferred to later work.
A falsifiable extension
MEASURABLE OBJECTInduction accuracy on repeated random sequences before and after targeted ablation.EXPERIMENTAblate a candidate previous-token head; then insert an artificial one-step shifter and measure recovery.WEAKENING RESULTInduction-like behavior remains unchanged and no shifted-key path can be identified.PRIMARY CONFOUNDRedundant heads or MLP-mediated substitutes preserve the behavior after a single-path intervention.
The right ending is therefore not “transformers are databases,” but a ledger that records which statements belong to evidence, interpretation, and extension.
10 · Epistemic ledgerCONCEPTUAL JOB · keep derivation, interpretation, and extension visibly separatePAPER ANCHOR · Summary of Results; Where Does This Leave Us?; CJ synthesis
Three layers. No smuggling.
- Zero-layer models expose a bigram table in their weights.
- One-layer attention-only models factor into bigram and skip-trigram paths.
- The studied two-layer model contains induction heads reliant on K-composition with earlier heads.
- Repeated random tokens elicit the induction pattern in the paper’s experiment.
- Order-2 virtual-head paths have a small aggregate effect in the paper’s term-importance analysis.
- Induction heads constitute a more powerful and general in-context learning algorithm than one-layer copying.
- Virtual attention heads may matter more in larger, deeper models despite being weak here.
- Path-based circuit analysis may scale at least partially beyond these toy systems.
- Positive eigenvalues are a useful but incomplete signature of matching and copying.
- Induction’s shift-match-copy structure resembles pointer arithmetic over addressable context.
- That motif may be useful when designing experiments on agent memory and retrieval.
- No identity between the toy circuit and modern agent memory is claimed.
With the claims sorted, the vocabulary can become compact rather than slippery.
11 · GlossaryCONCEPTUAL JOB · make the circuit vocabulary reusablePAPER ANCHOR · Notation; Transformer Overview; Two-Layer sections
A field guide to the residual stream
Residual stream
The shared vector channel that components read from and add their outputs to.
Path expansion
Algebraically expanding layer products into a sum of end-to-end routes through the model.
QK circuit
The query-key computation that sets attention scores: where a head reads.
OV circuit
The output-value computation that maps an attended token to a change in logits: what a head writes.
Bigram
A next-token relationship involving the present token and the token predicted after it.
Skip-trigram
A factored relationship of the form source … destination → output.
Composition
A later attention head reading information written by an earlier attention head.
K-composition
Composition in which the later head's key reads the earlier head's output.
Previous-token head
A head whose attention pattern mainly selects the token one position back.
Induction head
A head that finds an earlier occurrence of the current token and copies the token that followed it.
Virtual attention head
An implicit end-to-end term created when two heads V-compose.
Eigenvalue positivity
The paper's imperfect summary statistic for whether a matrix behaves like matching or copying.
The framework’s payoff is not a new metaphor for attention. It is a practical method for turning parameters into candidate programs.
12 · ConclusionCONCEPTUAL JOB · state the mechanism in one durable sentencePAPER ANCHOR · Summary of Results
Two layers are enough to turn a table into a program.
Path expansion makes the small model readable because it changes the unit of analysis: not a layer, neuron, or attention map, but an end-to-end route from tokens to logits. In one layer, those routes encode skip-trigrams. In two, an earlier head can shift information so a later head retrieves by relation rather than memorized plausibility. That is the induction circuit—and the paper’s first clean example of a transformer implementing a general algorithm in context.
The citation map below pins each claim to the primary source and records the paper’s post-publication correction.
13 · Source notesCONCEPTUAL JOB · make every major claim auditablePAPER ANCHOR · Primary source and correction
The argument, pinned to the paper
- Zero-layer result: “Zero-Layer Transformers.”
- One-layer path expansion and QK/OV split: “The Path Expansion Trick” and “Splitting Attention Head terms into Query-Key and Output-Value Circuits.”
- Skip-trigram interpretation and large entries: “Interpretation as Skip-Trigrams” and the linked head-dump tables.
- Three compositions: “Three Kinds of Composition” and “Path Expansion of Attention Scores QK Circuit.”
- Induction algorithm: “Function of Induction Heads” and “How Induction Heads Work,” including the Id ⊗ Aᵖʳᵉᵛ ⊗ W term.
- Copying / matching detection: “Checking the Mechanistic Theory” and the two-dimensional OV/QK eigenvalue-positivity comparison.
- Ablation values: “Term Importance Analysis”: order 0 −1.8 nats; order 1 −5.2 nats across 24 terms; order 2 −0.3 nats across 144 terms.
- Limits: “Where Does This Leave Us?” and “Additional Intuition and Observations: MLP Layers.”
- Correction: “Correction: Attention Head Composition Diagram.” The bug affected one visualization; the importance of previous-token K-composition remained, with additional composition revealed.
Elhage, Nelson, Neel Nanda, Catherine Olsson, et al. “A Mathematical Framework for Transformer Circuits.” Transformer Circuits Thread, Anthropic, 22 Dec. 2021.
Read the original Transformer Circuits paper ↗