- Prequential description length splits into program size $|M^*|$ and residual error.
- Reservoir ridge solve gives closed-form spectral $S^\phi$.
- Rule 110 ranks #1 among all 256 ECA rules.
- NCA epiplexity gradient ascent generates emergent solitons.
- Unsupervised MNIST encoder organizes around digit classes.
- Epiplexity RL bonus improves 9/10 benchmarks in Table 1.
CONTEXT JAMMING EXPLAINER / arXiv:2607.18433v1
INTELLIGENCE FROM
LEARNABLE NOVELTY
One bounded observer. Three classical faces of intelligence.
Total surprise conflates what can be learned with what cannot. By isolating the learnable portion—epiplexity—Zhang & Levin (2026) show that solitons, unsupervised category clusters, and stable agent exploration emerge from a single scalar gradient.
01 · Two mirror-image failures
The Noisy Television and the Dark Room
PAPER-DERIVED RESULTMaximizing total surprise traps an agent in front of static TV noise. Minimizing total surprise freezes an agent in a dark room. Both pathologies stem from the exact same failure: treating unlearnable entropy as novelty.
ILLUSTRATIVE MODEL — demonstrates the decomposition principle of Eq. 1–4
TOTAL SURPRISE L8.94 bitsEPIPLEXITY S^ϕ6.18 bitsRESIDUAL NOISE2.76 bits
⚡ BALANCED REGIME: High learnable structure fraction. Program reuse drives gradient ascent.
L = 8.94 bits · S^ϕ = 6.18 bitsFull epiplexity uses the closed-form reservoir ridge solution (Eq. 9). This interaction shows the qualitative split between learnable program length |M*| and residual noise.
DO NOT CONFUSE Novelty is not raw entropy. Total surprise $L(X)$ counts every bit of unpredictable data stream; epiplexity $S^\phi(X)$ counts only the subset that a bounded observer $\phi$ can compress into reusable program structure.
02 · What a bounded mind can actually carry away
Epiplexity: The Learnable Part of Surprise
PAPER-DERIVED RESULTIn prequential coding, the total description length of a sequence x^(1:T) under a model class M splits into two terms: the length of the optimal model program |M*|, and the residual coding cost of the remaining errors.
Epiplexity is defined as the asymptotic per-step program length $|M^*_\phi|$ accessible to an observer $\phi$. It measures the reusable, structured complexity the observer extracts from the environment.
03 · A closed-form score from random features
The Reservoir Estimator
PAPER-DERIVED RESULTEvaluating program length for deep models usually requires expensive inner training loops. Zhang & Levin overcome this by using a fixed, random reservoir feature map $\Phi(X)$ paired with a closed-form ridge readout $W_\lambda$.
Because the reservoir weights stay frozen, the effective spectral description length is given directly by the regularized log-determinant of the feature covariance matrix.
ILLUSTRATIVE MODEL — same mathematical form as Eq. 9; scale is toy for interactivity
SPECTRAL S^ϕ4.84EFFECTIVE RANK4.01TOP-3 SV SUM7.45
Ridge Readout Equation:
S^ϕ = ½ log₂ det(I + η Φ Φᵀ) = ½ ∑ log₂(1 + η sᵢ²)Log-determinant prices feature capacity without requiring inner-loop weight updates.Regime: Complex-mixture · S^ϕ = 4.84Paper uses 256-channel convolutional reservoir on real ECA/MNIST data. Direction and comparative behavior are preserved in this deterministic toy matrix solve.
04 · Computation at the edge of chaos
Elementary Cellular Automata: Rule 110 Ranks Highest
PAPER-DERIVED RESULTWithout human supervision or prior complexity definitions, the reservoir epiplexity score ranks all 256 Elementary Cellular Automata rules. It places Rule 110—the only proven Turing-complete rule—at the absolute top.
PAPER-DERIVED VALUES for reference scores; ILLUSTRATIVE for live proxy
PAPER RANK#1 (Top)PAPER MEAN S^ϕ3.84 ± 0.12CATEGORYEdge of Chaos
Rule 110: Edge of Chaos
Proven Turing-complete. Supports gliders, localized structures, and complex collisions.
| Rule | Category | Mean S^ϕ |
|---|---|---|
| Rule 110 | Edge of Chaos | 3.84 ± 0.12 |
| Rule 54 | Edge of Chaos | 3.12 ± 0.15 |
| Rule 30 | Chaotic | 2.45 ± 0.18 |
| Rule 3 | Periodic | 1.15 ± 0.08 |
| Rule 2 | Trivial | 0.42 ± 0.05 |
| Rule 1 | Trivial | 0.05 ± 0.01 |
Selected: Rule 110 · Paper S^ϕ = 3.84Live score is a lightweight proxy. Full ranking used 512 samples × 10 reservoir draws with 256-channel conv reservoir (Table 2). Rule 110 remains highest under all tested hyperparameters.
DO NOT CONFUSE The ECA ranking is not hand-tuned. Rule 110 scores highest because its glider interactions generate rich, reusable feature subspaces for the fixed observer, unlike chaotic Rule 30 or periodic Rule 3.
05 · Solitons emerge from a single scalar
Neural Cellular Automata Learn to Compute by Gradient Ascent on Epiplexity
PAPER-DERIVED RESULTWhen a 1D Neural Cellular Automaton (NCA) updates its local rules by gradient ascent on $S^\phi(X)$, coherent traveling wave packets—solitons—emerge spontaneously from random noise.
ILLUSTRATIVE MODEL — reproduces qualitative soliton phenomenology (Figure 3)
REGIMESOLITONSASCENT BIAS80%COLLISION SURVIVALHIGH
Emergent Solitons (Figure 3):
Without any explicit target shape or reward, gradient ascent on epiplexity drives local NCA rules to synthesize coherent traveling wave packets that pass through each other unscathed.
State: Emergent Soliton WaveTrue gradient ascent on Eq. 9 through a trainable NCA produces the behavior shown in Figure 3 across all seeds. This lab lets you explore the resulting regime.
06 · Categories appear when redundancy is expensive
Unsupervised MNIST Encoder Organizes Around Digit Classes
PAPER-DERIVED RESULTMaximizing spectral description length on unlabeled MNIST images forces an encoder to shed redundant background pixels and retain the principal factor that separates images: digit class identity.
CONCEPTUAL RECONSTRUCTION — Section 4.2 & Figure 4 MNIST experiment
LINEAR PROBE ACC58.8%READOUT PENALTY λ3.00UNSUPERVISEDYES
Unsupervised Category Emergence:
No class labels ever entered training in the paper. Maximizing learnable novelty forces the encoder to discard shared background redundancy and retain the dominant recoverable factor: digit identity.
Probe Accuracy: 58.8% · λ = 3Real experiment used 64-dim unit-norm codes + width-2048 reservoir, λ=3, η=30. Class separation emerges without human supervision.
07 · Exploration that never collapses
Epiplexity as Intrinsic Reward Improves Nine of Ten RL Tasks
PAPER-DERIVED RESULTAdding the trajectory increment $\Delta S^\phi$ as an intrinsic curiosity bonus outperforms standard exploration baselines across 9 out of 10 reinforcement learning benchmarks in Table 1.
PAPER-DERIVED DATA from Table 1; ILLUSTRATIVE trajectory visualizer
BASELINE SCORE-45MAGNITUDE BONUS-82.1EPIPLEXITY BONUS142.5
Deceptive Maze (Deceptive Geometry):
Magnitude bonus traps agent in noisy high-norm dead ends; epiplexity finds exit.
Deceptive Maze: Epiplexity Bonus = 142.5 vs Magnitude = -82.1Epiplexity intrinsic bonus improves 9 of 10 RL benchmarks by rewarding trajectory program expansion without collapsing into high-norm noise.
08 · The boundary that must move
Limits of the Frozen Observer
AUTHOR INTERPRETATIONA fixed observer eventually saturates. As an agent discovers complex structures, the reservoir’s feature space reaches capacity (S^ϕ → const). Open-ended growth requires the observer itself to evolve.
“AN OBSERVER THAT CANNOT GROW WILL EVENTUALLY MISTAKE NEW HIGHER-ORDER STRUCTURE FOR IRREDUCIBLE NOISE.”
09 · Making the measurer grow with what it measures
Co-Evolving Observers (Context Jamming Extension)
CONTEXT JAMMING EXTENSION (STRUCTURAL ANALOGY, NOT A PAPER CLAIM)
What happens when we apply this insight to agentic AI research systems? If an agent’s evaluation harness or memory index remains static, its discovery loop eventually plateaus. Co-evolving the observer enables open-ended exploration.
CONTEXT JAMMING EXTENSION — structural analogy for open-ended capacity expansion
OBSERVER GROWTHENABLEDFINAL DIMENSION48 dimsSATURATION PREVENTEDYES
Open-Ended Intelligence Question:
The paper explicitly notes in the Discussion that a frozen reservoir eventually saturates. Allowing the evaluation harness or memory index to grow alongside the data stream sustains open-ended discovery.
Mode: Co-Evolving CapacityThis demonstrates the open-endedness question the paper raises for LLMs and agentic systems, not a result established by the paper.
Where the Analogy Breaks (4 Key Differences)
- 1. Paper reservoir is an untrained, frozen 256-channel convolution; real LLM or agentic evaluators possess deep, trainable, multi-layer parameters.
- 2. Real agentic observers perform dynamic in-context learning and zero-shot reasoning, whereas the paper’s observer is strictly a linear ridge readout on fixed features.
- 3. Observer growth is explicitly identified in the paper as a future theoretical challenge (Discussion), whereas Context Jamming models it as a continuous architectural expansion.
- 4. Biological observers (e.g. Levin lab bio-electric networks) undergo continuous physical morphogenesis and structural remodeling, unlike fixed matrix operators.
Falsifiable Research Questions
- Q1: Does expanding an LLM reward-model reservoir at detected S^ϕ plateaus prevent policy collapse during long-horizon RLHF?
- Q2: Can spectral description length on frozen multi-head attention keys serve as an unsupervised signal for context-window compression?
- Q3: Do agentic research pipelines that automatically generate new evaluation metrics exhibit soliton-like self-sustaining discovery loops?
- Q4: Is there a critical ratio between observer feature dimension and data manifold rank where unsupervised representation learning switches from noise-fitting to digit/concept clustering?
10 · Epistemic ledger & Glossary
Clean Separation of Evidence & Field Guide
- Intelligence can be viewed as gradient ascent on learnable novelty.
- Noisy-TV and dark-room are symmetric failures of conflating surprise.
- Frozen random features suffice as a universal bounded observer.
- Future open-ended intelligence requires co-evolving observers.
- LLM reward models & agent evaluation harnesses as expanding reservoirs.
- Co-evolving observer capacity in multi-agent research loops.
- Attention-key spectral description length for context compression.
Compact Glossary
Epiplexity (S^ϕ)
The learnable fraction of prequential description length extracted by a bounded observer ϕ. Isolates reuseable structural programs from irreducible noise.
Prequential Description Length
The total code length required to transmit a sequential data stream using an optimal predictive model family.
Reservoir Computer
A fixed, randomly initialized non-linear feature extractor. Because its weights stay frozen, its effective description length can be computed in closed form.
Spectral Description Length
A closed-form measure of feature subspace complexity derived from the log-determinant of the regularized feature covariance matrix.
Noisy TV Pathology
The failure mode of curiosity agents that get trapped by unlearnable, irreducible random noise (high surprise, zero learnable structure).
Dark Room Pathology
The failure mode of entropy-minimizing agents that freeze in static environments to avoid any novel input (zero surprise, zero growth).
Edge of Chaos
The dynamical boundary between rigid periodic order and chaotic randomness, where complex computation, long-range correlations, and solitons thrive.
Soliton
A localized wave packet or structure that maintains its shape while propagating at constant velocity and surviving collisions with other solitons.
Neural Cellular Automata (NCA)
A differentiable spatial lattice model updated by local neural rules. Optimizing NCA parameters via epiplexity gradient ascent yields emergent solitons.
Linear Readout Ridge Solve
The closed-form L2-regularized linear regression solution W_λ = (Φ^T Φ + λ I)^(-1) Φ^T Y, which prices model capacity without inner training loops.
Bounded Observer
An information-processing agent defined by finite memory, receptive field, and feature dimension, determining what counts as learnable structure vs noise.
Co-Evolving Observer
A proposed extension where the observer’s own capacity and feature mappings expand as the environment or data stream saturates current features.
11 · Source notes & References
Argument-to-Source Mapping
- Pathologies & Decomposition: Abstract, Section 1, Figure 1a (Surprise split into learnable vs residual).
- Prequential DL & Epiplexity: Section 2, Equations 1–4 (L(x) = |M*| + residual; S^ϕ limit definition).
- Reservoir Estimator: Section 3, Equations 5–9, Appendix E (spectral log-det $S^\phi = \frac12 \sum \log_2(1 + \eta s_i^2)$).
- ECA Rule 110 Ranking: Section 4.1, Figure 2, Table 2 (Rule 110 scores highest at $S^\phi = 3.84 \pm 0.12$).
- NCA Solitons: Section 4.1, Figure 3, Appendix G (emergent traveling waves under epiplexity gradient ascent).
- MNIST Unsupervised Encoder: Section 4.2, Figure 4 (digit class cluster emergence at $\lambda=3, \eta=30$).
- RL Intrinsic Curiosity: Section 4.3, Table 1 (9 of 10 continuous and discrete environments improved).
- Frozen Observer Limitations: Discussion Section (reservoir saturation and LLM co-evolution remarks).
Zhang, Y., & Levin, M. (2026). Intelligence from Learnable Novelty. arXiv:2607.18433v1 [cs.LG].