CONTEXT JAMMING

Field notes from inside the context window.

CONTEXT JAMMING / EXPERIMENT 01 / JULY 2026

Filtering Training Data to the Universal Weight Subspace

Neural networks keep arriving at the same narrow geometry. The experiment asks a blunt question: why make the optimizer rediscover the road every time?

Enter the geometryRead Kaushik et al. ↗
EVIDENCE BOUNDARY

One controlled mechanism check. One real digits MLP. No ViT result. No 7B result. The page keeps those facts separate.

UPDATED 2026-07-16Added cost-economics synthesis, ranked scoring avenues, stability protocol, and pre-registered kill criteria (Sections 14–17). Cost memo record 187be4ef….
Training signal entering a low-rank universal subspaceMany noisy gradient arrows enter from the left. Eight rust directions form a narrow shared corridor that carries an aligned update to the right.GRADIENTALIGNMENTFULL TRAINING SIGNALTOP-k SHARED DIRECTIONSΔW ∈ span(Uk)
CONCEPTUAL RECONSTRUCTIONThe corridor is the hypothesis made operational—not a measured loss landscape.
SECTION 01 / 17THESIS / OPERATIONALIZATION

The hidden geometry of training

A billion parameters. A much smaller road.

The parameter count describes the room. It does not tell us where training walks. Kaushik and colleagues stacked more than 1,100 trained models—ViTs, Mistral LoRAs, LLaMA-3-8B models, ResNets—and found sharp, layer-wise spectral decay inside shared architectures.

The original result is about weights. This project turns it into a training intervention: measure whether each example's gradient points into the recovered basis, then filter, reweight, or constrain accordingly.

The room is huge. The traffic keeps choosing the same eight exits.— The thesis, compressed
FRAMINGESTABLISHED THEORY / INTERPRETIVE REFRAME

Why prediction is compression

The information-theoretic frame

A model's cross-entropy loss is a bit-cost. H(P, Q)—the expected number of bits needed to encode outcomes drawn from the true distribution P using a code built from the model's distribution Q—decomposes into the data's irreducible entropy plus the excess cost of using the wrong code:

SHANNON 1948 / 1950CROSS-ENTROPY DECOMPOSITION
H(P,Q) = H(P) + D_{\mathrm{KL}}(P\Vert Q)
Shannon, "A Mathematical Theory of Communication" (1948); Shannon, "Prediction and Entropy of Printed English" (1950)—the roughly one-bit-per-character English prediction experiment that made the point empirically. Training that minimizes cross-entropy loss is training that minimizes a compression cost.

Sanderson's "Reinventing Entropy" series ↗ walks the same identity from the compression side: a good predictor is a good compressor, and a good compressor has found the structure the data actually has.

SECTION 02 / 17PAPER / ALGORITHM 1 + APPENDIX B.1

The universal weight subspace hypothesis

Stack the models. Center the layers. Watch the variance collapse.

PAPER FINDING

For each same-shaped layer, flatten the parameters from many same-architecture models into rows. Subtract the feature-wise mean. Run PCA—the paper's practical order-1 HOSVD case—and retain the smallest rank that crosses a variance threshold.

ORDER-1 HOSVD / PCAKAUSHIK ET AL. · ALG. 1
X_c=X-\mu,\qquad X_c=U\Sigma V^\top,\qquad U_k=V_{1:k}^\top
The page's implementation operates layer by layer and can analyze absolute weights or updates from a common reference.
REAL SMALL-SCALE MEASUREMENTFC2.WEIGHT · TEN SOURCE MODELS

How much geometry survives?

Choose a rank. Rust bars are retained directions; pale bars are the residual. The eighth direction crosses the pre-registered 90% variance threshold.

94.9%cumulative centered variance
PC123.1%
PC216.8%
PC312.4%
PC410.9%
PC510.1%
PC69.4%
PC76.4%
PC85.9%
PC95.1%
Component ratios are read from the downloadable serialized subspace. This is one middle layer of a digits MLP—not an LLM layer.
SECTION 03 / 17PROJECT HYPOTHESIS / NOT A PAPER RESULT

Why pre-filtering matters

The optimizer pays for directions the model may later abandon.

01

Score before spending

Measure how much of each example's gradient energy falls inside the recovered basis.

02

Protect coverage

Rank inside labels or task strata. Raw global filtering can erase a minority class.

03

Constrain what remains

Project updates toward the shared basis—or learn only coefficients over frozen directions.

UNPROVEN AT SCALE“Waste” is the hypothesis under test. A discarded direction may contain the very novelty a new task needs.

SECTION 04 / 17IMPLEMENTED KERNEL / ILLUSTRATIVE DEMO

The gradient alignment scorer

Turn every training example into a geometric question.

PER-EXAMPLE SCOREIMPLEMENTED
s_i=\frac{\lVert U_k^\top g_i\rVert_2^2}{\lVert g_i\rVert_2^2}
g is the selected per-example layer gradient. The score lies between zero and one and is invariant to gradient scale.

A high score says direction, not usefulness. Easy examples, mislabeled examples, and duplicated examples can still score highly. Geometry becomes one feature in the sampling decision—not the whole decision.

ILLUSTRATIVE BATCHCALIBRATED TO THE REAL SCORE RANGE
Y00.0005DROP
Y10.0178KEEP
Y20.0161KEEP
Y30.0115KEEP
Y40.0012KEEP
Y00.0005KEEP
Y10.0158KEEP
Y20.0008DROP
Y30.0023KEEP
Y40.0011KEEP
Y00.0017KEEP
Y10.0066DROP
Y20.0130KEEP
Y30.0185KEEP
Y40.0035KEEP
Y00.0106KEEP
Y10.0151KEEP
Y20.0115KEEP
Y30.0010DROP
Y40.0007DROP
15 / 20 examples retainedmean retained alignment 0.0094label coverage protected

The displayed examples are deterministic teaching inputs, not hidden rows from the digits dataset. Their scale is bounded by the observed real-data minimum and maximum. The class-coverage switch encodes a failure found during the smoke run.

SECTION 05 / 17CONTROLLED ABLATION

Four training regimes

One baseline. Three ways to intervene.

01

Baseline

Uniform data. Full gradients. The control arm.

no intervention
02

Filtered

Keep the highest-alignment examples inside every class.

60% kept per class
03

Importance

Sample all examples, but visit aligned examples more often.

group-normalized sampler
04

Hard constraint

Project most of each update back into the recovered basis.

85% residual removal
SECTION 06 / 17CONTROLLED SYNTHETIC / FIVE SEEDS

Controlled mechanism check

When the geometry is planted, the scorer finds it.

8 / 8rank recoveredknown planted basis
100%selected scoreinformative examples
3.8e−15rejected scoreorthogonal noise
87.5%filtered alignmentbaseline: 33.0%
Selected mean1.0000
Rejected mean3.83e-15

The scorer sees the planted boundary almost perfectly. That proves the instrument can detect geometry deliberately placed in the data.

The control proves that the instrument detects signal deliberately placed in a known subspace. It does not prove that real models contain the same clean boundary.

SECTION 07 / 17REAL SMALL-SCALE / THREE SEEDS

Real-data results on digits

The result was positive in one window—and mixed everywhere else.

92.37%constrained IID @ 5,040baseline: 91.70%
85.93%constrained OOD @ 5,040baseline: 85.26%
+0.74 ppfiltered final OODIID cost: −1.19 pp
64.8%constrained alignmentbaseline: 9.0%
REAL SMALL-SCALE RESULTSMEAN CURVES · THREE DETERMINISTIC SEEDS
Training accuracy against examples seenMean curves compare baseline, filtered, importance-sampled, and subspace-constrained training.4055708510002.5k5k7.5k10kEXAMPLES SEENACCURACY (%)
Curves come directly from the hash-verified experiment record. Toggle the split and methods; no smoothing or interpolation is applied.

Loading verified result record…

REAL SMALL-SCALE RESULTS

The constraint changed direction. It did not clean the spectrum.

Toggle the target-layer update spectrum. The constrained run places far more energy inside the recovered universal basis—but its matrix effective rank rises from 7.77 to 11.54. Those are different measurements.

Baseline—effective rank (participation ratio)—spectral Shannon entropy, bits
Hard constraint—effective rank (participation ratio)—spectral Shannon entropy, bits
NOT MET

Both measures rose under constraint. The intervention did not concentrate the spectrum by either definition of "effective rank."

Normalized target-layer update singular valuesThe selected baseline or constrained mean singular-value spectrum is shown on a logarithmic scale.10−010−110−210−310−410−5SINGULAR-VALUE INDEX
Baseline · normalized to the leading singular value
SECTION 08 / 17EPISTEMIC LEDGER

What improved—and what did not

The geometry moved. The generalization claim did not clear the bar.

POSITIVE

Matched-budget checkpoint

The hard constraint improved both IID and noisy OOD accuracy at 5,040 examples seen.

MIXED

Final filtering tradeoff

Filtered data gained 0.74 OOD points and lost 1.19 IID points. Robustness and fit moved apart.

NOT MET

Cleaner spectra

Effective rank rose under the constraint. More basis alignment did not mean sharper matrix decay.

OPEN

Scale and universality

Ten MLP source models do not establish a ViT, LoRA, or frontier-pretraining result.

SECTION 09 / 17REAL SMALL-SCALE / THIRTY REFITS PER RANK

Leave-one-source-out stability

Seven directions stayed close. The weakest one nearly disappeared.

12.95°rank-4 mean angleacross directions
8.90°rank-8 mean anglelooks reassuring alone
71.17°rank-8 mean maximumweakest direction moves
89.81°rank-8 worst maximumnearly orthogonal
MEASURED NEGATIVE FINDING

The average hid the edge.

For each of three independently generated ten-checkpoint collections, the experiment removed one source, refit centered PCA, and compared principal angles with the full basis. That produced thirty comparisons at each measurable rank.

At rank eight, most directions can remain close enough to pull the mean down while the weakest included direction rotates toward orthogonality. A scorer that uses all eight directions inherits that sensitivity.

Rank nine is NOT IDENTIFIABLE after holdout. Nine centered held-in checkpoints support at most eight independent PCA directions. The record does not pad the basis or change the protocol to manufacture an answer.

SECTION 10 / 17PYTORCH PACKAGE / VERIFIED KERNELS

Implementation

Extract the basis once. Then choose where to intervene.

01

Same-architecture checkpoints

→
02

Layer-wise centered PCA

→
03

Per-example gradient scores

→
04

Filter / reweight / constrain

→
05

Measure accuracy + spectra

MINIMAL EXTRACTION PATH
subspace = extract_layer_subspaces(
    checkpoints,
    parameter_names=["fc2.weight"],
    variance_threshold=0.90,
    max_rank=8,
    reference_state=initial_state,
)

scores = score_examples(model, x, y, subspace)
chosen = select_top_fraction_per_group(scores, y, 0.60)

loss.backward()
project_gradients_(model, subspace, strength=0.85)
HASH-VERIFIED ARTIFACT

Download the recovered digits subspace

PyTorch payload containing the basis, layer mean, shape, variance ratios, source-model count, and threshold metadata.

Download digits_subspace.ptDownload the mini-packageSHA-256 · f507d7bc5ac700088d…
u1
u2
u3
×
=
PAPER METHOD + IMPLEMENTED HOOK

Freeze the directions. Learn the coefficients.

\Delta W_t \approx U_k\alpha_t

The expensive object is the shared basis. A new task changes only the small coefficient vector. The local package implements this parametrization; the 7B run did not execute on this machine.

SECTION 11 / 17CONTEXT JAMMING EXTENSION

TUCG + CAIRN

A narrow basis is not yet a cognitive geometry—or a memory cell.

TUCG

Sweep the Goldilocks zone

Test k = 16, 24, and 32 across layers. Numerical coincidence is not evidence; held-out behavior and subspace stability decide.

Open TUCG →
CAIRN

Record provenance per update

Log example IDs, score, basis hash, coefficient delta, residual energy, and contradiction outcomes. Then ask whether cleaner attachments actually follow.

Open CAIRN →
SECTION 12 / 17FAILURE MODES / UNKNOWN

Limitations and open questions

The missing experiments are the point.

  1. 01

    Discovery cost

    How few source models can recover a stable basis? Ten models cap centered PCA rank at nine.

  2. 02

    Novelty suppression

    Filtering toward yesterday's geometry may delete the example that creates tomorrow's capability.

  3. 03

    Layer transport

    A score in one middle layer may not predict a useful whole-model update.

  4. 04

    Selection bias

    Alignment ranking can collapse class or task coverage unless the sampler protects it explicitly.

  5. 05

    Hardware and carbon

    No calibrated power sensor was available. Wall time was recorded; energy was left null.

  6. 06

    Medium scale

    The 7B entry point is implemented but unexecuted. This 16 GiB host failed the 32 GiB safety floor.

SECTION 13 / 17LITERATURE SYNTHESIS / NO NEW MEASUREMENT

Decision memo

The mechanism is credible. The economics narrow the path.

MEASURED IN CITED PAPERS

The systems pieces exist.

Low-rank optimization, proxy selection, batched gradient features, and selective backward passes have all been demonstrated. None is a UWS alignment result.

ARITHMETIC EXTRAPOLATION

The useful boundary is late.

Reuse the candidate forward pass. Backpropagate only to a late layer. Contract there. Reject before paying for the rest.

UNTESTED PROPOSAL

The scale path is a proxy.

Recover an architecture-specific basis in a 25–100M model, then test whether its example ranking transfers to the target.

SECTION 14 / 17ARITHMETIC EXTRAPOLATION / NOT A MEASUREMENT

The economics of scoring

A scoring method has to beat the backward pass it hopes to avoid.

COMMON COST MODEL2,048-TOKEN SCORING UNIT
F\approx2PT,\qquad B\approx2F,\qquad \text{train}=3F
ARITHMETIC EXTRAPOLATION Dense-transformer, matmul-dominant accounting—not measured wall time.
OFFLINE BREAK-EVENKEEP FRACTION p
cF+3pF<3F\quad\Longrightarrow\quad p<1-\frac{c}{3}
The online gate is more favorable because it reuses the forward pass and can skip most backward work.
0.410100M forward TFLOPsordinary step: 1.23
4.101B forward TFLOPsordinary step: 12.3
28.77B forward TFLOPsordinary step: 86.0
2,048tokens per scoring unitshared assumption
ARITHMETIC EXTRAPOLATIONPER MILLION 2,048-TOKEN SCORING UNITS
Result units
Accounting mode
Target scale
VERDICTSCORING PAYS
BREAK-EVEN KEEP FRACTION98.4%
COMPUTE SAVED20,951 PFLOPs

Online total = cF + 1.9pF; the candidate forward is reused and rejected examples skip most backward work. Matmul-dominant estimate; attention, optimizer, utilization, and packing effects change wall time.

Full-model, exact per-example gradients are economically dead for one-pass pretraining. Random projections solve storage, not the cost of obtaining the backward signal.
SECTION 15 / 17LITERATURE SYNTHESIS + COST ARITHMETIC

Eight ways to score cheaply, ranked

The cheapest gradient is the one you stop computing.

01ARITHMETIC EXTRAPOLATION

Online last-block alignment gate

cF · 100M / 1B / 7B
1.13 / 1.13 / 1.13 F
Max keep at break-even
98.4% · online

Strongest direct economics. Reuse the forward pass, stop backward at a late layer, and skip the rest for rejected examples.

02ARITHMETIC EXTRAPOLATION

Full-gradient proxy scoring

cF · 100M / 1B / 7B
.75 / .30 / .043 F
Max keep at break-even
75% / 90% / 98.6%

Best offline scaling economics. Transfer of this ranking remains unproved.

03UNTESTED PROPOSAL

Online amortized scorer

cF · 100M / 1B / 7B
1.06 / 1.06 / 1.06 F
Max keep at break-even
97% · online

A tiny head can be cheap enough. The unresolved variable is ranking drift.

04ARITHMETIC EXTRAPOLATION

Exact one-last-block score

cF · 100M / 1B / 7B
1.10 / 1.10 / 1.10 F
Max keep at break-even
63% · offline

Viable only if one layer predicts the useful ranking.

05ARITHMETIC EXTRAPOLATION

Structured JL last-block sketch

cF · 100M / 1B / 7B
1.15 / 1.15 / 1.15 F
Max keep at break-even
62% · offline

Storage-efficient. The partial backward still dominates.

06ARITHMETIC EXTRAPOLATION

Fixed 100M embedding encoder

cF · 100M / 1B / 7B
1.0 / .10 / .014 F
Max keep at break-even
67% / 96.7% / 99.5%

Very cheap at scale; no demonstrated bridge from embeddings to alignment.

07ARITHMETIC EXTRAPOLATION

Full target per-example gradients

cF · 100M / 1B / 7B
≥3 / ≥3 / ≥3 F
Max keep at break-even
NONE · OFFLINE

Batching can reduce overhead and memory. It cannot make a target-sized backward free.

—UNTESTED PROPOSAL

Streaming or robust basis extraction

cF · 100M / 1B / 7B
≈0 marginal scoring F
Max keep at break-even
NOT APPLICABLE

Necessary infrastructure. Layer selection still decides whether the basis fits.

ARITHMETIC EXTRAPOLATION

Contract the factors. Never materialize the full gradient.

For a linear layer, the activation–error outer product supplies the gradient. Contract those factors directly against each basis matrix during a partial backward, then skip the remaining backward for rejected examples.

MEASURED IN CITED PAPER LoGra reports up to 6,500× attribution throughput over EKFAC at Llama-3-8B scale—an influence-computation result, not a UWS measurement.

LoGra [M] ↗ and Selective Backprop [M] ↗ provide systems evidence whose trick transfers. Neither paper measured UWS filtering.

LAST-BLOCK CONTRACTIONNOT A UWS MEASUREMENT
G_i=D_iA_i^\top,\qquad \langle G_i,U_j\rangle=\operatorname{tr}(D_iA_i^\top U_j^\top)
For language-model sequences, cross-token terms complicate the denominator. Token scoring, a structured sketch, or an estimator is still required.
1B

A 100M proxy breaks even after dropping only 10%.

7B

A 100M proxy breaks even after dropping roughly 1.4%.

112 GB

All-layer bf16 k=8 bases at 7B. Memory alone forces layer selection.

The decisive scientific question is no longer whether gradient alignment can be computed. It is whether its ranking survives layer restriction, proxy transfer, training drift, and uncertainty in the basis.
SECTION 16 / 17UNTESTED PROPOSAL / STABILITY PROTOCOL

Fixing the 89.81° problem

Treat the basis as a distribution before treating it as a road.

LINKED MEASUREMENT / SECTION 09

The existing leave-one-source-out result stays unchanged: the worst rank-eight angle reached 89.81°. This section is a proposed response to that negative result, not a repair of the record.

DAVIS–KAHAN / CITED THEORY

Centered PCA from N=10 sources caps rank at nine. Rank eight sits one direction below that algebraic ceiling. Rotation is governed by covariance error relative to the eigengap; more sources help only when a real gap exists. Davis–Kahan [M] ↗

CONCEPTUAL RECONSTRUCTIONBOOTSTRAP PROJECTOR AVERAGE
Confidence-weighted universal subspace directionsSeven stable directions remain strongly weighted while a rotating eighth direction is down-weighted.U1U2U3U4U5U6U7U8STABLE DIRECTIONS · WEIGHT NEAR 1ROTATING EDGE · DOWN-WEIGHTED
The diagram explains the proposed mechanism. It is not a fitted confidence spectrum.
CONFIDENCE-WEIGHTED PROJECTORUNTESTED PROPOSAL
C=\mathbb{E}_b[P_b],\qquad s_i^{\mathrm{robust}}=\frac{g_i^\top Cg_i}{\lVert g_i\rVert_2^2}
UNTESTED PROPOSAL Stable directions receive weight near one. A rotating marginal direction is automatically down-weighted.
  1. 01

    Use exact-architecture groups and same-base LoRA deltas.

  2. 02

    Sweep N = 12 / 20 / 32 / 50.

  3. 03

    Report bootstrap + LOSO angle distributions at every N.

  4. 04

    Retain only directions whose 95% bootstrap UCB on LOSO angle is under 30°.

  5. 05

    Stop when the interval crosses 30°, even below 90% cumulative variance.

PROPOSALSUNTESTED PROPOSAL / NOT A RESULT

Two new scores, and a name for a third

SDEF, SSER, and the constrained regime

UNTESTED PROPOSAL

SDEF — Subspace-Differential Entropy Filtering

The gradient-alignment score (Section 4) asks a gradient-space question. SDEF asks the same question in function space: how many more bits does this example cost to predict once the model can only use the shared basis?

REGISTERED PROPOSALSUBSPACE ENTROPY GAP
\Delta H_i = H\!\left(P_i, Q_{\mathrm{UWS}}\right) - H\!\left(P_i, Q_{\mathrm{full}}\right)\ \geq 0
Qfull is the unrestricted model; QUWS is the same model with its weights projected onto the top-k universal basis before the forward pass. ΔH is the excess bit-cost the restriction imposes on example i—never negative, since a restricted model cannot predict better than the full one it was restricted from.

The brief this proposal was drafted from wrote the gap the other way— H(P,Qfull)−H(P,QUWS)—which is always ≤ 0 and cannot produce the "large ΔH ⇒ relies on the noise subspace" behavior it describes. The sign above is the one that matches that behavior.

TypeH(P,Qfull)H(P,QUWS)ΔHAction
Core-structureLowLowSmallKeep
Memorized-nicheLowHighLargeDown-weight / drop
Pure noiseHighHighSmallDrop
Novel / edge-of-competenceModerateModerateModerateSample

Core-structure and pure-noise examples both produce a small ΔH—the subspace restriction barely changes their cost—but for opposite reasons: one is cheap for every model, the other is expensive for every model. ΔH alone cannot tell them apart; H(P,Qfull) does. The two scores are gradient-space and function-space views of the same hypothesis, not competing measurements—where they agree, either is redundant with the other; where they disagree, the disagreement is itself informative about what the basis captures.

ILLUSTRATIVE RECONSTRUCTIONNOT MEASURED · SYNTHETIC EXAMPLES

Sixteen synthetic examples, four per typology bucket above. Each bar pair is H(P,Qfull) and H(P,QUWS) in bits; the gap between them is ΔH. Core-structure and pure-noise examples both track closely, for opposite reasons. Memorized-niche examples diverge sharply.

Core-structure
ΔH 0.03
Memorized-niche
ΔH 2.44
Pure noise
ΔH 0.20
Novel / edge-of-competence
ΔH 1.19
Core-structure
ΔH 0.13
Memorized-niche
ΔH 2.62
Pure noise
ΔH 0.09
Novel / edge-of-competence
ΔH 1.31
Core-structure
ΔH 0.16
Memorized-niche
ΔH 2.92
Pure noise
ΔH 0.20
Novel / edge-of-competence
ΔH 0.81
Core-structure
ΔH 0.06
Memorized-niche
ΔH 3.13
Pure noise
ΔH 0.05
Novel / edge-of-competence
ΔH 1.31
H(P,Qfull) H(P,QUWS), same color at 40% opacity

These are deterministic teaching values, not measurements—no ΔH has been computed from a real model here. They illustrate the typology defined above, not a result.

Cost: SDEF requires a second forward pass with projected weights on top of whatever pass produced Qfull. It is folded into the break-even calculator above as a fourth avenue.

UNTESTED PROPOSAL

SSER — Spectral Shannon Entropy Regularization

Direct response to the cleaner-spectrum criterion marked NOT MET above: the hard-constraint regime raised effective rank instead of lowering it, and raised the entropy of the singular-value distribution too (Section 7 spectral-entropy gauge). Constraint alone did not clean the spectrum. SSER makes concentration a training objective instead of a hoped-for side effect.

REGISTERED PROPOSALSPECTRAL ENTROPY PENALTY
p_i = \frac{\sigma_i}{\sum_j \sigma_j},\qquad \mathcal{L} = \mathcal{L}_{\mathrm{CE}} + \lambda H(p)
σ are a layer's singular values; H(p) is their Shannon entropy in bits. Minimizing the combined loss pushes weight toward lower spectral entropy—fewer, more dominant directions—directly, rather than hoping a filtering or projection side effect produces it.
SECTION 17 / 17ROADMAP / PRE-REGISTERED FAILURE

Next experiments, now with kill criteria

Make the next claim falsifiable before making it large.

01 · CREDIBILITY-MOVING

ViT / LoRA study

Use ≥20 same-base adapters per architecture and k ∈ {4, 8, 16}. Compare last-block with multi-layer gold scores on 5,000 examples. Gate 20 / 40 / 60%; run five seeds; report Spearman ρ, top-20% overlap, IID/OOD, and realized GPU time.

02 · TRANSFER

ViT-Tiny → ViT-S

Recover architecture-specific bases—never transport U across widths. Test rank correlation and top-quantile overlap against loss-, embedding-, target-, and random-selected baselines.

03 · DRIFT

Amortized scorer

Label 2% of each candidate window exactly. Refresh every 250 steps. Track rolling ρ, top-quantile recall, refresh cost, and calibration under distribution shift.

04 · ≥32 GB

7B audit protocol

A 48 GB-class device is safer. Score 50,000 sequences with a 100M proxy, audit 2,000 exactly, then run matched-token random / proxy-aligned / loss-selected / hybrid LoRA arms with scoring FLOPs separate.

05 · FUNCTION-SPACE

SDEF field validation

Compute ΔH alongside the existing gradient-alignment score on the same held-out set. Report Spearman ρ and top-20% overlap between the two rankings before proposing ΔH as a standalone filter.

06 · DIRECT INTERVENTION

SSER λ-sweep

Sweep λ against the digits setup. Report effective rank, spectral entropy, and IID/OOD accuracy at each λ against the unconstrained and hard-constraint baselines already measured.

UNTESTED PROGRAM / PRE-REGISTERED

Kill criteria before downstream accuracy.

Online last-block gate

Kill if ρ < 0.50 against multi-layer gold, top-20% recall < 40%, or a 40% drop saves no measured GPU time.

Proxy scoring

Kill if median proxy-to-target ρ < 0.40, top-20% recall < 35%, or selection misses random's confidence interval at two keep fractions.

Amortized scorer

Kill if rolling ρ < 0.60, correlation falls by more than 0.10 between refreshes, or refresh plus inference consumes over 10% of saved backward FLOPs.

Shared basis itself

Kill any direction whose 95% bootstrap UCB on LOSO angle exceeds 30°, even if it improves in-sample explained variance.

SDEF

Kill if ΔH-ranking correlates with the existing gradient-alignment score at ρ < 0.50, top-20% overlap < 40%, or the second forward pass pushes total scoring cost past the full-gradient baseline (3F) with no measured accuracy gain over the gradient-only score.

SSER

Kill if no λ in the sweep reaches effective rank at or below the unconstrained baseline (7.77) without more than 0.5 percentage points of IID accuracy loss on the digits setup.

REFERENCES ADDED WITH THE COST MEMOARXIV LINKS · VERIFIED 2026-07-16
Read the paper PDF ↗Inspect raw digits JSONInspect stability JSONInspect cost memo JSONRead the agent summaryInspect controlled JSON

MEASURED DATA · 3cc738c29955c0ac… · COST MEMO · 187be4eff9183a0461ea7115e227b5e53db908bf6d2b7937c07caf580b0f9497 · GATE verify_20260715T225801Z_2c4bd74f