neuron-undershoot

Measures whether the typical neuron among the neurons that respond to an object's picture in Gemma 4 12B base falls below its usual level after a chat has mentioned the object and moved on. A plain-text chat transcript names or describes an object ("Think of what a hamster looks like. Just reply OK."), the base model reads "Assistant: OK", and the activity of that object's neurons over the next tokens is compared with the same chat mentioning no object, in units of the spread of the no-object chats. The fall is called the undershoot. Three stages test whether it exists, what lifts it, and how it depends on repeated mentions and on delay. One replication repeats Stage 0 in another model and with neurons chosen another way; another repeats Stage 1 on fresh objects.

Controls, per template: 16 neutral phrases ("the idea of fairness"), 16 placeholders ("something") and 16 object-free messages.

What it does

1. Pick objects and neurons. Catalog image labels (200 objects for Stage 0, 190 further ones that Stage 0 did not use for Stage 2) and, for each, up to 64 neuron poles that respond to its image label most reliably (objects keep at least 32 after neurons shared with other objects are removed).
2. Build prompts. Each object is mentioned by name or by a description that never uses the name (Stage 0: six message templates; Stage 2: a list of note lines), and paired with controls that mention no object.
3. Run the model and record the neurons (the input to every down_proj) at every token, with batch size 1.
4. Score. For each pole, the difference between the mention prompt and the no-object bank, divided by the bank's spread at that token; averaged over the window, then over the object's poles, then over objects. Significance comes from sign-flips over the 16 object categories.

Terms used below. A pole is one sign of a neuron's activation, so each neuron has two; an object's set is its poles, signed so that positive means more of the object. About half of the catalog sets' poles (sign −1) respond to the object by going negative; the localizer's sets are all positive. "Up" always means toward the pole's response to the object. E1c adds each pole's raw elevation, which lowers the raw activation of the sign −1 poles. The no-object bank is the controls' activations at the same token, and the SD is their standard deviation; a score in SD is the gap to the bank's mean divided by that spread. The off-target sets of an object are the sets of eight objects from other categories, scored on the same prompts. The reply onset is the ":" after "Assistant", where the model starts its reply. Tail B is a variant in which the text continues in the same user turn instead of ending the turn. A localizer picks an object's neurons from separate sentences that name it. Holm is the multiple-comparison correction of the p-values within a family of tests.

The stages ask:

• Stage 0, baseline. Is the set below the level after prompts that mention no object? Is that specific to the mentioned object's own set? Does it survive fp32? (stages/e0_baseline)
• Stage 1, relevance. Does a later question that needs the idea lift the set with no new object input (E1m, the retro-cue arm)? Is a late rise ordinary prediction of the name, with or without a copy source (E1r, the riddle)? Does expecting the object back change the undershoot (E1b, the expectation prefix)? Does pushing the set toward its response to the name by hand produce a later dip (E1c, forced activation)? (stages/e1_relevance)
• Stage 2, dose and lag. Does the response to the name shrink with the number of earlier mentions? Does the undershoot deepen with mentions and recover with distance, and does it step when the mention leaves Gemma's local attention window of 1,024 tokens? (stages/e2_dose_lag)
• Replications. Stage 0 in Qwen3-8B-Base and in Gemma with neurons chosen by a text localizer instead of the catalog; Stage 1 on the 190 objects of Stage 2. (replications)

docs/method.md has the full design: the estimator, the controls, the tests (P0.1 and so on) and how their results are read. docs/replications.md has the two replications.

Quick start

Python 3.12 or newer (the pinned numpy needs it). The tests need no GPU and no model weights; one scenario downloads Gemma's tokenizer (32 MB) from the Hub, so the first run needs network access (offline, with the tokenizer not cached, that scenario is skipped). They were run on CPU with Python 3.12, torch 2.14.1 and transformers 5.20.1 (the pin is 5.22.1, the version of the GPU run). The experiments need an NVIDIA GPU: Gemma 4 12B is a 23.9 GB checkpoint in bf16 and about twice that in fp32, so the plan is one 80 GB card. --device cpu works, slowly. Model revisions are pinned in neuron_undershoot/text.py and listed in docs/data.md.

git clone https://github.com/logitsml/neuron-undershoot
cd neuron-undershoot
pip install -e ".[test]"

Run the tests.

python -m pytest        # scorers on synthetic activations, runners on a tiny random model, replication pipelines, input hashes

Print the numbers in the Results section from the shipped score files.

python scripts/summarize_results.py

Run a stage. Each command runs the forward passes (resumable: run it again after an interruption), the integrity check and the scorer. Raw activations go to runs/, scores to results/, overwriting the shipped score files (--results DIR keeps them). Run one stage at a time; Stage 1 starts only if Stage 0's verdict is a concept-specific undershoot or P0.1 and P0.5 hold (--force overrides). Stage 0's instruction-tuned arm needs google/gemma-4-12B-it, which bash gpu/setup.sh --with-it downloads.

bash scripts/run_stage.sh e0 --device cuda
bash scripts/run_stage.sh e1 --device cuda
bash scripts/run_stage.sh e2 --device cuda

The script calls python from the current environment (PYTHON=path/to/python changes it). --limit 8 runs a smoke test of a few prompts and skips the scoring.

Run Stage 2 and the replications in one command on a CUDA GPU with 80 GB (gpu/README.md).

bash gpu/setup.sh
bash gpu/run_all.sh

Run the checks on Stage 0 (after run_stage.sh e0). Each writes results/e0_baseline/<check>.json, replacing the shipped file (--json FILE writes elsewhere).

for c in adversarial estimator_variants rank_recheck most_below name_probability; do
  python stages/e0_baseline/checks/$c.py runs/e0_baseline
done

Rebuild the prompts from the catalog (the shipped files are enough to run the stages). The builders write to runs/rebuilt/.

python scripts/fetch_catalog.py                            # external/image_l12plus.parquet (columns of about 11 GB of layer files), digest-checked
python stages/e0_baseline/build_prompts.py                 # sets.json, prompts.json
python stages/e1_relevance/build_prompts.py
python stages/e1_relevance/build_forced.py
python stages/e2_dose_lag/build_prompts.py
python replications/stage0_localizer/build_prompts.py --model qwen        # tokenizer only
python replications/stage1_fresh/build_prompts.py

Compare the rebuilt files with the shipped ones by sha256 (docs/data.md).

Repository layout

README.md, LICENSE, LICENSE-data, NOTICE, pyproject.toml
neuron_undershoot/        text.py: the prompt texts and builders' helpers, shared by all stages; model revisions
stages/
  e0_baseline/            Stage 0
    data/                 classified, selection, descriptions, objects, sets, prompts (JSON)
    select_objects.py     order the included labels, stratified by category
    merge_descriptions.py leak-check the descriptions, write objects.json
    build_prompts.py      sets and prompts (needs the catalog table)
    run.py                forward passes; also runs Stage 1's prompts (--phase e1)
    check_chunks.py       integrity check of saved chunks; also used for Stage 1
    score.py              tests P0.1, P0.2, P0.4-P0.6 and the verdict
    checks/               adversarial, estimator variants, rank re-check, most-below, name probability
  e1_relevance/           Stage 1: build_prompts, build_forced, run_forced, score, explore, data/
  e2_dose_lag/            Stage 2: build_prompts, note_lines, run, check_chunks, score, data/
replications/
  stage0_localizer/       build_prompts, localize (text localizer), score, data/
  stage1_fresh/           build_prompts, elevations, build_forced, score, store, data/
gpu/                      run_gpu.py (one runner for every format), setup.sh, run_all.sh, README.md
scripts/                  fetch_catalog.py, run_stage.sh, summarize_results.py
tests/                    planted-signal scorer tests, tiny-model runner tests, input hashes, shell-script checks, scenarios/
docs/                     method.md, replications.md, data.md, results.md, related-work.md,
                          figures/ (Mermaid sources, build_charts.py, render.mjs), images/ (rendered light and dark SVGs)
results/                  score files of every stage and replication (JSON)
runs/, external/          raw activations and the catalog table; git-ignored

python docs/figures/build_charts.py rewrites the chart sources from results/; npm install pele@1.13.0 && node docs/figures/render.mjs redraws the figures in docs/images, both from the repository root.

How it works

For one set pole and one token, the score is (mention − control) / SD of the no-object bank, with the SD floored at 5% of the median SD over neurons and the result clipped to ±6; poles are signed so that positive means more of the object. Stage 0 averages the score over the tokens of a window and over the poles (the mean over poles is primary, the median over poles co-primary) and reports the raw gap in activation units next to them, because the three can disagree; the replications make the median primary. The primary window is the seven tail tokens from " Just" to the second " OK", leaving out the reply-onset ":". All prompts of an arm are padded so that the tail starts at the same position, and the bank is built from prompts that mention no object.

Stage 1 has four arms. E1m reads the set at the final reply onset after one of five cues; E1r after a riddle that needs the name, with and without the name earlier in the chat; E1b over the tail after a prefix of four example exchanges; E1c adds a multiple of each pole's elevation to the down_proj input on a no-object sentence and reads the filler after it.

Stage 2's prompts hold one user turn of twelve note lines with 1, 2, 4 or 8 mentions, followed by the closing "Just reply OK". Every line is fixed to 12 tokens, so that the number and form of the mentions never move a token. The lag arm replaces the second turn with a long neutral text, so that the mention and the start token leave the 1,024-token window at known positions. docs/method.md gives the constants and how the results are read.

Relation to prior work

Question	Prior work	Here
Which neurons respond to an object's picture	neuron-lens labels Gemma 4 12B's neurons by image class	The sets are taken from the catalog; this study follows their activity at later tokens, not at the object's words
Is activity below baseline under suppression	Kowalski et al.: below the no-instruction level when the model is told not to think of a concept	No suppression instruction; a plain mention and "Just reply OK"
Repeated words in LLMs	Pace: decay of MLP output norm over 20 repeats at one token in Qwen3-4B, no baseline; Ren and Wang: output priming, base against instruct	Neurons of the object's own set, position-matched first-mention control, off-target sets, tests
Suppression of an already-mentioned token	Copy suppression: lowers the logit at the position where the token is predicted	A dip at non-output tokens; E1r compares with and without a copy source at matched P(name)
Adaptation	Vinken et al.: a hand-installed suppression state in a CNN	Stage 2 asks whether a pretrained LLM shows dose and lag effects without one
Wording of the prompt	vlens decodes the object a model is thinking of after "Think of X"	The closing wording of Stage 0's prompts follows it; no neurons are read there

Details and the remaining works are in docs/related-work.md.

Results

Numbers are standard deviations (SD) of the no-object bank; negative is below the no-object level. Brackets are 95% intervals clustered by object category, except those of E1c and P0.4, which resample objects. Stages 0 and 1 ran in bf16; Stage 2 and the replications ran in fp32 on one H100 80 GB. Every number below is read from, or computed from, the score files; python scripts/summarize_results.py prints them with their file and key, and docs/results.md holds the same tables.

Stage 0

Across 200 objects, the typical neuron (median over poles) of an object's set sits 0.127 SD below its level in chats that mention no object after "Just reply OK": −0.127 [−0.138, −0.116] (P0.1_median; 90% of objects below 0; exact sign-flip p < 0.001 over 16 categories). The mean over poles is −0.121 [−0.141, −0.099]. The largest difference between two of the three kinds of no-object control is 0.027 SD (P0.6, passed: each 90% interval lies inside ±0.1), so the no-object level does not depend on which control is used.

• The object's own set is lower than other sets. After the object's own mention the set is lower than after eight other-category objects' mentions (−0.096 [−0.116, −0.071], P0.2), and lower than eight off-target sets on the same prompts (−0.090 [−0.107, −0.069], P0.5; the off-target sets themselves are at −0.032).
• Total activity is not lower. The raw gap in activation units is +0.020 of the peak response (P0.1_raw_share_of_peak), so the undershoot is a statement about the typical neuron and not about the set's total.
• Layer profile. At the primary window layers 12-23 give −0.265, layers 24-35 −0.111 (185 objects have a pole there) and layers 36-47 −0.023 [−0.055, +0.009]. At the reply-onset token the late layers rise instead (+0.407 in layers 36-47).
• Wording, templates and precision. The undershoot appears for named (−0.106) and described (−0.137) objects and in all six templates (−0.070 to −0.155). fp32 and bf16 differ by +0.0008 [−0.0004, +0.0021] on identical prompts (25 objects).
• Context. The instruction-tuned model reading the raw text gives −0.052 (the base model on the same items −0.104); with its chat template the set stays above its no-object level (+0.333).
• The name. Log P of the object's own name rises by 6.0 nats at the window (logP_own_name_delta) while the neurons dip.
• No turn change. With the text continuing in the same user turn (tail B, 100 objects) Stage 0 gives +0.046 [+0.004, +0.082], no dip. Stage 2's tail B and the Gemma replication below do show one (−0.137 and −0.096).

Checks on the estimator (stages/e0_baseline/checks/).

• The median over poles is stable, the mean is not. Under the primary estimator and six variants of it the median is below zero in every one (−0.040 to −0.174, 82% to 90% of objects), and the own set stays below the off-target sets (−0.029 to −0.124). The mean over poles ranges from −0.147 to +0.288 and is positive without clipping. Averaging the window before standardising gives −0.012 or −0.037, with the own set no lower than the other objects of its category (own −0.037, same category −0.119); resampling the no-object prompts as well as the objects gives an interval of [−0.098, +0.011], and 12 of 16 categories are below 0. Stage 0's headline therefore rests on the median.
• About half of an object's neurons are below the no-object mean. The median object has 54% of its neurons below it on most chats, and 66% of objects have most of theirs below. A no-object prompt scored as a mention already puts 66% of objects "below zero" by −0.003 on the rank-percentile scale (the typical pole's percentile within the no-object bank, minus ½), against −0.074 for the own set, so counts of objects below zero need that reference.
• The name becomes much more probable and stays improbable. The median chance of the object's own name at the window rises from 1 in 50 million (2.0e-08, no-object prompts) to 1 in 67,000 (1.5e-05), and is higher in 99.2% of prompts; no window position has a median chance above 0.0017 (1 in 600).

Each bar is the median over poles at the primary window after "Just reply OK", for one choice of objects and neuron sets. A row gives the study or model, where the neuron sets come from (catalog or text localizer) and the number of objects. Stage 0 and Stage 2 are Gemma 4 12B base, Stage 2 on 190 objects that Stage 0 did not use; the two lower bars are the Stage 0 replication with localizer-chosen sets, in Gemma 4 12B base and in Qwen3-8B-Base.

Stage 1

E1m and E1r use 100 objects each, E1b 80 and E1c 52. Stage 1 values are means over poles; the replication table below uses the median.

• A colour question lifts the set, mostly whatever it asks. Averaged over layers 12-35, a question about the object's colour lifts the set at the reply onset by +0.598 [+0.460, +0.741] compared with "just reply OK", and by +0.120 [+0.062, +0.193] compared with an unrelated colour question. The difference from the unrelated question is −0.031 in layers 12-23, +0.487 [+0.311, +0.710] in 24-35 and +0.640 [+0.560, +0.743] in 36-47. After "just reply OK" the set at the reply onset is −0.323 [−0.393, −0.256] below the no-object histories.
• Expecting a re-mention raises the set. Four example exchanges before the test item either ask for each mentioned object again by name or do not. With them asking, the set stands +0.141 [+0.070, +0.205] higher over all layers, mostly in layers 36-47 (+0.476 above the paired placeholder against +0.270 without the expectation). In layers 12-35 the set is below the placeholder either way (−0.126 against −0.189).
• No sign of an output-side copy-suppression pattern in layers 12-35. At matched log P(name) the set is higher with the name already in context (intercept +0.151 [+0.070, +0.256]); layers 36-47 are unresolved (−0.398 [−0.815, +0.066]).
• Forcing the set by hand is followed by a small, dose-dependent later dip. Adding the own set's elevation (its activation on the object's name) to a no-object sentence leaves the set −0.017, −0.034 and −0.071 below its no-injection level on the filler that follows, at 0.5×, 1× and 2× (1×: [−0.045, −0.025]; none of 200,000 random sign patterns was as extreme). Another family's set at 1× and 2× gives +0.000 and −0.001. The positive control, log P of the own name, rises by +0.405 nats [+0.287, +0.542] at 1×.

In each group, left to right (blue, then gold): colour cue minus "just reply OK"; colour cue minus the unrelated colour cue (mean over poles, per layer band). Layers 24-35 leave out the one object with no pole there (n = 99).

Stage 2

The undershoot after one mention in the notes format has the sign and size of Stage 0's: −0.126 [−0.147, −0.100] (median over poles −0.095 [−0.118, −0.069]; own minus off-target −0.110 [−0.132, −0.082]). By layer third it is −0.179, −0.046 and −0.126.

• The response to a repeated name shrinks. Per doubling of the number of mentions the object's set responds 0.701 SD less (slope −0.701 [−0.835, −0.581], Holm p = 0.00009; own minus off-target −0.694). Against the no-object bank the response is 2.90 at the first mention and 1.67 at the eighth, 43% lower. The slope is −1.229 in layers 12-23, −0.824 in 24-35 and −0.349 in 36-47, and −0.667, −0.696 and −0.741 for the three filler types.
• The fall does not need identical wording, and descriptions do not produce it. Repeating the name in varied frames gives a response at the eighth mention 1.323 below the first (1.906 below with identical lines). Three descriptions without the name raise the later response to the name by 2.844 instead.
• The undershoot does not deepen with repeats. Its slope is +0.0077 per doubling [+0.0040, +0.0116], which differs from 0 (Holm p = 0.0035) but lies inside ±0.05, so it is equivalent to flat: −0.126, −0.113, −0.106 and −0.102 after 1, 2, 4 and 8 mentions. It is not specific to the object (own minus off-target +0.0042).
• It stays below the no-object level over 859 tokens, a little shallower. 500-859 tokens after the mention it is −0.080 [−0.105, −0.053] (72% of objects below 0; own minus off-target −0.098). The slope over log2 of the delay, for the mean of one and eight mentions, is +0.024 [+0.009, +0.036], inside ±0.05; for eight mentions alone it is +0.040 [+0.024, +0.055], which is not inside.
• Part of the dip lifts when the mention leaves the local attention window. The step is read from the 64 tokens just before the exit, which are a trough. For one mention they sit at −0.157, against about −0.09 earlier in the view, and the level 20 to 84 tokens after the exit is −0.083, about the level earlier in the view. For eight mentions the levels are −0.093 before the first mention leaves and −0.008 after the last one has. The step is +0.074 [+0.042, +0.104] (one mention) and +0.085 [+0.056, +0.111] (eight), Holm p = 0.005. When only the start token passes, it does not rise (+0.007 [−0.021, +0.037] for one mention; +0.025 [−0.028, +0.079] for eight). With the mention placed later in the prompt the steps are +0.065 (Holm p = 0.004) and +0.063 (Holm p = 0.014).
• It appears without a turn change. With the text continuing in the same turn the undershoot is −0.137 after one mention and −0.108 after eight, about the same as after the turn change (difference +0.011 [−0.004, +0.027]).

The scorer's verdict is "mixed": the response falls, the undershoot by dose rises slightly and by delay is flat, so the undershoot is not called habituation-like.

The first chart is the within-object difference from the first mention, clipped per pole, so it is larger than the difference of the two absolute responses above (2.90 and 1.67); the second is the undershoot after the last mention. The third averages the one-mention and eight-mention prompts of the lag arm in 64-token windows; one mention leaves the local window at token 1,182 (eight mentions between about 1,098 and 1,182) and the start token at token 1,024. The windows centred at 1,096 and 1,160 are the trough.

Replications

Stage 0 on localizer sets. Both models pass the two primary tests (R0.1, own set below the no-object level; R0.2, own minus off-target sets).

	Qwen3-8B-Base	Gemma 4 12B
Objects kept of 200	188	197
R0.1, own set	−0.030 [−0.040, −0.019]	−0.114 [−0.129, −0.097]
R0.2, own minus off-target sets	−0.033 [−0.043, −0.022]	−0.090 [−0.107, −0.071]
Window averaged first, mean over poles	+0.076	−0.130
No turn change (tail B)	+0.015 [−0.010, +0.035]	−0.096 [−0.122, −0.071]

• Gemma with sets that never touch the catalog shows the undershoot in every layer third and in 16 of 16 categories. Its size is close to Stage 0's (median −0.114 against −0.127). With Stage 0's own scorer the layer thirds are −0.193, −0.088 and −0.084, against −0.265, −0.111 and −0.023 for the catalog sets: weaker in layers 12-23 and 24-35, and present in layers 36-47, where the catalog sets show none. The undershoot does not need the catalog's neuron sets; the objects and prompts are still Stage 0's.
• Qwen3-8B-Base shows it weakly. The effect is about a quarter of Gemma's. The mean over poles of the window-averaged score is positive (+0.076) while the median over poles is −0.031, so a minority of poles rises strongly. The effect is a small typical-neuron dip.

Stage 1 on 190 fresh objects. Every test meets its criterion (rejection at its family's level after Holm correction, or equivalence for the partner arms); R4 reports a sign, which is positive.

Test	Value	Holm p
R1, colour cue minus the unrelated colour cue, layers 24-35 / 36-47	+0.593 [+0.401, +0.808] / +1.063 [+0.834, +1.314]	0.0003 / 0.00005
R2, "just reply OK" against the no-object histories, layers 24-35	−0.344 [−0.464, −0.209]	0.0005
R3, expecting a re-mention minus not, layers 12-23 / 24-35	+0.044 [+0.011, +0.075] / +0.101 [−0.007, +0.221]	0.032 / 0.049
R4, intercept at matched log P(name), layers 12-35	+0.248 [+0.102, +0.404] (positive)	0.0012
R5a, later undershoot after the own set is pushed toward its response to the name 1×	−0.025 [−0.035, −0.017] typical neuron; −0.069 [−0.088, −0.050] average neuron	0.00006
R5b, slope over dose	−0.029 [−0.037, −0.021]	0.00003

R3 in layers 24-35 passes narrowly: its Holm p is 0.049 and its two-sided interval includes 0. Another family's set at 1× and 2× is equivalent to 0 within ±0.02 (R5c), and the positive control rises by +1.524 nats [+1.176, +1.909] (R5d).

Limits

• One main model. Stages 0 to 2 use Gemma 4 12B base and one catalog's image labels. Qwen3-8B-Base appears only in the Stage 0 replication, where the effect is about a quarter of Gemma's and its mean over poles changes sign when the window is averaged before standardising. The result is for the base model reading plain text. Chat-tuned Gemma 4 12B reading the same text dips half as much (−0.052 against −0.104 for base on the same items), and in its chat format the set sits above its no-object level (+0.333, 50 objects).
• Small effect, many analysis choices. The undershoot is about a tenth of the no-object spread, and the estimator's choices move the mean over poles (see the checks above). The median over poles is the stable quantity. The hamster in the design figure is a favourable example: 65% of its 63 poles are below the no-object mean, against 54% for the median object.
• The baseline is defined by three kinds of control (neutral phrases, placeholders, object-free messages). P0.6 checks that they agree; if they do not, there is no "below baseline" claim.
• Hand-written templates and LLM-written descriptions, with an automated check that a description does not contain the object's name or another pool object's name.
• Stage 2 differs in format. Its note lists are not Stage 0's single sentence, one prompt per object and condition makes per-object estimates noisy, its off-target sets are not matched on layer profile or pole count, its no-object bank has a filler sentence where the mention frame would be (so the undershoot after one mention mixes frame and concept, and the own-minus-off-target contrast removes the frame part), and the lag arm has 60 objects and one filler type.
• Stage 0's no-turn-change control differs from the later runs. It gives +0.046 where Stage 2's tail B and the Gemma replication give a dip.
• Stage 1's cue and riddle results are single-position readouts in one model, and the forced activation adds content to the residual stream, not only the set's activity.
• No causal test. Nothing here ablates attention or clamps the set, so the accounts of the dip, including the attention-window step, are separated by the patterns they imply and not by intervention.

Data, licenses and credits

The code, documentation and figures are MIT-licensed (LICENSE), © 2026 logitsml. The data files in stages/*/data/, replications/*/data/ and results/ are derived from, or built with the objects of, the neuron-lens catalog (dataset, CC BY 4.0) and carry that license (LICENSE-data, CC BY 4.0). The labels of the catalog's media come from source collections with their own terms (NOTICE). The models, Gemma 4 12B and Qwen3-8B-Base, are Apache-2.0 and are not redistributed. The object descriptions were written by an LLM. The closing wording of the prompts follows vlens. The figures are drawn with pele. Related work is credited in docs/related-work.md.