Repository navigation
Type manifold frames: vector writes, covector reads (DiM push, LEACE ablation) - #77
Merged
Merged
Conversation
…vectors The whitened fit took the Fisher directions Σ⁻¹Xᵀa, QR'd them into a Euclidean-orthonormal basis and both read and wrote along it. Σ⁻¹δ is the LDA read covector, not a write: a 2-node push went along Σ⁻¹δ instead of the difference of means (cos 0.26–0.83 on cached SmolLM2, Qwen3.5-4B and Gemma-3-4B fits), the `!` ablation left 18–46% of the concept's Mahalanobis norm in the class-mean gap, and the monitor read with Σ⁻²δ. A LayerSubspace now carries `basis` (write rows, Mahalanobis-orthonormal) and `read` (their Σ⁻¹ images, exactly dual). The whitened fit is PCA in the whitened space mapped back with the right types: rows Xᵀa/√μ, reads aᵀ(Σ⁻¹X)/√μ. A 2-node fit writes δ and reads Σ⁻¹δ (PCA@2 ≡ DiM holds again), a full `!` is LEACE against neutral, and reduced coordinates are whitened coordinates, so the foot solve, σ tube and share are Mahalanobis. The kernel, AffineOp, synthesis (Gram-Schmidt on write/read pairs), the curved-overlap lowering, fold, DLS, node statistics, the monitor's curved reads and topology scoring read through `read` and write through `basis`. Transfer re-orthonormalizes in the target metric (Cholesky), and a non-isometric map rescales a curved fit's scalar tube to the volume-preserving isotropic value instead of refusing. Fitted tensors gain `layer_<L>.read`; MANIFOLD_FIT_POLICY_VERSION 1 → 2, so every fit refits. Browser seam: affine-fisher-fit and affine-ablation-lowering move to -v2 (new `read` field) with a regenerator in the parity module.
…m in the dual metric
A `jlens/` atom (W_U[v] J_l) and an `slens/` atom (K_l ẽ_v) score a residual
by a dot product: they are covectors. Registered as plain profiles under
typed frames, `!jlens/<word>` would pin the Σ⁻¹u reading rather than the
token's own. The resolvers now register the raised vector Σu
(LayerWhitener.apply), the write that moves the reading furthest per unit of
Mahalanobis displacement; folded, it reads with u again, so a push raises the
token's reading and `!` zeroes exactly that reading (LEACE). On real models
Σu is mostly u restricted to the neutral span (cos about 0.15 with u; no
concentration in the rogue channels), so jlens/slens push calibration moves.
The token dictionaries of `decompose` take the same typing: atoms whiten by
Σ^{+1/2} (LayerWhitener.whiten_dual) while targets and neuron writes whiten
by Σ^{-1/2}, so a token's score is its reading of the residual over that
reading's neutral spread, and a raised atom decomposes to its own token. Read
atoms are nearly collinear in that metric, so the pursuit's k×k solve runs in
fp64 and selection stops once the residual is negligible.
Raising needs the whitener while resolving, so opening a steering scope now
makes the neutral baseline resident first (through the composer's unguarded
ensure); a cold session's generate(steering="... jlens/x") used to refuse.
Browser seam: the slens and unit-reads units goldens regenerate (the S atom
profile and `unit find` of a jlens atom).
…ed along 4 → 2
Typed frames write along δ, which moves a concept's reading furthest per
whitened unit and disrupts more per unit, and curved surfaces now pass
through the true centroids rather than their projections onto the covector
span. Both gains were re-fit to keep the documented α bands by matching the
old code's neutral-text NLL cost (teacher-forced):
- affine: six bipolar concepts on SmolLM2-360M, gemma-3-4b, Llama-3.1-8B and
gemma-4-12b at old α ∈ {0.25, 0.5, 1}; equal-cost α was 0.47–0.48× the old
one, stable across α (per model 0.44–0.55), so 16 → 8. At matched cost the
δ write moves pole preference more for most concepts.
- curved: spectral emotions%{happy,angry,calm} on SmolLM2-360M and
gemma-4-12b; equal-cost along 0.35–0.80× (geo-mean ~0.55), so 4 → 2.
- jlens atoms (no per-term gain): on gemma-3-4b the old α ≈ 0.3 sweet spot
maps to α ≈ 0.1; the note says so.
The lowering goldens regenerate (only their gain-dependent fields), with
regenerators for the rank-one and composed lowerings added to the parity
module; the units goldens and the term-layers golden's curved groups follow.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Manifold frames were mistyped. The whitened fit computed the Fisher directions
Σ⁻¹Xᵀa(LDA read covectors), QR'd them into a Euclidean-orthonormal basis, and both read and wrote along it. So:Σ⁻¹δinstead of the difference of meansδ(cosine 0.26–0.83 on the cached SmolLM2, Qwen3.5-4B and Gemma-3-4B fits);!projected Euclidean-orthogonally offΣ⁻¹δ, leaving 18–46% of the concept's Mahalanobis norm in the class-mean gap (LEACE leaves 0);Σ⁻²δ(correlation ≈ 0.8 with the LDA read).Commit 1: typed frames
LayerSubspacecarriesbasis(write vectors, Mahalanobis-orthonormal) andread(theirΣ⁻¹images, exactly dual). The whitened fit is PCA in whitened space mapped back with the right types: rowsXᵀa/√μ, readsaᵀ(Σ⁻¹X)/√μ. A 2-node fit writesδand readsΣ⁻¹δ(PCA@2 ≡ DiM holds again), and a full!is LEACE against neutral. Reduced coordinates are whitened coordinates, so the foot solve, σ tube and share are Mahalanobis. The kernel,AffineOp, synthesis, curved-overlap lowering, fold, DLS, node stats, the monitor's curved reads, topology scoring and transfer all follow the typing.Commit 2: lens atoms are reads
jlens//slens/atoms are covectors. They register their raised vectorΣu, so!jlens/xzeroes the token's own reading. The token dictionaries ofdecomposewhiten byΣ^{+1/2}. Opening a scope now makes the neutral baseline resident before resolving.Breaking
MANIFOLD_FIT_POLICY_VERSION1 → 2: every fitted tensor refits. Tensors gainlayer_<L>.read.affine-fisher-fitandaffine-ablation-loweringmove to-v2(newreadfield). The slens and unit-reads units goldens regenerate. drowse.ai's Fisher fit, affine lowering, ablation, S atoms and archive reader need porting before its pin moves.Calibration
The
δwrite disrupts more per whitened unit, and curved surfaces now pass through the true centroids, so both gains were re-fit. Each new gain was chosen to match the old code's neutral-text NLL cost at the old operating points, keeping the documented α bands. Measured teacher-forced, main (old fits) vs this branch:_SUBSPACE_GAIN16 → 8. Six bipolar concepts on SmolLM2-360M, gemma-3-4b, Llama-3.1-8B and gemma-4-12b at old α ∈ {0.25, 0.5, 1}. The equal-cost α was 0.47–0.48× the old one, stable across α (per model 0.44–0.55). At matched cost theδwrite moves pole preference more for most concepts._MANIFOLD_ALONG_GAIN4 → 2. Spectralemotions%{happy,angry,calm}on SmolLM2-360M and gemma-4-12b. The equal-costalongwas 0.35–0.80×, geometric mean about 0.55.Checks
CPU suite 6031 passed; ruff clean; pyright 0 errors; GPU suite (SmolLM2, MPS) passes; perf gates passed before the gain change (gains don't touch the hot path).