Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,11 @@ few massive-activation channels, so a fallback would be a wrong answer, not a
degraded one. The deliberate Euclidean exceptions are listed in
`drowse/geometry/AGENTS.md`. Neutral activations are cached fp32: fp16
overflows late layers, and `covers_all` must mean finite factors everywhere.
Frames are typed: a `LayerSubspace`'s `basis` rows are write vectors
(Mahalanobis-orthonormal) and its `read` rows the dual covectors (`Σ⁻¹` of
them), so a 2-node push is DiM, a full `!` is LEACE and the monitor reads the
LDA score. Never write along a read row or read with a write row
(ARCHITECTURE §5.3).

**Exclusive GPU.** One reentrant `_gen_lock` serializes all model work, because
MPS's single global command buffer is not thread-safe. Collaborators that run
Expand Down
74 changes: 49 additions & 25 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,10 +86,10 @@ A manifold is a set of labeled nodes, a domain or discovered layout, and a
per-model fit at selected residual-stream layers. A fitted layer is a
`LayerSubspace` (`geometry/subspace.py`) in one of two geometries:

- **Flat**: a neutral-anchored affine frame (`mean`, orthonormal `basis`, real
per-layer node coordinates). A rank-1, 2-node flat fit is the ordinary
bipolar steering vector; a 1-node flat fit is a monopolar ray from the
neutral mean.
- **Flat**: a neutral-anchored affine frame (`mean`, a Mahalanobis-orthonormal
`basis` of write rows with their dual `read` rows, real per-layer node
coordinates). A rank-1, 2-node flat fit is the ordinary bipolar steering
vector; a 1-node flat fit is a monopolar ray from the neutral mean.
- **Curved**: the same enclosing frame plus an `r³` polyharmonic RBF surface
through the node centroids, a neutral-origin foot, and optionally a learned
tube thickness σ(z).
Expand Down Expand Up @@ -299,9 +299,26 @@ the neutral mean into the span, node coordinates are measured from it, and
coordinate 0 means neutral. The basis comes from the node-centroid scatter
around the nodes' own mean, never around neutral: anchor-centering would inject
the neutral offset as a spurious axis and break PCA@2 ≡ difference-of-means.
With a whitener the basis maximizes `vᵀS_b v / vᵀΣv` (the Fisher/LDA
objective), which divides each direction by its background variance so the
rogue channels cancel the way they cancel in a difference of means.

**Frames are typed: writes are vectors, reads are covectors.** A frame's
`basis` rows are what steering adds to the residual stream, and its `read`
rows are what measures a coordinate (`(h − mean)·readᵀ`). The two are
different objects whenever the metric is not Euclidean: the read that best
separates two classes is the LDA covector `Σ⁻¹δ`, while the write that moves
that reading furthest per unit of Mahalanobis displacement is `Σ·Σ⁻¹δ = δ`,
the difference of means itself. So the whitened fit is PCA in the whitened
space mapped back with the right types. The eigenvectors `a` of the whitened
Gram `G = XΣ⁻¹Xᵀ` rank the axes by whitened between-node variance (the
Fisher/LDA objective `vᵀS_b v / vᵀΣv`, which is what keeps the rogue channels
from choosing the axes). The write rows are `Xᵀa/√μ`, combinations of the node
displacements, Mahalanobis-orthonormal by construction, and the read rows are
their `Σ⁻¹` images, exactly dual (`read·basisᵀ = I`). A 2-node fit therefore
writes along `δ` (its push is DiM), reads with `Σ⁻¹δ`, and its full ablation
is LEACE. Every frame obeys the same contract: fitted, folded, synthesized
and transferred. Reduced coordinates are whitened coordinates, so every
reduced norm (the foot solve, the σ tube, the share) is a Mahalanobis norm.
Before fit policy 2 the basis was the QR of the covector rows `Σ⁻¹Xᵀa`, so
steering wrote along LDA directions and the monitor read with `Σ⁻²δ`.

### 5.4 Discovering and fitting geometry

Expand All @@ -323,8 +340,8 @@ curved fits learn the σ tube from the retained rows.
Each fit bakes a per-layer Mahalanobis `share`, the cross-layer budget the
hooks normalize at apply time. Discriminative layer selection (Selective
Steering, Dang & Ngo 2026) keeps a flat axis at a layer only when the node
projections relative to neutral straddle zero: same-side layers encode
intensity, not polarity. `--no-dls` opts out. Curved fits have no polarity and
readings relative to neutral (through the axis's read row) straddle zero:
same-side layers encode intensity, not polarity. `--no-dls` opts out. Curved fits have no polarity and
skip DLS.

A monopolar concept (`extract(concept)` with no baseline) is a genuine 1-node
Expand All @@ -345,12 +362,12 @@ in `drowse/steering/AGENTS.md`. In brief:
|---|---|
| `0.3 warm` | push toward a pole or label |
| `a~b`, `a\|b` | keep / remove the Mahalanobis component of `a` shared with `b` |
| `!a` | mean-ablate `a` (`h' = h − α(h·d̂ − μ·d̂)d̂`) |
| `!a` | LEACE-ablate `a` toward neutral (`h' = h − α((h − μ)·Σ⁻¹d / d·Σ⁻¹d)·d`) |
| `along[,onto] M%label`, `M%x,y` | move toward a node or coordinates of a manifold |
| `x{12-20}`, `x{4-8,20}`, `x{workspace}` | act only at these layers (any term kind) |
| `@response`, `@after`, `@first:N`, … | restrict the token phase or window |
| `@when:probe>t` | fire only while a read-side scalar holds |
| `jlens/<word>`, `sae/<id>` | instrument atoms in reserved namespaces |
| `jlens/<word>`, `sae/<id>` | instrument atoms in reserved namespaces (a lens atom is a read `u`, steered along its raised vector `Σu`) |
| `neuron/L8n344`, `!head/L20h17` | model units: a neuron push along its write (a one-layer profile through the one kernel), a unit mean-ablation (`UnitAblationTerm` → a `UnitOp` pre-hook replacing the unit's activation by its neutral mean) |

### 6.2 Resolution and composition
Expand Down Expand Up @@ -379,8 +396,10 @@ collapse axes, and affine `%` terms contribute their fitted basis and target.
All affine terms of one trigger become one `SynthesizedSubspace` per layer
(`geometry/inject.py::synthesize_subspace`), so terms sharing a layer
superpose instead of colliding; partial, repeated, negative and non-orthogonal
ablations get exact per-axis coefficients. Each curved `%` term stays its own
term. Two curved spans may share a layer only when they are orthogonal
ablations get exact per-axis coefficients. All of it runs in the Mahalanobis
metric: one Woodbury `Σ⁻¹` gives each row's read, and Gram-Schmidt works on
(write, read) pairs. Each curved `%` term stays its own term. Two curved spans
may share a layer only when they are Mahalanobis-orthogonal
(`OverlappingManifoldError` otherwise), and the merged affine span is
orthogonalized against them, the curved term owning the overlap.

Expand All @@ -407,9 +426,11 @@ in-subspace residual, off-subspace residual) and applies two operations:
`onto = 1` lands on the surface's typical set rather than its zero-thickness
wire. It is vacuous for flat terms.

`H_o`, the off-subspace residual, is always kept verbatim. It contains every
composing neighbor's span, which is why a vector and several orthogonal
manifolds compose with zero cross-talk.
The split reads with the frame's `read` rows and writes back along its
`basis` rows, so `H_o`, the off-subspace residual, is the
Mahalanobis-orthogonal complement `(I − basisᵀ·read)(h − mean)`. It is always
kept verbatim. It contains every composing neighbor's span, which is why a
vector and several orthogonal manifolds compose with zero cross-talk.

On a flat subspace the foot is the projected coordinate and the kernel reduces
to an analytic slide, with no foot solve, RBF or norm cap (a flat fit cannot
Expand Down Expand Up @@ -838,9 +859,9 @@ references are:
|---|---|---|
| expression grammar | `steering/grammar.py` | `tests/fixtures/browser_steering_expression.json` |
| sampler | `generation/decode.py` | `sampling-parity-v1.json` |
| affine lowering and the kernel | `steering/hooks.py`, `geometry/inject.py` | `affine-*-lowering-v1.json`, `post-block-rank-one-v2.json` |
| affine lowering and the kernel | `steering/hooks.py`, `geometry/inject.py` | `affine-rank-one-lowering-v1.json`, `affine-composed-lowering-v1.json`, `affine-ablation-lowering-v2.json`, `post-block-rank-one-v2.json` |
| curved kernel, SAE and lens token directions, gates | `geometry/inject.py`, `sae/backend.py`, `lens/jlens.py`, `steering/triggers.py` | `structured-program-parity-v1.json` |
| Fisher fit, DLS, topology and RBF | `geometry/subspace.py`, `fit/capture.py::compute_dls_axes`, `geometry/topology.py`, `geometry/rbf.py` | `affine-fisher-fit-v1.json`, `dls-axes-v1.json`, `topology-rbf-parity-v1.json`, `topology-adversarial-v1.csv` |
| Fisher fit, DLS, topology and RBF | `geometry/subspace.py`, `fit/capture.py::compute_dls_axes`, `geometry/topology.py`, `geometry/rbf.py` | `affine-fisher-fit-v2.json`, `dls-axes-v1.json`, `topology-rbf-parity-v1.json`, `topology-adversarial-v1.csv` |
| measurement envelope | `readout/measurements.py` | `measurement-envelope-v2.json` |
| loom ops and events | `loom/tree.py`, `loom/ops.py` | `loom-operation-transcript-v1.json` |
| generation-settings composition | `loom/recipe.py::compose_call` | `recipe-compose-v1.json` |
Expand Down Expand Up @@ -910,15 +931,18 @@ publisher identity.

Cross-model transfer (`geometry/procrustes.py`, `geometry/transfer.py`) fits a
per-layer affine alignment from shared neutral activations, maps means as
points and bases as directions, re-bakes the share in the target model's
Mahalanobis metric, and writes the `from-<source>` variant. Curved transfer
rejects maps that would make a scalar tube anisotropic. Discover layouts are
points and write rows as directions, re-orthonormalizes the frame in the
target model's Mahalanobis metric (recomputing its reads there), re-bakes the
share in that metric, and writes the `from-<source>` variant. A curved fit's
scalar tube becomes the volume-preserving isotropic approximation when the map
is not an isometry between the two metrics. Discover layouts are
model-specific, so transfer maps fitted activation geometry, not a universal
coordinate system.

A fitted 2-node flat manifold exports as a llama.cpp control-vector GGUF
(`folded_directions = δ̂_L · share_L`). llama.cpp's uniform scale reproduces
that share profile, which approximates but does not equal drowse's live affine
(`folded_directions`: each layer's difference-of-means write row scaled to
Mahalanobis norm `share_L`). llama.cpp's uniform scale reproduces that share
profile, which approximates but does not equal drowse's live affine
injection. Multi-rank and curved fits are rejected.

## 13. Concurrency and consistency
Expand Down Expand Up @@ -1074,8 +1098,8 @@ executable statement of why a boundary is strict.
- **Per-target coherence.** Whitened-unit targets unify push *strength* across
targets, but not *coherence*: a hard persona such as `hacker` shatters at
roughly half the effective gain a robust concept tolerates (about 2×).
`_SUBSPACE_GAIN = 16.0` is the coherence-first compromise and is tagged a
prototype; tune α down per target.
`_SUBSPACE_GAIN` (8.0 since fit policy 2) is the coherence-first
compromise; tune α down per target.
- **Low-α drift.** A steering trajectory can pass through neighboring attractor
basins at low displacement (`personas%hacker` surfacing a cyber-security
cluster before locking into the persona). That is signal about the model's
Expand Down
19 changes: 13 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,8 +109,10 @@ assigns steering to roles; raw mode is one buffer for base models.

In Drowse, you extract concepts as **manifolds** or **subspaces**.

- A 1D flat subspace is just a steering vector.
- A higher-rank flat subspace is a group of orthogonal steering vectors.
- A 1D flat subspace is just a steering vector: the difference of the two
poles' means.
- A higher-rank flat subspace is a group of steering vectors, orthogonal in
the model's Mahalanobis metric.
- A curved manifold fits a nonlinear surface and lets you steer along it.

### Bundled concepts
Expand Down Expand Up @@ -162,14 +164,14 @@ across Python, YAML, OpenAI, Ollama, or the native API.
| leading number | Steering coefficient; omitted terms default to `0.5` |
| `~` | Keep the component shared with another direction (Mahalanobis) |
| `\|` | Remove the component shared with another direction (Mahalanobis) |
| `!` | Mean-ablate a direction or a unit; a bare `!x` ablates fully |
| `!` | Ablate a direction toward neutral (LEACE: reads with the whitened covector, edits along the direction) or mean-ablate a unit; a bare `!x` ablates fully |
| `%label` or `%x,y,…` | Choose a named node or coordinates on a manifold |
| `{12-20}`, `{4-8,20}`, `{workspace}` | Act only at these layers (after the selector, before `@`) |
| `@prompt`, `@response`, `@after`, `@thinking`, … | Restrict the token phase where a term applies |
| `@first:N`, `@after:N` | Restrict a term to a counted decode window |
| `@when:<probe><op><value>` | Apply a term only while a live probe gate is true |
| `jlens/<word>`, `jlens/#<id>`, `sae/<id>` | Steer toward saying a token (its R-lens direction) or along an SAE decoder row |
| `slens/<word>`, `slens/#<id>` | Steer toward having read a token: its S-lens footprint at each layer |
| `jlens/<word>`, `jlens/#<id>`, `sae/<id>` | Steer toward saying a token (the whitener-raised vector of its R-lens read) or along an SAE decoder row |
| `slens/<word>`, `slens/#<id>` | Steer toward having read a token: its S-lens footprint at each layer, raised to a vector |
| `neuron/L8n344`, `head/L20h17` | A unit: push along a neuron's write, or mean-ablate a neuron or head (`!head/L20h17`) |

A bare pole or node label such as `formal` or `pirate` resolves to its concept
Expand Down Expand Up @@ -204,7 +206,12 @@ Drowse first has the model answer a shared set of baseline prompts as each conce
then it takes the resulting hidden states and fits them to either a curved manifold
or a flat subspace.

Layer allocation uses a Mahalanobis metric estimated from neutral activations.
Fitting, layer allocation and reads use a Mahalanobis metric estimated from
neutral activations. Each fitted frame keeps two row sets: write rows, the
directions steering adds (for a two-pole concept, the difference of means),
and read rows, the whitened covectors that measure a coordinate (the LDA
direction). A full `!` ablation reads with one and edits along the other,
which is LEACE.
Discriminative layer selection removes flat axes that fail to straddle the neutral
baseline across the fitted nodes.

Expand Down
8 changes: 5 additions & 3 deletions drowse/fit/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ must stay deterministic.
| Constant | Bump invalidates |
|---|---|
| `MANIFOLD_CAPTURE_VERSION = 3` (`pipeline.py`) | all fitted tensors and all cached captures |
| `MANIFOLD_FIT_POLICY_VERSION = 1` (`geometry/manifold.py`) | fitted tensors only |
| `MANIFOLD_FIT_POLICY_VERSION = 2` (`geometry/manifold.py`; 2 typed the frames: vector write rows, dual read rows) | fitted tensors only |
| `CAPTURE_CACHE_FORMAT = "5"` (`capture_cache.py`) | capture shards |
| `NEUTRAL_CAPTURE_VERSION = 2` (`store/neutral.py`; 2 added the mid means) | neutral caches |

Expand Down Expand Up @@ -180,7 +180,8 @@ identity, but never uses them.

**Flat**: `fit_affine_subspace` with `n_components` = the layout width (a flat
subspace *is* its layout span, so `max_subspace_dim` does not apply) and
`orient_to=0` (node 0 positive, so a 2-node fit is `+δ̂`: PCA@2 ≡ DiM). DLS
`orient_to=0` (node 0 positive, so a 2-node fit writes `+δ` and reads
`Σ⁻¹δ`: PCA@2 ≡ DiM). DLS
then prunes axes and `subspace_share` is baked on the survivors. The display
layout is translated onto neutral (`_neutral_centered_layout`) so `% 0,…,0`
reads as neutral; steering is unchanged. No origin is stored (neutral's foot
Expand All @@ -194,7 +195,8 @@ the σ-field, and each layer stores `origin`, the neutral foot that seeds the
solve. There is no DLS, since a curved fit has no polarity.

**DLS** (`compute_dls_axes`; Dang & Ngo 2026, Selective Steering, Eq. 9) keeps
axis `d̂_r` at layer L iff `{(c_i − ν_L)·d̂_r}` straddles zero. Same-side axes
axis `r` at layer L iff the node readings `{(c_i − ν_L)·p_r}` through its read
row `p_r` (`LayerSubspace.read`) straddle zero. Same-side axes
encode intensity, not polarity, and at K=2 this is exactly the bipolar
opposite-sign test. Edge cases: `layer_means=None` (`dls=False`) keeps every
axis; a zero-norm axis drops; a layer with no ν keeps every checkable axis; if
Expand Down
13 changes: 7 additions & 6 deletions drowse/fit/capture.py
Original file line number Diff line number Diff line change
Expand Up @@ -655,11 +655,12 @@ def compute_dls_axes(

The N-node generalization of the bipolar opposite-sign test (Dang & Ngo
2026, Eq. 9) — the form a flat *subspace* of any node count uses. Each
layer carries a basis ``[R, D]`` (its ``LayerSubspace.basis`` rows) and
``node_centroids[L]`` is the ``[K, D]`` stack of that layer's per-node
mean activations. Every row ``d̂_r`` is kept iff the node projections
relative to the neutral baseline, ``{(c_i − ν)·d̂_r : i}``, **straddle
zero** (both a negative and a positive present) — the axis then
layer carries its read rows ``[R, D]`` (``LayerSubspace.read``, the
covectors that read an axis's coordinate) and ``node_centroids[L]`` is
the ``[K, D]`` stack of that layer's per-node mean activations. Every
axis ``r`` is kept iff the node readings relative to the neutral
baseline, ``{(c_i − ν)·p_r : i}``, **straddle zero** (both a negative and
a positive present) — the axis then
discriminates node *position* across the baseline rather than encoding a
common offset / intensity every node shares.

Expand All @@ -682,7 +683,7 @@ def compute_dls_axes(

Args:
node_centroids: per-layer ``[K, D]`` stack of node mean activations.
bases: per-layer ``[R, D]`` basis (rows are the candidate axes).
bases: per-layer ``[R, D]`` read rows (one per candidate axis).
layer_means: per-layer neutral baseline; ``None``/empty disables.
"""
if layer_means is None or not layer_means:
Expand Down
2 changes: 1 addition & 1 deletion drowse/fit/naturalness.py
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ def trajectory_naturalness(
``node_coords`` ``(K, n)`` seed the inverse map. Returns ``(T,)``.
"""
h = to_hellinger(traj_dists) # (T, V)
coords = (h - behavior.mean) @ behavior.basis.T # (T, R)
coords = (h - behavior.mean) @ behavior.read.T # (T, R)
pos, _ = invert_parameterization(behavior, domain, coords, node_coords)
curve_h = behavior.eval_at(domain.embed(pos)) # (T, V)
# In Hellinger space ||h_a - h_b||^2 = 2 - 2*BC, so the Bhattacharyya
Expand Down
4 changes: 2 additions & 2 deletions drowse/fit/node_stats.py
Original file line number Diff line number Diff line change
Expand Up @@ -214,7 +214,7 @@ def compute_store_reduced_covariances(
for offset, size in zip(store.offsets, store.node_sizes, strict=True)
]
for idx, sub in layer_subs.items():
basis_t = sub.basis.to(device="cpu", dtype=torch.float32).transpose(0, 1)
read_t = sub.read.to(device="cpu", dtype=torch.float32).transpose(0, 1)
rank = sub.rank
rows = store.flat_rows(idx)
reduced_rows = torch.empty(store.total_rows, rank, dtype=torch.float32)
Expand All @@ -230,7 +230,7 @@ def compute_store_reduced_covariances(
# the operation bounded and out-of-place even when the source row
# store is already fp32 (where ``.to(float32)`` would alias it).
torch.sub(rows[start:end], mean, out=centered)
reduced_rows[start:end] = centered @ basis_t
reduced_rows[start:end] = centered @ read_t

for k, (start, end) in enumerate(boundaries):
size = end - start
Expand Down
Loading
Loading