Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 35 additions & 12 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,26 +28,49 @@ jobs:
- uses: actions/setup-python@v7
with:
python-version: "3.12"
cache: pip
cache-dependency-path: pyproject.toml
- run: python -m pip install --upgrade pip
# Install dev extras so pyright can resolve runtime/test dependencies
# while it analyzes the package, tests, examples, and maintained scripts.
- run: pip install -e ".[dev]"
- uses: astral-sh/setup-uv@d0cc045d04ccac9d8b7881df0226f9e82c39688e # v6.8.0
with:
enable-cache: true
cache-dependency-glob: pyproject.toml
# CPU torch: the default Linux wheel drags in ~2.5 GB of CUDA libraries
# that neither pyright nor the CPU suite touches.
- run: uv pip install --system torch --index-url https://download.pytorch.org/whl/cpu
- run: uv pip install --system -e ".[dev]"
- run: pyright

test:
# The CPU suite in four parallel shards (a runner has 4 vCPUs; the suite is
# ~330 CPU-seconds locally, a third of it the route tests). The last shard
# takes every directory the others don't name, so a new one is never skipped.
test-shard:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
shard:
- "tests/server"
- "tests/lens tests/units tests/sae tests/cli"
- "tests/fit tests/session tests/model tests/browser tests/loom"
- "tests --ignore=tests/server --ignore=tests/lens --ignore=tests/units --ignore=tests/sae --ignore=tests/cli --ignore=tests/fit --ignore=tests/session --ignore=tests/model --ignore=tests/browser --ignore=tests/loom"
steps:
- uses: actions/checkout@v7
- uses: actions/setup-python@v7
with:
python-version: "3.13"
cache: pip
cache-dependency-path: pyproject.toml
- run: python -m pip install --upgrade pip
- run: pip install -e ".[dev]"
- run: pytest -q -m "not gpu" -n auto
- uses: astral-sh/setup-uv@d0cc045d04ccac9d8b7881df0226f9e82c39688e # v6.8.0
with:
enable-cache: true
cache-dependency-glob: pyproject.toml
- run: uv pip install --system torch --index-url https://download.pytorch.org/whl/cpu
- run: uv pip install --system -e ".[dev]"
- run: pytest -q -m "not gpu" -n auto ${{ matrix.shard }}

# The one check branch protection names.
test:
if: always()
needs: test-shard
runs-on: ubuntu-latest
steps:
- run: test "${{ needs.test-shard.result }}" = success

webui:
runs-on: ubuntu-latest
Expand Down
20 changes: 11 additions & 9 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,11 +209,12 @@ get wrong raises `RequestError` (400, text verbatim) or a subclass
failures after parsing are `SteeringCompositionError` (422). A bare
`ValueError` is a broken invariant: the server logs it and answers a scrubbed
500. The wire `code` is the exception class name. The CLI maps 404 → exit 1,
other 4xx → 2, 5xx → 1. `tests/invariants/test_user_message.py` enforces the
family.
other 4xx → 2, 5xx → 1.

**Persisted formats are exact.** Readers accept only the current version and
never migrate (a derived cache recomputes instead). These froze at 6.0.0:
**Persisted formats are versioned.** Readers accept only the current version and
never migrate (a derived cache recomputes instead); the fitted sidecar reader
ignores keys the contract has retired, so dropping provenance never strands a
fit. These froze at 6.0.0:
`manifold.json` v10, `template.json` v2, `.drowse` v1, loom `tree_format` 2
with token sidecar 3 (token rows carry measurement envelope v2; a node's
`finish_reason` is one of `stop`, `length`, `stop_sequence`, `cancelled`),
Expand Down Expand Up @@ -309,10 +310,11 @@ fit loop, `lens/fit.py`, is the reference pattern).
tiny generated checkpoint (`tiny_session`); route tests that need no behavior
use the autospec `tests.support.server.mock_session`. GPU tests (`gpu` marker)
load `HuggingFaceTB/SmolLM2-360M-Instruct` unless `DROWSE_TEST_MODEL` names
another; `gpu_gemma` needs gated Gemma access. Every bug fix lands with a
regression test. A test never touches the real home (an autouse fixture sets a
scratch `DROWSE_HOME`). CI runs ruff, pyright, the CPU suite on Python 3.13
(`-n auto`) and the webui build with a dist-drift check. Details: `tests/AGENTS.md`.
another; the GPU suite is one end-to-end smoke plus the throughput gates. Add a
regression test when a bug could quietly come back, not as a ritual. A test
never touches the real home (an autouse fixture sets a scratch `DROWSE_HOME`). CI runs ruff, pyright, the CPU suite on Python 3.13
(four parallel shards, `-n auto`, CPU torch via uv) and the webui build with a
dist-drift check. Details: `tests/AGENTS.md`.

## State and environment

Expand All @@ -333,4 +335,4 @@ downloads, and Python saves only to an explicit `LoomTree.save(path)`.
| `DROWSE_STRICT_MODEL` | OpenAI/Ollama 404 on a model-name mismatch |
| `DROWSE_MAX_REQUEST_BYTES` | request body cap (default 64 MiB) |
| `DROWSE_MANIFOLD_CAPTURE_CACHE_GB` | fit capture cache bound (default 8) |
| `DROWSE_TEST_MODEL`, `DROWSE_TEST_SAELENS`, `DROWSE_TEST_LIVE_TOKENIZERS` | test knobs (`tests/AGENTS.md`) |
| `DROWSE_TEST_MODEL`, `DROWSE_TEST_LIVE_TOKENIZERS` | test knobs (`tests/AGENTS.md`) |
3 changes: 1 addition & 2 deletions drowse/fit/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -375,8 +375,7 @@ corpora `tests/session/test_generate_responses.py`, session bake
`tests/session/test_session_bake.py`, neutral cache
`tests/store/test_neutral_cache.py`, neutral unit statistics
`tests/units/test_unit_stats.py`, DLS golden
`tests/browser/test_browser_parity.py`, GPU `tests/gpu/test_smoke.py` and
`tests/gpu/test_session.py`.
`tests/browser/test_browser_parity.py`, GPU `tests/gpu/test_smoke.py`.

## Deliberately absent

Expand Down
16 changes: 2 additions & 14 deletions drowse/fit/pipeline.py
Original file line number Diff line number Diff line change
Expand Up @@ -798,16 +798,9 @@ def accepts(self, sidecar: "ManifoldSidecar") -> bool:
return bool(sidecar.sae_full_coverage)

def base_metadata(
self, *, method: str, fitted_layers: list[int], subspace_metric: str,
self, *, method: str, fitted_layers: list[int],
) -> dict[str, Any]:
"""The sidecar keys every fit writes.

``subspace_metric`` is a basis *label*: the monopolar fold keeps the
raw δ̂ basis (``concept − ν`` cancels common-mode by differencing, like
DiM) and reports ``euclidean``; every other fit selects a whitened
basis. The share is always Mahalanobis — the whitener gate guarantees
it.
"""
"""The sidecar keys every fit writes."""
ident = self.identity
metadata: dict[str, Any] = {
"method": method,
Expand All @@ -820,9 +813,6 @@ def base_metadata(
"baseline_prompts_sha256": ident.baseline_prompts_sha256,
"fitted_layers": fitted_layers,
"fit_policy_version": MANIFOLD_FIT_POLICY_VERSION,
# Provenance only (nothing branches on these at load).
"share_metric": "mahalanobis",
"subspace_metric": subspace_metric,
}
backend = self.sae_backend
if backend is not None:
Expand Down Expand Up @@ -1342,7 +1332,6 @@ def _fit_monopolar(
else "manifold_monopolar"
),
fitted_layers=list(fit_layers),
subspace_metric="euclidean",
)
metadata.update({
"fit_mode": mf.fit_mode,
Expand Down Expand Up @@ -1448,7 +1437,6 @@ def _fit_nodes(
metadata = inputs.base_metadata(
method=geometry.method,
fitted_layers=sorted(layer_subs),
subspace_metric="mahalanobis",
)
# Diagnostic layer profile: the whitened between-node spread
# ``tr(G_L)`` — surfaced by `manifold show`, consumed by nothing.
Expand Down
1 change: 0 additions & 1 deletion drowse/geometry/fold.py
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,6 @@ def fold_directions_to_subspace(
node_kinds=[None],
mahalanobis_share=mahalanobis_share,
)
manifold.metadata["share_metric"] = "mahalanobis"
manifold.validate_runtime_geometry()
return manifold

Expand Down
2 changes: 0 additions & 2 deletions drowse/lens/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -733,5 +733,3 @@ the per-layer softmax probability `p_l(v)`.
(`LensHarness`, `lens_instrument(**kw)`, `lens_kwargs(**kw)`,
`FakeCapture`), which builds a real `LensInstrument` with session-shaped
dependencies. Generation-boundary tests use `tiny_session`.
`tests/gpu/test_jlens_gpu.py` micro-fits a real model (SmolLM2-360M or
`DROWSE_TEST_MODEL`).
5 changes: 3 additions & 2 deletions drowse/model/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -446,5 +446,6 @@ and stitcher against the Jinja fakes in `tests/support/chat_templates.py`
`test_role_template_drift.py` checks `SEAT_HEADERS` against real tokenizers in
the local HF cache; `DROWSE_TEST_LIVE_TOKENIZERS=1` permits downloads (run it
before a release). `tests/support/tiny_models.py` supplies genuine 4-layer
models with real family templates for CPU end-to-end tests; real-weight checks
are `tests/gpu/test_model_numerics_gpu.py` (`DROWSE_TEST_MODEL`).
models with real family templates for CPU end-to-end tests;
`tests/model/test_model_numerics.py` carries `mps` params for real-device
numerics.
4 changes: 2 additions & 2 deletions drowse/readout/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -284,8 +284,8 @@ rows at its own rank, one batched foot solve (`invert_stacked`) per rank
(one for a rank-uniform fit), and the share-weighted means as a left fold in
layer order. Every kernel computes a layer as the per-layer read did, so a
reading does not depend on how many layers ran together: bit-identical to the
per-layer read on MPS (`tests/gpu/test_curved_stack_gpu.py`; on CPU
`tests/readout/test_monitor_curved_stack.py` pins it to tolerance). A
per-layer read (`tests/readout/test_monitor_curved_stack.py` pins it to
tolerance on CPU). A
reduction batched across layers moves values by an ulp, and an off-surface
state's foot to another basin, so keep reductions per layer. A read that saw only some of
a probe's layers uses a cached `_CurvedStack.subset`. Curved feet warm-start
Expand Down
3 changes: 1 addition & 2 deletions drowse/sae/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -346,5 +346,4 @@ instrument). `tests/support/sae.py` provides `MockSaeBackend`,
section and recorded registry effects) and `make_resident(runtime, backend,
layer=...)`. Session-level tests load a mock through `session.sae.load` with
`drowse.sae.runtime.resolve_sae_backend` monkeypatched. Tests that need SAELens
`importorskip` it. The real end-to-end run, `tests/gpu/test_sae_lens_smoke.py`,
is opt-in via `DROWSE_TEST_SAELENS=1` and is marked `gpu_gemma`.
`importorskip` it.
6 changes: 3 additions & 3 deletions drowse/session/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -393,6 +393,6 @@ the root `AGENTS.md` "Errors" rule; `user_message()` defaults to
`tests.support.engine.engine_over`, `tests.support.extraction.make_extraction`.
Route tests use `tests.support.server.mock_session`, an autospec that fails on
removed or renamed members.
- GPU: `tests/gpu/test_session.py`, and `tests/gpu/test_smoke.py` (owns the
throughput regression), on `HuggingFaceTB/SmolLM2-360M-Instruct` unless
`DROWSE_TEST_MODEL` names another model.
- GPU: `tests/gpu/test_smoke.py` (one end-to-end smoke and the throughput
gates), on `HuggingFaceTB/SmolLM2-360M-Instruct` unless `DROWSE_TEST_MODEL`
names another model.
1 change: 0 additions & 1 deletion drowse/session/extraction.py
Original file line number Diff line number Diff line change
Expand Up @@ -393,7 +393,6 @@ def _fit_vector(
metadata={
"method": "manifold_pca",
"name": ret_name,
"share_metric": manifold.metadata.get("share_metric"),
},
)
self._events.emit(ManifoldExtracted(
Expand Down
2 changes: 1 addition & 1 deletion drowse/store/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ provenance only, and an unknown key raises. Provenance is all-or-nothing against
| `method` | `fit_mode` | required block |
| --- | --- | --- |
| `manifold_pca`, `manifold_sae` | authored | |
| `manifold_monopolar[_sae]` | pca | 1-node ray, `subspace_metric = euclidean` |
| `manifold_monopolar[_sae]` | pca | 1-node ray (`concept − ν`) |
| `manifold_discover_{pca,spectral,auto}` | its own | auto: `resolved_fit_mode`, `topology_winner`, `topology_candidates` |
| `manifold_discover_sae` | any discover | |
| `manifold_procrustes_transfer` | any | `source_model_id`, `source_model_fingerprint` |
Expand Down
3 changes: 0 additions & 3 deletions drowse/store/manifold_authoring.py
Original file line number Diff line number Diff line change
Expand Up @@ -453,9 +453,6 @@ def build(staging: Path) -> None:
"method": method, "fit_mode": "baked", "nodes_sha256": nodes_sha,
"model_fingerprint": item.model_fingerprint,
}
share_metric = item.manifold.metadata.get("share_metric")
if share_metric:
meta["share_metric"] = share_metric
if item.components:
meta["components"] = item.components
if method == "merge":
Expand Down
6 changes: 0 additions & 6 deletions drowse/store/manifold_lifecycle.py
Original file line number Diff line number Diff line change
Expand Up @@ -457,10 +457,6 @@ def transfer_manifold(
metadata["source_model_id"] = from_model
metadata["source_model_fingerprint"] = source_model_fingerprint
metadata["model_fingerprint"] = target_model_fingerprint
# The share was re-baked in the target metric; ``subspace_metric``
# keeps the source value (the basis was selected there and only
# rotated here).
metadata["share_metric"] = "mahalanobis"
if transfer_quality_estimate is not None:
metadata["transfer_quality_estimate"] = float(transfer_quality_estimate)
sidecar_path = out_path.with_suffix(".json")
Expand Down Expand Up @@ -502,8 +498,6 @@ def manifold_fit_summary(sidecar: ManifoldSidecar, stem: str) -> dict[str, Any]:
("hyperparams", sidecar.hyperparams),
("diagnostics", sidecar.diagnostics),
("node_spread", sidecar.node_spread_per_layer),
("share_metric", sidecar.share_metric),
("subspace_metric", sidecar.subspace_metric),
("resolved_fit_mode", sidecar.resolved_fit_mode),
("topology_winner", sidecar.topology_winner),
("topology_candidates", sidecar.topology_candidates),
Expand Down
16 changes: 10 additions & 6 deletions drowse/store/manifold_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -619,8 +619,6 @@ class ManifoldSidecar:
fit_policy_version: Optional[int] = None
# ``mahalanobis`` on an ordinary fit; a monopolar fold labels its raw-δ̂
# basis ``euclidean`` (a basis-selection label, not a metric fallback).
share_metric: Optional[str] = None
subspace_metric: Optional[str] = None
# Curved-fit summaries: GCV smoothing ``{L: {lambda, edf, gcv}}`` and the
# fuzzy-manifold tube ``{L: {sigma_mean, sigma_min, sigma_max, lambda}}``.
rbf_smoothing_per_layer: dict[str, Any] = field(default_factory=dict)
Expand Down Expand Up @@ -839,11 +837,17 @@ def _non_negative_int(value: Any) -> bool:
def validate_sidecar_payload(
data: Any, *, location: str = "manifold sidecar",
) -> dict[str, Any]:
"""Validate every field of the exact current fitted-sidecar contract."""
"""Validate every field of the current fitted-sidecar contract.

Keys the contract no longer names (retired provenance) are dropped, so
removing a field never strands an existing fit; every current field must
be present.
"""
from drowse.geometry.domains import validate_domain_spec

if not isinstance(data, dict) or set(data) != SIDECAR_FIELDS:
raise ManifoldFormatError(f"{location} does not match the current exact schema")
if not isinstance(data, dict) or not SIDECAR_FIELDS <= set(data):
raise ManifoldFormatError(f"{location} does not match the current schema")
data = {key: data[key] for key in SIDECAR_FIELDS}
validate_manifold_format_version(data["format_version"], location=location)
try:
validate_domain_spec(data["domain"])
Expand Down Expand Up @@ -881,7 +885,7 @@ def validate_sidecar_payload(
nullable_strings = (
"nodes_sha256", "sae_release", "sae_revision", "sae_fingerprint",
"model_fingerprint", "model_source_fingerprint", "capture_sha256",
"share_metric", "subspace_metric", "resolved_fit_mode",
"resolved_fit_mode",
"topology_winner", "bake_policy", "source_model_id",
"source_model_fingerprint",
)
Expand Down
1 change: 0 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,6 @@ addopts = ["--strict-markers"]
xfail_strict = true
markers = [
"gpu: requires a CUDA or MPS backend (deselect with `-m \"not gpu\"`)",
"gpu_gemma: requires explicit gated Gemma model and artifact access",
"perf: wall-clock throughput test; skipped under xdist, run with `pytest -m perf -p no:xdist`",
]
filterwarnings = [
Expand Down
Loading
Loading