feat(MODEL-TEXT-FLINT): the paged-engine arm — continuous batching on CPU, five defects fixed, the first throughput numbers

.agents/issues/MODEL-TEXT-FLINT/ISSUE-LOCAL-01M5W2QQBJ32HRB29G1KVQJS77.md

@@ -0,0 +1,19 @@
+ID: ISSUE-LOCAL-01M5W2QQBJ32HRB29G1KVQJS77
+Title: Flint CPU paged-engine serving arm (MODEL-TEXT-FLINT)
+Row: MODEL-TEXT-FLINT
+State: CLOSED
+Kind: feature
+GitHub: -
+Mirror: PENDING
+Availability: FULL
+Created: 2026-10-26
+Updated: 2026-10-27
+Closed: 2026-10-27
+
+## Problem
+
+Flint serves on CPU only through the single-sequence reference forward. RESOLVED (2026-10-27, waves W-a..W-e on row/FLINT-PAGED): the paged arm serves the published Q4_K_M through the full LLMEngine with continuous batching; the synthetic gate gates the house band, the real-weights paged-engine gate adjudicates three continuously-arriving requests against the reference forward at margins 0.036-0.114 nats; the first CPU throughput numbers are recorded (docs/benchmarks/flint-cpu-paged.md). Three shared defects were found and fixed along the way (each closed with its own issue): the cpu_paged_attn sink seed, the missing flint.context_length mapping, and the multi-group slot-mapping write. Docs updated (USAGE/FEATURES/benchmarks).
+
+## Resolution
+
+Landed 2026-10-27 on row/FLINT-PAGED in five committed waves (W-a KV-group gate; W-b paged arm + synthetic gate, which caught the shared cpu_paged_attn sink seed ISSUE-LOCAL-01M5YMB0XZ8723N0688YHMKP3N, the arm's double-added residual, and the in-place QK norm; W-c the real-weights paged-engine gate with continuous arrival, margins 0.036-0.114 nats inside the 2.5-nat band, which caught ISSUE-LOCAL-01M5YRQ1SWC93MDG52QZGDFBY4 (context_length) and ISSUE-LOCAL-01M5Z079R7RFSCPB1F4P9C94S5 (per-group slot mapping); W-d the first CPU throughput numbers (prefill 7.44 tok/s, decode 1.26 tok/s, docs/benchmarks/flint-cpu-paged.md); W-e USAGE/FEATURES/benchmark docs). Gates: test_flint_paged, test_flint_paged_engine, test_flint_gguf, test_flint_forward, test_ops_paged_attn(_neon), test_paged_attn_route, test_mimov2_* green; commit style + trailer checkers green; check-agent-record OK.

.agents/issues/MODEL-TEXT-FLINT/ISSUE-LOCAL-01M5YMB0XZ8723N0688YHMKP3N.md

@@ -0,0 +1,19 @@
+ID: ISSUE-LOCAL-01M5YMB0XZ8723N0688YHMKP3N
+Title: cpu_paged_attn seeds the attention sink at exp(0) instead of exp(sink - m)
+Row: MODEL-TEXT-FLINT
+State: CLOSED
+Kind: bug
+GitHub: -
+Mirror: PENDING
+Availability: FULL
+Created: 2026-10-27
+Updated: 2026-10-27
+Closed: 2026-10-27
+
+## Problem
+
+PagedAttentionKernel (src/vt/cpu/cpu_paged_attn.cpp) seeds the softmax running max at the per-head sink and the denominator at exp(0)=1, in a TWO-PASS form that never rescales the seed when pass 1 raises m past the sink. Whenever any score exceeds the sink, the sink term silently becomes exp(m-m)=1 instead of exp(sink-m) — a larger term than the sink deserves — which shrinks every value weight. MiMoV2's large learned sinks always stay the max, so every existing gate passed; Flint's small per-head sinks expose it: attention output off ~6%, model logits off ~2 on the flint-Q4_K_M paged gate. The online-softmax arms (cpu_mla_attn.cpp, cuda_mla_attn.cu) rescale their seed as m grows and never had the defect. Fix: seed the denominator at exp(sink - m_final) in both the NEON and scalar lanes; bit-identical when the sink is the max (exp(0)==1.0f).
+
+## Resolution
+
+Fixed 2026-10-27 in the same flow (worktree /tmp/flint-paged, branch row/FLINT-PAGED): both lanes seed std::exp(sink_p[h] - m) after pass 1. The flint paged synthetic gate reproduced the defect red-first (mixed-batch max logit gap 1.21; L0 attention output -0.128 vs hand-computed -1.139 over identical bf16 streams) and runs green with the fix (L0 exact vs the reference control).

.agents/issues/MODEL-TEXT-FLINT/ISSUE-LOCAL-01M5YRQ1SWC93MDG52QZGDFBY4.md

@@ -0,0 +1,19 @@
+ID: ISSUE-LOCAL-01M5YRQ1SWC93MDG52QZGDFBY4
+Title: FlintHfConfigFromGguf never maps flint.context_length, so paged-engine serving sizes its buffers at max_model_len 0 and segfaults
+Row: MODEL-TEXT-FLINT
+State: CLOSED
+Kind: bug
+GitHub: -
+Mirror: PENDING
+Availability: FULL
+Created: 2026-10-27
+Updated: 2026-10-27
+Closed: 2026-10-27
+
+## Problem
+
+FlintHfConfigFromGguf (flint_gguf_weights.cpp:262) sets every architecture field EXCEPT max_position_embeddings; every other GGUF config builder (qwen3_gguf_weights.cpp:440, qwen3_5:1019, qwen4_exp:110, muse_glimmer:426) maps <arch>.context_length onto it. The engine resolves max_model_len = 0 => config.max_position_embeddings = 0, and the runner's InputBatch sizes token_ids_cpu rows with it — the first add_request writes through a zero-size row and SEGFAULTS (vllm-bench and any FromModelDir serve on the published flint-Q4_K_M.gguf; found by the W-c/W-d waves of MODEL-TEXT-FLINT). The reference/token arms never noticed: they never build an engine.
+
+## Resolution
+
+Fixed 2026-10-27 in the same flow (flint_gguf_weights.cpp): c.max_position_embeddings = OptInt(gguf, flint.context_length, 0), mirroring qwen3/qwen3_5/qwen4_exp/muse_glimmer; the FromModelDir serve then resolves a real max_model_len (auto-fit 4064 on the default pool) and the first add_request no longer segfaults. Red-first: vllm-bench on the published Q4_K_M segfaulted in InputBatch::add_request before the fix; serves after.

.agents/issues/MODEL-TEXT-FLINT/ISSUE-LOCAL-01M5Z079R7RFSCPB1F4P9C94S5.md

@@ -0,0 +1,19 @@
+ID: ISSUE-LOCAL-01M5Z079R7RFSCPB1F4P9C94S5
+Title: Multi-group paged serving writes every group's cache with the TARGET group's slot mapping
+Row: MODEL-TEXT-FLINT
+State: CLOSED
+Kind: bug
+GitHub: -
+Mirror: PENDING
+Availability: FULL
+Created: 2026-10-27
+Updated: 2026-10-27
+Closed: 2026-10-27
+
+## Problem
+
+MakeCommonAttentionMetadata (src/vllm/v1/attention/backend.cpp) selects ONE group's slot_mapping (step.slot_mapping[kv_cache_group_id]) into the single CommonAttentionMetadata the model forward receives, and GPUModelRunner gathers per-group block tables onto the multi_kv channel. The groups' pools allocate DIFFERENT block ids for the same positions (measured on flint-Q4_K_M: the full-attention group's first allocated block is 1, the sliding-window group's is 2), so any multi-group forward that writes its non-target groups' caches with the CAM's slot mapping puts every key in the wrong page; the pages the group's own block table names are still zero and the layer's attention output is exactly 0. Measured on the Flint paged engine (MODEL-TEXT-FLINT W-c): every sliding-window layer's attention output sum was exactly 0.0; after deriving slot = table[pos/bs]*bs + pos%bs from the layer's OWN published group table, the outputs came back. MiMoV2 (mimo_v2.cpp:364) reads attn_meta.slot_mapping for every layer the same way and is exposed identically whenever its groups' block ids diverge.
+
+## Resolution
+
+Fixed 2026-10-27 in the Flint paged arm in the same flow (worktree /tmp/flint-paged, branch row/FLINT-PAGED): when multi_kv is present the arm derives each layer's slot mapping from that layer's OWN published group block table (slot = table[pos/block_size]*block_size + pos%block_size) and writes the cache with it. Red-first evidence: every sliding layer's attention output sum exactly 0.0 with the CAM mapping on flint-Q4_K_M; nonzero after the fix. The engine-side single-CAM seam stays as found; MiMoV2 (mimo_v2.cpp:364) is exposed identically whenever its groups' ids diverge — the engine-side single-CAM seam belongs to the engine row that owns it and stays as found here.

.agents/specs/flint-paged.md

@@ -0,0 +1,185 @@
+# Flint — CPU paged-engine serving arm (`MODEL-TEXT-FLINT`)
+
+Row: `MODEL-TEXT-FLINT`
+State: WAVE
+Issue: ISSUE-LOCAL-01M5W2QQBJ32HRB29G1KVQJS77
+
+## Scope
+
+Bring the CPU Flint arm from the single-sequence reference forward to the
+shared paged engine: continuous batching through the LLMEngine/runner, a
+synthetic-gated paged forward arm, a real-weights paged gate against the
+reference forward, the first honest CPU throughput numbers for
+flint-Q4_K_M, and the serving docs. CPU only; the P150 device belongs to
+another session.
+
+## Upstream anchors
+
+- vLLM implements no flint; the reference for numerics stays the CPU
+  reference forward (`FlintModel::Forward`, spec
+  `.agents/specs/flint-cpu-gguf.md`) and, for serving behavior, vLLM's
+  paged engine conventions already mirrored by the shared seams
+  (`dense_attn::AttnBlock`, `vt::PagedAttention`, `v1` KV cache groups).
+- llama.cpp PR #30928 handles sinks in its paged path by folding the
+  per-head sink into the softmax denominator; our equivalent is
+  `PagedAttentionArgs::attn_sink`, implemented on CPU
+  (`src/vt/cpu/cpu_paged_attn.cpp:207`).
+
+## What already exists (verified at base e4918e77)
+
+- `MakeFlintKVCache` (flint.cpp:196) already emits the two KV cache groups:
+  `FullAttentionSpec` for the full layers, `SlidingWindowSpec`
+  (window 513) for the sliding layers, derived from `layer_types`, which
+  the GGUF loader builds from `flint.attention.sliding_window_pattern`
+  (flint_gguf_weights.cpp:320). W-a is therefore verification + the red
+  gate, not new allocation code.
+- Per-head sinks, window masking, and the CPU paged attention kernel.
+- `dense_attn::StepInputs`/`BuildStepInputs`, `ResidentWeight`, `KvSlice`,
+  `WriteKvCache`, `AttnQkNormRope` (per-head QK RMSNorm + NeoX RoPE),
+  `MoeRouterTopK` (sigmoid + renorm), `MoeSiluMul`, `MoeCombine`,
+  `SharedExpertGate` — the mimo_v2.cpp hybrid pattern is the template.
+
+## Design
+
+W-a (paged KV): the hybrid split above; a focused check that the two
+groups carry the right layer membership, window, and dims.
+
+W-b (paged arm): `FlintModel::ForwardPaged` over vt ops, reached through
+`ForwardFlintForCausalLM` when `input.attn_kv` is non-empty:
+
+- embed → per-layer: input RMSNorm (add+norm) → q/k/v `MatmulBT` →
+  `AttnQkNormRope` on sliding layers only (full layers: QK norm without
+  RoPE) → `WriteKvCache` → `vt::PagedAttention` with
+  `attn_sink` on EVERY layer and `AttentionWindow{W-1, 0}` on sliding
+  layers only → o_proj → add → post RMSNorm (add+norm) → dense SiLU MLP
+  or sigmoid top-k renormalized MoE + ungated shared expert → final norm
+  → untied lm_head into f32 logits.
+- The reference forward stays byte-identical; it remains the oracle.
+
+W-c (real-weights gate): flint-Q4_K_M through the full paged engine,
+batch > 1 with continuous arrival; greedy chains compared against the
+reference forward's chains within the house near-tie band.
+
+W-d (bench): `vllm-bench` on flint-Q4_K_M, decode + prefill tok/s,
+single prompt + a small concurrency sweep, recorded under
+`docs/bench-evidence/` and the benchmark record.
+
+W-e (docs): `docs/USAGE.md` Flint serving commands + container pins;
+`docs/FEATURES.md` row.
+
+## Risks
+
+- QK-norm + RoPE numerics must match the reference's f32 head stream;
+  the synthetic gate asserts the sinks softmax and the window boundary
+  token-exactly against hand-computed probes.
+- The full-attention layers carry NO RoPE — a wrong preamble silently
+  degrades; the position-independence probe (test 4 of the reference
+  gate) is ported to the paged arm.
+
+## Tests
+
+- `test_flint_paged.cpp`: KV group shape/membership; red-first paged-arm
+  synthetic gate (single-batch paged arm vs reference forward chains;
+  window boundary; sink probe; MoE renorm).
+- Real gate: reference-vs-paged greedy chains on flint-Q4_K_M.
+- Bench evidence files.
+
+## Gates
+
+Focused ctest sets (`flint`, `engine`, `runner`, `paged`), commit style +
+trailers over base, `check-agent-record`, preflight.
+
+## Stop conditions
+
+NEEDS_CONTEXT if the shared paged attention cannot express per-head
+sinks or the hybrid window split — checked false at base; sinks and
+windows both exist on CPU.
+
+## Findings
+
+W-b (2026-10-26/27, worktree /tmp/flint-paged): the paged arm LANDED and
+the synthetic gate runs green and GATES: the hybrid KV groups, a mixed
+2-request batch through `ModelRegistry::Forward` with continuous decode,
+window masking, sinks, and the house 2.5-nat near-tie band against the
+reference forward (request 0 replays the reference chain token-for-token;
+the one request-1 divergence sits inside the band).
+
+The gate caught THREE real defects:
+
+1. The per-head QK RMSNorm ran IN-PLACE over the head view and corrupted
+   the normed stream (q/k sums off ~5%); the arm now writes qn/kn.
+2. THE SHARED KERNEL (ISSUE-LOCAL-01M5YMB0XZ8723N0688YHMKP3N, fixed in
+   the same flow): `cpu_paged_attn.cpp` seeded the softmax denominator at
+   exp(0) with the running max seeded at the sink, in a two-pass form
+   that never rescales the seed once a score exceeds the sink — the sink
+   term silently became exp(m-m)=1 instead of exp(sink-m). MiMoV2's big
+   learned sinks always stay the max, so every earlier gate passed;
+   Flint's small sinks exposed it. Both lanes now seed exp(sink - m)
+   (bit-identical when the sink IS the max). Measured red-first: L0
+   attention output -0.128 vs hand-computed -1.139 over identical bf16
+   streams; exact after the fix.
+3. The paged arm passed a LIVE residual buffer to the fused
+   `vt::RmsNorm(residual=...)` on BOTH norms, so the second call added
+   the residual again (residual += x) and every MLP input was
+   rmsnorm(2x). The reference keeps ONE residual stream (x itself), so
+   the arm now runs plain norms + explicit adds.
+
+Contract alignment (the dead agent's ~3-nat "unexplained residual" was
+defects 2+3, not conditioning): the arm mirrors the reference's rounding
+exactly at every stage the dump can see — projections round to bf16
+before the head norm (MatmulBf16's own final store), the head norm runs
+f32-in/f32-out with one bf16 rounding after (the FLINT_REF_BF16_HEAD
+control), silu-mul consumes the same bf16 g/u. FLINT_CPU_DUMP now dumps
+every paged layer ([FLINT-PAGED] L<l> <stage>) for like-for-like stage
+sums against the reference.
+
+W-c (2026-10-27): the REAL-WEIGHTS paged-engine gate
+(`test_flint_paged_engine`) drives the published Q4_K_M through the full
+LLMEngine stack with three continuously-arriving requests; every chain
+adjudicates against the reference forward inside the 2.5-nat band (measured
+margins 0.114 / 0.036 / 0.042 nats; request 0's text replays the reference
+exactly). Two engine-side defects surfaced and were fixed in the same flow
+(both closed with red-first evidence): the GGUF config never mapped
+`flint.context_length` (ISSUE-LOCAL-01M5YRQ1SWC93MDG52QZGDFBY4 — segfault on
+the first add_request), and the multi-group cache write used the target
+group's slot mapping (ISSUE-LOCAL-01M5Z079R7RFSCPB1F4P9C94S5 — every sliding
+layer's attention output exactly 0.0). The arm now consumes the
+MultiKvCacheIndex by name and derives each layer's slots from its own
+group's table.
+
+W-d (2026-10-27): the FIRST CPU throughput numbers, `vllm-bench` on the
+published container at 32 threads on the shared aarch64 box: prefill 7.44
+tok/s (512 in), decode 1.26 tok/s (TPOT 797 ms, c=1), 0.97 tok/s aggregate
+at c=4. Baseline, not a parity claim — no vLLM denominator exists for flint.
+[docs/benchmarks/flint-cpu-paged.md], evidence
+[docs/bench-evidence/flint-paged-20261027/].
+
+W-e (2026-10-27): `docs/USAGE.md` Flint serving section with the container
+pins; `docs/FEATURES.md` row updated; the benchmark record is
+`docs/benchmarks/flint-cpu-paged.md`.
+
+## Outcome
+
+The row landed as five committed waves on `row/FLINT-PAGED` (base e4918e776):
+W-a the hybrid KV-group gate, W-b the paged arm + synthetic gate, W-c the
+real-weights paged-engine gate, W-d the first throughput numbers, W-e the
+docs. Measured: the numbers above. Rejected: token-exact paged-vs-reference
+agreement as the gate form — the paged path rounds the head stream and the
+KV cache through the model dtype bf16 where the reference keeps its f32
+score path, so the house near-tie band is the correct contract (measured
+margins sit two orders of magnitude inside it). Each default's reason: the
+per-layer slot derivation lives model-side (not engine-side) because the
+single-CAM seam is the engine's and the arm already reads the group tables;
+`consumes_multi_kv` mirrors MiMoV2/DeepSeek-V4; the reference forward stays
+byte-identical as the oracle.
+
+## Owed
+
+- CUDA/ROCm device arms: refused by name in the registry switch (existing
+  behavior).
+- The llama.cpp token gate for the paged arm is NOT owed: the reference
+  forward is the oracle for paged batching per row direction.
+- The engine-side per-group slot-mapping seam (one CommonAttentionMetadata
+  per step) is owned by the engine row; the closed multi-group slot-mapping
+  issue in this row's directory records the finding, and MiMoV2 is exposed
+  identically.

docs/BENCHMARKS.md

@@ -14,6 +14,7 @@ to the file stem.
 | `llama-cpp-cpu` | llama.cpp on CPU, including the RPi5/A76 gating arm that does not transfer | Results | [Detail](benchmarks/llama-cpp-cpu.md) |
 | `kolibri1-cpu-dequant-cache-default` | Kolibri-1 on CPU with the dequant cache on by default: decode, prefill and the n-gram spec-decode serving number | Results | [Detail](benchmarks/kolibri1-cpu-dequant-cache-default.md) |
 | `mlx-lm-apple-m4` | MLX-LM on Apple M4, warm, batch 1, 6 interleaved runs | Results | [Detail](benchmarks/mlx-lm-apple-m4.md) |
+| `flint-cpu-paged` | Flint (RecursiveSI) Q4_K_M on CPU through the paged engine: the first prefill/decode numbers | Results | [Detail](benchmarks/flint-cpu-paged.md) |
 | `dwarfstar-gguf` | DwarfStar vs ds4 on DeepSeek-V4-Flash GGUF, GB10, byte-exact | Results | [Detail](benchmarks/dwarfstar-gguf.md) |
 | `speculative-decoding` | MTP / DFlash / n-gram / DSpark rows against vLLM | Results | [Detail](benchmarks/speculative-decoding.md) |
 | `tt-capture-default-decode` | Qwen3-0.6B decode on the P150 with captured decode as the shipped default | Results | [Detail](benchmarks/tt-capture-default-decode.md) |