Repository navigation
gate(MODEL-TEXT-FLINT): the llama-cpp-flint oracle is gateable -- and the CPU keep-quant arm is named, not assumed - #3521
Merged
Conversation
The developer set llama.cpp as the peer floor for Flint CPU throughput. The row pins llama.cpp PR #30928 as the llama-cpp-flint oracle, measures its tg rate on flint-Q4_K_M.gguf like-for-like with our serving harness (identical artifact, same box, same core and thread budget), measures our keepquant arm under the same budget, and records the gap. Kernel work files its own row; a numerics defect found by the golden cross-check is fixed here. The oracle file and the AGENTS.md registry row land with the gateability measurement, because the pin contract admits only gateable=yes-with-evidence or gateable=no-with-a-GitHub issue, and this row delivers the measurement. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:zai/glm-5.3-flash [maki]
… Q4_K_M artifact — gated, floor 26.29 tok/s at 16 threads The FLINT-CPU-PARITY gateability measurement (ISSUE-LOCAL-01M61G9R7D4EAM54RKXMES0Z6J): llama.cpp PR #30928 head 1b44e3077 served flint-Q4_K_M.gguf (sha256 20c89707f762af73..., the same file both sides) on the shared aarch64 box, bench zone taskset -c 32-63, governor schedutil. Greedy goldens on three fixed prompts (temperature 0, seed 42, finish reason length) and llama bench tg rates, 3-rep medians: tg200 26.29 / tg400 25.66 tok/s at 16 threads, 26.38 / 24.69 at 32 threads. Those two cells are the row's like-for-like floor; nothing else is quotable as the floor. The pin flips to gateable=yes with the evidence directory as its record. The AGENTS.md registry row lands in the same change; the pre-existing draft row and the committed row were merged into ONE row — committing both would have duplicated the llama-cpp-flint registry id. One harness quirk is recorded: the container's chat-template metadata carries a \x00 escape llama.cpp's lexer rejects, so the golden legs pass an explicit raw-passthrough chat template with --jinja; numerics are unaffected. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:zai/glm-5-3-flash [maki]
…by name The keepquant CPU serving leg found the defect: VT_FLINT_KEEP_QUANT=1 loads the block-carrying tree, whose only consumer is the Tenstorrent device forward, and the paged CPU arm — the serving path — carried no audit. The packed bytes reached a bf16 matmul and the engine died on the opaque 'vt: matmul: inner dims mismatch at src/vt/ops.cpp:150' (the pre-fix capture is quoted in docs/bench-evidence/flint-cpu-parity-20261029/report.md). A safe throw, never a silent misread, but an unimplemented arm must be refused with a message that names the missing part. FlintForwardPaged now carries the same named audit the reference forward has (flint.cpp, the BF16-OWNED block); the post-fix keepquant-refusal.log records the refusal firing from the serving path with the full reload instruction. Consequence for the row: the keepquant serving number on CPU is not producible, so the keepquant-vs-floor gap stays unrecorded pending a CPU keep-quant dispatch arm, which files its own row. The shexp check verified the loader reads ONE double-width shared expert (expert_shared_count=2 as 2 slots of 512, never two experts). Issue closed with the dated evidence. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:zai/glm-5-3-flash [maki]
Evidence directories reference raw logs, and the global *.log rule in .gitignore drops those files at git add time without an error. No gate checks evidence-directory contents, so a report can land alone while the logs it cites are lost. This row's logs survive only because the worktree still exists; the readback, paged-regression, and numa evidence dirs on main hold README assertions only, their raw logs gone with removed worktrees. The report records the completeness note beside the log list. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:zai/glm-5-3-flash [maki]
lu-zero
added a commit
that referenced
this pull request
Oct 29, 2026
Both rows landed 2026-10-29: FLINT-CPU-PARITY as 19979ac (PR #3521) and FIX-EVIDENCE-LOG-GATE as 007c67b (PR #3522). The parity outcome records the gated oracle, the measured floor, and the row's central discovery -- the CPU keep-quant serving arm does not exist, the defect is repaired as a named refusal, and the dispatch arm files its own row -- together with the shexp verification and the labeled bf16 bound. The evidence-gate outcome records the whitelist, the wired checker, the 46-entry red-before across nine dirs, and the named known gaps (the unguarded CI wiring above all). The parity issue was already closed with its resolution; this commit adds the missing ## Outcome sections and the evidence-gate close. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:zai/glm-5-3-flash [maki]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Row FLINT-CPU-PARITY gates the llama.cpp Flint oracle and measures the
like-for-like CPU floor for the shipped Q4_K_M artifact — and in doing so
surfaced a capability gap and a defect, both repaired or named here.
The oracle. llama.cpp PR #30928 head 1b44e3077 (the model author's own
Flint support, the only llama.cpp that loads flint) serves the shipped
artifact — /mnt/models/flint-GGUF/flint-Q4_K_M.gguf, sha256 20c89707...927e,
same file recorded — and flips .agents/oracles/llama-cpp-flint.md to
gateable = yes with the AGENTS.md registry row. Goldens: three prompts,
greedy, raw-passthrough --jinja (the container's chat-template metadata
carries a \x00 escape that crashes llama.cpp template init; template-only,
numerics neutral, documented).
The floor (llama.cpp, bench zone, 3-rep medians, governor schedutil as
observed and logged): 26.29 / 25.66 tok/s at 200/400 completions at 16
threads; 26.38 / 24.69 at 32.
The gap and the defect. vllm.cpp has no CPU keep-quant serving arm:
VT_FLINT_KEEP_QUANT=1 loads a block tree consumed only by the Tenstorrent
forward; the paged serving arm died on an opaque "matmul: inner dims
mismatch" (ops.cpp:150). The defect is fixed here: FlintForwardPaged refuses
a keep-quant tree BY NAME (the arm, the env var, the reload instruction, the
TT arm), and the review's M1 mutation reproduces the opaque crash without
it. The bf16 paged arm passes through untouched (11/11 engine assertions
with the real artifact, margins 0.036-0.114 nats in the 2.5-nat band).
Measured and clearly labeled as a different kernel class — a bound, not a
parity gap: 3.25/3.28 tok/s @16T. The CPU keep-quant dispatch arm — the real
lever under the 26.29 floor — is a named follow-up row. The shexp mapping is
verified correct: one rank-2 projection of width
shared_expert_intermediate_size = expert_shared_count x
moe_intermediate_size (one double-width shared expert as two slots, never
two experts).
Fresh review (MERGEABLE): all six claims verified; M1 and M2 RED (the
refusal is load-bearing; the pin contract enforces gateable). Three
characterized detection gaps are recorded, not papered over: no checker
compares evidence content against the pin's sha256; check-oracle-pins
compares registry ids as sets so duplicate rows are invisible; deleting a
committed raw log trips no gate — the last is owned by the
FIX-EVIDENCE-LOG-GATE row, the first two are named candidates.
The 9 raw measurement logs are committed (evidence-completeness note in
report.md): a global *.log gitignore had been silently dropping evidence
logs at git add time across rows — the incident and its systemic fix live in
row FIX-EVIDENCE-LOG-GATE.
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:zai/glm-5-3-flash [maki]