Repository navigation
Conversation
Align patch, LayerNorm, rotary, head-64 attention and vision GEMM boundaries with the pinned BF16 reference. Keep legacy/JEMM behavior and optional ABI compatibility. The original 280-answer whole-model precision gate still fails; this is the bounded vision stage for InferMatrixLab#114.
hsliuustc0106
left a comment
There was a problem hiding this comment.
Review draft — #209 "omnijev: add opt-in BF16 reference vision arithmetic"
Verdict: COMMENTED. All four PRs in this series are OPEN and explicitly DRAFT; this is a first review of a draft, not an approval and not a completed validation. The human self-review checklist in the body is intentionally left unchecked by the contributor and must stay untouched by reviewers.
Provenance and pins
- Repository:
ThinkFlowLab/system1-omniresolves via rename redirect toInferMatrixLab/system1-omni(same repo, API id1386454853). CI run URLs therefore appear under the InferMatrixLab name; both API hostnames return identical runs. - Base (GitHub
baseRefOidand body pin, verified identical):f1e674f2d324f5bb4e1ba2162551d0b633f20be0("Add a Valen-Decision-1.5 reference worker (#204)"). - Head:
6410e7634fd5e077000810ae07dc2f251b4f044d(matchesorigin/pr/209and GitHubheadRefOid). Authorpsv666(new author in this series), head repopsv666/system1-omni, branchcodex/omnijev114-vision-20261120. - Diff: 17 files, +841/−18 (GitHub and local
git diff --statagree). Body's "730 changed authored-code lines, 859 total" reproduces exactly (code-file numstat +712/−18 = 730; total 841+18 = 859). - Local checkout used for skill/contract loading: branch
skill-add-new-model-evidence-pointer@ca5f69b5c104ff2ed0c64ed2f195b49bec92e32c(note: this is ahead of the PR base; see contract-provenance note below).
Position in the series (cross-reference)
Verified by git merge-base --is-ancestor: 209 ⊂ 211 ⊂ 212 is a clean linear stack (one commit each, first-parent chains verified). #215 is NOT a superset: git merge-base origin/pr/212 origin/pr/215 is the shared base f1e674f2d, and 209/211/212 are not ancestors of 215. The series is actually two families: the OmniJev stack #209→#211→#212, and a separate Vela carrier PR #215 (own base branch codex/vela152-coexist-base, see the #215 review). Review order applied: #209 → #211 → #212 → #215.
Template gates
- Four required contributor answers: present and explicit (#209 carries them; #211/#212/#215 restate and link back). Necessity answer ("Is a PR needed?"): present, "Yes", with scope of authorization. Draft decision: explicit ("this PR stays draft").
- Self-review checklist: present, all four boxes unchecked — correct for a draft; not filled by this review.
- Evidence pins: base/head SHAs correct (verified against GitHub metadata and local git). Pinned artifact hashes (official code
14dbec4f7…, checkpointffe5f436e…, base851bf6e8…, export manifest SHA256) are author-reported; the manifest/vocabulary/geometry pins match the constants in code (src/models/omnijev/native/src/export.rs:CHECKPOINT_REVISION,BASE_REVISION).
Findings
No P0/P1/P2 findings. The contract questions resolve cleanly:
- Opt-in mechanism: environment variable
OMNIJEV_VISION_NUMERICSparsed inExecutor::load(src/models/omnijev/native/src/executor.rs:120-131at head);NotPresent→legacy; unknown values fail startup (tested intests/omnijev/window_graph.rs::vision_profile_is_explicit_and_rejects_unknown_settings, including"Reference","reference "variants). Library callers useExecutor::load_with_vision_numerics. Default is unchanged legacy behavior; opt-out default preserved. - Opt-in is device-guarded:
VisionModel::enable_omnijev_numerics(src/models/qwen3_5/native/src/vision/model.rs:231-263) requires unadapted 4B (hidden 1024, no LoRA), selection before first forward, rejects bounded-cache accounting, requires the cuDNN-9 capability probe, and is idempotent. All four guards are exercised intests/qwen3_5/vision_omnijev.rs(real-weight,#[ignore]-gated). - BF16 rounding locations: BF16 outputs at every operator boundary retained; FP32 rotary angles (
cs1_omnijev_vision_rope, unusedsidocumented insrc/backends/cuda/qwen3_5/ops.h:272); Welford LayerNorm kernel; framework-style GEMMs via a dedicated cuBLASLt handle with Torch's 1 MiB heuristic budget (the body documents that 32 MiB changes merger split-K and fails the exact-stage gate — a measured, not assumed, rationale); head-64 FlashAttention with FP32 accumulation → BF16 output (src/backends/cuda/qwen3_5/reference_vision_attention.cu:198-214, scale0.125f, rows ≤ 4608, heads == 16, dh == 64 guards). - Parity evidence is measured-then-declared: kernel-level tests assert bitwise
torch.equalagainst pinned-Torch SDPA/layer_norm (tests/qwen3_5/test_vision_omnijev.py, pinned Torch 2.14.0 enforced); stage-level test compares BF16 bytes against a frozen oracle (tests/qwen3_5/vision_omnijev.rs); legacy-operator byte preservation is tested against an immutable pre-change library. The declared corpus result (177/280 within 0.01, max drift 0.1758, 3 protected flips, strict gate exits 1) is repeated consistently in body,recipe/omnijev/native.mdandsrc/models/omnijev/README.md, and the PR honestly keeps the 280-answer gate failed and stays draft. - ABI: purely additive at ABI 9 — five new optional symbols (
cs1_reference_patch_create_v2,cs1_omnijev_vision_norm/rope/attention_workspace_floats/attention,src/backends/cuda/qwen3_5/ops.h:267-276). The original 27B constructor is preserved as a wrapper calling v2 with hidden 1152 (reference_patch.cu:63-66); old libraries remain loadable (optionalresolve, explicit rebuild-instruction error viaApi::omnijev_vision()). The V-pointer offsetqkv.at(h*2*2)is correct becauseDeviceBuffer::atis byte-offsetting (src/models/qwen3_5/native/src/cuda.rs:767-770), matching the kernel's compact-Q/K / V-in-QKV layout (also verified bitwise from Python at rows 4..4608). - Informational (no action required):
cudaFuncSetAttribute's return incs1_omnijev_vision_attentionis unchecked (reference_vision_attention.cu:203-204); a failure still surfaces through the subsequentcudaGetLastErroron launch, so the error path is covered. The 1 MiB budget and parity claims are explicitly scoped to the pinned stack (Torch 2.14.0/cuDNN 9.24/sm_89) — the docs state this.
Evidence provenance
- Source inspected: full 17-file diff at
f1e674f2d..6410e7634; ops.h/cuda.rs/executor.rs/vision model.rs/reference*.cu in depth; ops.h cross-checked against the Rust extern signatures. - Commands run (personally): ref/ancestry verification;
git diff --stat/--numstatper PR; tree/SHA pin checks; numstat authored-line reproduction; binary/non-source file scan (none committed — no run dumps, no fixtures, matching the body claim). - Observed CI (Actions API, both repo hostnames, exact head_sha
6410e7634…):CI(rust, benchmarks, kev-export, cuda-contract, cuda-compile jobs per.github/workflows/ci.yml) andDocs— both success, run 44674667802 / 44674667780. Fork repo (psv666/system1-omni) has 0 runs for this SHA. These are CPU-level checks; no GPU model test ran. - Author-reported (not independently verifiable here): all L20 GPU measurements (40-boundary bitwise match, 280-answer table, protected-case margins), the evidence archive (release asset
omnijev114-vision-public-evidence.tgz, SHA25685e39a9d…, 197 files — not downloaded/verified), CUDA library/operator hash pins.
Unverified checks / evidence gaps
- No local GPU and no cargo in this environment: the weight-bearing Rust suites (
vision_omnijev.rs,vision_omnijev_reference.py) and the 280-answer replay are author-reported only. Missing hardware is an evidence gap, not a PASS. - The declared corpus gate fails (177/280) by design and the PR says so; acceptance of #114 remains blocked. No merging decision is requested or implied here.
Skill load record
system1-omni-review SKILL.md loaded from /Users/hsliu/thinkflowlab/system1-omni/.agents/skills/system1-omni-review/SKILL.md, SHA-256 58be3bc60806c4c5af7038a0c6a80fadb94f3360d714556a3e965c8903dde108, last touched by commit ef927963, working tree matches HEAD. Also read: references/repository-contracts.md (SHA-256 d6ae9650…), self-review/SKILL.md (SHA-256 a795b008…), docs/architecture.md and CONTRIBUTING.md sections at the refs named above. Provenance note: the "Large code changes" CONTRIBUTING section exists at the local checkout (ca5f69b5) but not in fork/main:CONTRIBUTING.md; the 3,000-authored-line requirements live in .agents/skills/self-review/SKILL.md at canonical main. Not load-bearing for #209 (859 total lines).
Purpose
OmniJev's legacy BF16 vision arithmetic drifts from the pinned official tower through normalization, attention and GEMM reduction boundaries. Add an explicit
OMNIJEV_VISION_NUMERICS=referencepath that matches all 40 observed visual boundaries on the original protected PNG case, while retaininglegacyas the default. This is the bounded vision stage of #114; the original full 280-answer precision gate still fails, so this PR stays draft.Required human contributor answers
The contributor confirmed these answers and the draft-PR decision during preparation; the public scope is RFC #114. Private preparation records are retained with the contributor.
Is a PR needed? Yes: the contributor authorized necessary implementation, L20 validation, self-review and a reviewable draft PR for this stage.
Observed failure: Original
video.png/noul_region, unchanged question/region: official unmerged BF160.5443, legacy native0.4997, drift0.0446and a protected true→false decision. The strict gate actually exits 1. Raw before/after results and failing intermediate stage regression are linked below.Summary
Reuse the existing reference kernels/GEMM handle; the 1 MiB heuristic budget matches the actual Torch addmm boundary. A 32 MiB budget selects a different split-K in the merger: the intermediate exact-stage regression fails there, and the single budget control makes it exact. No change to
gemm.cu, language/LoRA execution, weights, calibration, corpus or gates.Add separately resolved optional ABI-9 symbols, retaining old-library loading and JEMM's 1152/head-72 guards. The original patch constructor still fixes 1152. Reject adapted/incorrect layout, bounded-cache accounting and late profile selection. Keep the existing trace events; add opt-in selected-block operator tracing and root tests/oracle generator. This builds on the native worker and reuse implementation from merged #129/#134, preserving their author attribution.
Test Plan
Base / head:
f1e674f2d324f5bb4e1ba2162551d0b633f20be0/6410e7634fd5e077000810ae07dc2f251b4f044d. 17 files; 730 changed authored-code lines, 859 total. No generated run dumps or matrices committed.Pinned artifacts: official code
14dbec4f71e194852c8d7b88ab36ef639493f400; checkpointffe5f436eaf22e20e2f041f8e74e121fd057a6cb; Qwen3.5-4B851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a; export manifest SHA2562bab1a7d6ac96859fa07af932f1bafb448acd4abe21a36d9b204e5e2a56c6dfe. Original BF16 unmerged reference, 248079 vocabulary, 602112 pixels, calibration and all original 28 requests/280 answers remain fixed.Hardware/runtime: one L20 GPU0 (sm_89), driver 580.178.04; Rust 1.100, nvcc 12.8, native cuBLASLt 120803 and cuDNN 92400. Actual native maps also load cuDNN's bundled cuBLASLt13. Official Python 3.10.12, Torch 2.14.0+cu130, torchvision 0.29.0+cu130, Transformers 5.17.0, PEFT 0.19.1, Pillow 12.3.0, no FLA. Full lock/library hashes are included. These stacks are not claimed library-identical. Cooperative GPU lock and occupancy checks were used; no performance campaign.
Test Result
--locked --all-targets --features omni-laya/serve -- -D warnings), ordinary workspace tests, workspace release build, test discovery and strict MkDocs pass. Ordinary tests: 427 passed, 122 ignored; skipped device tests are not counted as executed passes. CUDA backend/checker tests: 27 + 68 pass; the existing Laya ABI-declaration warning remains.0.5525against official0.5443(drift0.0082); all 40 visual boundaries, including all 24 blocks and merger, are bitwise exact on this case.Unrounded: 177/280 within 0.01, max
0.17586451768875122, same three protected failures; all raw results round back to production. The original strict full gate exits 1. The previous documented baseline (113/280 within 0.01, max 0.3066, four protected flips) remains separately retained; the JPEG failure set changes and does not uniformly improve.Remaining protected cases:
demo-jpg/board.jpg/q8margin 0.0397 (check first→safe);gate-jpg/arcade.jpg/long_startmargin 0.0935 (third→second);gate-jpg/web.jpg/one_tokenmargin 0.0746 (click OK→click Close). All 140 PNG decisions agree, but 20 PNG answers still exceed 0.01. In the original 12-questiondemo-png/video.png/q2Score request, confidence remains0.5613460540771484versus0.5786595344543457(raw drift0.017313480377197266) with the same harmless decision. Its full context/raw/control pointers are preserved for the next stage. Remaining errors are not attributed wholesale to JPEG or LoRA; JPEG input factors and language/selected-readout/merged-vs-unmerged residuals need separate controls.Demo / evidence
Numerical evidence archive, 485264 bytes, 197 inventoried files. SHA256
85e39a9d95762dec3cc34447ffb40d2e12562a0ae8d416810138ecc24fa786da. Contains the protocol, all original gate rows/raw numeric results, failures, source helpers, dependency lock, actual runtime/library hashes and reproduction commands. Screenshot bytes, embedded image requests, weights, credentials and private preparation discussions are omitted; complete replay inputs/BF16 tensors remain in the contributor's private archive. Maintained test commands and profile scope are documented in the recipe and root tests README.Video: N/A, numerical backend change. Performance/startup/memory improvements: unmeasured. Other GPUs/Metal, full 27B/Cua checkpoints and whole unmerged-language parity: unverified. No ready/merge request or issue-closing claim.
Merge Danger
Door: two-way. Revert the change or select
legacy; exports and request/response schema are unchanged.Blast Radius: Vision. New arithmetic is selected explicitly for unadapted OmniJev4B; shared optional bindings and the patch-constructor wrapper have device compatibility coverage. This does not qualify real 27B models or enable the new profile by default. Whole #114 acceptance remains blocked by the disclosed original-gate failures.
Self-review
Agent self-review/precheck completed against the full pinned diff; contributor/maintainer review remains separate. Kept draft. Human checklist is intentionally left for the contributor: