Skip to content

omnijev: add eager unmerged reference Score readout - #211

Draft
psv666 wants to merge 2 commits into
InferMatrixLab:mainfrom
psv666:codex/omnijev114-language-20261120
Draft

psv666 wants to merge 2 commits into
InferMatrixLab:mainfrom
psv666:codex/omnijev114-language-20261120

Conversation

@psv666

@psv666 psv666 commented Nov 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

OmniJev's fixed selected-readout path rejects unmerged adapters and FP32 GDN. Add an explicit eager one-shot Score library API that runs the existing normal full-row reference path and gathers selected final-norm hidden rows. The original full-context PNG q2 now meets the unchanged 0.01 answer gate. This is not an all-type worker or completion of #114.

explicit unmerged-reference/1 artifact
  original BF16 base + restored special rows + 400 FP32 LoRA tensors
Executor::load_one_shot_reference
  Model::load_with_lora / enable_reference_text_numerics
score_one_shot(complete prepared request, question index)
  normal forward_multimodal per row -> selected zq/u -> ordinal head -> calibrated answer

Stacked on draft #209 at 6410e7634fd5e077000810ae07dc2f251b4f044d; main remains f1e674f2d324f5bb4e1ba2162551d0b633f20be0. The new commit is 72af54fbca76fe34fc525cc4e5d49a7bbe89bd36, tree 17fc7ebb41ef696b7e0324e3679e467f194f288f. The incremental diff against #209 is 12 files / 571 changed lines, including 483 authored-code lines. The main-based PR also contains the unchanged #209 prerequisite; its authorship is retained.

The old merged export/worker/defaults remain. The new profile rejects Graphs, prefix reuse, whole-request execution, Choice/Noul and LM targets. The existing fixed/window guards remain intact. No CUDA implementation, ABI, ops.h or Cargo/lockfile changes.

Required human contributor answers

The contributor's explicit shared intent and necessary-draft decision are recorded in the preceding stage's human-answer section. They remain applicable to this bounded successor of RFC #114:

  1. Outcome: Advance reliable native model support to reproducible real-weight acceptance with complete numerical evidence.
  2. Highest-leverage variable: Trustworthy support coverage and numerical consistency before performance optimization.
  3. Why this moves it: Reuse the existing implementation and resolve the demonstrated reference/readout gap with a small native slice.
  4. Users and need: Model/application/agent developers need stable native interfaces, reliable output and reproducible deployment on the target hardware.

PR needed: Yes; the contributor authorized necessary implementation, L20 verification, self-review and a reviewable draft for each bounded stage. This PR is kept draft while all-type/full-corpus acceptance remains outstanding.

Evidence

Actual observed failure and result, preserving all 12 original questions, prefix 581, usage 1214, image and checkpoint identity:

Original demo-png / video.png / q2 Score Raw confidence Maximum raw answer-field delta from normative branch
Official BF16 unmerged original branch 0.5613460540771484 0
Previous native merged fixed path, reference vision 0.5786595344543457 0.017313480377197266
New native eager unmerged one-shot path 0.5640108585357666 0.002664804458618164

The baseline is freshly red twice; the candidate is repeated exactly at hidden/head boundaries. Maximum returned-field delta is 0.0027. The protected Score decision remains harmless (reference raw top-two margin 0.5176341654616531). Original 0.01 and protected-margin gates are unchanged.

The new selected zq/u match every BF16 value in a freshly run official unmerged plain-row control. That control is diagnostic; the normative original branch remains the gate oracle. Actual loaded 426 base, 400 FP32 adapter and 25 FP32 head tensors all match the new export's dtype/shape/bytes. With identical [581,2560] input, the existing reference base GEMM, LoRA A/B GEMMs and final scaled/cast projection match actual PEFT exactly, including [581,8192] output. The production ordinal head versus actual official same-hidden head differs by at most 5.960464477539063e-8; raw finishing is independently replayed and rounds back to the production result.

Frozen artifacts: official source 14dbec4f71e194852c8d7b88ab36ef639493f400, checkpoint ffe5f436eaf22e20e2f041f8e74e121fd057a6cb, base 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a. Python 3.10.12 / Torch 2.14.0+cu130 / Transformers 5.17.0 / PEFT 0.19.1, no FLA; 64-package hash lock is unchanged. One L20 GPU0, Rust 1.100 / native CUDA 12.8. This is numerical validation, with no performance claim.

Test plan and result

  • cargo fmt --all --check; workspace locked Clippy/all-targets/omni-laya/serve/-D warnings; ordinary workspace tests; locked release workspace build; strict MkDocs; test discovery; exporter AST: pass. Ordinary tests: 428 passed / 0 failed / 125 ignored. Ignored checks are not counted as device passes.
  • Named L20 tests: new reference_selected_rows_preserve_full_row_and_recover, new unmerged_reference_original_q2_gate, existing text_readout_equals_forward_fixed and existing OmniJev prefix_reuse_matches_plain_passes_bit_for_bit: pass. Selected rows cover order, duplicates, last-row equivalence, invalid rows/LM-target rejection, scratch growth, restoration and existing prefix/fixed guards.
  • Explicit reference_graph_is_rejected_before_loading with Graph requested: pass, before artifact/CUDA loading. A real old worker process rejects the unmerged format at startup with expected exit 1, before readiness.
  • 62 original asset/corpus hashes freshly pass. New export manifest SHA256: f5ffba7e53e3289b0019a90bcb487483f7bc33e122c2d57ceaafafe2cf5e33bf; immutable CUDA library SHA256: b1f08a2177ffe5aa5701c710eec3a9dd71c0efe43b1b6b12bc6182f2c0e061e5.

Unsupported/unrun: new-profile all-type/full-280 replay and HTTP, reference Graph/reuse, JPEG repair, other GPUs/Metal, whole real-weight Cua/Open-Jev models, and performance. The previous #209 full-280 gate remains failed; this stage does not relabel its old results as a fresh candidate run.

Demo / evidence and successor

Maintained reproduction and mode boundaries are in recipe/omnijev/README.md; all tests/helpers are under root tests/. No image/video demo is applicable to this library-only numerical slice. Original requests, model tensors, raw logs, exact source/library hashes and all failed controls are retained in the contributor's private evidence archive; they are not committed or redistributed here.

LM features need a separate slice: pinned mso/branch.py::branch_forward batches the original request's 35 rows in one chunk, producing [35,33,2560] suffix hidden. Its option-token projection input is [112,2560], then zq input [31,2560], with vocabulary 248079 and BF16 logits cast to FP32 before log-softmax. These shapes/order are derived from the exact original compiled rows and pinned source, not presented as an LM same-hidden measurement. Current Model::vocabulary_logprobs gathers up to 64 targets per call under the fixed-row-independence contract. Reusing that grouping with a reference handle has not been validated; the new API therefore rejects LM targets. The successor needs explicit grouping/ownership plus actual same-hidden projection/log-softmax and six-feature validation before all-type/full-280/HTTP acceptance. No new CUDA need is asserted by this stage.

Merge Danger

Door: two-way. The new artifact and library entry points are explicit, and existing worker defaults remain merged.

Blast Radius: bounded. Shared Qwen gains a guarded full-row readout API; OmniJev owns artifact/profile/Score orchestration. Existing default consumer regressions pass, but complete #114 acceptance and new-mode serving remain outstanding. Keep draft; do not close #114 or advance media/performance gates.

Self-review

Agent-assisted full incremental diff/architecture/numerical/lifetime/artifact review completed; it does not replace human or maintainer review. Human checklist remains for the contributor:

  • I have reviewed the full diff and addressed the issues I found.
  • I have checked architecture and scope.
  • I have reviewed the reported checks and unverified coverage.
  • I have checked the documentation and numerical claims against the evidence.

Align patch, LayerNorm, rotary, head-64 attention and vision GEMM boundaries with the pinned BF16 reference. Keep legacy/JEMM behavior and optional ABI compatibility. The original 280-answer whole-model precision gate still fails; this is the bounded vision stage for InferMatrixLab#114.

@hsliuustc0106 hsliuustc0106 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review draft — #211 "omnijev: add eager unmerged reference Score readout"

Verdict: COMMENTED. The PR is OPEN and explicitly DRAFT; this is a first review, not an approval or completed validation. The contributor's human checklist is deliberately unchecked and stays untouched.

Provenance and pins

  • Base: main @ f1e674f2d324f5bb4e1ba2162551d0b633f20be0 (matches GitHub baseRefOid and body).
  • Head: 72af54fbca76fe34fc525cc4e5d49a7bbe89bd36 (matches origin/pr/211); tree 17fc7ebb41ef696b7e0324e3679e467f194f288f — verified via git rev-parse 72af54fbc^{tree}.
  • Diff vs base: 25 files, +1389/−41 (GitHub and local agree). Incremental vs #209: 12 files, +548/−23 = 571 changed lines — matches the body exactly; "483 authored-code" is plausible from the numstat mix (code files dominate; docs are the remainder) and was not separately re-derived.
  • Author psv666, head repo psv666/system1-omni, branch codex/omnijev114-language-20261120.

Position in the series (cross-reference)

Second commit of the verified linear stack 209 ⊂ 211 ⊂ 212 (first-parent chain 6410e763 → 72af54fb from base f1e674f2; ancestry re-checked with git merge-base --is-ancestor). Not contained in #215 — see the stack note in the #209 body and the #215 review. Review order applied: #209 → #211 → #212 → #215.

Template gates

  • The four required contributor answers and the necessity answer are restated inline with a link to #209's section; the draft decision is explicit. Acceptable for a stacked successor; the answers are restated, not just referenced.
  • Checklist: four boxes, all unchecked — correct for draft.
  • Evidence pins: all verified (base, head, tree, #209 head pin 6410e7634…, incremental compare link, per-stage artifact hashes are author-reported pins consistent with the #209 set).

Findings

No P0/P1/P2 findings.

  • Score finishing path: Executor::score_one_shot (src/models/omnijev/native/src/executor.rs:265-307 at head) runs the normal full-row reference path per row (vision forward once, then one unpadded forward per option row via the shared row builder, which routes to forward_multimodal_reference_readout), gathers zq/u, applies Heads::ordinal (FP32 sequential theta accumulation, sigmoid CDF, differences, max(1e-8) floor — pre-existing production finishing, unchanged), then contract::answer with calibration and elapsed time including preparation_seconds. Vision and language are synchronized on both success and error paths before results are released.
  • Unmerged semantics: a separate artifact format omnijev-unmerged-reference/1 is loaded only by Executor::load_one_shot_reference → export::load_reference, which requires the manifest LoRA profile {rank 32, alpha 64, scale 2, float32, 400 tensors} and validates the complete per-layer module mapping and shapes before any device allocation (export.rs::validate_reference_adapter). The loaders are dual-strict: the merged loader rejects an export that contains adapter_model.safetensors; the reference loader rejects a merged manifest. The language model is loaded with Model::load_with_lora(..., 2.0) plus enable_reference_text_numerics() and requires fp32_gdn — FP32 LoRA/unmerged arithmetic as described.
  • Fail-closed surface: full-request execute/execute_rows, prefix reuse, Graphs (env checked before artifact/library access; reference_graph_is_rejected_before_loading proves rejection before loading anything) and LM targets (targets.is_empty() guard in forward_multimodal_reference_readout) are all rejected. Negative paths are tested (tests/omnijev/worker.rs::unmerged_reference_original_q2_gate asserts execute/execute_rows/out-of-range/nonscore rejections on the real request).
  • float_repr / CPython parity discipline (compensated-sum lesson): no new float-reduction claim is made beyond measurement. The body reports the head boundary measured against an official same-hidden control (max 5.960464477539063e-8) and raw finishing independently replayed to round back to the production result; the normative original branch remains the oracle and the plain-row control is explicitly labeled diagnostic. The CPU FP64 log-softmax approximation disclaimer appears in the successor (#212), and this PR's API returns no logprobs (logprobs: Vec::new()). Tolerances: the original unchanged 0.01 gate is hard-asserted in the test; the measured max returned-field delta is 0.0027 against official 0.5613 — measured-then-declared, no tolerance was relaxed.
  • No CUDA/ABI changes: the delta contains no .cu, ops.h or Cargo/lockfile changes (verified from the file list), matching the body claim.
  • Informational: score_one_shot is a one-Score-question library API by design (documented in body and recipe/omnijev/README.md); repetition determinism is asserted (zq/u/probabilities equal across two calls). The evidence (original request, oracle) is deliberately private and hash-pinned (a34c2d13… in the test), so these numbers cannot be re-derived by a reviewer without the contributor's archive — recorded as author-reported.

Evidence provenance

  • Source inspected: full incremental diff 6410e7634..72af54fbc (all 12 files); executor.rs, export.rs, model.rs (qwen) in depth; heads.rs::ordinal and contract finishing read at the head.
  • Commands run (personally): ancestry/tree pins; incremental diff stat; numstat reproduction; binary-file scan (none); docs/body cross-check of the measured numbers (0.5640 vs 0.5613, 0.0027 delta — consistent across body, recipe README, and test thresholds).
  • Observed CI (both repo hostnames, exact head_sha 72af54fbc…): CI success (44683034925) and Docs success (44683034946). Fork repo: 0 runs.
  • Author-reported: all L20 measurements (freshly-red baseline, same-hidden zq/u bitwise match vs plain-row control, 426/400/25 tensor byte checks, PEFT GEMM equality, 62 asset hashes, export manifest and library SHA256s).

Unverified checks / evidence gaps

  • GPU suites (unmerged_reference_original_q2_gate, reference_selected_rows_preserve_full_row_and_recover, and the existing text_readout_equals_forward_fixed / bit-for-bit prefix reuse tests) are author-reported; no GPU here. The Rust CPU weight-free suites could not run locally (no cargo); observed CI covers the workspace checks at this exact head.
  • All-type serving, full-280, JPEG, HTTP are explicitly unrun in this stage — the PR says so and stays draft. Nothing here should be read as closing #114.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[RFC]: Native Rust/CUDA support for OmniJev-4B v1.1 and community help

2 participants