Repository navigation
Docs: refresh README quickstart, architecture and release guidance - #533
Conversation
Align the shared diagrams, paths and import guidance with the current package tree. Document the checkout-scoped knowledge grants and distinguish the released CLI from task entry on main. Agent-assisted-by: Codex (author) Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
zengchuang-hw
left a comment
There was a problem hiding this comment.
PASS
初次独立检视 1b826b5f → 23be5b72,逐项看了六处文档/图示变更与 #531 的修订:三个上游 Draft 标注仍保留,#10184 更新为已合入;Find 只说明当前注册的 Cannbot,MindIE 的 CLI/reader 单列。旧 import 路径的兼容边界、图示资源、当前架构链接、CLI 参数和授权路径也与源码对齐。无批准前必修项。
非阻塞:可把 v0.2.6 标为 preview,和发布页措辞一致。已静态核对源码、文档链接及 11 个上游 PR 状态;未执行 CLI、渲染图示或查询 CI。
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed base 1b826b5f9666a8b734000e1eb2990e582cdef876 and head 23be5b7267ccb3d45e05d623a830fb13912b5a57.
Independent third-party technical verify (cursor[bot]; collaborator permission none — not a write+ eligible Approver under docs/agent-collaboration.md; does not complete dual tip / merge-ready).
Verified against governing #531 slice and tip tree:
- README Results table:
#10184marked merged;#9698/#9745remain draft;#10243open — matches the PR claim of rechecked upstream statuses. - Architecture /
docs/contracts/current-design.md/ contributing: host inspection importer andharness/data/repo_packspath align withsrc/inferrsi/apps/host/inspection. - SVG assets remain well-formed; Ready tip green (full-check 3.10/3.11, declarations, installed-package, dashboard-browser, pr-references).
Peer tip Approve: zengchuang-hw. Author is Dong1017 → Dong ineligible to Approve; this is cursor first formal tip vote as independent verify.
Merge still needs write+ independent Approves / steward integration — not claimed here.
amy-why-3459
left a comment
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed base 1b826b5f9666a8b734000e1eb2990e582cdef876 and head 23be5b7267ccb3d45e05d623a830fb13912b5a57.
Docs-only refresh for #531. Verified:
- The v0.2.6 release exists (published 2026-11-20) and the README highlights match its notes (paired knowledge-on/off reports and service-reuse evaluation).
- Upstream statuses rechecked: vLLM-Omni #10184 is merged; #9698 and #9745 remain drafts; #10243 is open, matching the 2026-11-20 table.
apps/workflows/improvement.py,apps/host/inspection.py, andharness/data/repo_packspath updates match this checkout.skills enteris correctly described as introduced after v0.2.6.
Accuracy and performance comparisons are not applicable (documentation only). No outstanding CHANGES_REQUESTED on this head.
Non-blocking: the v0.2.6 bullet could note that the GitHub release is a preview, consistent with the release page.
e520d79
|
Thanks for the first-screen quickstart and the merge onto current main. Reviewed The merge conflict in One first-screen wording issue to fix before I can approve:
Accuracy and performance comparisons are not applicable (documentation only). No outstanding CHANGES_REQUESTED on this head. |
zengchuang-hw
left a comment
There was a problem hiding this comment.
NO PASS
增量检视 23be5b7267ccb3d45e05d623a830fb13912b5a57 → e520d79f385d266ee13e04283b0e9dadf725f6e4,本轮 base 6bdc86ee,承接上轮检视。之前对 Draft、MindIE 和 CANNBot Find 的区分仍成立;v0.2.6 能力表述和 quickstart 入口也已核对。
批准前必做:README 首屏新加的 “Every recorded run also produces a local evidence dashboard” 把独立的 dashboard 说成了 run 的默认产物。实际 run 返回的是 report.html(src/inferrsi/apps/cli/__init__.py:772–783),截图所示的 dashboard 由另一个命令打开或导出(同文件 :614–628)。建议改成 “Open a recorded run in the local evidence dashboard with inferrsi dashboard --input <run-directory>.” 将自动生成的报告与 dashboard 区分。
已核对六个文档/图示变更及相关 CLI 源码;未执行 CLI 或测试。
Use aligned result tables with explicit baselines, preserve validation limits, and link the benchmark and historical records. Refresh the README banner and collaboration guidance. Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com> Agent-assisted-by: Codex (author)
Agent-assisted-by: Codex (author) Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
|
Reviewed The previous first-screen wording issue is resolved. This README now tells readers to generate a dashboard from a recorded run directory with This increment is larger than that wording fix. Please address the following before I can approve:
Non-blocking:
Accuracy and performance comparisons are not applicable as new experiment evidence (documentation only); the points above are about how existing numbers are transcribed and curated. No outstanding CHANGES_REQUESTED on this head. |
zengchuang-hw
left a comment
There was a problem hiding this comment.
NO PASS
增量检视 e520d79f385d266ee13e04283b0e9dadf725f6e4 → c242ac5e7f7b3db09201a82021b58937049de4b2,承接上轮检视。已经明确 dashboard 是单独运行的命令,上一轮措辞问题已修复。
批准前必做:README 的新示例从已有运行目录执行 inferrsi dashboard --input . --output report.html。但普通 inferrsi run 已生成 report.html,而 apps/views/dashboard.py:383–411 的导出会拒绝覆盖非 dashboard HTML。请用 inferrsi dashboard --input . 默认输出 dashboard.html,或显式指定 --output dashboard.html,保留原报告。
已静态核对本轮 README/结果文档/图示差异、CLI 实际输出及 dashboard 覆盖保护;未运行 CLI 或测试。
- Add emoji to section headings and keep every previous anchor id so existing links into the README still resolve. - Merge the Baseline column into the measured-change column, taking the result tables from five columns to four for narrow screens, and shorten the validation cells. - LFM3-Omni-2B: report +108.1%, which is what the listed 0.285 -> 0.593 req/s endpoints give; the details page notes the upstream +108.4% comes from unrounded means. - DeepSeek W4A8 prefill: "Input throughput" instead of "Input throughout the run". - Restore the three below-20% vLLM-Omni campaigns to the details page in their own section, with their draft and merged labels intact, and stop describing the page as only showing 20%+ tables. - Point the six-run DeepSeek baseline at the comment that states 39.28 / 44.07 / 56.86 ms/token. - Dashboard example: use `inferrsi dashboard --input .` and its dashboard.html default, since `run` already writes report.html and the exporter refuses to overwrite a document it did not generate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
|
PR review verdict — DO NOT PASS Reviewed The four items from my previous comment are addressed:
The dashboard example now uses Please fix the following before I can approve:
Non-blocking: the PR description still describes the earlier Accuracy and performance comparisons are not applicable as new experiment evidence (documentation only). No outstanding CHANGES_REQUESTED on this head. |
- README: the legend says the fire mark is the largest gain in its table, so only the 4.22x row, the +158% row and the +381% row keep it. - optimization-results: the sentence about the LFM3 percentage cited "the rounded means above", which sat under the 950PR step-reduction table. Move it into the LFM3 row itself. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Dong1017 <55723162+Dong1017@users.noreply.github.com>
amy-why-3459
left a comment
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed 43d313545eb40a26d891a0720054963c78159784 → a777b007bbd4e4c9074c46f93c4091222c2e4c00 (PR base 6bdc86ee339c3007f15833434065adf893bff953).
Both items from my previous comment are resolved:
- The LFM3-Omni-2B rounding note (“Computed from the rounded means; upstream quotes +108.4% from its unrounded values.”) now sits in the LFM3 row of
docs/optimization-results.md(L61), and the glossary after the 950PR step table only defines TTFA / RTF / WER. README.mdnow has exactly one 🔥 per performance table that carries one, each on its largest gain: Image and video−76.3%(4.22×, larger than 3.77× / −73.45%), Speech+158%(Qwen3-TTS 1.7B, above +108.1% / +100.6%), and Standalone engines+381%. This matches the legend at L90.
The increment is a single docs-only commit (2 files, +5/−7); knowledge-model-policy is success on this head, and there are no open CHANGES_REQUESTED or inline threads on it. Accuracy and performance comparisons are not applicable (documentation only).
Non-blocking (unchanged from last time): please refresh the PR description, which still describes the earlier e520d79f README.
zengchuang-hw
left a comment
There was a problem hiding this comment.
NO PASS
增量检视 c242ac5e7f7b3db09201a82021b58937049de4b2 → a777b007bbd4e4c9074c46f93c4091222c2e4c00,承接上轮。Dashboard 示例已改为默认写 dashboard.html,不再撞上原报告。
批准前必做:README 新图例把 ✅ 定义为通过质量检查,却给 DeepSeek V4.1 Flash 三组 W4A8 吞吐复测加了同一标记。已核对吞吐数字,但 纯 P 原始证据明确保留长输出精度未验证,延迟门限内的吞吐复测不能作为整行质量通过。请移除这个 ✅,或按场景注明实际覆盖的质量范围。吞吐值不需要改。
已对照本轮 README、结果文档和原始证据;未运行 CLI 或复测。
|
Thanks @zengchuang-hw. Fixed in 13c1d4a. The DeepSeek V4.1 Flash W4A8 row under Text serving no longer carries ✅. Its validation cell now reads: "Throughput and latency rechecked at each scenario's latency limit; decode rechecked once. Long-output accuracy not verified." The three throughput figures, their baselines and the +54.45% / +36.12% / +20.80% conclusion are untouched. Your reading of the source matches mine: case19-pure-p.md still lists cross-restart, long-run and long-output accuracy as pending verification, so a latency-bounded throughput recheck cannot support the legend's "quality checks passed" meaning. I also re-audited every other ✅ in the README results tables. Each remaining ✅ in the performance tables names its output-level check (14/14 quality comparisons, 9 and 44 pixel-identical outputs, 30/30 numerical checks, bit-identical audio, audio and contract checks), so none of them rests on throughput-only evidence. The onboarding LFM3-Omni-2B ✅ covers the two real-weight serving tests named in the same cell. The PR description now describes head 13c1d4a, including the architecture link, and the branch was pushed without force. No other rows or numbers changed. @zengchuang-hw @amy-why-3459 could you take another look once CI is green? |
amy-why-3459
left a comment
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed a777b007bbd4e4c9074c46f93c4091222c2e4c00 → 13c1d4a37c684fa5c4b26f25442715228c6a3822 (PR base 6bdc86ee339c3007f15833434065adf893bff953).
The only change is the DeepSeek V4.1 Flash row under Text serving in README.md (L111): the ✅ is removed and the cell now scopes the evidence to throughput and latency rechecks, with “Long-output accuracy not verified.” This matches case19-pure-p.md, which says cross-restart, long-run and long-output accuracy are still pending, and it keeps the ✅ legend consistent. The +54.45% / +36.12% / +20.80% figures and baselines are unchanged.
The PR description now describes this head. knowledge-model-policy is success on 13c1d4a3, and no CHANGES_REQUESTED or other blocking review is open on this head. Accuracy and performance comparisons are not applicable (documentation only).
# Conflicts: # docs/architecture.md
e1d9065
zengchuang-hw
left a comment
There was a problem hiding this comment.
PASS
增量检视 13c1d4a37c684fa5c4b26f25442715228c6a3822 → e1d9065ff9881c43cfb478fa0a8aa235a018d4a1,承接上轮检视。本轮同步了已落地的 #535;以 fb343547f5c72155a8554038773866327da8d446 为基线,实际 PR 仍是 8 个文档和图示文件。合并保留了 Anthropic 集成与 artifact helpers 的新归属、README 对 DeepSeek 长输出精度的限定,以及默认输出 dashboard.html 的示例。
本轮没有批准前必做项。非阻塞:主干继承的架构表仍有一行旧 Model 职责,与下文的包迁移说明略重复,可以另行归并到 Apps。已静态核对精确 head、提交范围、实际差异和相关路径;未运行 CLI、实验或检查 CI。
amy-why-3459
left a comment
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed 13c1d4a37c684fa5c4b26f25442715228c6a3822 → e1d9065ff9881c43cfb478fa0a8aa235a018d4a1 (new PR base fb343547f5c72155a8554038773866327da8d446, i.e. main after #535).
The increment is only the merge of main (#535). Against the new base the PR still changes the same 8 documentation/diagram files; the patches for README.md, docs/optimization-results.md, docs/contracts/current-design.md, docs/contributing.md and the three SVGs are byte-identical to the previously approved 13c1d4a3 diff. The only resolved file is docs/architecture.md: the merge drops this PR's earlier Package table (which still listed Model/Infra packages) and keeps main's post-#535 table and text, while keeping this PR's intro, migration-list/import-gate links and the harness/data/repo_packs fix. Relative links and anchors in README.md and docs/architecture.md resolve on this head, and none of the PR's files reference the removed inferrsi.infra / inferrsi.model paths.
Accuracy and performance comparisons are not applicable (documentation only). No CHANGES_REQUESTED or other blocking review is open on this head; full-check was still running when I looked.
Non-blocking (inherited from main, not introduced here): docs/architecture.md L98 still has a Model row although #498/#535 removed that package, and its provider adapters are now listed under Apps; docs/contributing.md L103 still says "Harness/Model/Infra plane". Both could be folded into a follow-up.
Closes #531
What changed
docs/optimization-results.mdfor the full curated record. One 🔥 per table marks the largest comparable gain, and ✅ appears only where the linked evidence includes output-quality checks. The DeepSeek V4.1 Flash W4A8 row states its scope explicitly: throughput and latency were rechecked at each scenario's latency limit, while long-output accuracy remains unverified, so it carries no quality-pass mark.skills enterafter v0.2.6 and the current knowledge entry points.inferrsi dashboard --input .with the defaultdashboard.htmlso the dashboard never overwrites thereport.htmlwritten byrun; keep one 🔥 per performance table; and align every ✅ mark with the scope of its cited evidence.Verification
maininto this branch and resolved the one conflict indocs/architecture.md, keeping the currentapps/integrations/cannbot/layout. All eight files in the diff are documentation or diagram assets; no code changes.inferrsi --help,benchmarks list(returnsmoe-forward-npu,qwen21-denoise-npu,qwen3-tts-npu), the CPUdoctorcheck against a vLLM-Omni checkout, the READMEfindquery and bothcontextexports (five records each). Confirmed thecannbot,mindieandskills entercommand surfaces.benchmarks listoutput.docs/optimization-results.mdcarries those labels.src/inferrsi/onmain.The issue owner is submitting this revision for review and closure on merge. Verification covered documentation, CLI behavior and upstream PR metadata; performance and quality figures retain their original campaign evidence. GUI plugin installation was outside this run.