Repository navigation
test: 增加 Deep Research 取数基准与可核验性能证据 - #4
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
问题与结果
此前“数据获取 9.6s → 5.2s、降低 46%”缺少原始记录、版本和计时边界。本 PR 为 Trail Deep Research 补充真实接口性能基准及长期可核验的证据,给出可复现的新数据,不追认原数字。
在 Windows / Bun 1.4.2 / Longbridge CLI 0.30.0 上,以同一
ResearchRunner的串行 Executor 闸门作为实验对照,和现有最多 4 路 worker 并发进行 30 对测量:60 次正式运行、220 次能力调用全部成功;另有 6 次预热,不入统计。30 对规范化数据哈希均一致。70.4% 按未舍入的两组中位数计算,不是逐对降幅均值。
改动
scripts/bench/:真实公开数据取数基准、独立离线审计、额外实时引用映射验证,以及复现说明。保存协议、环境、源码哈希、逐次轨迹和数据快照;禁止私人账户/交易能力,保留所有失败尝试。docs/benchmarks/:证据报告、简历候选表述及 15 个面试追问,明确计时、对照、个人贡献和推断边界。docs/benchmarks/evidence/2026-10-19/:持久保存原始 320 KB ZIP、30 对 CSV、66 次含预热运行日志、汇总/审计、预检失败及正确性记录。ZIP SHA-256 为911cad12785451b02016d7c9d4adaf3f44471a421702b6ccd42c74ee5e2ae437,测量数据和原计时源码保持不变。完整证据索引与离线核验步骤
可复现测试报告
环境:Windows 10.0.26100 / AMD Ryzen 9 7945HX / Bun 1.4.2;真实接口测量基线
324dc4ed958f73a0637a68e2e3d2334a6d7d1ba9。本地桌面负载未隔离,CLI/远端缓存状态不可控。8 项跳过:7 项依赖未纳入仓库的私人账户 fixtures;1 项因本机 Windows 无符号链接权限而跳过。全套本地测试采用显式源码目录,避免
artifacts/中两份解包副本重复计入;未省略任何仓库测试目录。此前测量阶段的 34 项定向检查保留为历史记录,新增归档回归测试后,以此处的 6 项/全套结果为准。原 ZIP 已解包校验 60 个逐文件哈希,0 不一致,并对解包后的数据独立审计通过。之后将 ZIP 字节哈希和 30 对结果加入离线回归测试。
范围与已知限制
fetching → synthesizing的数据获取阶段,使用单调时钟;不包含真实 LLM 综合、模型规划、UI 或持久化恢复写盘。后续综合使用已有本地规则,模型调用为零。market.intraday报item.price.toFixed is not a function,research.events返回Longbridge request failed。原始失败保留,未在本 PR 修复;正式结论只覆盖上述三个固定任务。