Repository navigation
ci: verify Fetch installations and clarify benchmark usage - #13
Conversation
lanyue-llk
changed the title
|
🔍 OpenCodeReview found 1 issue(s) in this PR.
ℹ️ All inline comments overlapped with existing reviews; nothing new was posted. |
![lexmount-ocr[bot]](https://avatars.githubusercontent.com/u/277637559?s=60&v=4){kind=small}
lanyue-llk
left a comment
lanyue-llk
left a comment
There was a problem hiding this comment.
Reviewed commit 6e945f3.
The change consolidates maintenance and packaging while preserving Fetch/Search distributions, imports, scoring and frozen editions. That separation is appropriate; I found no newly introduced report-instance coupling or need for an additional class hierarchy.
Two existing findings were independently reproduced:
- [P2] Makefile interpreter selection: both isolated wheel-test environments select Python 3.11 from .python-version, including in the other CI matrix jobs. Pass the matrix interpreter explicitly to both uv venv commands.
- [P2] Answer token boundaries: with answer_match_rule=published-answers/2, reference 150 and delivered text id150 receive factual-evidence credit; public evaluate()/task_acceptance() also accept it. A standalone 150 is a positive control, while 150% is rejected. This defect is inherited from the imported scorer: its provenance hash is unchanged. Fix the shared boundary rule with paired accepted/rejected cases through the scoring-revision process, preserving historical identities. The default frozen factual-v1 tasks do not select this matcher.
Validation: make check and make package-check passed. Both source and independently installed wheels passed Fetch's 2,579 tests plus 321 subtests and Search's 92 tests plus 2 subtests. All 24 imported source/definition hashes matched provenance. Positive/negative CLI examples, Search subdirectory setup, and wheel license notices also passed. No further actionable defect was confirmed. The CI interpreter gap should be corrected before relying on the advertised multi-version wheel coverage.
lanyue-llk
left a comment
lanyue-llk
left a comment
There was a problem hiding this comment.
Re-reviewed 45772fc, including the full PR and changes since the previous review. One new P2 scoring regression is described inline.
The earlier interpreter-selection and numeric-start findings are fixed, including Apollo _11 and the decimal/negative fixed-prefix examples. The new provenance receipt preserves the imported identities and records unreleased corrections; the frozen datasets and release catalog remain unchanged. No report-instance coupling or unnecessary abstraction was found.
Validation passed: make check; source-archive-to-wheel installation and full tests with PACKAGE_PYTHON=3.12 (both isolated environments actually used 3.12.10). Fetch: 2,579 tests plus 321 subtests; Search: 100 tests plus 2 subtests in both source and installed-wheel runs. GitHub's Python 3.11–3.14 CI jobs also passed. These tests miss the internal quantity-operator scenario below; add paired public-scorer and task-acceptance controls for it.
lanyue-llk
changed the title
There was a problem hiding this comment.
This review found 1 issue(s), including 1 blocking issue(s).
lanyue-llk
changed the title
lanyue-llk
left a comment
lanyue-llk
left a comment
There was a problem hiding this comment.
Reviewed all six changed files at a352e2a. No Fetch runtime, CI or packaging regression was found. Source tests, builds and installed-wheel tests now consistently use BENCH_PYTHON. Python 3.12.10 make check and make package-check both passed: 2,579 tests and 321 subtests, plus catalog validation. Accepted/rejected README examples passed, and the catalog remains 2.0.1 with 550 entries. Current Python 3.11–3.14 CI and OCR statuses passed.
One cross-repository documentation correction is needed in docs/versions.md:37: the claim that both repositories use bench-vMAJOR.MINOR.PATCH for new editions is not supported by Search's release APIs, which generate and validate bench-MAJOR.MINOR.PATCH even for new versions. The generator/validator reproduction and minimal documentation fix are recorded in the companion Search PR review: lexmount/lexbench-websearch#3 (review). Keep Fetch's existing convention and describe Search's supported convention separately. No version bump or historical catalog change is needed.
There was a problem hiding this comment.
ℹ️ Review skipped · a02704f
No reviewable content was selected for this run.
There was a problem hiding this comment.
This review found 1 issue(s), including 1 blocking issue(s).
Summary
Clarify Web Fetch task suites, runnable examples, version identities and related-project attribution in aligned English/Chinese READMEs. Keep this repository independent from Web Search, with one Fetch package and its existing frozen catalog.
Make the selected Python interpreter explicit for source tests, builds and isolated wheel tests. An interpreter chosen only during uv sync can otherwise be replaced by .python-version on later commands. Use one Make entry for installed-package verification, and run PR CI once instead of duplicating branch-push and PR runs.
Package versions, benchmark editions, existing tags and historical evidence are unchanged. No Search source or family workspace remains in the diff.
Python CI runs only on 3.11. Existing test and package-validation steps remain unchanged.
Validation