[lanyue-llk]

Problem

A task can require records, facts and resources together. On current main, an answer with valid records can pass despite missing a required fact or resource. A perfect tree match can also replace another failed requirement.

Change

The shared scorer combines every declared requirement across body, structure, resource and file outputs. Undeclared checks are skipped; missing required evidence cannot pass. Title, comment and post checks appear in the same result inventory. Optional literal and length diagnostics no longer reject an otherwise complete answer merely because its output kind is body.

flowchart LR
    A[Task requirements] --> B[Shared scorer]
    C[Answer and delivered files] --> B
    B --> D[Facts, records, body, patterns, resources and files]
    D --> E{Every declared check passes?}
    E -->|Yes| F[Pass]
    E -->|No| G[Fail or missing required evidence]

Loading

The branch is rebased on main, including merged #2, #5, #7, #10, #11 and #13. Existing task files, frozen references, release bindings and tags are unchanged. version identifies a changed scorer as unreleased and returns the published catalog separately; strict release verification still rejects a scorer mismatch.

The task-authoring guidance from #4 is retained in CONTRIBUTING without rewriting 385 frozen prompts. Site-specific main-content reference revisions are outside this scorer change: the scorer enforces the requirements that a task actually declares. It does not establish complete coverage for every existing page.

Compatibility

Required facts and resources now affect acceptance even alongside a passing record contract. Recompute the complete affected saved cohort before using this scorer in a report; do not relabel old scores with the new scorer. New task scope or prompts need a separate input revision and the corresponding benchmark release. This PR does not publish a release or update reports.

Validation

• Same-answer comparison against main: complete answers pass in both; missing facts/resources incorrectly pass on main and fail here; missing records fail in both.
• Added accepted/rejected scenarios across four output kinds for disabled text diagnostics, plus paired executable regression controls.
• Tree-version tests now satisfy the independent fact requirement, so they isolate changes to tree acceptance.
• Public version-command tests distinguish released and unreleased scorers and retain strict mismatch rejection.
• make check: 2,770 tests and 321 subtests passed; all five suites validated, 550 tasks.
• make package-check: source archive to wheel, clean installation, the same 2,770 tests and 321 subtests, and suite validation passed.

[lanyue-llk]

Reviewed all 16 changed files at a1c4413f34aea6da059244685f6e617064127751, including the shared scorer, task predicates, contract validation, version commands, regression controls and documentation. No actionable defect introduced by this PR was confirmed.

The change fixes acceptance composition: satisfying a record or tree requirement can no longer substitute for other declared requirements. I independently compared identical answers against main (5fd55e3) and this head through evaluate_response:

Scenario	Main	PR
Complete records, fact and resource	Pass	Pass
Required fact missing	Pass	Fail
Required resource missing	Pass	Fail
Required record missing	Fail	Fail
Perfect tree match, required fact missing	Pass	Fail

The implementation extends the existing shared scorer and keeps requirement selection in one helper. This is a suitable abstraction for the current scope; introducing a class hierarchy would add complexity without addressing a demonstrated extension need. The CLI remains thin, strict catalog verification still rejects scorer mismatches, and the frozen task definitions and release bindings are unchanged. Historical metric handling and disabled-diagnostic scenarios were also checked. All registered file tasks explicitly disable text scoring, so the new default length check does not affect those tasks.

Validation completed independently:

• make check: 2,770 tests and 321 subtests passed; all five suites validated, 550 tasks.
• make package-check: source archive built into a wheel, clean installation, the same tests and suite validations passed.
• git diff --check: passed.

The compatibility guidance is appropriate: changed acceptance results require a complete rescore of the affected saved cohort before report use.