Separate factual and style failures in write-naturally evals #3

Problem

The writing evals can penalize factual omissions twice, as both fidelity and readability failures. Some exact-format assertions also reject readable outputs that preserve the required behavior.

The public suite in PR #2 demonstrates this:

• D7's candidate is readable, but readable-on-first-pass fails because it omits max_completion_tokens. That omission belongs to factual fidelity.
• D7 checks the entire formula code block with contains. Indenting the same formula inside a numbered list changes the match without changing its terms or order.
• C1 has a hard two-question cap. Check that this limit reflects the requested reader experience rather than treating every threshold failure as a demonstrated writing defect.

The Humanizer cases already have separate style criteria. The candidate/source boundary was also corrected after C2 and H4 graders quoted source-only wording; the full suite still needs a rerun after that correction.

Proposed work

• Separate factual and style criteria where the shared fallback still mixes them.
• Check D7's displayed formula terms and order while allowing harmless Markdown indentation. Keep missing or altered terms as failures.
• Review hard formatting and count thresholds against the request supplied to the writer.
• Calibrate with clear-but-incomplete outputs, factually complete-but-unreadable outputs, indented formulas, and formulas with changed or missing terms.

Acceptance

• A readable output missing a required field fails fidelity and passes readability.
• An unreadable output retaining all required facts fails readability.
• Harmless indentation preserves a formula pass; changed terms or order fail.
• Known-good and deliberately faulty controls receive the expected grades before rerunning the broader suite.

Writer: gpt-6.1-sol; grader: gpt-5.6-luna; reasoning: medium. Use fictional public fixtures for reproduction.

Additional coverage lives in the private eval repository.