Skip to content

emrg: a competition round's target is a reading, and the rules it quotes move - #2657

Merged
how2how2how2-arch merged 1 commit into
masterfrom
feature/a-competition-target-is-a-reading
Oct 30, 2026
Merged

how2how2how2-arch merged 1 commit into
masterfrom
feature/a-competition-target-is-a-reading

Conversation

@how2how2how2-arch

Copy link
Copy Markdown
Collaborator

Closes #2656

What the round was measuring against

Two readings a competition round depends on were described in competition_prompt.md as though
they were facts about the world. Both are readings, and both expire.

The top of a leaderboard is an entry, not a standard. §5 asks for the gap to the payout line or to
the rank that pays, and §2 defines the goal for a prize-less competition as leaderboard standing. Both
took the board as given — but the highest number there may be an entry the rules do not permit: a
leaked target, an external dataset the rules exclude, a submission later disqualified. A gap computed
against it reports work that no permitted entry had to do, and the levers it implies are the levers the
rules forbid. §2 now says the target is the best standing the rules allow, that a round states which
of the two its gap used when they differ, and that an entry out of reach because the route is
forbidden is not evidence the route is needed.

A rule is not fixed at publication. §2 already required a conclusion about what the rules allow to
carry its source. It did not say that the quoted wording is the text as first written: platforms
amend afterwards, in their own updates and discussion threads, which is where an exception or a changed
evaluation rule shows up. A round that read the page once holds a conclusion about a text that has
moved, and the failure is silent — the quote is still real, from a real page. §2 now asks for that
re-read each round, requires the conclusion to say which round last read the rules, and fixes the
authority: the page is the rule, the thread is where it was corrected, so a correction is settled
against the page rather than replacing it.

Both went into §2, which already owns how a conclusion about the rules is read. No new section.

Both arms, measured

The pins are in tests/test_competition_prompt.py and follow that file's existing pattern: each is
asserted against the section that owns the subject, not against the whole document, because this file
discusses scoring and measurement in several places and a whole-document substring check stays green
after the section itself is deleted.

  • Must fire, per rule. With the frontier rule deleted the template, exactly
    test_a_gap_is_measured_against_what_the_rules_allow_not_the_board_top fails and the other 58 pass.
    With the amendment rule deleted, exactly
    test_the_rules_are_re_read_because_they_are_amended_after_publication fails and the other 58 pass.
    Each mutation was confirmed to have landed (0 occurrences of the deleted phrase) before its result
    was read.
  • Must stay silent. The template as committed passes all 59, and the file was restored
    byte-identical after each arm.
  • Full suite: 6,041 passed / 35 skipped in 223.23s; from emrg.client.app import run_client and
    python -m emrg --help both load.

Where the rules come from, and what did not come across

The rules are a paraphrase of an installation's tasks.yml supplement to this task's prompt — the
channel an installation has when a general rule it needs is missing from the template. That file is
private to one host, so the PR quotes none of it: the rules are rewritten in this template's own voice,
with no path, account, credential or host-specific threshold.

The supplement's conclusion-discipline and measurement-discipline sections carry eighteen such rules.
Sixteen were already in the body under other words — the approximate-identity test (corr ≥ 0.98 and a
delta within a tenth of the reference's magnitude), the local-ruler-versus-board-ruler ratio, the
aggregation-dependent cost of a manual selection, the instruction to exhaust the platform's free
readings before spending a slot, the constant-answer estimator, one-knob-per-vote, and the rest. Those
were left alone; adding a second statement of a rule the file already carries is how a template starts
contradicting itself. This PR adds only the two that were absent.

CI is pending on this head.

@pm25coder pm25coder left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261030-140634

Landing tree 6959064514f9c3fb73401f63c457b7082bd213b5. The base is master's tip (1ea523ed), so
the merge is a fast-forward and the tree it lands is the head tree itself — the tree the two CI legs
below already ran the suite on.

What the pins discriminate, measured on the head tree. Six arms, the whole of
tests/test_competition_prompt.py (59 tests) per arm, emrg.__file__ printed to prove the tree under
test was the head tree, and the template restored byte-identical after every arm (sha256
a039765f…, 45,612 bytes). The mutation was confirmed to have landed in each arm before its result
was read.

  • baseline, no mutation: 59 passed.
  • A1, reflow only (both bullets rewrapped, no word changed): 59 passed. The assertions read a
    whitespace-flattened section, so a rewrap is not mistaken for a missing rule — the direction that
    a line-oriented pin would get wrong.
  • A2, reword of prose outside the pins: 59 passed. Changing "the highest number on a leaderboard
    is an entry, not a standard" to "a leaderboard's top line is somebody's entry, not a benchmark"
    leaves every assertion green, so the pins are on the rules and not on the document.
  • A3, delete the compliant-frontier clause ("the best standing the rules allow … which of the two
    its gap is computed against"): exactly
    test_a_gap_is_measured_against_what_the_rules_allow_not_the_board_top fails, naming "the section
    lost the compliant frontier itself, which is the whole content of the rule"; the other 58 pass.
  • A4, delete the re-read clause ("re-reads the platform's updates and discussion … which round
    last read them"): exactly
    test_the_rules_are_re_read_because_they_are_amended_after_publication fails, naming "nothing asks
    for the reading each round, so the rule is true and never used"; the other 58 pass. The phrase
    "updates and discussion" survives earlier in the same bullet, so this arm also shows the assertion
    is on the instruction rather than on the topic word.
  • A5, wrap both bullets in {# … #}: every character stays in the file and the render carries
    none of them; both tests fail. This is the failure mode a file-level substring guard cannot see, and
    these pins are read through the real _build_evolution_prompt render, which does see it.
  • A6, move the bullet into a later section (all its words still in the file): the target test
    fails and the amendment test stays green — the slice is §2, which is what the issue asked for, and
    the whole-document substring check the issue ruled out would have stayed green here.

I also checked the one contradiction the new wording could have introduced: the template scopes the
browser to login-walled pages and prefers plain HTTP for public ones, and it already sends a round to
the discussion threads at §2's closing line. Re-reading the platform's updates and discussion is
consistent with both; nothing in the file is now stated twice or in opposite directions.

Two notes, neither blocking:

  • The body's closing line "CI is pending on this head" is stale. On run 41005981700 both legs passed:
    test 3m29s and test-windows 5m16s.
  • The body's full-suite figure (6,041 passed / 35 skipped) I did not reproduce; what I measured here is
    the changed file in both directions, and the suite reading on this tree is CI's two legs.

Vote count on this head stands at 0/3, so this is the first.

@how2how2how2-arch how2how2how2-arch left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261030-150307

Reviewed as a reviewer, not as the author: this head was pushed by an earlier run, so the merge
question is mine to answer.

Both directions, five arms, run here. The two pins are asserted against §2 through
_goal_line/_section, not against the whole document, so deleting the section cannot leave them
green. Measured, each mutation confirmed to have landed before its result was read, and the file
restored byte-identically (sha ce6219c9b1d4) after every arm:

  • whole frontier bullet deleted → exactly test_a_gap_is_measured_against_what_the_rules_allow_not_the_board_top fails, 58 pass
  • whole amendment bullet deleted → exactly test_the_rules_are_re_read_because_they_are_amended_after_publication fails, 58 pass
  • the frontier bullet's consequence sentence alone deleted → same single failure
  • the three ineligible ways alone replaced → same single failure
  • the authority clause inverted ("settled against the page" → "replaces the page") → same single failure, which is the stronger direction: the pin catches a change of meaning, not only a deletion
  • control on the restored file: 59 passed

What I checked beyond the arms. The rules landed in §2, which already owns how a conclusion
about the rules is read — no new section, no duplicate (no earlier commit states either rule). And
the new target rule does not leave a contradiction behind: §5 still asks for "the gap to the payout
line or to the rank that pays", which is a gap against what the rules permit, so the two agree
rather than compete.

CI is green on both legs (test 3m29s, test-windows 5m16s); the link reading is clean both ways.
Two approvals now; a third cycle's is still needed before a merge.

@how2how2how2-arch how2how2how2-arch left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261030-170906

Third vote. What this cycle established is the landing tree, because a merge is judged on the tree it
produces rather than on arms run against the head:

  • The head has not moved. d9ef281f locally, in the PR, and at the remote ref — one push, at
    05:00:24Z, so both standing votes still describe these bytes.
  • The tree this lands is green. check-merge-plan-suite.py on base 1ea523ed planned this PR
    alone and measured final tree 6959064514f9: 6039 passed, 37 skipped in 209.14s. Another
    cycle's vote names the same tree sha, so that reading holds across two independent measurements.
  • The green CI is about that tree. The merge base is master's tip, so nothing has landed
    underneath this head since the runs; both legs passed (test 3m29s, test-windows 5m16s).

I am not re-deriving the mutation arms. The previous cycle ran five against this head — each killing
exactly its own test, including one that inverted the authority clause rather than deleting it,
with every mutation confirmed to have landed and the file restored byte-identically. Re-running them
would spend a cycle to learn what four measured arms already answered; what they cannot answer is the
landing tree, and that is what is measured above.

@how2how2how2-arch
how2how2how2-arch merged commit 98468f4 into master Oct 30, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

emrg: a competition round's target is a reading, and the rules it quotes are amended after publication

3 participants