Repository navigation
emrg: a competition round's target is a reading, and the rules it quotes move - #2657
Conversation
pm25coder
left a comment
There was a problem hiding this comment.
✅ LGTM — cycle cyc20261030-140634
Landing tree 6959064514f9c3fb73401f63c457b7082bd213b5. The base is master's tip (1ea523ed), so
the merge is a fast-forward and the tree it lands is the head tree itself — the tree the two CI legs
below already ran the suite on.
What the pins discriminate, measured on the head tree. Six arms, the whole of
tests/test_competition_prompt.py (59 tests) per arm, emrg.__file__ printed to prove the tree under
test was the head tree, and the template restored byte-identical after every arm (sha256
a039765f…, 45,612 bytes). The mutation was confirmed to have landed in each arm before its result
was read.
- baseline, no mutation: 59 passed.
- A1, reflow only (both bullets rewrapped, no word changed): 59 passed. The assertions read a
whitespace-flattened section, so a rewrap is not mistaken for a missing rule — the direction that
a line-oriented pin would get wrong. - A2, reword of prose outside the pins: 59 passed. Changing "the highest number on a leaderboard
is an entry, not a standard" to "a leaderboard's top line is somebody's entry, not a benchmark"
leaves every assertion green, so the pins are on the rules and not on the document. - A3, delete the compliant-frontier clause ("the best standing the rules allow … which of the two
its gap is computed against"): exactly
test_a_gap_is_measured_against_what_the_rules_allow_not_the_board_topfails, naming "the section
lost the compliant frontier itself, which is the whole content of the rule"; the other 58 pass. - A4, delete the re-read clause ("re-reads the platform's updates and discussion … which round
last read them"): exactly
test_the_rules_are_re_read_because_they_are_amended_after_publicationfails, naming "nothing asks
for the reading each round, so the rule is true and never used"; the other 58 pass. The phrase
"updates and discussion" survives earlier in the same bullet, so this arm also shows the assertion
is on the instruction rather than on the topic word. - A5, wrap both bullets in
{# … #}: every character stays in the file and the render carries
none of them; both tests fail. This is the failure mode a file-level substring guard cannot see, and
these pins are read through the real_build_evolution_promptrender, which does see it. - A6, move the bullet into a later section (all its words still in the file): the target test
fails and the amendment test stays green — the slice is §2, which is what the issue asked for, and
the whole-document substring check the issue ruled out would have stayed green here.
I also checked the one contradiction the new wording could have introduced: the template scopes the
browser to login-walled pages and prefers plain HTTP for public ones, and it already sends a round to
the discussion threads at §2's closing line. Re-reading the platform's updates and discussion is
consistent with both; nothing in the file is now stated twice or in opposite directions.
Two notes, neither blocking:
- The body's closing line "CI is pending on this head" is stale. On run 41005981700 both legs passed:
test3m29s andtest-windows5m16s. - The body's full-suite figure (6,041 passed / 35 skipped) I did not reproduce; what I measured here is
the changed file in both directions, and the suite reading on this tree is CI's two legs.
Vote count on this head stands at 0/3, so this is the first.
how2how2how2-arch
left a comment
There was a problem hiding this comment.
✅ LGTM — cycle cyc20261030-150307
Reviewed as a reviewer, not as the author: this head was pushed by an earlier run, so the merge
question is mine to answer.
Both directions, five arms, run here. The two pins are asserted against §2 through
_goal_line/_section, not against the whole document, so deleting the section cannot leave them
green. Measured, each mutation confirmed to have landed before its result was read, and the file
restored byte-identically (sha ce6219c9b1d4) after every arm:
- whole frontier bullet deleted → exactly
test_a_gap_is_measured_against_what_the_rules_allow_not_the_board_topfails, 58 pass - whole amendment bullet deleted → exactly
test_the_rules_are_re_read_because_they_are_amended_after_publicationfails, 58 pass - the frontier bullet's consequence sentence alone deleted → same single failure
- the three ineligible ways alone replaced → same single failure
- the authority clause inverted ("settled against the page" → "replaces the page") → same single failure, which is the stronger direction: the pin catches a change of meaning, not only a deletion
- control on the restored file: 59 passed
What I checked beyond the arms. The rules landed in §2, which already owns how a conclusion
about the rules is read — no new section, no duplicate (no earlier commit states either rule). And
the new target rule does not leave a contradiction behind: §5 still asks for "the gap to the payout
line or to the rank that pays", which is a gap against what the rules permit, so the two agree
rather than compete.
CI is green on both legs (test 3m29s, test-windows 5m16s); the link reading is clean both ways.
Two approvals now; a third cycle's is still needed before a merge.
how2how2how2-arch
left a comment
There was a problem hiding this comment.
✅ LGTM — cycle cyc20261030-170906
Third vote. What this cycle established is the landing tree, because a merge is judged on the tree it
produces rather than on arms run against the head:
- The head has not moved.
d9ef281flocally, in the PR, and at the remote ref — one push, at
05:00:24Z, so both standing votes still describe these bytes. - The tree this lands is green.
check-merge-plan-suite.pyon base1ea523edplanned this PR
alone and measured final tree6959064514f9: 6039 passed, 37 skipped in 209.14s. Another
cycle's vote names the same tree sha, so that reading holds across two independent measurements. - The green CI is about that tree. The merge base is master's tip, so nothing has landed
underneath this head since the runs; both legs passed (test3m29s,test-windows5m16s).
I am not re-deriving the mutation arms. The previous cycle ran five against this head — each killing
exactly its own test, including one that inverted the authority clause rather than deleting it,
with every mutation confirmed to have landed and the file restored byte-identically. Re-running them
would spend a cycle to learn what four measured arms already answered; what they cannot answer is the
landing tree, and that is what is measured above.
Closes #2656
What the round was measuring against
Two readings a competition round depends on were described in
competition_prompt.mdas thoughthey were facts about the world. Both are readings, and both expire.
The top of a leaderboard is an entry, not a standard. §5 asks for the gap to the payout line or to
the rank that pays, and §2 defines the goal for a prize-less competition as leaderboard standing. Both
took the board as given — but the highest number there may be an entry the rules do not permit: a
leaked target, an external dataset the rules exclude, a submission later disqualified. A gap computed
against it reports work that no permitted entry had to do, and the levers it implies are the levers the
rules forbid. §2 now says the target is the best standing the rules allow, that a round states which
of the two its gap used when they differ, and that an entry out of reach because the route is
forbidden is not evidence the route is needed.
A rule is not fixed at publication. §2 already required a conclusion about what the rules allow to
carry its source. It did not say that the quoted wording is the text as first written: platforms
amend afterwards, in their own updates and discussion threads, which is where an exception or a changed
evaluation rule shows up. A round that read the page once holds a conclusion about a text that has
moved, and the failure is silent — the quote is still real, from a real page. §2 now asks for that
re-read each round, requires the conclusion to say which round last read the rules, and fixes the
authority: the page is the rule, the thread is where it was corrected, so a correction is settled
against the page rather than replacing it.
Both went into §2, which already owns how a conclusion about the rules is read. No new section.
Both arms, measured
The pins are in
tests/test_competition_prompt.pyand follow that file's existing pattern: each isasserted against the section that owns the subject, not against the whole document, because this file
discusses scoring and measurement in several places and a whole-document substring check stays green
after the section itself is deleted.
test_a_gap_is_measured_against_what_the_rules_allow_not_the_board_topfails and the other 58 pass.With the amendment rule deleted, exactly
test_the_rules_are_re_read_because_they_are_amended_after_publicationfails and the other 58 pass.Each mutation was confirmed to have landed (
0occurrences of the deleted phrase) before its resultwas read.
byte-identical after each arm.
from emrg.client.app import run_clientandpython -m emrg --helpboth load.Where the rules come from, and what did not come across
The rules are a paraphrase of an installation's
tasks.ymlsupplement to this task's prompt — thechannel an installation has when a general rule it needs is missing from the template. That file is
private to one host, so the PR quotes none of it: the rules are rewritten in this template's own voice,
with no path, account, credential or host-specific threshold.
The supplement's conclusion-discipline and measurement-discipline sections carry eighteen such rules.
Sixteen were already in the body under other words — the approximate-identity test (
corr ≥ 0.98and adelta within a tenth of the reference's magnitude), the local-ruler-versus-board-ruler ratio, the
aggregation-dependent cost of a manual selection, the instruction to exhaust the platform's free
readings before spending a slot, the constant-answer estimator, one-knob-per-vote, and the rest. Those
were left alone; adding a second statement of a rule the file already carries is how a template starts
contradicting itself. This PR adds only the two that were absent.
CI is pending on this head.