Skip to content

Print every T1 round's decision and cover the trigger write path (#3253) - #3254

Merged
Birfy merged 1 commit into
ThinkFlowLab:mainfrom
HugoXK:dev/5-aq-t1-write-tests
Nov 20, 2026
Merged

Birfy merged 1 commit into
ThinkFlowLab:mainfrom
HugoXK:dev/5-aq-t1-write-tests

Conversation

@HugoXK

@HugoXK HugoXK commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #3253

Owner: Dev-5
Reviewer: Reviewer
Tier: T3 (.github/scripts/**, from risk-gate.mjs)
Claim-Id: d5cd33f4f8c14080b9611d209cd0d942
Bot: Dev-5

What changed

file change
.github/scripts/auto-queue-t1.test.mjs six new tests: the write path hermetically, the stdout parity, and the dead assertion replaced
.github/scripts/auto-queue-t1.mjs the round's summary now also goes to stdout, unconditionally; one comment recording why SELF_PATHS does not list the test file

The five write-path cases run the real CLI end to end in execute mode. A stand-in gh first on PATH
serves the reads (pulls?state=open, the files, the check runs, the queue query and the head re-read)
from the environment and records every argv to a log file: a process is spawned, but nothing reaches
the network, needs a token, or writes into the checkout. act() stays private — the tests drive
main(), which is the code CI would run, not a copy of one function out of it.

The five cases, one per test:

  • execute mode: exactly two writes — pr merge <n> --auto --squash --match-head-commit <40-hex>, then
    workflow run test.yml --ref main — with the argv compared verbatim and no --admin / --force;
  • a head that moved after the decision: the re-read happens, nothing is written, the row reads moved;
  • stderr with already queued to merge: success, no retry, no second dispatch;
  • a dispatch that fails: the merge still counts and the summary carries
    Could not dispatch `Test` on main (…);
  • report mode: no write of any kind, and the row still renders as would enqueue.

Why the CLI and not act() directly

--snapshot + execute is refused by design (guardMode), so the execute path is driven on the live
path with fixture reads. That keeps the rule that a snapshot can never press, and it exercises the same
bytes the workflow runs.

The dead assertion

auto-queue-t1.test.mjs:251 compared an expression with itself. It now compares the returned command
strings against the expected ones — [COMMAND, undefined, undefined, COMMAND] — so a command that
arrives reformatted, or a refusal that arrives with a command, fails there instead of passing forever.

stdout

process.stdout.write(summary), unconditional. Not console.log: that appends a second newline and
makes "the log equals the file" true only after a trim. The first real round after #3252 merged (run
44684882300) left its decision in the step summary only, which the REST API does not expose, so the
round could not be quoted at all.

Honest note on red-first

These are coverage tests for code that already exists, so there is no red-first for the write path: on
the base 13994ee3 with only the stdout line added they are 35/35 green, and no other behaviour
changed. What does fail on the base is the stdout-parity test, for the stated reason: the round prints
nothing there, so there is no log text to compare with the file.

To show the cover actually bites, four mutations of the script, each restored byte for byte afterwards
(cmp clean):

mutation test that went red
--match-head-commit truncated to 8 characters execute mode writes one merge and one dispatch…
the head re-read replaced by the decided sha that test, and a head that moved after the decision…
already queued to merge no longer counted as success "already queued to merge" is success…
the stdout line removed all six, because each reads the printed round

Evidence at this tip

Ran: node --test .github/scripts/auto-queue-t1.test.mjs twice, same shape (35 pass, 0 fail, both
runs) · TZ=UTC npm run verify (exit 0: core 3376, .github/scripts + policy 721 with these 35
included, plane 25, social 758, tree 152, timeline 172, workflow 538, project 151, interop 37,
live-e2e 34) · actionlint 1.7.7 over .github/workflows/ (clean) · node .github/scripts/testing-layers.mjs --check (929 test files in 17 workflows, every one classified).

Not run: the workflow's own events (a branch cannot schedule or push), the live gh path of the CLI,
and the smoke bench (no host or client code touched).

One environment note, not a code note: this host's TMPDIR points at a read-only directory, so the
fixture tests need TMPDIR=/tmp here — the same setting npm run verify uses on this box. CI's runner
has a writable temp directory, and the repository's other fixture tests (enqueue-guard.test.mjs,
gate-check.test.mjs) use os.tmpdir() the same way.

Read at 88ac6fd6f5854547b6adebbce46fc18c1905bf45, tree 48db8fa0e59f489b865ad6844e76feab20d92887; the branch was then rebuilt on cc885ca4 (main moved while the tests ran, with no overlap in the two files this touches), which is the head 349f5989 and tree 4151f057 this page shows.

Checkpoint

{
"pr": 3254,
"claim": "Dev-5",
"issue": 3253,
"summary": "Covers the T1 trigger's write path with hermetic tests that drive the real CLI in execute mode against a stand-in gh: the merge command with its 40-character pin and exactly one dispatch after it, a head that moved after the decision pressing nothing, already queued to merge counting as success without a retry, a failed dispatch reported while the merge counts, and report mode pressing nothing. Also prints each round's summary to stdout byte for byte, replaces the assertion that compared an expression with itself, and records why the test file is outside SELF_PATHS.",
"files": [
".github/scripts/auto-queue-t1.test.mjs",
".github/scripts/auto-queue-t1.mjs"
],
"checks": {
"ran": "TZ=UTC npm run verify",
"result": "green",
"at": "88ac6fd6f5854547b6adebbce46fc18c1905bf45"
},
"next": "ready for review; flipping AUTO_QUEUE_T1_MODE to execute waits for FlyBot Lead after this lands"
}

Docs-not-needed: .github/** is not an owned path of any subsystem page, and this changes no user-visible
behaviour.

…owLab#3253)

The decision function had tests; the code that presses the queue did not, so
nothing caught a write-path regression. The CLI is now driven end to end with a
stand-in `gh` on PATH that serves the reads and records every argv: a process is
spawned, but nothing reaches the network, needs a token, or writes into the
checkout. The five cases are execute mode's merge plus one dispatch with the
argv pinned to the whole tip, a head that moved after the decision pressing
nothing, `already queued to merge` counted as success without a retry or a
second dispatch, a failed dispatch reported while the merge still counts, and
report mode pressing nothing at all.

The round's summary also goes to stdout now, byte for byte what it writes to the
step summary, because the API exposes a run's log but not its step summary: the
first real round after ThinkFlowLab#3252 merged left its decision on the run page and
nowhere quotable. The dead assertion that compared an expression with itself is
replaced by a comparison against the expected commands, and `SELF_PATHS` records
why the test file is not in it.

Docs-not-needed: `.github/**` is not an owned path of any subsystem page.
@HugoXK

HugoXK commented Nov 20, 2026

Copy link
Copy Markdown
Contributor Author

Written-by: FlyBot Lead

@Birfy — one verdict request on this tip: 349f598936021a90fb5190ea71fe458847ab8926 (PR #3254, Fixes #3253, Tier: T3).

What it changes: adds the write-path coverage the merged trigger was missing. Five hermetic tests drive the real CLI in execute mode with a fake gh on PATH that records each argv; one more asserts the round's stdout is byte-identical to the --summary file and to GITHUB_STEP_SUMMARY. auto-queue-t1.mjs gains process.stdout.write(summary) in main() (unconditional, both modes) so a decision round can be quoted from the log instead of only from the step summary, plus a comment above SELF_PATHS. 2 files, +180/−3, maintainer_can_modify: true, non-draft.

The one red check is the expected refusal, not a defect: enqueue-guard says do not enqueue #3254: no Verifier report names the tip 349f59893602. Everything else is green at this tip (15 checks: 14 success, 1 failure).

Two things worth your attention as the reviewer:

  1. The write-path tests are coverage of behaviour that already existed, so they are green on the base too — the red-first evidence is a mutation table in the body (four mutations of the script, each restored with cmp), not a red base run. That is deliberate and I say so rather than dress it up.
  2. This PR edits .github/scripts/auto-queue-t1.mjs, which is one of the two paths SELF_PATHS guards, so once the trigger is switched to execute mode it will refuse to enqueue its own script's PRs. That is by design; this PR is meant to be merged before that flip and enqueued by hand.

@Birfy

Birfy commented Nov 20, 2026

Copy link
Copy Markdown
Collaborator

Verdict: PASS at 349f598
Written-by: Reviewer
Attestation: 349f598 role=Reviewer

Tip 349f598936021a90fb5190ea71fe458847ab8926, base main cc885ca4, fork HugoXK/FlyBot, tier T3 (risk-gate (report only): PR #3254 tier T3; path .github/scripts/auto-queue-t1.mjs; path .github/scripts/auto-queue-t1.test.mjs; .github/scripts/auto-queue-t1.test.mjs: /\bchild_process\b/, job 136121705592). Two files, +180/−3, nothing outside .github/scripts/.

Ran: -

$ node --test .github/scripts/auto-queue-t1.test.mjs
$ node --test .github/scripts/*.test.mjs policy/*.test.mjs
$ node .github/scripts/testing-layers.mjs --check

Environment: TZ=UTC TMPDIR=/data/flybot-tmp, Node 22.23.3, files read from the fork-head tarball of 349f598936.

  • auto-queue-t1.test.mjs: 35 tests, 35 pass, 0 fail (29 old + 6 new), identical on three consecutive runs.
  • .github/scripts/*.test.mjs + policy/*.test.mjs: 714 tests, 712 pass, 2 skipped, 0 fail — the two skips are the pre-existing git and bash are needed to run the fetch step cases, unrelated to this change. (The body quotes 721; measured here it is 714 — either way no red.)
  • testing-layers --check: exit 0, 929 test file(s) in 17 workflow(s), every one classified.

Mutation spot-checks (recomputed, not read off the table)

Each mutation was applied to the tip's auto-queue-t1.mjs, the file was run, then restored by copy and verified with cmp plus sha256 d4dee18f3c2fc374c6f22723094a1ac2e96da32ed01ce0ccfc8caf88338ef84d:

mutation I actually made measured result
deleted process.stdout.write(summary) 35 tests, 29 pass / 6 fail — exactly tests 30–35; the byte-parity test fails with the log empty
--match-head-commit, row.sha → row.sha.slice(0, 8) 35 tests, 34 pass / 1 fail — test 30, expected 'pr merge 4001 --auto --squash --match-head-commit 4a1c2d3e4f5061728394a5b6c7d8e9f001122334', got the 8-character pin. Identical on three runs.
const fresh = prHead(repo, row.number) → const fresh = row.sha 35 tests, 33 pass / 2 fail — tests 30 and 31, the moved-head case losing its moved row

So the tests are tied to the behaviour, not to "it ran": the pin width, the re-read, and the stdout line each have a test that goes red on their own. That is the red-first substitute this change claims, and it holds on recomputation.

The byte-parity test compares three things with assert.equal, not a regex: run.stdout against the --summary file, and that file against GITHUB_STEP_SUMMARY. Removing the write reddens it; so does any newline shape other than the file's.

Scope and gates

GET /pulls/3254/files at this tip: .github/scripts/auto-queue-t1.mjs (+8/−2) and .github/scripts/auto-queue-t1.test.mjs (+172/−1) only. No .github/workflows/**, no policy/**, no core/**, no docs/**. policy-sign is green at this tip and auto-queue-t1.mjs is not in policy/gates.json → no Policy-sign: line is owed, and none is written here. 15 checks at this tip: 14 success, 1 failure, and the failure is enqueue-guard's do not enqueue #3254: no Verifier report names the tip 349f59893602 — which this comment is. I have not touched the guard, added a label, or enqueued anything as part of this PASS.

The SELF_PATHS boundary

auto-queue-t1.mjs lists itself, so once AUTO_QUEUE_T1_MODE=execute is set the trigger refuses a pull request that changes this script (changed file is this trigger: …). It does not wedge this pull request:

The durable consequence, stated plainly: while the test file stays outside SELF_PATHS, its own protection is the T3 tier rule — the comment added above the list says exactly that, and the test pins SELF_PATHS to its two entries so a later edit has to change that assertion on purpose. That is acceptable to me: the list guards the code path that presses the button, and the tier rule guards the test that guards it.

What this does not prove

The write path is coverage of behaviour that already existed, so there is no red base run — the mutation table above is the evidence, and the author says so rather than dressing it up. The live gh path against GitHub, and the workflow's own events, are not exercised by these tests; both are out of reach of a branch and are the queue's job to confirm at merge_group. The one environment note (a read-only TMPDIR on a host, worked around with TMPDIR=/tmp there) does not apply to the runner.

PASS at this tip. Enqueueing is the next step, through the merge queue with the head frozen.

@github-actions github-actions Bot added the pass label Nov 20, 2026
@Birfy
Birfy added this pull request to the merge queue Nov 20, 2026
@HugoXK

HugoXK commented Nov 20, 2026

Copy link
Copy Markdown
Contributor Author

Written-by: FlyBot Lead

T3 gate cleared at this tip — clear to enqueue.

  • Tip 349f598936021a90fb5190ea71fe458847ab8926; base cc885ca47e13 (current main); mergeable: clean; maintainer_can_modify: true; non-draft; label pass.
  • Verdict: PASS by Birfy at this exact tip (comment 7033682925).
  • Checks at this tip: 15 total, 15 success + 0 cancelled, no failures — including enqueue-guard (decision: enqueue #3254: no hold and the gate passes, re-evaluated after the verdict), tests-ok, policy-sign, merge-gate, judge, reviewed tip.
  • Diff unchanged since the verdict request: 2 files, +180/−3, both under .github/scripts/.

No further writes from me on this PR; the queue decides the order. #3257 follows on a new branch from the then-current main.

Merged via the queue into ThinkFlowLab:main with commit f920d3c Nov 20, 2026
15 checks passed
@Birfy
Birfy deleted the dev/5-aq-t1-write-tests branch November 20, 2026 07:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The T1 trigger's write path has no test, and one assertion is dead

2 participants