fix: make the Jest wiring guard read the runner's test list reliably

[eleshar]

Linked issues

Fixes #3575

Related: #3552 and #3496 (the guard this hardens), #3572 (the separate timing-boundary flakes), #3487 (made Jest a required gate, which is why a false positive here now blocks arbitrary PRs).

Context

• Severity/Impact: Medium-High. The guard is a required gate since fix: ci - run Jest on every PR and fail only on new failures #3487, and it fails on a red build that a re-run hides while naming innocent files. It is a false-positive generator, not a missed regression.
• Affected versions/environments: scripts/validation/__tests__/test-wiring.test.js on develop since f58448f45d. Both nested listings it spawns are affected: the root jest --listTests and the pr-agent jest-vm.js --listTests pass-through.

Reproduction

• Steps:
  1. Scope the parent run to a subset, which the root config supports: JEST_IGNORE_PATTERN='<rootDir>/scripts/automation/' npx jest --config .jest.config.cjs scripts/validation/__tests__/test-wiring.test.js.
  2. The nested child inherits that variable, so it resolves a different configuration than the one the guard is asserting about: it reports 230 test files instead of 282.
  3. The guard then reports the 50 healthy files under scripts/automation/ as undiscovered.
• Expected vs Actual: expected the guard to describe the default root runner. Actual — identical symptom to the flake in test: test-wiring spawns a nested Jest and reports innocent files as undiscovered under load #3575:

  - Array []
  + Array [
  +   "scripts/automation/__tests__/phase-3-integration.test.js",
  ...
  

• The load-dependent truncation reported in test: test-wiring spawns a nested Jest and reports innocent files as undiscovered under load #3575 was not reproduced on demand: 45 child invocations across two stress setups (8-way CPU contention; a concurrent full parent suite) always returned the full 282. What is reproducible is the class of failure — the guard cannot tell a bad read from a real break — and that is what this change fixes.

Root Cause

1. The guard read the child's stdout. Jest only writes the list to stdout via console.log; its --outputFile path uses a single synchronous fs.writeFileSync. A stdout pipe is asynchronous in Node, so a payload can be cut short if the process exits before the pipe drains, and newline-delimited paths carry no completeness marker. A clipped list is therefore indistinguishable from a genuinely shorter one, and the only thing the guard could do with it was blame files.
2. The child inherited the caller's environment, including the JEST_* discovery overrides the root config reads (JEST_IGNORE_PATTERN, JEST_TEST_MATCH_1..6). The guard was asserting "the default root runner discovers every file" using a runner configured by whoever invoked the test. This is a deterministic false positive, and it is the same wrong diagnosis as the reported flake.
3. expect(result.status).toBe(0) was the only check on the child. A child that failed, was killed, or wrote nothing produced a confusing downstream diff rather than "the listing did not work".

Fix Summary

• Read the list with --listTests --json --outputFile <tmpfile>:
  • --outputFile makes Jest write with one synchronous writeFileSync instead of console.log, so the payload cannot be clipped by a pipe the process exits before draining.
  • --json makes the payload an array, so a partial file fails to parse. Incompleteness becomes a detectable state instead of a silently short list — this replaces the size assertion the issue suggested, which cannot distinguish a clipped listing from a genuine break and is, once the listing is known complete, implied by the existing check.
  • Verified: the child now writes 0 bytes to stdout.
• Strip the discovery overrides from the child's environment so the listing always describes the default root configuration.
• Throw with the honest diagnosis for every way the listing can fail to be obtained: spawn error, non-zero exit (with status, signal and stderr), missing output file, unparseable JSON, non-array payload.
• Applied the same helper to the pr-agent nested listing, which had the identical stdout pattern plus a bare expect(result.status).toBe(0).
• Temp file is removed in a finally block; the counter and pid keep concurrent workers from colliding.

Verification

• Tests added/updated to cover the bug
• Manual verification steps — N/A, no UI; behaviour verified by executing the guard under scoped, healthy and fault-injected conditions
• Negative/edge cases checked

Evidence:

• The test: test-wiring spawns a nested Jest and reports innocent files as undiscovered under load #3575 symptom no longer reproduces. The JEST_IGNORE_PATTERN run that deterministically failed before now passes; same for a JEST_TEST_MATCH_1 scoped run.
• Fault injection A — genuine exclusion still fails, and names the file. Adding a real coverage/fault-injection.test.js (a directory the root config ignores) makes the guard fail with + "coverage/fault-injection.test.js". The assertion was not weakened or removed.
• Fault injection B — a failed listing reports itself. Pointing the child at a non-existent config makes the guard fail with the nested Jest listing exited with status 1, so its result is unknown, naming no suites.
• 20 consecutive full-suite runs pass (the issue's first acceptance criterion): SUMMARY pass=20 fail=0, every run 14 todo, 5551 passed, 5565 total.
• Clipped payload is rejected, not silently short: a truncated listing file raises SyntaxError on JSON.parse, which the helper converts into "the listing is incomplete rather than short".
• eslint 0 problems and prettier --check clean on the changed file (it was clean at HEAD, so the change introduces no new lint or format debt).
• semgrep scan --config p/security-audit --config p/secrets --config p/php: 0 findings. markdownlint-cli2 CHANGELOG.md: 0 issues. actionlint: not applicable, no workflow files changed.
• Changelog gate: 10 failing entries with and without the new entry — 0 new failures.

Risk & Rollback

• Risk level: Low. The guard's invariant is unchanged: every test file on disk, minus the documented standalone and .jest-skip/ exclusions, must be discovered. Only how the listing is obtained and how failures are reported changed.
• The child now runs with the discovery overrides removed. That is the point of the fix, and it is the configuration the guard's own comment claims to assert.
• compareDiscovered was inlined and the suggested size assertion dropped as provably redundant; the diff is the smallest change that makes the read trustworthy.
• Rollback plan: revert the single commit. No configuration, schema or persisted data is involved.

Changelog

Added under ## [Unreleased] → ### Fixed in CHANGELOG.md:

Wiring Guard Stops Blaming Healthy Tests — The test wiring guard now reports an incomplete test listing as such, instead of naming suites that are in fact discovered. (#3575)

[coderabbitai]

Warning

Review limit reached

Next included review available in 32 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: lightspeedwp/.github/.coderabbit.yml

Review profile: CHILL

Plan: Advanced

Run ID: 32f7ed3c-12e8-41a2-8527-ec4d02c0911b

📥 Commits

Reviewing files that changed from the base of the PR and between 10bed6d and 352d5a9.

📒 Files selected for processing (2)

• CHANGELOG.md
• scripts/validation/__tests__/test-wiring.test.js

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

PR Template Routing

Branch Type: fix
Scope: test-wiring-listing-integrity-3575
Template: pr_bug.md
Labels Applied: type:bug

This PR was automatically routed based on the branch naming strategy.

[github-actions]

📋 Changelog Quality Validation

Metric	Count
✅ Passing	111
❌ Failing	10
🆕 New failures in this PR	0
📦 Pre-existing failures	10

Status

✅ Validation PASSED - No new failures introduced by this PR.
Note: 10 pre-existing failure(s) remain in the Unreleased section.

No action required.

[qodo-code-review]

PR Summary by Qodo

Make Jest wiring test listings reliable

🐞 Bug fix 🧪 Tests 📝 Documentation 🕐 20-40 Minutes

AI Description

• Read nested Jest test lists from validated JSON output files instead of stdout.
• Clear inherited discovery overrides so checks always inspect default Jest configurations.
• Report child-process and malformed-listing failures without blaming healthy test suites.

Diagram

graph TD
  A["Wiring guard"] --> B["Clean discovery env"] --> C["Nested Jest runner"] --> D["JSON output file"] --> E["Parse and validate"] --> F["Compare test paths"]

Loading

High-Level Assessment

The PR's approach is appropriate because it retains real CLI-level coverage while using Jest's native synchronous output-file path and JSON as a completeness check. Continuing to parse stdout cannot distinguish truncation from a valid short list, while resolving Jest configuration through internal APIs would bypass the root and pr-agent launchers that this guard must verify.

Files changed (2) +115 / -24

Bug fix (1) +114 / -24

test-wiring.test.jsHarden nested Jest test-list collection +114/-24

Harden nested Jest test-list collection

• Replaces stdout parsing with temporary JSON output files for both root and pr-agent Jest listings. Clears inherited discovery overrides, validates child execution and JSON shape, emits targeted failure diagnostics, and always removes temporary output.

scripts/validation/tests/test-wiring.test.js

Documentation (1) +1 / -0

CHANGELOG.mdDocument reliable wiring-guard diagnostics +1/-0

Document reliable wiring-guard diagnostics

• Adds a fixed-item entry explaining that incomplete Jest listings are now diagnosed directly instead of being reported as undiscovered healthy suites.

CHANGELOG.md

[linear-code]

GIT-2360