Skip to content

Replace monolithic Windows validation with a compiled contract runner - #956

Merged
coneilen merged 46 commits into
mainfrom
coneilen-compiled-validation-runner
Nov 3, 2026
Merged

coneilen merged 46 commits into
mainfrom
coneilen-compiled-validation-runner

Conversation

@coneilen

@coneilen coneilen commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Replace the monolithic PowerShell Windows validation contract with a normal compiled C# entry that cannot report success with missing coverage. Preserve the native validation preconditions and lifetime observations exposed by actual CI, without weakening required checks or executing the old monolith.

Changes

  • Pin SDK 10.0.403 and integrate verified installed/private SDK resolution with the Windows bootstrap.
  • Share complete production bindings between their real consumers and the compiled contracts; validate source/reference fingerprints and loaded assembly identity.
  • Require independent unit manifests, exact counts/groups, suite-result provenance, same-invocation composite proof, and a normal full invocation before emitting success markers.
  • Preserve real PowerShell semantics through protected named adapters and owned suspended/job-based child supervision, raw-byte capture, bounded deadlines, independent EOF proof, primary errors, and UNKNOWN lifetime retention.
  • Switch the dedicated PR job and scheduled hardening fallback to the compiled full command, including an owned Pester 5 prerequisite and measured 25-minute execution / 40-minute job caps.
  • Preserve the main product baseline's caller registrations instead of registering unshipped Cards feature tests.
  • Preserve failing smoke's nonzero exit, activate the existing visible workspace before exercising its actions, and refuse requested actions that never ran. Six native controls cover both stderr producers, successful exits, and missing activation/focus effects.
  • Finalize endpoint captures before reading them, but observe the product's daemon lifetime before job teardown can manufacture the expected result.
  • Retain hidden evidence only under the two explicit .ci-validation-runner upload roots; classify console failures by sanitized suite ID and exception type without leaking raw payloads.
  • Isolate only owned nonactivating pointer fixtures. A real unrelated same-process HWND control verifies recipient refusal before any pointer messages; production recipient/DPI guards remain unchanged.
  • Restore the owned graph surface after Delete selection opens the loop workspace, reacquire the same card before Fit Canvas, and clear the completed stress-job reference only after successful removal. Twenty additional required guards preserve the original nine visibility and 21 Select contracts.
  • Distinguish a control's immediate hit-test container from its exact GA_ROOT input owner in the shared UIA helper. Five real hidden Win32 topology controls preserve strict PID, ancestry, recipient and exact-root admission; physical flat/nested clicks remain a separate interactive proof, not an inferred canonical result.
  • Publish the current compiled assembly once at the real Windows task boundary when its environment entry is absent, rather than relying on a local preset. Preserve supplied/empty/missing state, current-worktree loader checks, native child inheritance and exception-safe process-only restoration.
  • Keep application-product edits restricted to three App.zig functions: App.run smoke-exit/reporting, runSmokeWorkspaceActions prerequisites, and smokeContractPassed's requested-but-unexecuted guard. No DaemonClient, Workspace, AppFont, GraphCanvas, shared Swift, or unrelated Cards feature changes.

Test plan

RED: & .\Tools\windows\ValidationRunner\bin\Debug\net10.0-windows\ValidationRunner.exe --self-test -> actual behavioral run: 22 executed, 21 failed with "RED: coverage admission is not implemented"; original output retained
GREEN: & .\Tools\windows\ValidationRunner\bin\Release\net10.0-windows\ValidationRunner.exe --self-test -> 65 executed, 65 passed, 0 failed, 0 skipped on local commit 90479b3
REGRESSION: & .\Tools\windows\ValidationRunner\bin\Release\net10.0-windows\ValidationRunner.exe --repository-root $PWD --artifact-root (Join-Path $base 'pr956-90479b38-full') -> 605/605 units, 2094 direct checks, exit 0 on local commit 90479b3

  • From Tools\windows\ValidationRunner: dotnet --version selected 10.0.403; dotnet build -c Release completed with 0 warnings and 0 errors.
  • Normal unfiltered command: & .\Tools\windows\ValidationRunner\bin\Release\net10.0-windows\ValidationRunner.exe --repository-root $PWD --artifact-root $a, where $a is an owned evidence directory. Local commit 90479b38 passed 605/605 required units, 2,094 direct checks, exit 0, complete=true, fullInvocation=true, in 1,003.87 seconds. All 146 changed source-file hashes remained unchanged. The derived 465-case proof covers 126 original templates and is not counted again as direct execution.
  • Repeated the normal compiled build/run on the authoritative Cards feature tree after guarded source integration: 578/578, 2,008 direct checks, exit 0, stderr 0, in 981.03 seconds. All 1,133 unleased original files, the original index/HEAD, and the old monolith source were preserved. No old monolith execution occurred.
  • First-head hosted CI observed two missing feature registrations, absent hidden artifacts, and a held-root exit observation failure. Its compiled job stopped after 141 units/1,010 checks without an uploaded report. A faithful local canonical run reproduced the Delete abort with a CRLF adapter; this is not represented as recovered hosted stderr. Original failures are retained.
  • Focused fixes: the original Delete LF/CRLF adapters each passed 30 cases; the four-combination matrix passed 120 cases and rejected two structural mutants. The expanded producer guards passed 50/50, including 20 new navigation/lifecycle controls, in direct and LF/CRLF canonical runs. Topology passed 39 cases. Ownership passed 70 cases plus 44 mandatory native cases. Handoff adapter regression passed 13 cases.
  • The new native smoke fixture passed all six controls. Successful requested smoke retained exit 0; both original stderr-file and owned-capture failure paths returned nonzero with the real daemon diagnostic. Requested-but-unexecuted actions and an omitted focus effect were refused.
  • The isolated Wire payload passed 57/57, including all original 56 cases and the new real wrong-root overlay refusal. Its selected host invocation intentionally remained incomplete and nonzero; that is not claimed as a full-root pass.
  • The final local unit evidence passed 63 complete/disjoint sections (22/17/24); the affected App shard and shard 2 ran serially with stable owned caches. The App suite reported 1,587 passed, 18 existing skipped, 0 failed; the formerly failing close-box key case explicitly reported OK, not a skip. The native accessibility executable passed 259 checks. Earlier concurrent key-case and expired-cache failures remain retained and are not rewritten as passes.
  • Serial native integration with an exclusively owned short root passed smoke, unassisted daemon handoff, six controls, scrubbed startup, and wire roundtrip. The renewed-foreground runs then exposed the repaired Delete scenario and later reached the complete edge create/edit/cancel workflow before the nested node-sheet click refused. All failures and their cleanup evidence remain retained; none is a full integration pass.
  • The nested-click RED was independently reproduced in an ordinary compiled fixture: flat click passed, nested click refused the wrong immediate-container foreground owner. The corrected helper passed five real hidden topology/strict-negative controls. Actual corrected flat/nested input GREEN remains unproven pending the prepared user-launched foreground host; no synthetic keys, input-queue attachment, generic same-PID fallback, or foreground-policy change was added.
  • Hosted head 5a7a3bd4 passed the compiled full 604/604, 2,087 direct checks and retained its scoped hidden artifact. Its unit 1 and native integration jobs then failed the same absent-assembly-environment prerequisite at GDI startup and scrubbed startup. Earlier local commands had a preset and did not exercise that CI path.
  • The new scoped caller regression passed 7/7 production-function/native-child controls. The final exact local unit 1 command ran with GRAPHCODE_VALIDATION_ASSEMBLY explicitly absent, passed 17/63 sections, and produced GDIPLUS_STARTUP_LIVE: PASS; executed=1; observations=2; original absence was restored afterward. Updated three-shard manifests independently passed 63 complete/disjoint sections (22/17/24). The strict fixture-directory loader was not weakened.
  • pwsh -NoLogo -NoProfile -File Tools\tdd\Tests\TddEvidence.Tests.ps1 passed.
  • pwsh -NoLogo -NoProfile -File Tools\windows\validate.ps1 -Task privacy passed.
  • git diff --check passed; every branch commit has its DCO sign-off and required co-author trailer.

Limits: the current compiled full is green and unchanged native unit-input evidence covers all 63 shard sections, but actual corrected local nested-click input and local native integration remain unverified. The user explicitly approved a PR #956-only exception permitting these verified fixes to be pushed with that limitation disclosed. This does not waive hosted native integration, any required check, or strict up-to-date enforcement: all must pass on the new head before normal merge. These contracts and controlled fixtures do not establish hardware/glyph/visual parity or a merge result.

Checklist

  • I have read the Contributing Guidelines
  • I have signed off my commits (git commit -s) per the DCO
  • Tests pass locally (make test)
  • Code follows the existing style (make check)
  • I added the test/contract before the implementation and observed the intended RED failure

make test and make check are macOS-only and were not run on this Windows host. The completed Windows commands and remaining native/final-source limits are distinguished above.

coneilen and others added 30 commits November 2, 2026 13:41
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
…merge

Do not execute the legacy monolithic runner. Preserve source-only baseline dependencies for compiled contract migration.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Preserve monotonic exit/drain deadlines, independent stream faults, primary errors and UNKNOWN retention. Add strict typed multiproject protocol and controlled native fixtures. Keep twelve legacy and production-linkage obligations explicitly incomplete.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Keep incomplete coverage nonzero; preserve owned supervision, suite provenance, and root console marker isolation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Run real supervisor teardown through the same coordinator as controlled fake authority cases. Preserve original primary objects, independently observe root/job/stream proofs, share one cleanup deadline, and continue later close actions after failures.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
…racle

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Run exact receipt7/capture14/tree4/provider4/pre-UIA10/retention6 families through the compiled owned-child transport. Keep mixed and newly reconciled startup/tree assertions mandatory incomplete, with immutable 126-template/192-assertion source correspondence. Expose host terminal retention hold so process exit cannot close unproved jobs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Target production bindings at .NET 8 and fixture bindings at netstandard2.1; keep host .NET 10. Prevent process exit while retained UNKNOWN authority remains.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
…nkage

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Observe the exact UNKNOWN sentinel incrementally in both shared production readers. Retain the original lease immediately, cancel controller waits only, and refuse termination or close after the latch. Add fake-only retention-decision tests and exact fragmented marker controls. Verify startup diagnostics and mixed source boundaries through protected linked-production adapters.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
…ondence

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
…rity

Force explicit inherited stdout/stderr with a closed redirected stdin, without pumps or changing root-exit versus stream-EOF order.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Separate raw EOF proof from text decode failure and observe live retention markers independently of codec errors. Add safe controlled BOM/malformed/supplementary-byte regressions and execute the original eight gate capture modes plus native held-tree and real shared-linkage obligations. All165 selected ownership checks pass; the full mixed family remains mandatory incomplete.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Keep the production native delegate and existing fallback/refusal order; remove reliance on arbitrary preloaded type presence.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
… source provenance

Preserve root registry admission, source/ref fingerprints and loaded MVID checks, normal compiled fixtures, owned mock regression suites, and positive provider/source contract counts. Incomplete canonical coverage remains nonzero.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
…ived proof

Allow repeated exact suite selectors for bounded diagnostics while keeping all unselected mandatory units incomplete.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Port production-backed diagnostic, CI partition, aggregate, section, and Zig resolver cases with typed compiled CI topology guards and exact execution inventories. Preserve genuine pre-cutover CI source failures rather than accepting the legacy runner.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Exercise seven original producer AST predicates and nine negative controls through the protected host transport. Record existing UI and host ownership without counting those cases twice.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
…nert transports

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Add actual PowerShell startup-event, stream identity, complete receipt, protocol, and real production-stub pipe families. Bind the independent UI policy and native fixture deliverables with exact expanded counts. Map all126 original templates onto133 unique dependencies and465 original expanded cases; keep two new focus failures separate. The frozen host composite schema must admit actual current results before root success.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Preserve the dedicated fail-closed Windows 2022 PR job and single non-PR hardening fallback. Install the exact SDK, build from the global.json directory, invoke the normal compiled host, and require fresh complete positive coverage with exactly one full PASS and execution marker. Keep compatibility entry compiled-only and preserve evidence artifacts.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Let the apphost inherit the selected private SDK runtime without changing normal arguments or persistent settings. Restore exact process environment state after success or failure and exercise the production helper in sixteen targeted controls wired into topology validation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
coneilen and others added 16 commits November 2, 2026 15:15
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Reuse loaded or available Pester 5 and save pinned 5.7.1 only when missing, scoped to workspace modules and the current job environment. Guard prerequisite ordering and gating, and reject a missing loader without changing canonical execution or deadlines.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
…failures

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
…ndary

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Preserve exact native dry-run exit status and no-effect source controls, required SDK regression counts, and non-double-counted composite coverage.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
The integration host measured a successful unfiltered run at 973.70 seconds. Give both canonical execution steps 25 minutes and their containing jobs 40 minutes without changing commands, gates, or child-process deadlines.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Opt into hidden files only under the explicit .ci-validation-runner artifact root in both canonical workflows. Add source coverage and missing/false negative controls so the hosted evidence-loss regression cannot silently recur.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
A real native stress reproducer observed400 empty job censuses before original process objects signalled. Preserve zero-time refusal for explicit finalization, but share one bounded scope-exit deadline across job quiescence and every held root before any launch closes. Add required44-case native helper coverage and70-case lifecycle regression including40 real module-owned roots; preserve no-close timeout and sibling negative controls.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Keep validation scoped to the main product baseline, preserve native window and smoke-exit behavior, finalize endpoint captures in the correct order, and observe daemon lifetime before job teardown. Add bounded native smoke controls and non-activating owned pointer fixtures without weakening recipient checks. Classify failing suites without printing private exception payloads.

Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use the existing owned Show in Graph route after selection opens the loop workspace, reacquire the same card before Fit Canvas, and preserve the original pointer and selection guards. Clear the completed PowerShell stress job reference only after successful removal so cleanup cannot remove it twice.

Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ease

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
…t resolution

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Build and validate once for a genuinely missing assembly environment entry, preserve supplied and empty semantics, and restore only the process publication after the task. Exercise native child inheritance and failure identity, and keep the strict fixture loader unchanged.

Signed-off-by: Colin Neilens <coneilen@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@coneilen
coneilen merged commit 25deacb into main Nov 3, 2026
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant