Skip to content

Run the xUnit v3 benchmark flavor on Microsoft.Testing.Platform - #94

Merged
woksin merged 1 commit into
mainfrom
specifications-77-bench-xunit-mtp-trx
Oct 25, 2026
Merged

woksin merged 1 commit into
mainfrom
specifications-77-bench-xunit-mtp-trx

Conversation

@woksin

@woksin woksin commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

The benchmark's xUnit v3 rows on Microsoft.Testing.Platform now actually run on that platform and produce results, so the native runner can be compared with them (part of #77).

The xunit-v3-mtp project never switched on UseMicrosoftTestingPlatformRunner,
so its executable used xUnit's console runner and rejected --report-trx.
Turn the platform on and reference the TRX report extension.
@woksin woksin self-assigned this Oct 25, 2026
@woksin

woksin commented Oct 25, 2026

Copy link
Copy Markdown
Contributor Author

Reviewer notes

  • Benchmarks run 40173461224 had every xunit-v3-mtp row INVALID (exit 3, unknown option: --report-trx). The generated project never set UseMicrosoftTestingPlatformRunner, so the executable used xUnit's own console runner. It also lacked the TRX extension.
  • The fix applies to the xunit-v3-mtp flavor only. No shipped package changes.
  • Local: harness unit tests 19/19; smoke run (FLAVORS=xunit-v3-mtp N=5 F=1 R=1) gives all four rows valid.
  • Review: full round 1 passed with no findings.
  • After merge, I re-dispatch the Benchmarks workflow for the Graduate the native runner from experimental #77 numbers.

@woksin
woksin merged commit 1fd416b into main Oct 25, 2026
3 of 4 checks passed
@woksin
woksin deleted the specifications-77-bench-xunit-mtp-trx branch October 25, 2026 07:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant