Skip to content

test(stdlib): table-driven call-table differential tests against CPython 3.14 - #715

Merged
espetro merged 3 commits into
masterfrom
work/difftab586
Oct 27, 2026
Merged

espetro merged 3 commits into
masterfrom
work/difftab586

Conversation

@espetro

@espetro espetro commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Closes #586

What changed

  • Adds tests/test_stdlib_call_table.py, a table-driven differential sweep over the complete fixed-signature stdlib call table: 153 forms (18 MATH_FUNCTIONS, 23 WAVE_FUNCTIONS, 104 REST_FIXED, 8 SPECIAL_FUNCTIONS).
  • The forms are batched into 12 tiny programs. Each program compiles once with --emit bin --call run, runs as a native binary, and runs the same source under CPython 3.14 (sys.executable). Comparison is stdout line-by-line plus exit status. Each probe emits exactly one stdout line, so a mismatch maps back to its table row. platform.python_version is compared on the 3.14 line rather than the patch, because the binary pins its declared target while the CI oracle can run a different 3.14 patch.
  • A coverage test asserts that the table inventory, the probe set and the form count (153) agree. A new table row without a probe fails the test instead of being skipped silently.
  • Three rows diverge and are xfail with their tracking issue: fix(unicodedata): mirrored returns bool where CPython 3.14 returns int #712 unicodedata.mirrored returns bool instead of int, fix(platform): platform.platform output differs from CPython 3.14 #713 platform.platform output shape, fix(posixpath): realpath raises on a missing target where CPython returns a path #714 os.path.realpath on a missing target raises RuntimeError instead of resolving lexically.
  • Those are bugs against the parity contract, so they are filed as issues rather than added to docs/reference/divergences.md. The xfail references the issue, and the batch passes once the row is fixed.

Test output

uv run pytest tests/test_stdlib_call_table.py -n 2 -q --no-header -rxX:

XFAIL tests/test_stdlib_call_table.py::test_call_table_group[host] - host: recorded divergence: issue #713 (platform.platform output) (platform.platform: native='Darwin-25.6.0-arm64' cpython='macOS-26.6.2-arm64-arm-64bit-Mach-O')
XFAIL tests/test_stdlib_call_table.py::test_call_table_group[paths_divergent] - paths_divergent: recorded divergence, exit status native=1 cpython=0: issue #714 (os.path.realpath on a missing target)
XFAIL tests/test_stdlib_call_table.py::test_call_table_group[text] - text: recorded divergence: issue #712 (unicodedata.mirrored return kind) (unicodedata.mirrored: native='True' cpython='1')
10 passed, 3 xfailed in 28.50s

The nine passing batches cover 119 forms; the three xfail batches execute 34 more forms, with the three divergent rows named above. Total wall time stays under the 90 second budget with -n 2.

CI

The first run (ded3134) passed ruff and compat2 and test shards 1/3 and 2/3. Shard 3/3 failed because platform.python_version() prints the pinned target patch (3.14.9) while that runner's oracle is 3.14.7. be75ba9 compares the 3.14. line instead. In the follow-up run ruff and compat2 passed; the test shards were still queued when this body was updated, so the final CI result is not claimed here.

$ uv run ruff check
All checks passed!
$ uv run ruff format --check
809 files already formatted

Not done

@espetro espetro added this to the v0.7.0 milestone Oct 26, 2026
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Deploying pyscriptc with  Cloudflare Pages  Cloudflare Pages

Latest commit: c6bf041
Status:🚫  Build failed.

View logs

Walk every fixed-signature row in MATH_FUNCTIONS, WAVE_FUNCTIONS, REST_FIXED and SPECIAL_FUNCTIONS (153 forms), batch them into thirteen programs, and compare native stdout plus exit status against CPython 3.14.

Rows with a tracked difference are xfail with the issue reference: #712 unicodedata.mirrored return kind, #713 platform.platform output, #714 os.path.realpath on a missing target. The coverage test fails if a table row has no probe.
The compiled binary pins its declared target (3.14.9) while the CI oracle can run a different 3.14 patch (3.14.7). Compare the 3.14 line so the differential row stays meaningful on both hosts.
@espetro
espetro merged commit 23cfa93 into master Oct 27, 2026
32 of 33 checks passed
@espetro
espetro deleted the work/difftab586 branch October 27, 2026 21:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test: table-driven stdlib call-table differential tests against CPython 3.14

1 participant