feat: warn when running on an unsupported Python version - #133
Conversation
Merging this PR will degrade performance by 1.17%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ⚡ | Memory | test_multiprocessing_map[10] |
4.1 MB | 3.2 MB | +27.26% |
| ⚡ | WallTime | test_multiprocessing_map[1000] |
59.2 ms | 49.8 ms | +18.96% |
| ⚡ | WallTime | test_multiprocessing_map[100000] |
419.4 ms | 364.9 ms | +14.96% |
| ⚡ | WallTime | test_sum_of_squares[sum_of_squares_sum_labmda_power] |
350.4 µs | 312.5 µs | +12.13% |
| ⚡ | WallTime | test_multiprocessing_map[10000] |
59.8 ms | 56.2 ms | +6.36% |
| ⚡ | WallTime | test_sudoku[initial_grid0] |
8.3 µs | 7.9 µs | +4.97% |
| ⚡ | WallTime | test_open_knight_tour[1] |
4.3 µs | 4.1 µs | +4.66% |
| ⚡ | Memory | test_multiprocessing_map[100000] |
6.2 MB | 6 MB | +3.63% |
| ⚡ | WallTime | test_open_close_fd[10] |
96.8 µs | 93.4 µs | +3.61% |
| ⚡ | WallTime | test_make_lowpass |
5.6 µs | 5.5 µs | +2.75% |
| ⚡ | WallTime | test_make_peak |
6.4 µs | 6.3 µs | +2.7% |
| ⚡ | WallTime | test_hamilton_cycle[graph0] |
25.5 µs | 24.8 µs | +2.56% |
| ⚡ | WallTime | test_color[graph0-3] |
31.4 µs | 30.7 µs | +2.39% |
| ⚡ | WallTime | test_open_close_fd[100] |
901.6 µs | 880.6 µs | +2.37% |
| ⚡ | WallTime | test_recursive_fibo_20 |
5.3 ms | 5.2 ms | +2.1% |
| ⚡ | WallTime | test_pipe_communication[100] |
6.3 ms | 6.2 ms | +2.09% |
| 👁 | WallTime | test_noop_pass_decorated |
658.4 ns | 804.6 ns | -18.16% |
| 👁 | WallTime | test_make_lowshelf |
7.3 µs | 7.7 µs | -5.18% |
| 👁 | WallTime | test_make_highpass |
5.5 µs | 5.7 µs | -3.82% |
| 👁 | WallTime | test_generate_sum_of_subsets_soln[nums0-9] |
26.1 µs | 29.3 µs | -10.98% |
| ... | ... | ... | ... | ... | ... |
ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.
Tip
Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.
Comparing cod-3400-warn-when-running-pytest-codspeed-with-unsupported-python (14b0bb9) with master (c7e9981)
e09ec2d to
2345b56
Compare
|
@greptileai review |
2345b56 to
4e49265
Compare
Greptile SummaryThe PR adds a pytest-header notice when the active interpreter falls outside the supported Python 3.9–3.15 range.
Confidence Score: 5/5The PR appears safe to merge because no blocking failure remains. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| src/pytest_codspeed/plugin.py | Adds supported-version bounds, warning formatting, and inclusion of the warning in pytest's report header. |
| tests/test_pytest_plugin.py | Adds tests for supported and unsupported versions, GitHub Actions annotations, and header integration. |
Reviews (2): Last reviewed commit: "feat: warn when running on an unsupporte..." | Re-trigger Greptile
GuillaumeLagrange
left a comment
There was a problem hiding this comment.
could use a hint of deslop for the tests, olgtm.
I wont highjack this one for the perf trampoline, let's merge this I'll make another PR
Print a warning in the pytest header when the interpreter is outside the version range pytest-codspeed claims to support, so customers upgrading Python are told their measurements run on an untested interpreter instead of getting silent, unvalidated results. On GitHub Actions the warning is emitted as a `::warning` workflow command so it becomes an annotation shown outside the job log; on GitLab CI, which has no annotation mechanism, the line is coloured instead. Mirrors the `warnCi` helper of the codspeed-node counterpart, including the percent-encoding of workflow-command syntax. The bounds mirror `requires-python` and the `Programming Language :: Python` classifiers in pyproject.toml, and the CI matrix. Refs COD-3400
65a48fb to
14b0bb9
Compare
Print a warning in the pytest header when the running interpreter is outside the
Python version range pytest-codspeed claims to support:
Customers upgrading Python currently get silent, unvalidated measurements: nothing
tells them the interpreter they benchmark on was never tested. Surfacing it in the
header means it lands in the CI log right under the existing
codspeed: <version>line, next to the other notices.
The bounds live in
pytest_codspeed.utilsand mirrorrequires-pythonplus theProgramming Language :: Pythonclassifiers inpyproject.tomland the CI matrix.The issue did not specify a list, and the sources disagreed: the classifiers and CI
say 3.15, while the README badge says 3.14 and the docs badge 3.13. The badges are
stale (docs still claimed 3.13 well after 3.14 classifiers shipped), so the bounds
follow the classifiers. Warning on 3.15 would contradict the package's own metadata
and redden the 3.15 CI legs.
test_get_python_version_warning_supportedasserts no warning fires on theinterpreter running the suite, so adding a Python version to the CI matrix without
bumping the constant fails loudly instead of silently warning users.
Not included: the "warning on the Run page" half of the issue is platform-side. The
plugin already ships full interpreter metadata via
get_environment_metadataandcollect_and_write_python_environment, so no extra plumbing is needed here.Refs COD-3400