Skip to content

Install CI dependencies from uv.lock - #136

Merged
puddly merged 6 commits into
puddly:devfrom
TheJulianJES:tjj/ci-uv-sync
Oct 13, 2026
Merged

puddly merged 6 commits into
puddly:devfrom
TheJulianJES:tjj/ci-uv-sync

Conversation

@TheJulianJES

@TheJulianJES TheJulianJES commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

CI currently builds the cached venvs with:

  • uv pip install -e . --group dev for Linux, and with
  • pip install -e . --group dev for macOS and Windows.

Those both ignore the uv.lock file that's tracked in the repo, so library versions will be whatever the newest version was when the venv cache was last rebuilt. This is also what caused ruff 0.17 to randomly appear in #128 and #129. I don't think this is intentional?

This PR changes that to use the uv.lock to create the CI venv from (with uv sync), similar to the recommended uv sync approach described in the README for the local environment/development. uv.lock is also added as a cache key.

To not revert CI to old library versions temporarily, I ran uv lock --upgrade and committed that here. If wanted, this can also be dropped from this PR and become a separate dependency upgrade PR, e.g. with: dev...TheJulianJES:tjj/uv-lock-upgrade. Might be cleaner?

pytest-cov and pytest-github-actions-annotate-failures were added to the dev group (installed by uv sync by default, so also in CI). For zigpy, we have a separate ci group to not install pytest-github-actions-annotate-failures (and pytest-xdist) in local environments but the package just doesn't do anything then when not in a GitHub runner.

Lastly, the venv folder in CI is now the standard for uv: .venv, not venv. As prek config runs with uv run, this will no longer cause a second venv to be created just for prek in CI.

Notes / questions

  • uv.lock upgrade: the second to last commit is only uv lock --upgrade and can be dropped from this PR, to upgrade the lockfile in a separate PR instead. Without it, CI would move back to the (older) locked versions for some packages, e.g. pytest 9.1.1 → 9.0.3 or prek 0.5.5 → 0.3.13. The prek hooks and the ESPHome tests were also checked with those older versions.
  • Lockfile format: the lockfile changes were generated with the latest uv (0.13.0), which moves uv.lock from revision = 3 to revision = 5. That's most of the upgrade diff. Revision changes are backwards compatible, so older uv versions can still read it ("Changes to the revision will not cause older versions of uv to error", per the uv docs). uv 0.11.20 reads it fine.
  • dev vs. ci group: pytest-github-actions-annotate-failures is only needed in CI. It's inert outside GitHub Actions (it only activates when GITHUB_ACTIONS=true), so having it in dev doesn't hurt. The zigpy repos (zigpy, zha, zha-device-handlers, zigpy-ota) put it in a separate ci group instead, with CI running uv sync --group ci. Should we do the same here, or keep it in dev for simplicity?
  • Dependabot (suggestion): to keep uv.lock (and the pinned GitHub Actions) up to date, a Dependabot config with grouped updates could be added, similar to the zha repo: the uv ecosystem with all Python dependencies in a single group and a short cooldown, plus github-actions. That way, there are no spammy PRs for every single dependency bump, but one bigger PR that can be merged every few weeks.

Detailed AI-generated PR description

AI-generated PR description (CLICK TO EXPAND)

Why

Since #132, the prek hooks and uv sync for development use the versions pinned in uv.lock, but the CI test venvs still resolve fresh. Because those venvs are cached by a hash of pyproject.toml, setup.py and the Cargo files, the versions they contain depend on when the cache was last rebuilt, so CI is neither reproducible nor actually up to date. With this PR, CI, the hooks and local development all use the same locked versions, and updates happen through explicit lock bumps.

For comparison: zigpy, zha, zha-device-handlers and zigpy-ota use uv sync --group ci via zigpy/workflows, and HA Core / aioesphomeapi / python-otbr-api install exact == pins from requirements files.

Changes in detail

  • prepare-base (Linux): uv sync --locked with UV_PYTHON set to setup-python's python-path, so the venv always uses the interpreter its cache key refers to. The explicit pip/setuptools/prek install is gone; prek, uv and setuptools (via setuptools-rust) come from the dev group.
  • macOS/Windows: same uv sync --locked step (plus setup-uv) instead of python -m venv + pip install. The editable build still puts the Rust extension in-tree, where the extension cache picks it up, and the Rebuild Rust extension step works in the uv-created venv.
  • Cache keys: uv.lock is added to all base venv keys and to the prek key (prek itself now comes from the lock, so a prek version change needs a fresh hook cache).
  • dev group: adds pytest-cov (used by all test jobs; harmless locally since --cov isn't in the default options) and pytest-github-actions-annotate-failures (see the ci group question above).
  • venv → .venv: .venv is uv's default project environment, which uv sync creates and uv run uses (and what zigpy/workflows and the uv sync dev setup use). Previously, the prek job activated the cached venv, but the hooks' uv run --frozen --group dev ignored it (VIRTUAL_ENV=venv does not match the project environment path .venv and will be ignored) and built a second environment in .venv. Now they reuse the cached one.
  • serialx-compat/coverage jobs: uv pip install instead of pip install, since uv-created venvs have no pip. coverage now comes from pytest-cov's locked dependency.
  • Lockfile upgrade (last commit): 47 packages updated, one new transitive dependency (backports-zstd). Notably mypy 2.1.0 → 2.4.0, pylint 4.0.5 → 4.1.3, pytest 9.0.3 → 9.1.1, pytest-asyncio 1.3.0 → 1.4.0, prek 0.3.13 → 0.5.5, aioesphomeapi 46.6.0 → 46.10.0, uv 0.11.14 → 0.13.0. All prek hooks (incl. the new mypy and pylint) pass without code changes.

Testing

  • This PR changes every venv cache key, so its CI rebuilds all venvs and runs the new steps on Linux, macOS and Windows.
  • Locally: a CI-style uv sync --locked venv passes the test suite (Python 3.14, with --cov) and all prek hooks with the upgraded lockfile. Without the upgrade commit, the prek hooks and the ESPHome tests were checked as well.

The base venv was built with `uv pip install`, which ignores the lockfile and
resolves the newest versions at the time the cache is created. Use
`uv sync --locked` instead, so CI runs the locked versions (and fails if the
lockfile is out of date), and include `uv.lock` in the cache key.

Venvs created by uv don't include pip, so use `uv pip` in the remaining jobs.
`pytest-cov` and `pytest-github-actions-annotate-failures` were installed
unpinned in every test job. Add them to the `dev` group so they come from
`uv.lock` like everything else (the annotation plugin only activates on
GitHub Actions).
Like the Linux base venv, install the locked dependencies instead of resolving
them with pip at cache creation time.
prek itself now comes from `uv.lock`, so a prek version change needs a fresh
hook environment cache.
`uv lock --upgrade` with uv 0.13.0. Notably mypy 2.4.0, pylint 4.1.3,
pytest 9.1.1, pytest-asyncio 1.4.0, prek 0.5.5, aioesphomeapi 46.10.0 and
uv 0.13.0.
@TheJulianJES
TheJulianJES marked this pull request as draft October 13, 2026 00:40
Comment thread .github/workflows/ci.yml Outdated
@codecov

codecov Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.36%. Comparing base (fc11b40) to head (b16d2ab).

Additional details and impacted files
@@           Coverage Diff           @@
##              dev     #136   +/-   ##
=======================================
  Coverage   94.36%   94.36%           
=======================================
  Files          25       25           
  Lines        4362     4362           
=======================================
  Hits         4116     4116           
  Misses        246      246           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

It's uv's default project environment, so `UV_PROJECT_ENVIRONMENT` isn't
needed. It also lets the prek hooks (`uv run --frozen --group dev`) reuse
the cached venv instead of creating a second environment in `.venv`.
@TheJulianJES
TheJulianJES marked this pull request as ready for review October 13, 2026 00:50
@puddly
puddly merged commit 467c854 into puddly:dev Oct 13, 2026
31 checks passed
@TheJulianJES
TheJulianJES deleted the tjj/ci-uv-sync branch October 13, 2026 18:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants