Skip to content

[FIX] Stop tox 4.64's .venv redirect from breaking every CI run - #2297

Merged
ritwik-g merged 2 commits into
mainfrom
fix/ci-tox-venv-redirect
Sep 24, 2026
Merged

ritwik-g merged 2 commits into
mainfrom
fix/ci-tox-venv-redirect

Conversation

@muhammad-ali-e

Copy link
Copy Markdown
Contributor

What

CI went red on every branch at once on 2026-09-24, with no commit to blame. perf/executor-registry-import passed on 09-23 04:11 and failed on 09-24 09:06 with byte-identical errors and no new commits; the same happened on UN-3487, UN-3494, UN-4172 and main. This unblocks all of them.

Why it broke

uv tool install tox --with tox-uv is unpinned and resolves at job time, so tox 4.63.0 → 4.64.1 landed in CI on its release day (2026-09-24):

green (09-23) red (09-24)
tox 4.63.0 4.64.1
virtualenv 21.11.0 21.12.0
uv 0.12.18 0.12.18

tox 4.64.0 ships a PEP 832 .venv redirect: after each run it writes a file named .venv at the project root containing the path of a tox env, so editors can find an interpreter. It skips this when a real .venv directory already exists — which is why no local checkout noticed, and only CI, which starts clean, broke.

The repo root is also a uv workspace. uv reads .venv as the project environment and refuses a file:

error: Project virtual environment directory `/home/runner/work/unstract/unstract/.venv` cannot be used because expected directory but found a file

That is exactly the three root-workdir invocations:

  • the unit-rig group (workdir: .)
  • the e2e-smoke group (no workdir → root), whose failure then SKIPs the other seven e2e groups
  • the final uv run --with coverage[toml] coverage combine

Both tiers exit 2, and report fails behind them. Every Python test passed throughout — the unit tier logged 970 passed, 1 skipped and the e2e job brought every container up Healthy before this fired.

Why disable the feature rather than move to a fixed version

  • There is no newer tox. 4.64.1 is the latest release and it keeps the feature — 4.64.1 only fixes an unrelated --override bug. The version-only option is pinning back to 4.63.0, which forfeits fixes and re-breaks on the next deliberate bump.
  • Following the redirect would be worse than rejecting it. It aims at .tox/<env> — ~97 packages of pytest/testcontainers/coverage — while the groups pointed at that root need the ~269-package workspace environment. A uv that honoured PEP 832 would run unit-rig and e2e-smoke in the wrong environment silently. Today's loud failure is the better outcome.
  • Nothing here wants it. The feature exists so editors can find an interpreter; a local checkout already has a real .venv directory, which tox leaves alone.

tox and tox-uv are pinned as well, so the next release is a bump we make rather than one CI makes for us.

Testing

Verified locally against the real versions, in a clean worktree with no .venv:

check result
tox 4.64.1, before the change .venv is a 9-byte file containing .tox/rig; uv run at the root reproduces the CI error verbatim
tox 4.64.1, after no .venv file written; uv run at the root succeeds
tox 4.63.0, after accepts venv_redirect without warning — safe either side of the bump

tox -e rig -- validate passes on both tox versions (OK — 22 groups, 19 critical paths).

Notes

  • No Jira ticket — filed as a CI hotfix. Happy to attach one.
  • SonarCloud Code Analysis is red on this PR as it is on every commit for the past several days, including the four before the breakage. Separate, pre-existing, out of scope here.
  • npx canceled due to missing packages and no YES option: ["vitest@5.0.1"] in the unit tier is also pre-existing — it is present in the green 09-23 run too, and is non-fatal. Worth its own ticket.

🤖 Generated with Claude Code

CI went red on every branch at once on 2026-09-24, with no commit to blame:
a branch that passed on 09-23 failed on 09-24 with byte-identical errors.

`uv tool install tox --with tox-uv` is unpinned and resolves at job time, so
tox 4.63.0 -> 4.64.1 landed in CI on its release day. tox 4.64.0 ships a PEP
832 `.venv` redirect: after each run it writes a *file* named `.venv` at the
project root, holding the path of a tox env, so editors can find an
interpreter. It skips this when a real `.venv` directory already exists, which
is why no local checkout noticed and only CI, which starts clean, broke.

The repo root is also a uv workspace. uv reads `.venv` as the project
environment and refuses a file:

  error: Project virtual environment directory `.../.venv` cannot be used
  because expected directory but found a file

That is exactly the three root-workdir invocations: the unit-rig group, the
e2e-smoke group (whose failure then SKIPs the other seven e2e groups), and the
final `uv run --with coverage[toml] coverage combine`. Both tiers exit 2 and
report fails behind them. Every Python test in the unit tier passed throughout.

Disabling the redirect is the fix, not a workaround. Nothing in this repo wants
it: the redirect aims at .tox/<env>, ~97 packages of pytest/testcontainers/
coverage, while the groups pointed at that root need the ~269 package workspace
environment. A uv that honoured PEP 832 would run them in the wrong env
silently, which is worse than today's loud failure.

There is no newer tox to move to instead -- 4.64.1 is the latest release, and
it keeps the feature (4.64.1 only fixes an unrelated override bug). The
version-only alternative is pinning back to 4.63.0, which forfeits fixes and
re-breaks on the next deliberate bump.

tox and tox-uv are now pinned as well, so the next release is a bump we make
rather than one CI makes for us.

Verified locally against the real versions:
  * tox 4.64.1 before the change: `.venv` is a 9-byte file containing
    `.tox/rig`, and `uv run` at the root reproduces the CI error verbatim
  * tox 4.64.1 after: no `.venv` file, and `uv run` at the root succeeds
  * tox 4.63.0 after: accepts the key without warning, so the change is safe
    either side of the bump

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Comment thread .github/workflows/ci-test.yaml Fixed
Comment thread .github/workflows/ci-test.yaml Fixed
Comment thread .github/workflows/ci-test.yaml Fixed
@greptile-apps

greptile-apps Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

via Greptile

RetriggerConfidence Score: 5/5

The PR appears safe to merge; no actionable new failure was established.

Summary

The PR disables tox’s .venv redirect and pins the CI tox installation. Since the previous review, it also adds --no-build to that installation in all three jobs.

Reviews (2) · Last reviewed commit: "[FIX] Install the CI toolchain from whee..."

Sonar flagged the three `uv tool install` lines this PR touches (C Security
Rating on New Code, blocking the quality gate): omitting --no-build permits a
source distribution's setup script to execute during install. The lines are
older than this PR, but editing them made them new code, so a dormant rule
became a gate failure.

Worth fixing rather than waiving. A CI runner installing the test toolchain is
exactly where an sdist build hook would be cheapest to abuse, and the flag
costs nothing here: tox, tox-uv, uv, virtualenv and the rest of the tree all
publish wheels.

Verified with uv 0.6.14 -- the version setup-uv pins, and so the one that
actually runs this command -- that --no-build is supported there, that the
resolution is byte-identical (uv==0.12.18, virtualenv==21.12.0,
tox-uv==1.36.0), and that the resulting tox 4.64.1 runs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@sonarqubecloud

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown
Contributor

Unstract test results

Per-group results

Status Group Tier Passed Failed Errors Skipped Duration (s)
✅ e2e-api-deployment e2e 3 0 0 0 17.3
✅ e2e-coowners e2e 1 0 0 0 1.4
✅ e2e-etl e2e 1 0 0 0 18.6
✅ e2e-login e2e 2 0 0 0 1.2
✅ e2e-prompt-studio e2e 1 0 0 0 9.6
✅ e2e-smoke e2e 2 0 0 0 1.1
✅ e2e-workflow e2e 1 0 0 0 16.5
❌ frontend unit 0 1 0 0 0.0
✅ integration-backend integration 598 0 0 26 57.5
✅ integration-connectors integration 1 0 0 7 8.3
❌ integration-workers integration 159 5 0 1 55.5
❌ ui e2e 0 1 0 0 0.0
✅ unit-backend unit 1318 0 0 1 45.1
✅ unit-connectors unit 63 0 0 0 9.8
✅ unit-core unit 237 0 0 0 3.0
✅ unit-platform-service unit 15 0 0 0 2.6
✅ unit-rig unit 120 0 0 0 4.5
✅ unit-runner unit 10 0 0 0 3.0
✅ unit-sdk1 unit 587 0 0 0 32.1
✅ unit-workers unit 1362 0 0 1 125.2
TOTAL 4481 7 0 36 412.1

Critical paths

⚠️ Critical paths not yet covered

  • workflow-execution-fan-out — Multi-file workflow execution fans out to file-processing workers and rejoins. (declared coverage: no groups declared)
✅ Covered critical paths
  • auth-login — covered by e2e-login
  • adapter-register-llm — covered by integration-backend
  • workflow-author — covered by integration-backend
  • co-owner-manage — covered by integration-backend, e2e-coowners
  • workflow-create-execute — covered by e2e-workflow
  • api-deployment-provision — covered by integration-backend
  • api-deployment-auth — covered by integration-backend
  • api-deployment-run — covered by e2e-api-deployment
  • mcp-server-auth — covered by integration-backend
  • mcp-platform-auth — covered by integration-backend
  • platform-key-whoami — covered by integration-backend
  • prompt-studio-author — covered by integration-backend
  • prompt-studio-fetch-response — covered by e2e-prompt-studio
  • connector-register-test — covered by integration-backend
  • pipeline-etl-execute — covered by e2e-etl
  • usage-aggregate-read — covered by integration-backend
  • usage-token-tracking — covered by e2e-api-deployment
  • callback-result-delivery — covered by e2e-api-deployment

@ritwik-g
ritwik-g merged commit 05e00a5 into main Sep 24, 2026
10 checks passed
@ritwik-g
ritwik-g deleted the fix/ci-tox-venv-redirect branch September 24, 2026 09:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants