[AAASM-6251] 🐛 (ci): Audit the project env, not uvx's own - #351
Open
Chisanan232 wants to merge 1 commit into
Open
Chisanan232 wants to merge 1 commit into
Chisanan232 wants to merge 1 commit into
Conversation
The gate ran `uvx pip-audit`. `uvx` gives the tool its own throwaway environment and pip-audit audits whichever environment it runs in, so the job audited pip-audit's ~29 dependencies and never looked at the 194 packages the preceding `uv sync --frozen` had just installed. It was therefore incapable of failing on a project dependency, and it did not: it was green on every commit while this repository carried 24 open Dependabot alerts against uv.lock, one of them critical. Run the audit through `uv run --frozen --with` so it resolves the locked project environment, and allowlist the one advisory that has no fixed release (nltk PYSEC-2026-3740) in the format the job already documented. Proven on one identical tree, the pre-bump lock at 7cad098: `uvx pip-audit --strict --desc` exits 0 with "No known vulnerabilities found", while the new command exits 1 with "Found 24 known vulnerabilities, ignored 1 in 3 packages" (pyjwt, urllib3, virtualenv). refs AAASM-6251 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



What changed
.github/workflows/ci.yaml'sdependency-auditjob now runs its audit throughuv run --frozen --with pip-auditinstead ofuvx pip-audit, and allowlists thesingle advisory that has no fixed release.
Why
uvxdeliberately builds the tool it runs its own throwaway environment, andpip-auditwith no requirements file and no explicit package argument audits theenvironment it is itself running in. So
uv sync --frozeninstalled 194 projectpackages, and the next step then installed ~29 unrelated packages and audited
those. The gate has been structurally incapable of failing on a project
dependency for its entire lifetime.
It is not a theoretical gap. Job 110761168336 on
mainat0d2ac609:That job was green on every commit through 2026-10-02 while this repository
carried 24 open Dependabot alerts against
uv.lock, one of them critical.The job's comment also asserted that "pip-audit resolves the synced
environment". That was false as written, so the comment is corrected too.
How to verify (text only — no visual artifacts)
Differential, back to back on one identical tree. Detached checkout of
7cad098bac5f3abbff95423d94fa0f6fe2542c24, the commit immediately before thissweep's four security bumps, whose
uv.lockcarries virtualenv 21.6.1,pyjwt 2.13.0, urllib3 2.7.0 and gitpython 3.1.59.
uv sync --frozenfirst, then:uvx pip-audit --strict --desc(old)No known vulnerabilities founduv run --frozen --with pip-audit pip-audit --strict --desc --ignore-vuln PYSEC-2026-3740(new)Found 24 known vulnerabilities, ignored 1 in 3 packagesPackages flagged by the new command:
pyjwt,urllib3,virtualenv. This isthe anti-vacuity proof: the fixed gate fails on a knowingly vulnerable locked
set that the old gate passed, and the allowlist is narrow enough that only
nltkis suppressed while the other three still bite.
Green on current
mainfor the real reason. Same command againstmain'slock:
No known vulnerabilities found, 1 ignored, exit 0. Without the allowlistflag it is
Found 1 known vulnerability in 1 package—nltk 3.10.3 PYSEC-2026-3740, reported with an empty fix column.The allowlisted advisory is genuinely unfixable today.
uv tree --frozen --invert --package nltkgivesnltk v3.10.3 -> llama-index-core v0.14.23 -> agent-assembly (group: dev), and the PyPI JSON API reports 3.10.3 as the newestnltk release, so no bump can satisfy it. The flag carries a dated rationale and
an explicit removal condition in the workflow.
Gate wiring is unchanged and still blocking.
dependency-auditremains inci-success'sneedslist, andci-successstill fails on afailureresult.No required check was weakened, removed or made non-blocking.
Lint.
actionlint .github/workflows/ci.yamlexits 0. The YAML parses and thegate's
run:block, extracted verbatim from the committed file and executed as ascript, reproduces the exit codes above.
pre-commit run --files .github/workflows/ci.yamlexits 0 (every hook reports no matching files).Scope note
The pip-audit finding set is not a superset of Dependabot's. gitpython
3.1.59 was flagged by Dependabot but is not reported by pip-audit on that tree.
Neither surface subsumes the other; both stay necessary.
A second tempting fix,
--path .venv/lib/python<X.Y>/site-packages, isdeliberately not used: it is silent on a path that does not exist. Locally
the venv is
python3.14while CI'ssetup-pythonpins 3.12, anduvx pip-audit --path .venv/lib/python3.12/site-packagesexited 0 withNo known vulnerabilities foundagainst a lock holding 25 of them. That wouldreintroduce exactly this defect the day the interpreter version moved. The
workflow comment records why.
Closes AAASM-6251
🤖 Generated with Claude Code