Skip to content

chore: repository health check (tests, security, deps, CI) - #494

Draft
shenxianpeng wants to merge 9 commits into
mainfrom
chore/repo-health-check
Draft

shenxianpeng wants to merge 9 commits into
mainfrom
chore/repo-health-check

Conversation

@shenxianpeng

Copy link
Copy Markdown
Member

Summary

A repository health check: unit tests for the docs scripts, hardening of action.yml and the workflows, toolchain and lock file updates, and three bug fixes found along the way. There is one commit per topic. The action's inputs, outputs and behavior stay the same, except where a fix below says otherwise.

Coverage

  • Before: there were no tests. docs/*.py is the only Python code in the repository; the action's own logic is nushell inside action.yml.
  • After: docs/badge_hook.py and docs/gen_io_doc.py have 100% line and branch coverage (coverage.py with branch = true and source = ["docs"]). The only exclusion is one pragma: no cover on the Windows fallback of a test fixture.

Tests added

  • tests/test_gen_io_doc.py runs the script the way mkdocs-gen-files does. It checks that every input and output in action.yml is documented with its version, default and description. It also covers how defaults, the experimental flag and the permission badges are rendered, the annotations for undocumented fields and missing minimum-version entries, and that incomplete metadata fails the build.
  • tests/test_badge_hook.py covers every badge type, markers matched without case, and the error paths.
  • Supporting changes:
    • a test dependency group
    • pytest and coverage settings in pyproject.toml
    • a Run tests workflow that runs when action.yml, docs/*.py, docs/action.yml, tests/, pyproject.toml or uv.lock change
    • a short note in CONTRIBUTING.md (uv run --group test pytest)

Bug fixes

  • gen_io_doc.py annotations: the script printed ::warning file=docs/action.ymltitle=.... Without the comma between the properties, a missing minimum-version never became a proper annotation. The error for an undocumented field also said actions.yml instead of action.yml. Regression tests are added.
  • Cache key: the cache-key step wrote key=<hash>\n with a literal backslash-n, because $'...' strings do not process escapes in nushell. Every cache key ended in \n (visible in the actions/cache logs). Existing caches will miss once.
  • Quotes and backslashes in inputs: an input with a quote or a backslash could break a step. For example, auto-fix-commit-msg: "style: apply clang-format's fixes" failed with "Unbalanced delimiter", and style: file:C:\cfg\.clang-format failed with "unrecognized escape". The environment variable change below fixes this.

Security hardening

  • action.yml inputs: the nushell scripts now read every ${{ inputs.* }} value, and the github.* values they use, from INPUT_* and PR_* environment variables instead of having them expanded into the script text.
    • Each value is still passed as a single argument. extra-args stays one --extra-arg=... that cpp-linter splits on spaces, and an empty version still runs the LLVM script without arguments.
    • zizmor's template-injection findings in action.yml go from 45 to 0.
  • uv installer: the action checks the installer against a pinned SHA256 before running it, and a mismatch stops the step with an error annotation. With uv 0.12.21, the shell installer also checks the SHA256 of the archive it downloads; 0.9.5 printed "no checksums to verify".
  • uv sync --locked: the action now installs exactly what the hash-pinned lock file lists. A stale lock file fails the step instead of being re-resolved at run time.
  • Workflow permissions: the repository's default token is read-write. cpp-linter.yml, mkdocs-deploy.yml and pre-commit.yml now declare least-privilege permissions, as the other workflows here and in cpp-linter/cpp-linter already do: contents: read, plus pages: write and id-token: write for the docs deployment.
  • release.yml:
    • The workflow_dispatch tag input now reaches the script through an environment variable.
    • The branches filter is removed; GitHub ignores it for release events, and actionlint reported it as an error.
    • The shellcheck findings SC2006 and SC2034 are fixed.
    • The rolling tag's message now reads Retag v2 instead of the literal Retag $MAJOR_VERSION.
  • zizmor overall: findings go from 66 to 15. These remain on purpose:
    • unpinned-uses for the org's reusable workflows and cpp-linter-action@main, which is the org's convention
    • informational template-injection of step outputs in the test workflows
    • self-repository hints
    • dependabot-cooldown (see the decisions below)
  • Scans:
    • There are no open Dependabot or secret-scanning alerts, and code scanning is not set up.
    • pip-audit flagged urllib3 2.7.0 (installed by the action), click, pygments and virtualenv. All four are fixed by the lock refresh, and pip-audit is now clean.
    • bandit only reports the asserts in the docs generator.

Dependency updates

  • Action toolchain: nushell 0.106.1 → 0.116.0 and uv 0.9.5 → 0.12.21. Both are pinned in action.yml, and Dependabot does not manage them.
  • uv.lock: all indirect dependencies are refreshed, because Dependabot only bumps direct ones. Direct dependencies are left to Dependabot.
    • For the action: certifi 2026.7.22, cffi 2.1.1, charset-normalizer 3.5.2, idna 3.20, pycparser 3.0, pygit2 1.20.1 (on Python 3.11 and later), requests 2.34.2 and urllib3 2.8.0.
    • cpp-linter 1.14.1 imports and parses diffs the same way with pygit2 1.20.1.
  • Already current: actions/checkout v7.0.1, actions/cache v6.1.0, setup-nu v3.27, pre-commit-hooks v6.0.0, cpp-linter 1.14.1, clang-tools 1.3.0, and LLVM 12 to 23 in the self-test matrix.
  • Bot PR worth merging: chore(deps-dev): bump ruff from 0.16.6 to 0.16.9 in the dev group across 1 directory #483 (ruff 0.16.6 → 0.16.9). ruff is left untouched here so that PR does not conflict.

CI warning fixes

  • Fixed here: the actionlint errors in release.yml and the malformed annotation from gen_io_doc.py.
  • From outside this repository:
    • Release Drafter prints about 16 deprecation warnings (categories[*].labels, version-resolver.*.labels, exclude-labels). They come from cpp-linter/.github/.github/release-drafter.yml.
    • The docs build prints "MkDocs may break support…" (from properdocs, which mkdocs-gen-files 0.6.1 pulls in) and a Material for MkDocs notice about MkDocs 2.0. actions/deploy-pages emits a Node punycode DeprecationWarning. These come from the org's mkdocs.yml reusable workflow and upstream projects.
    • A notice says ubuntu-latest moves to Ubuntu 26.04 from 2026-10-19. The self-test uses ubuntu-latest on purpose; its first runs on the new image are worth watching, because older clang-format-N apt packages may be missing.

Items needing a maintainer decision

  • Remaining integrity gaps (unchanged):
    • https://apt.llvm.org/llvm.sh is downloaded and run with sudo without a pin. Pinning it would break when upstream updates the script, and vendoring it would need an update for every new LLVM release.
    • On Windows, the PowerShell uv installer does not check the archive's checksum.
    • hustcer/setup-nu does not check the nushell download.
    • clang-tools checks its static binaries against a SHA512SUMS file from the same release.
  • astral-sh/setup-uv: it would check uv on every platform, but it puts uv on the user's PATH, which the docs say the action does not do.
  • Dependabot cooldown: zizmor's dependabot-cooldown asks for at least 7 days. uv has 3, and github-actions uses the default.
  • Coverage gate: the tests are at 100%; fail_under = 100 in [tool.coverage.report] would keep them there.
  • requires-python = ">=3.10": Python 3.10 reaches end of life this month. The action runs on 3.12 (.python-version), so this only affects how the lock is resolved.

How it was verified

  • Tests: uv run --group test coverage run -m pytest gives 25 passed with 100% line and branch coverage, and every commit passes on its own.
  • Linters and docs: pre-commit run --all-files, actionlint (workflows and examples) and zizmor pass. mkdocs build --strict succeeds; the pages are identical apart from the CONTRIBUTING note and Pygments no longer escaping '.
  • Old against new action.yml: a local harness (not committed) ran every nushell step of the old and new action.yml in an Ubuntu 24.04 container under nushell 0.106.1 and 0.116.0, for 44 input scenarios. Stub uv, apt-get, brew and sudo commands recorded the arguments they received.
    • The inputs covered defaults, empty values, spaces, $, backticks, parentheses, backslashes, quotes, newlines and Unicode.
    • The paths covered apt, the LLVM script and brew, plus the PR-head checkout and the auto-fix commit and push for PR, fork, push and tag events against a real git remote.
    • Arguments, exit codes, outputs and resulting commits were identical, except for the two inputs that broke the old version (see Bug fixes).
  • End to end in the same container: a real uv install (checksum checked), uv sync --locked, clang-tools 18 from apt, and cpp-linter on docs/examples/demo, for a push event and a pull_request event (REST diff plus pygit2 1.20.1).
    • Outputs, the step summary and the annotations matched main, apart from the cache key.
    • A wrong installer checksum and a stale lock file each fail the setup step, as intended.
  • Not verified locally: macOS and Windows runners. The self-test matrix on this PR covers them.

Cover docs/gen_io_doc.py (the generated inputs-outputs page) and
docs/badge_hook.py (the md:* badges) with pytest, including the checks
that every action input and output is documented. Add a test dependency
group, pytest and coverage settings, and a workflow that runs the tests
when the action metadata, the docs scripts or the tests change.
…page

gen_io_doc.py printed `::warning file=docs/action.ymltitle=...` when an
input or output had no minimum-version: without the comma between the
properties GitHub does not create the intended annotation. The error for
an undocumented field also named actions.yml instead of action.yml.
Add regression tests for both messages.
`$'...'` strings do not process escapes in nushell, so the step wrote
`key=<hash>\n` with a literal backslash-n into $GITHUB_OUTPUT and every
cache key ended in `\n` (visible in the actions/cache logs). Use a
double-quoted interpolation so the line ends with a real newline. Existing
caches are not restored once, as their keys change.
The nu scripts in action.yml expanded `${{ inputs.* }}` (and a few
`github.*` values) directly into their source, so an input was parsed as
nushell code. A value with a quote or a backslash could break a step, e.g.
`auto-fix-commit-msg: "style: apply clang-format's fixes"` failed with
"Unbalanced delimiter" and `style: file:C:\cfg\.clang-format` failed with
"unrecognized escape", and an input could change what the script runs.

Map every input a script uses to an INPUT_* variable in the step's env:
and read it from there. Each value is still passed as one argument, as
before: `extra-args` stays a single `--extra-arg=...` that cpp-linter
splits on spaces, and an empty `version` still runs the LLVM install
script without arguments.
The repository's default GITHUB_TOKEN permissions are read-write, and
cpp-linter.yml, mkdocs-deploy.yml and pre-commit.yml did not set any, so
their jobs ran with a write token. Grant each job only what it uses: read
access to the contents, plus pages: write and id-token: write for the
docs deployment, as the other workflows already do.

In release.yml, pass the workflow_dispatch `tag` input through an
environment variable instead of expanding it into the script, and drop the
`branches` filter that GitHub ignores for release events (actionlint
error). The rolling tag's message now names the tag; it read
"Retag $MAJOR_VERSION" because the variable sat in single quotes.
Both were pinned to year-old releases (nushell 0.106.1, uv 0.9.5). The
uv.lock file is up to date for uv 0.12.21 (`uv lock --check`), and none of
the breaking changes in uv 0.10 to 0.12 apply to the `uv sync` and
`uv run` calls the action makes.
The action downloads uv's install script for the pinned UV_VERSION and runs
it. Pin the SHA256 of the shell and PowerShell installers next to
UV_VERSION and stop with an error annotation if the downloaded script does
not match. For uv 0.12.21 the shell installer also checks the SHA256 of the
uv archive it downloads (0.9.5 printed "no checksums to verify"), so on
Linux and macOS the whole chain is verified.
Dependabot only bumps the direct dependencies, so the packages they pull
in had not moved since the lock file was created. Upgrade every indirect
dependency in uv.lock (direct ones are left to Dependabot). For the action
itself this updates certifi, cffi, charset-normalizer, idna, pycparser,
pygit2 (1.20.1 on Python >= 3.11), requests and urllib3. It also picks up
fixes for published advisories: urllib3 2.8.0 (installed by the action),
click 8.5.0 and pygments 2.21.0 (docs), and virtualenv 21.14.2 (dev).

cpp-linter 1.14.1 still imports the pygit2 constants it uses with pygit2
1.20.1, and parses diffs the same way.
`uv sync` re-resolves and rewrites uv.lock when it does not match
pyproject.toml, which would install whatever versions are current at run
time instead of the reviewed, hash-pinned ones. Pass `--locked` so a stale
lock file fails the step instead. With a lock file that matches (as the
released action always ships), nothing changes.
@shenxianpeng shenxianpeng added the maintenance Maintenance updates label Oct 1, 2026
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Oct 1, 2026
@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Cpp-Linter Report ⚠️

Some files did not pass the configured checks!

clang-format (v16.0.6) reports: 2 file(s) not formatted
  • docs/examples/demo/demo.cpp
  • docs/examples/demo/demo.hpp
clang-tidy (v16.0.6) reports: 7 concern(s)

Have any feedback or feature suggestions? Share it here.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation maintenance Maintenance updates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant