Skip to content

Add an 'All checks' aggregate job to the pupmod PR Tests - #118

Merged
silug merged 1 commit into
simp:mainfrom
silug:pr-tests-aggregate-check
Oct 15, 2026
Merged

silug merged 1 commit into
simp:mainfrom
silug:pr-tests-aggregate-check

Conversation

@silug

@silug silug commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

GitHub only blocks a merge on status checks that branch protection lists by name. A failing job that isn't listed (Ruby Style, before any repo required it) leaves the PR red but mergeable. Listing every job doesn't scale either: each matrix combination is its own check (acceptance (docker_alma8), ...), so the list would have to follow every template change in every repo.

Change

  • all-checks job ('All checks') in both pupmod pr_tests.yml templates. It needs every other PR Tests job, runs with if: always(), prints each job's result, and fails unless every needed job succeeded. Failed, cancelled and skipped all fail it. (In this template a job is only skipped when a job it needs failed, so that's already a failure.) Jobs with continue-on-error report success to needs, so the experimental spec rows stay non-blocking. The job uses jq on the runner rather than a third-party action such as re-actors/alls-green, so the required check depends on no outside code.
  • merge_gha_workflows: modules without acceptance suites have the template's acceptance job removed (preserve_blocks: jobs.acceptance). That would leave needs: [..., acceptance] naming a missing job, which actionlint and GitHub reject (job "all-checks" needs job "acceptance" which does not exist). Any job the task removes is now also dropped from every other job's needs:. That covers a scalar, a flow sequence and a block sequence, and the needs: key goes when nothing is left.
  • 20261014-all-checks-job session: merges pr_tests.yml across the pupmod fleet (github-org repolist), keeping each repo's jobs.acceptance. latest.yaml now points at it.

Rollout order

  1. Merge the puppetsync/20261003-gate-ruby-style PRs first. Before that, this session's PRs would also carry their Ruby Style change.
  2. Run this session and merge its PRs.
  3. Then require the All checks status check in each repo's branch protection. Requiring it before the job exists leaves every PR waiting for a check that never reports.

The seven rubygem repos have their own, non-templated PR workflows. They need the same job added by hand, and rubygem-simp-beaker-helpers and rubygem-simp-rake-helpers need one per workflow (pr_tests.yml and pr_acceptance.yml), because needs: can't cross workflows.

Testing

  • New task specs: a removed job is pruned from scalar, flow and block needs:; a job the repo has stays listed; a module without acceptance gets an aggregate that needs exactly its remaining jobs.
  • rspec: 254 examples, 0 failures. Plan specs: 31 examples, 0 failures.
  • actionlint 1.7.7 on both templates and on the merge output for a module without acceptance: no new findings (only the existing SC2102 info on the acceptance job's beaker:suites[...] line). Without the needs: pruning, actionlint rejects the no-acceptance output.
  • bolt plan run puppetsync options='{"list_pipeline_stages": true}' lists the session's stages.

🤖 Generated with Claude Code

GitHub only blocks a merge on status checks that branch protection lists
by name, so a failing job that isn't listed doesn't stop a merge, and
listing every job (one check per matrix combination) would have to track
every template change in every repo.

Both pupmod pr_tests.yml templates gain an `all-checks` job ('All
checks') that needs every other job, runs with `if: always()`, and fails
unless each needed job succeeded. Branch protection can then require that
one check. Jobs with `continue-on-error` (the experimental spec rows)
still report success, so they stay non-blocking.

merge_gha_workflows removes the template's acceptance job from modules
that have none, which would leave `needs: [..., acceptance]` naming a
missing job, a workflow GitHub rejects. The task now drops any job it
removes from every other job's `needs:` (scalar, flow or block
sequence), and removes the `needs:` key when nothing is left.

Add the 20261014-all-checks-job session to roll the job out to the
pupmod fleet, and point latest.yaml at it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@hcaballero2 hcaballero2 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small thing on the aggregate job's if:; otherwise the merge logic and the rollout notes look right to me.

@silug
silug merged commit b38854d into simp:main Oct 15, 2026
2 checks passed
@silug
silug deleted the pr-tests-aggregate-check branch October 15, 2026 20:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants