Skip to content

Make the pupmod acceptance jobs survive Launchpad PPA outages - #126

Merged
michael-riddle merged 4 commits into
simp:mainfrom
michael-riddle:issue_122
Nov 3, 2026
Merged

michael-riddle merged 4 commits into
simp:mainfrom
michael-riddle:issue_122

Conversation

@michael-riddle

@michael-riddle michael-riddle commented Nov 2, 2026 •

Copy link
Copy Markdown
Member

Fixes #122.

The templates pin simp/gha-workflows v1.1.0 (simp/gha-workflows#18).

The pupmod acceptance jobs installed vagrant and vagrant-libvirt from ppa:evgeni/vagrant alone, so a Launchpad outage failed every acceptance job and, through the required All checks status, blocked merging. This keeps the PPA as the primary source and makes the setup survive its outages.

Workflow templates

In both pr_tests.yml templates, "Setup libvirt for Vagrant" becomes one step:

      - name: Setup libvirt for Vagrant
        uses: simp/gha-workflows/.github/actions/setup-vagrant-libvirt@v1.1.0

The setup-vagrant-libvirt action in simp/gha-workflows installs from the Actions cache, then the PPA (with retries), then HashiCorp's apt repository plus vagrant plugin install vagrant-libvirt, with a warning that the last is BSL-licensed. simp/gha-workflows#18 has the details. Because the logic lives in gha-workflows, a later fix is a tag bump that Renovate proposes per repo, not a fleet sync. Any other repo with a Vagrant/libvirt acceptance job, including Ruby projects, can use the same step.

A pull request can only restore caches saved from its own ref, its base branch or the default branch, so caches saved by pull requests don't help other PRs. The new vagrant_package_cache.yml template runs the action with cache-only: 'true' from the default branch. It runs twice a week, which also keeps the cache from being evicted after seven days without use, and on a push to the default branch that changes the workflow file, which seeds the cache when it lands. Other pushes don't run it, so merges during a Launchpad outage don't each get a failed run. It runs on ubuntu-24.04 because the PPA only publishes for noble, so it only helps acceptance jobs on ubuntu-24.04; see the next section.

Only modules whose acceptance job uses Vagrant get this workflow, so it isn't in the pupmod baseline's present_action_files. This sync adds it with ensure_files, and later merge_github_workflows runs refresh it wherever it exists.

Ubuntu 26.04 runners

ubuntu-latest is moving to 26.04 (resolute): the 2026-10-27 auditd acceptance run was on ubuntu-24.04, and test runs on 2026-11-02 got ubuntu-26.04. The PPA has no resolute builds, so the current step fails there. On 26.04 the action goes straight to HashiCorp's Vagrant 2.4.9 with a notice. Pinning the acceptance job to runs-on: ubuntu-24.04 would keep the MPL build until GitHub retires that image. runs-on is part of each repo's preserved acceptance block, so that would be a separate change.

Syncing to the fleet

jobs.acceptance is repo-owned during syncs (preserve_blocks), so a template change to it can't reach the repositories. merge_gha_workflows gains a template_steps option for steps the template stays authoritative for inside preserved blocks:

  • Each of a preserved job's steps with a listed name is replaced in place by the template's step of that name, reindented to match.
  • A listed step the job lacks goes in next to its template neighbours, so the repo's own steps keep their positions.
  • Comments and blank lines before the next step stay with it.
  • A job with none of the named steps is left alone.
  • A listed name the template's job lacks is an error, not a deletion. So is a name more than one step in a job has, since replacing both would drop what told them apart (an if:, say).
  • preserve_keys still applies to grafted steps, so a repo whose Renovate has bumped the action past the template's tag keeps its tag on the next sync. A later template change that needs a new tag together with new inputs reaches each repo as the inputs at sync time and the tag through Renovate.

The plan passes puppetsync.plans.sync.merge_github_workflows.template_steps through.

The new 20261102-vagrant-ppa-fallback config uses it, together with ensure_files for vagrant_package_cache.yml. ensure_files adds the file to every repo in the session, so the config is meant for the new pupmods_vagrant_acceptance repolist: the 38 pupmods whose acceptance job uses Vagrant. That is 36 from pupmods.yaml plus pupmod-simp-rngd and pupmod-simp-simp_authselect, which aren't in it. Both latest.yaml symlinks point at them. Run against the current pr_tests.yml of those 38 and of every repository in pupmods.yaml:

  • the 38 Vagrant repositories get exactly the new step (a 6-line change), with their matrices, suites and other steps unchanged;
  • the 25 podman-based repositories and the 21 without an acceptance job are unchanged;
  • a second pass changes nothing.

After this merges, run bolt plan run puppetsync config=20261102-vagrant-ppa-fallback repolist=pupmods_vagrant_acceptance to open the downstream PRs.

Testing

  • rspec (266 examples) and the BoltSpec plan specs (32) pass. New specs cover in-place replacement, steps kept between named steps, steps the job lacks, comments belonging to the next step, commented-out keys belonging to the step, reindentation, jobs without the named steps, the errors for a name the template lacks and for a name two steps share, the repo's tag surviving a later sync, idempotency and the plan passthrough.
  • End to end on a fork of pupmod-simp-aide, with the merged workflows and the action vendored as a local action:

🤖 Generated with Claude Code

The acceptance jobs in both pupmod PR Tests templates installed vagrant
and vagrant-libvirt from ppa:evgeni/vagrant alone, so a Launchpad outage
failed every acceptance job and, through the required 'All checks'
status, blocked merging. Ubuntu dropped both packages after 23.10, and
the PPA carries vagrant 2.3.7 (the last MPL-licensed release) with a
matching vagrant-libvirt.

The PPA stays the primary source, wrapped so its outages don't matter:

* The PPA's two .debs are kept in the Actions cache, keyed on the Ubuntu
  codename, architecture and the pinned package versions. On a cache hit
  the step installs them without contacting Launchpad; their
  dependencies come from the Ubuntu archive.
* add-apt-repository, apt-get update and the download are retried three
  times, 30s and then 60s apart.
* If the PPA is still down, the step installs Vagrant pinned to 2.4.9
  from HashiCorp's apt repository and vagrant-libvirt 0.12.2 with
  `vagrant plugin install`, and emits a warning saying so.

The PPA only publishes for noble, and ubuntu-latest is moving to 26.04
(resolute), where the old step fails on the PPA's missing Release file.
On a series the PPA doesn't publish for, the step goes straight to the
HashiCorp path with a notice.

The acceptance block is repo-owned during syncs (preserve_blocks), so a
template change to it never reached the fleet. merge_gha_workflows gains
a `template_steps` option: inside a preserved job, the steps with those
names are replaced by the template's, in template order, where the first
of them was. A job with none of them is left alone, so podman-based and
acceptance-free modules are unchanged. The new
20261102-vagrant-ppa-fallback config uses it to sync the change.

Fixes simp#122

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
silug
silug previously requested changes Nov 2, 2026
… default branch

- Install Vagrant with simp/github-action-setup-vagrant-libvirt, which
  restores the PPA's packages from the Actions cache, then tries the PPA,
  then HashiCorp. It caches every package the PPA provides, unpinned,
  with the key derived from their file names; refreshes only the PPA's
  index with --error-on=any; and removes the PPA's sources file directly.
- Add vagrant_package_cache.yml, which saves the cache from the default
  branch, where pull requests can restore it.
- merge_gha_workflows template_steps: replace named steps in place, keep
  comments that belong to the next step, and fail when the template
  lacks a named step instead of deleting it.
- Sync to a repolist of the 36 Vagrant repos only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The action moved from its own repo into simp/gha-workflows, so there is
one repo to manage. Pin it at v1.1.0, the release that adds it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@michael-riddle

Copy link
Copy Markdown
Member Author

The setup action moved out of its own repo and into simp/gha-workflows (simp/gha-workflows#18), so there is one repo to manage. The templates now use simp/gha-workflows/.github/actions/setup-vagrant-libvirt@v1.1.0. The review replies above that mention simp/github-action-setup-vagrant-libvirt refer to the same action, unchanged; this PR merges after gha-workflows#18 is merged and tagged v1.1.0.

@hcaballero2

Copy link
Copy Markdown
Contributor

I ran the task from this branch against the live pr_tests.yml of every repo with the PPA step. All 38 get exactly the same 6-line swap to the action, and a second pass changes nothing. It's compatible with gha-workflows#18 / v1.1.0. A few things for you to decide, since you have the most context:

1. Two repos missing from the repolist

pupmod-simp-rngd (default branch main) and pupmod-simp-simp_authselect have the same "Setup libvirt for Vagrant" step but aren't in pupmods_vagrant_acceptance.yaml. pupmods.yaml predates rngd and never had simp_authselect. Both merge cleanly, like the other 36. Add them here, or handle them separately?

2. The cache workflow while acceptance runs on ubuntu-latest

All 36 (38) acceptance jobs run on ubuntu-latest, which now resolves to 26.04. There the action goes straight to HashiCorp and never restores the noble cache that vagrant_package_cache.yml saves. Meanwhile that workflow:

  • runs on every push to the default branch, and by design fails while Launchpad is down, so each merge during an outage gets a failed run;
  • isn't in present_action_files, so later baseline runs won't deliver it, despite its "maintained with puppetsync" notice.

Options I see:

  • a. Keep it as a hedge for a future pin to ubuntu-24.04.
  • b. Drop it, and ensure_files, from this sync, and add it if acceptance is ever pinned to 24.04. No current job reads the cache, so nothing changes for them.
  • c. Pin acceptance to ubuntu-24.04 so the cache and the PPA are actually used. runs-on is inside the preserved block, so that would need its own change.

3. template_steps vs preserve_keys on a later sync

preserve_keys runs after the graft, so the repo's existing tag on the grafted step wins. I tested both directions:

  • A repo Renovate bumped to @v1.2.0 keeps it when the template still says @v1.1.0. That's good, and it's what letting the template win would break.
  • A template that moves to @v2.0.0 and adds a with: input gets the new input paired with the repo's old tag.

Neither matters for this sync. Is the current behaviour what you intend? If so, a line in the task header saying so would help the next person who uses template_steps.

Minor, and no current repo hits them: two steps with the same listed name both get replaced (the repo's if: is lost), and trailing commented-out keys at the step's key column stay behind under the new step.

- pupmods_vagrant_acceptance: add pupmod-simp-rngd (main) and
  pupmod-simp-simp_authselect, which have the Vagrant step but aren't in
  pupmods.yaml.
- vagrant_package_cache.yml runs on push only when the file itself
  changes, so it seeds the cache when it lands without a run (that fails
  during a Launchpad outage) on every merge. Its header says how it is
  delivered, since it isn't in the pupmod present_action_files.
- merge_gha_workflows: template_steps fails when a name matches more than
  one step in a job instead of replacing both, keeps commented-out keys
  at a step's key column with the step, and the header documents that
  preserve_keys still applies to grafted steps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@michael-riddle

Copy link
Copy Markdown
Member Author

@hcaballero2 thanks for running it against the live fleet. Addressed in bd824a8:

1. Missing repos: added pupmod-simp-rngd (branch main) and pupmod-simp-simp_authselect to pupmods_vagrant_acceptance.yaml. I re-ran the task against the live pr_tests.yml of all 38. Each gets only the 6-line swap, and a second pass changes nothing.

2. The cache workflow: we're keeping it (option a). It's a hedge for acceptance jobs on ubuntu-24.04, whether that's a pin later or a repo that already uses it. Your two concrete problems are fixed:

  • On push it now runs only when vagrant_package_cache.yml itself changes (paths:). It still seeds the cache when the file lands, but merges during a Launchpad outage no longer each get a failed run. The twice-weekly schedule is unchanged.
  • It stays out of present_action_files on purpose: that list applies to every pupmod, podman and no-acceptance ones included. This session delivers it with ensure_files, and later merge_github_workflows runs refresh it wherever it exists, so the "maintained with puppetsync" notice holds. The file's header now says so.

3. template_steps vs preserve_keys: yes, that's intended. A repo Renovate has bumped shouldn't be rolled back. The task header now documents it: a template change that needs a new tag and new inputs together reaches each repo as the inputs at sync time and the tag through that repo's Renovate PR. There's a spec for the bumped tag surviving a sync.

Minor:

  • Two steps sharing a listed name now fail the task (template_steps name more than one step in a job: ...) instead of both being replaced, so a repo's if: can't be lost silently.
  • Commented-out keys at the step's key column now stay with the step and are replaced with it. Comments outdented to the - column still belong to the next step.

Both have specs. rspec (266) and the plan specs (32) pass, and the PR description is updated.

@michael-riddle
michael-riddle dismissed silug’s stale review November 3, 2026 17:47

All requested changes are addressed in da7ace8, 3836513 and bd824a8, and every review thread is resolved. The current head is approved by hcaballero2. Dismissing so the PR can merge while silug is unavailable.

@michael-riddle
michael-riddle merged commit 0766592 into simp:main Nov 3, 2026
3 checks passed
@michael-riddle
michael-riddle deleted the issue_122 branch November 3, 2026 19:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make the pupmod acceptance jobs survive Launchpad PPA outages

3 participants