From a469f271ab520eb511f78eafb9ae8863bcecd110 Mon Sep 17 00:00:00 2001 From: Erik Berlin Date: Mon, 26 Oct 2026 09:10:47 -0700 Subject: [PATCH] =?UTF-8?q?=E2=AC=86=EF=B8=8F=20kettle-jem:=20update=20the?= =?UTF-8?q?=20Duel=20template=20for=20Duel=200.1.1?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Perf workflow now comments on a pull request only when Duel says to (by default when it fails), and posts a new comment instead of editing the last one. Duel 0.1.1 watches the benchmarks, .duel.rb and the version files itself, so .duel.rb only adds the workflow, and the modular Gemfile requires 0.1.1 or later. --- gems/kettle-jem/CHANGELOG.md | 1 + gems/kettle-jem/README.md | 4 +- .../lib/kettle/jem/templates/.duel.rb.example | 17 +++++--- .../.github/workflows/perf.yml.example | 41 ++++++++++++------- .../gemfiles/modular/duel.gemfile.example | 2 +- .../spec/integration/duel_template_spec.rb | 14 ++++++- 6 files changed, 54 insertions(+), 25 deletions(-) diff --git a/gems/kettle-jem/CHANGELOG.md b/gems/kettle-jem/CHANGELOG.md index 36dae2159..b6b4c4e4a 100644 --- a/gems/kettle-jem/CHANGELOG.md +++ b/gems/kettle-jem/CHANGELOG.md @@ -23,6 +23,7 @@ Please file a bug if you notice a violation of semantic versioning. - kettle-jem: corporate sponsor configuration now accepts the explicit target_href and image_src keys alongside legacy url and img_src, preserving existing configs while supporting direct, date-bounded static sponsors. - kettle-jem: opt-in Duel performance regression guards (https://github.com/sferik/duel-ruby), off by default. With `workflows.duel: true`, the full template generates the Perf workflow (`.github/workflows/perf.yml`), `gemfiles/modular/duel.gemfile` (Duel and StackProf on Ruby 3.4 or later only), a `perf` appraisal, the root Gemfile wiring, a `rake duel` task with a stub for older Rubies, the RuboCop include for `.duel.rb`, and the Duel README badge. It creates `.duel.rb` only when it is missing. While Duel is off, a project's own `.duel.rb`, `perf.yml`, and `duel.gemfile` are neither written nor removed. - kettle-jem: `actions/download-artifact` joins the template workflow pins, for the Perf workflow's Comment job. +- kettle-jem: the Duel template follows Duel 0.1.1, which it now requires. The Perf workflow's Comment job posts a new comment only when Duel says to (by default when a benchmark fails, as `c.comment` in `.duel.rb` chooses), and never edits one, so the pull request keeps every report. `.duel.rb` adds only the Perf workflow to what Duel watches, as Duel 0.1.1 watches the benchmarks, `.duel.rb`, and the version files itself. ### Changed diff --git a/gems/kettle-jem/README.md b/gems/kettle-jem/README.md index d5e406deb..60127b1ca 100644 --- a/gems/kettle-jem/README.md +++ b/gems/kettle-jem/README.md @@ -381,14 +381,14 @@ With Duel on, the full template generates: | File | What it holds | |------|---------------| -| `.github/workflows/perf.yml` | A read-only Duel job that compares each pull request with the main branch, each push to `main` with the commit before it, and, on a schedule, the last release. A separate Comment job, with only `pull-requests: write` and none of the branch's code, posts the report on the pull request. When the baseline's version file names another Ruby, the workflow installs that Ruby too. | +| `.github/workflows/perf.yml` | A read-only Duel job that compares each pull request with the main branch, each push to `main` with the commit before it, and, on a schedule, the last release. A separate Comment job, with only `pull-requests: write` and none of the branch's code, posts the report on the pull request as a new comment, when a benchmark fails by default (`c.comment` in `.duel.rb` chooses when), and again when a later run's outcome differs from its last comment's. When the baseline's version file names another Ruby, the workflow installs that Ruby too. | | `gemfiles/modular/duel.gemfile` | Duel and StackProf, on Ruby 3.4 or later only, so older Rubies install neither. | | `Appraisals` | A `perf` appraisal, whose generated `gemfiles/perf.gemfile` the Perf workflow bundles. | | `Gemfile` | `eval_gemfile "gemfiles/modular/duel.gemfile"`, for `rake duel` in local development. | | `Rakefile` | `rake duel`, with a stub task that only warns where Duel isn't installed. | | `.rubocop.yml` | `.duel.rb` in `AllCops/Include` (RuboCop skips dotfiles otherwise), merged with RuboCop's own patterns. | | `README.md` | The Duel badge, at the end of the badge row. | -| `.duel.rb` | Duel's configuration, created only when it is missing. It runs each version on the Ruby that its own version file names, and watches the benchmarks, `.duel.rb`, `mise.toml`, and the Perf workflow as well as `lib/` and `ext/`. After that it is the project's own. | +| `.duel.rb` | Duel's configuration, created only when it is missing. It runs each version on the Ruby that its own version file names, and watches the Perf workflow on top of what Duel watches already: `lib/`, `ext/`, the benchmarks, `.duel.rb`, and the version files. After that it is the project's own. | The benchmarks in `duel/` are the project's to write: see [Write a benchmark](https://github.com/sferik/duel-ruby#write-a-benchmark). diff --git a/gems/kettle-jem/lib/kettle/jem/templates/.duel.rb.example b/gems/kettle-jem/lib/kettle/jem/templates/.duel.rb.example index a40189d04..784bb1b83 100644 --- a/gems/kettle-jem/lib/kettle/jem/templates/.duel.rb.example +++ b/gems/kettle-jem/lib/kettle/jem/templates/.duel.rb.example @@ -4,8 +4,8 @@ # # Duel follows conventions, so set only what differs: benchmarks are in # duel/**/*_duel.rb, each version's lib/ is first on the load path, a -# change is compared when it touches lib/ or ext/, and releases are -# tagged v*. +# change is compared when it touches lib/ or ext/ (or the benchmarks, this +# file, or the version files), and releases are tagged v*. # # kettle-jem creates this file when it is missing, and leaves it alone # after that, so it is this project's to change. @@ -13,8 +13,13 @@ Duel.configure do |c| # Each version runs on the Ruby that its own version file names, so a # branch that upgrades Ruby is compared on each version's own Ruby. c.ruby = :version_file - # Compare a change that touches what's measured or how, not only lib/ and - # ext/: the benchmarks and this file, the Ruby that mise.toml names, and - # the Perf workflow. (Each version runs the candidate's benchmarks.) - c.watch = %w[lib ext duel .duel.rb mise.toml .github/workflows/perf.yml] + # Compare a change to the Perf workflow too, which decides how the code + # is measured. (Duel already watches lib/, ext/, the benchmarks, this + # file, and the version files.) + c.watch += %w[.github/workflows/perf.yml] + + # When the workflow comments on a pull request with the report: :always, + # :results (passed or failed), :failures (failed, or couldn't compare), + # or :never. + # c.comment = :failures end diff --git a/gems/kettle-jem/lib/kettle/jem/templates/.github/workflows/perf.yml.example b/gems/kettle-jem/lib/kettle/jem/templates/.github/workflows/perf.yml.example index 15933e687..fbb45e651 100644 --- a/gems/kettle-jem/lib/kettle/jem/templates/.github/workflows/perf.yml.example +++ b/gems/kettle-jem/lib/kettle/jem/templates/.github/workflows/perf.yml.example @@ -32,6 +32,11 @@ jobs: if: "github.event_name != 'push' || (!contains(github.event.head_commit.message, '[ci skip]') && !contains(github.event.head_commit.message, '[skip ci]'))" name: Duel runs-on: ubuntu-latest + # Whether the Comment job comments (post, change or none), and the + # run's status (passed, failed or neutral), from Duel. + outputs: + comment: ${{ steps.compare.outputs.comment }} + status: ${{ steps.compare.outputs.status }} env: # $BUNDLE_GEMFILE must be set at job level, so it is set for all steps # The perf appraisal: the gemspec's dependencies and Duel (Ruby 3.4 or later) BUNDLE_GEMFILE: ${{ github.workspace }}/gemfiles/perf.gemfile @@ -110,9 +115,11 @@ jobs: # Against the main branch for a pull request, what was there before # for a push, and the last release on a schedule (slow creep). Duel - # adds its report to the job's summary, and writes it to duel.md for - # the comment. + # adds its report to the job's summary, writes it to duel.md for the + # comment, and sets the step's outputs to say whether to comment, as + # c.comment in .duel.rb says, whatever the outcome. - name: Compare + id: compare env: DUEL_BASE: ${{ github.event_name == 'schedule' && 'last-release' || github.event_name == 'push' && github.event.before || '' }} # (Run by hand, it compares anyway: with nothing changed, the code @@ -137,15 +144,17 @@ jobs: if-no-files-found: ignore # none when the comparison didn't run retention-days: 1 - # The report, as a comment on the pull request: one comment, updated on - # every push. This job checks nothing out and runs none of the branch's - # code, so it alone can write. (A pull request from a fork gets no - # comment, as its token can't write one: the Duel job's summary has the - # report.) + # The report, as a new comment on the pull request, when Duel says to: + # always (post), or only when its last comment's status differs from + # this run's (change), so the latest comment is never stale. It never + # edits or deletes a comment. This job checks nothing out and runs none + # of the branch's code, so it alone can write. (A pull request from a + # fork gets no comment, as its token can't write one: the Duel job's + # summary has the report.) comment: name: Comment needs: duel - if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository }} + if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && (needs.duel.outputs.comment == 'post' || needs.duel.outputs.comment == 'change') }} runs-on: ubuntu-latest permissions: pull-requests: write # to comment on the pull request with the report @@ -163,13 +172,15 @@ jobs: env: GH_TOKEN: ${{ github.token }} PR: ${{ github.event.pull_request.number }} + COMMENT: ${{ needs.duel.outputs.comment }} + STATUS: ${{ needs.duel.outputs.status }} run: | - marker="" - { echo "$marker"; cat "$RUNNER_TEMP/report/duel.md"; } > "$RUNNER_TEMP/comment.md" - ids=$(gh api --paginate "repos/$GITHUB_REPOSITORY/issues/$PR/comments" \ - --jq ".[] | select(.user.login == \"github-actions[bot]\" and (.body | startswith(\"$marker\"))) | .id") - if [ -n "$ids" ]; then - gh api --method PATCH "repos/$GITHUB_REPOSITORY/issues/comments/${ids%%$'\n'*}" -F body=@"$RUNNER_TEMP/comment.md" > /dev/null - else + marker="\")" | tail -n 1) + if [ "$COMMENT" = post ] || { [ -n "$last" ] && [ "$last" != "$marker status=$STATUS -->" ]; }; then + { echo "$marker status=$STATUS -->"; cat "$RUNNER_TEMP/report/duel.md"; } > "$RUNNER_TEMP/comment.md" gh api --method POST "repos/$GITHUB_REPOSITORY/issues/$PR/comments" -F body=@"$RUNNER_TEMP/comment.md" > /dev/null fi diff --git a/gems/kettle-jem/lib/kettle/jem/templates/gemfiles/modular/duel.gemfile.example b/gems/kettle-jem/lib/kettle/jem/templates/gemfiles/modular/duel.gemfile.example index 7d13e8c9b..9c9666e4a 100644 --- a/gems/kettle-jem/lib/kettle/jem/templates/gemfiles/modular/duel.gemfile.example +++ b/gems/kettle-jem/lib/kettle/jem/templates/gemfiles/modular/duel.gemfile.example @@ -9,7 +9,7 @@ # Performance regression guards (see .duel.rb and duel/) # Duel requires Ruby 3.4 or later, so older Rubies install neither gem. if Gem::Version.new(RUBY_VERSION) >= Gem::Version.new("3.4") - gem "duel", "~> 0.1", ">= 0.1.0" # ruby >= 3.4 + gem "duel", "~> 0.1", ">= 0.1.1" # ruby >= 3.4 # For profiles of regressions (StackProf runs on CRuby only) platform :mri do diff --git a/gems/kettle-jem/spec/integration/duel_template_spec.rb b/gems/kettle-jem/spec/integration/duel_template_spec.rb index c53be35c8..24361399d 100644 --- a/gems/kettle-jem/spec/integration/duel_template_spec.rb +++ b/gems/kettle-jem/spec/integration/duel_template_spec.rb @@ -114,14 +114,26 @@ def with_project(name) expect(checkout.fetch("with")).to include("fetch-depth" => 0, "persist-credentials" => false) baseline_ruby = workflow_yaml.dig("jobs", "duel", "steps").find { |step| step["id"] == "baseline-ruby" } expect(baseline_ruby.fetch("with")).to include("bundler-cache" => true) + compare = workflow_yaml.dig("jobs", "duel", "steps").find { |step| step["name"] == "Compare" } + expect(compare.fetch("id")).to eq("compare") + expect(workflow_yaml.dig("jobs", "duel", "outputs")).to eq( + "comment" => "${{ steps.compare.outputs.comment }}", + "status" => "${{ steps.compare.outputs.status }}" + ) + expect(workflow_yaml.dig("jobs", "comment", "if")).to end_with(" && (needs.duel.outputs.comment == 'post' || needs.duel.outputs.comment == 'change') }}") + comment_step = workflow_yaml.dig("jobs", "comment", "steps").find { |step| step["name"] == "Comment on the pull request" } + expect(comment_step.fetch("env")).to include("COMMENT" => "${{ needs.duel.outputs.comment }}", "STATUS" => "${{ needs.duel.outputs.status }}") + expect(comment_step.fetch("run")).to include(%(marker=""), "--method POST") + expect(workflow).not_to include("PATCH") %w[actions/checkout ruby/setup-ruby actions/upload-artifact actions/download-artifact].each do |action| expect_pinned_action(workflow, action) end expect(files.fetch(:"gemfiles/modular/duel.gemfile")).to include(%(if Gem::Version.new(RUBY_VERSION) >= Gem::Version.new("3.4"))) expect_gem_dependency_declared(files.fetch(:"gemfiles/modular/duel.gemfile"), "duel") + expect(files.fetch(:"gemfiles/modular/duel.gemfile")).to include(%(gem "duel", "~> 0.1", ">= 0.1.1")) expect_gem_dependency_declared(files.fetch(:"gemfiles/modular/duel.gemfile"), "stackprof") - expect(files.fetch(:".duel.rb")).to include("c.ruby = :version_file") + expect(files.fetch(:".duel.rb")).to include("c.ruby = :version_file", "c.watch += %w[.github/workflows/perf.yml]\n", "# c.comment = :failures\n") expect(files.fetch(:Gemfile).scan(%(eval_gemfile "gemfiles/modular/duel.gemfile")).size).to eq(1) expect(files.fetch(:Gemfile).index("gemfiles/modular/duel.gemfile")).to be < files.fetch(:Gemfile).index("gemfiles/modular/x_std_libs.gemfile") expect(appraisals_eval_gemfile_paths(files.fetch(:Appraisals), "perf")).to eq(%w[modular/duel.gemfile modular/x_std_libs.gemfile])