⬆️ kettle-jem: update the Duel template for Duel 0.1.1

gems/kettle-jem/CHANGELOG.md

@@ -23,6 +23,7 @@ Please file a bug if you notice a violation of semantic versioning.
 - kettle-jem: corporate sponsor configuration now accepts the explicit target_href and image_src keys alongside legacy url and img_src, preserving existing configs while supporting direct, date-bounded static sponsors.
 - kettle-jem: opt-in Duel performance regression guards (https://github.com/sferik/duel-ruby), off by default. With `workflows.duel: true`, the full template generates the Perf workflow (`.github/workflows/perf.yml`), `gemfiles/modular/duel.gemfile` (Duel and StackProf on Ruby 3.4 or later only), a `perf` appraisal, the root Gemfile wiring, a `rake duel` task with a stub for older Rubies, the RuboCop include for `.duel.rb`, and the Duel README badge. It creates `.duel.rb` only when it is missing. While Duel is off, a project's own `.duel.rb`, `perf.yml`, and `duel.gemfile` are neither written nor removed.
 - kettle-jem: `actions/download-artifact` joins the template workflow pins, for the Perf workflow's Comment job.
+- kettle-jem: the Duel template follows Duel 0.1.1, which it now requires. The Perf workflow's Comment job posts a new comment only when Duel says to (by default when a benchmark fails, as `c.comment` in `.duel.rb` chooses), and never edits one, so the pull request keeps every report. `.duel.rb` adds only the Perf workflow to what Duel watches, as Duel 0.1.1 watches the benchmarks, `.duel.rb`, and the version files itself.
 
 ### Changed
 

gems/kettle-jem/README.md

@@ -381,14 +381,14 @@ With Duel on, the full template generates:
 
 | File | What it holds |
 |------|---------------|
-| `.github/workflows/perf.yml` | A read-only Duel job that compares each pull request with the main branch, each push to `main` with the commit before it, and, on a schedule, the last release. A separate Comment job, with only `pull-requests: write` and none of the branch's code, posts the report on the pull request. When the baseline's version file names another Ruby, the workflow installs that Ruby too. |
+| `.github/workflows/perf.yml` | A read-only Duel job that compares each pull request with the main branch, each push to `main` with the commit before it, and, on a schedule, the last release. A separate Comment job, with only `pull-requests: write` and none of the branch's code, posts the report on the pull request as a new comment, when a benchmark fails by default (`c.comment` in `.duel.rb` chooses when), and again when a later run's outcome differs from its last comment's. When the baseline's version file names another Ruby, the workflow installs that Ruby too. |
 | `gemfiles/modular/duel.gemfile` | Duel and StackProf, on Ruby 3.4 or later only, so older Rubies install neither. |
 | `Appraisals` | A `perf` appraisal, whose generated `gemfiles/perf.gemfile` the Perf workflow bundles. |
 | `Gemfile` | `eval_gemfile "gemfiles/modular/duel.gemfile"`, for `rake duel` in local development. |
 | `Rakefile` | `rake duel`, with a stub task that only warns where Duel isn't installed. |
 | `.rubocop.yml` | `.duel.rb` in `AllCops/Include` (RuboCop skips dotfiles otherwise), merged with RuboCop's own patterns. |
 | `README.md` | The Duel badge, at the end of the badge row. |
-| `.duel.rb` | Duel's configuration, created only when it is missing. It runs each version on the Ruby that its own version file names, and watches the benchmarks, `.duel.rb`, `mise.toml`, and the Perf workflow as well as `lib/` and `ext/`. After that it is the project's own. |
+| `.duel.rb` | Duel's configuration, created only when it is missing. It runs each version on the Ruby that its own version file names, and watches the Perf workflow on top of what Duel watches already: `lib/`, `ext/`, the benchmarks, `.duel.rb`, and the version files. After that it is the project's own. |
 
 The benchmarks in `duel/` are the project's to write: see
 [Write a benchmark](https://github.com/sferik/duel-ruby#write-a-benchmark).

gems/kettle-jem/lib/kettle/jem/templates/.duel.rb.example

@@ -4,17 +4,22 @@
 #
 # Duel follows conventions, so set only what differs: benchmarks are in
 # duel/**/*_duel.rb, each version's lib/ is first on the load path, a
-# change is compared when it touches lib/ or ext/, and releases are
-# tagged v*.
+# change is compared when it touches lib/ or ext/ (or the benchmarks, this
+# file, or the version files), and releases are tagged v*.
 #
 # kettle-jem creates this file when it is missing, and leaves it alone
 # after that, so it is this project's to change.
 Duel.configure do |c|
   # Each version runs on the Ruby that its own version file names, so a
   # branch that upgrades Ruby is compared on each version's own Ruby.
   c.ruby = :version_file
-  # Compare a change that touches what's measured or how, not only lib/ and
-  # ext/: the benchmarks and this file, the Ruby that mise.toml names, and
-  # the Perf workflow. (Each version runs the candidate's benchmarks.)
-  c.watch = %w[lib ext duel .duel.rb mise.toml .github/workflows/perf.yml]
+  # Compare a change to the Perf workflow too, which decides how the code
+  # is measured. (Duel already watches lib/, ext/, the benchmarks, this
+  # file, and the version files.)
+  c.watch += %w[.github/workflows/perf.yml]
+
+  # When the workflow comments on a pull request with the report: :always,
+  # :results (passed or failed), :failures (failed, or couldn't compare),
+  # or :never.
+  # c.comment = :failures
 end

gems/kettle-jem/lib/kettle/jem/templates/.github/workflows/perf.yml.example

@@ -32,6 +32,11 @@ jobs:
     if: "github.event_name != 'push' || (!contains(github.event.head_commit.message, '[ci skip]') && !contains(github.event.head_commit.message, '[skip ci]'))"
     name: Duel
     runs-on: ubuntu-latest
+    # Whether the Comment job comments (post, change or none), and the
+    # run's status (passed, failed or neutral), from Duel.
+    outputs:
+      comment: ${{ steps.compare.outputs.comment }}
+      status: ${{ steps.compare.outputs.status }}
     env: # $BUNDLE_GEMFILE must be set at job level, so it is set for all steps
       # The perf appraisal: the gemspec's dependencies and Duel (Ruby 3.4 or later)
       BUNDLE_GEMFILE: ${{ github.workspace }}/gemfiles/perf.gemfile
@@ -110,9 +115,11 @@ jobs:
 
       # Against the main branch for a pull request, what was there before
       # for a push, and the last release on a schedule (slow creep). Duel
-      # adds its report to the job's summary, and writes it to duel.md for
-      # the comment.
+      # adds its report to the job's summary, writes it to duel.md for the
+      # comment, and sets the step's outputs to say whether to comment, as
+      # c.comment in .duel.rb says, whatever the outcome.
       - name: Compare
+        id: compare
         env:
           DUEL_BASE: ${{ github.event_name == 'schedule' && 'last-release' || github.event_name == 'push' && github.event.before || '' }}
           # (Run by hand, it compares anyway: with nothing changed, the code
@@ -137,15 +144,17 @@ jobs:
           if-no-files-found: ignore # none when the comparison didn't run
           retention-days: 1
 
-  # The report, as a comment on the pull request: one comment, updated on
-  # every push. This job checks nothing out and runs none of the branch's
-  # code, so it alone can write. (A pull request from a fork gets no
-  # comment, as its token can't write one: the Duel job's summary has the
-  # report.)
+  # The report, as a new comment on the pull request, when Duel says to:
+  # always (post), or only when its last comment's status differs from
+  # this run's (change), so the latest comment is never stale. It never
+  # edits or deletes a comment. This job checks nothing out and runs none
+  # of the branch's code, so it alone can write. (A pull request from a
+  # fork gets no comment, as its token can't write one: the Duel job's
+  # summary has the report.)
   comment:
     name: Comment
     needs: duel
-    if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository }}
+    if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && (needs.duel.outputs.comment == 'post' || needs.duel.outputs.comment == 'change') }}
     runs-on: ubuntu-latest
     permissions:
       pull-requests: write # to comment on the pull request with the report
@@ -163,13 +172,15 @@ jobs:
         env:
           GH_TOKEN: ${{ github.token }}
           PR: ${{ github.event.pull_request.number }}
+          COMMENT: ${{ needs.duel.outputs.comment }}
+          STATUS: ${{ needs.duel.outputs.status }}
         run: |
-          marker="<!-- duel -->"
-          { echo "$marker"; cat "$RUNNER_TEMP/report/duel.md"; } > "$RUNNER_TEMP/comment.md"
-          ids=$(gh api --paginate "repos/$GITHUB_REPOSITORY/issues/$PR/comments" \
-            --jq ".[] | select(.user.login == \"github-actions[bot]\" and (.body | startswith(\"$marker\"))) | .id")
-          if [ -n "$ids" ]; then
-            gh api --method PATCH "repos/$GITHUB_REPOSITORY/issues/comments/${ids%%$'\n'*}" -F body=@"$RUNNER_TEMP/comment.md" > /dev/null
-          else
+          marker="<!-- duel"
+          # The first line of the last comment Duel posted here: its marker,
+          # with its status (none in Duel 0.1.0's, which counts as another).
+          last=$(gh api --paginate "repos/$GITHUB_REPOSITORY/issues/$PR/comments" \
+            --jq ".[] | select(.user.login == \"github-actions[bot]\") | .body | split(\"\n\")[0] | rtrimstr(\"\r\") | select(startswith(\"$marker status=\") or . == \"$marker -->\")" | tail -n 1)
+          if [ "$COMMENT" = post ] || { [ -n "$last" ] && [ "$last" != "$marker status=$STATUS -->" ]; }; then
+            { echo "$marker status=$STATUS -->"; cat "$RUNNER_TEMP/report/duel.md"; } > "$RUNNER_TEMP/comment.md"
             gh api --method POST "repos/$GITHUB_REPOSITORY/issues/$PR/comments" -F body=@"$RUNNER_TEMP/comment.md" > /dev/null
           fi

gems/kettle-jem/lib/kettle/jem/templates/gemfiles/modular/duel.gemfile.example

@@ -9,7 +9,7 @@
 # Performance regression guards (see .duel.rb and duel/)
 # Duel requires Ruby 3.4 or later, so older Rubies install neither gem.
 if Gem::Version.new(RUBY_VERSION) >= Gem::Version.new("3.4")
-  gem "duel", "~> 0.1", ">= 0.1.0" # ruby >= 3.4
+  gem "duel", "~> 0.1", ">= 0.1.1" # ruby >= 3.4
 
   # For profiles of regressions (StackProf runs on CRuby only)
   platform :mri do

gems/kettle-jem/spec/integration/duel_template_spec.rb

@@ -114,14 +114,26 @@ def with_project(name)
       expect(checkout.fetch("with")).to include("fetch-depth" => 0, "persist-credentials" => false)
       baseline_ruby = workflow_yaml.dig("jobs", "duel", "steps").find { |step| step["id"] == "baseline-ruby" }
       expect(baseline_ruby.fetch("with")).to include("bundler-cache" => true)
+      compare = workflow_yaml.dig("jobs", "duel", "steps").find { |step| step["name"] == "Compare" }
+      expect(compare.fetch("id")).to eq("compare")
+      expect(workflow_yaml.dig("jobs", "duel", "outputs")).to eq(
+        "comment" => "${{ steps.compare.outputs.comment }}",
+        "status" => "${{ steps.compare.outputs.status }}"
+      )
+      expect(workflow_yaml.dig("jobs", "comment", "if")).to end_with(" && (needs.duel.outputs.comment == 'post' || needs.duel.outputs.comment == 'change') }}")
+      comment_step = workflow_yaml.dig("jobs", "comment", "steps").find { |step| step["name"] == "Comment on the pull request" }
+      expect(comment_step.fetch("env")).to include("COMMENT" => "${{ needs.duel.outputs.comment }}", "STATUS" => "${{ needs.duel.outputs.status }}")
+      expect(comment_step.fetch("run")).to include(%(marker="<!-- duel"\n), %(echo "$marker status=$STATUS -->"), "--method POST")
+      expect(workflow).not_to include("PATCH")
       %w[actions/checkout ruby/setup-ruby actions/upload-artifact actions/download-artifact].each do |action|
         expect_pinned_action(workflow, action)
       end
 
       expect(files.fetch(:"gemfiles/modular/duel.gemfile")).to include(%(if Gem::Version.new(RUBY_VERSION) >= Gem::Version.new("3.4")))
       expect_gem_dependency_declared(files.fetch(:"gemfiles/modular/duel.gemfile"), "duel")
+      expect(files.fetch(:"gemfiles/modular/duel.gemfile")).to include(%(gem "duel", "~> 0.1", ">= 0.1.1"))
       expect_gem_dependency_declared(files.fetch(:"gemfiles/modular/duel.gemfile"), "stackprof")
-      expect(files.fetch(:".duel.rb")).to include("c.ruby = :version_file")
+      expect(files.fetch(:".duel.rb")).to include("c.ruby = :version_file", "c.watch += %w[.github/workflows/perf.yml]\n", "# c.comment = :failures\n")
       expect(files.fetch(:Gemfile).scan(%(eval_gemfile "gemfiles/modular/duel.gemfile")).size).to eq(1)
       expect(files.fetch(:Gemfile).index("gemfiles/modular/duel.gemfile")).to be < files.fetch(:Gemfile).index("gemfiles/modular/x_std_libs.gemfile")
       expect(appraisals_eval_gemfile_paths(files.fetch(:Appraisals), "perf")).to eq(%w[modular/duel.gemfile modular/x_std_libs.gemfile])