Repository navigation
Add Duel performance regression guards #780
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+614
−7
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| # frozen_string_literal: true | ||
|
|
||
| # Duel's configuration: https://github.com/sferik/duel-ruby/blob/main/docs/configuration.md | ||
| # | ||
| # Duel follows conventions, so set only what differs: benchmarks are in | ||
| # duel/**/*_duel.rb, each version's lib/ is first on the load path, a | ||
| # change is compared when it touches lib/ or ext/ (or the benchmarks, this | ||
| # file, or the version files), and releases are tagged v*. | ||
| # | ||
| # kettle-jem creates this file when it is missing, and leaves it alone | ||
| # after that, so it is this project's to change. | ||
| Duel.configure do |c| | ||
| # Each version runs on the Ruby that its own version file names, so a | ||
| # branch that upgrades Ruby is compared on each version's own Ruby. | ||
| c.ruby = :version_file | ||
| # Compare a change to the Perf workflow too, which decides how the code | ||
| # is measured. (Duel already watches lib/, ext/, the benchmarks, this | ||
| # file, and the version files.) | ||
| c.watch += %w[.github/workflows/perf.yml] | ||
|
|
||
| # When the workflow comments on a pull request with the report: :always, | ||
| # :results (passed or failed), :failures (failed, or couldn't compare), | ||
| # or :never. | ||
| # c.comment = :failures | ||
| end |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,186 @@ | ||
| name: Perf | ||
|
|
||
| # Performance regression guards: Duel runs the benchmarks in duel/ on the | ||
| # baseline and on this code side by side, and fails if this code is slower. | ||
| # See .duel.rb and https://github.com/sferik/duel-ruby | ||
|
|
||
| # The Duel job runs this branch's code, so it only reads. Commenting is a | ||
| # job of its own, which runs none of it. | ||
| permissions: | ||
| contents: read | ||
|
|
||
| on: | ||
| pull_request: | ||
| push: | ||
| branches: | ||
| - 'main' | ||
| schedule: | ||
| - cron: "0 4 * * *" | ||
| # Allow manually triggering the workflow. | ||
| workflow_dispatch: | ||
|
|
||
| # Cancels a pull request's previous runs that have not yet completed. Every | ||
| # other run is a group of its own, so none is cancelled: each push to main | ||
| # is compared with the commit before it, and a cancelled run would skip | ||
| # that comparison. | ||
| concurrency: | ||
| group: "${{ github.workflow }}-${{ github.event_name == 'pull_request' && github.ref || github.run_id }}" | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| duel: | ||
| if: "github.event_name != 'push' || (!contains(github.event.head_commit.message, '[ci skip]') && !contains(github.event.head_commit.message, '[skip ci]'))" | ||
| name: Duel | ||
| runs-on: ubuntu-latest | ||
| # Whether the Comment job comments (post, change or none), and the | ||
| # run's status (passed, failed or neutral), from Duel. | ||
| outputs: | ||
| comment: ${{ steps.compare.outputs.comment }} | ||
| status: ${{ steps.compare.outputs.status }} | ||
| env: # $BUNDLE_GEMFILE must be set at job level, so it is set for all steps | ||
| # The perf appraisal: the gemspec's dependencies and Duel (Ruby 3.4 or later) | ||
| BUNDLE_GEMFILE: ${{ github.workspace }}/gemfiles/perf.gemfile | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| fetch-depth: 0 # the baseline comes from history, tags included | ||
| persist-credentials: false | ||
|
|
||
| # .duel.rb sets c.ruby = :version_file, so each version runs on the Ruby | ||
| # that its own version file names (.ruby-version, .tool-versions, or | ||
| # mise.toml, the first that names one, as Duel reads them). When the | ||
| # baseline's names another Ruby than this one's, install that Ruby too, | ||
| # and link it where Duel finds it. The baseline is the one the Compare | ||
| # step uses: the main branch for a pull request, the commit before a | ||
| # push, the last release on a schedule (the newest v* tag before HEAD, | ||
| # as Duel finds it), and where HEAD left the default branch when run by | ||
| # hand. | ||
| - name: Find the baseline's Ruby | ||
| id: baseline | ||
| env: | ||
| BEFORE: ${{ github.event.before }} | ||
| DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} | ||
| run: | | ||
| case "$GITHUB_EVENT_NAME" in | ||
| pull_request) base="origin/$GITHUB_BASE_REF" ;; | ||
| push) base="$BEFORE" ;; | ||
| schedule) base=$(git describe --tags --abbrev=0 --match 'v*' HEAD^ 2>/dev/null) || base= ;; | ||
| *) base=$(git merge-base HEAD "origin/$DEFAULT_BRANCH" 2>/dev/null) || base= ;; | ||
| esac | ||
| ruby_of() { | ||
| for file in .ruby-version .tool-versions mise.toml .mise.toml; do | ||
| text=$(git show "$1:$file" 2>/dev/null) || continue | ||
| case "$file" in | ||
| .ruby-version) version=$(printf '%s\n' "$text" | awk 'NF { print $1; exit }') ;; | ||
| .tool-versions) version=$(printf '%s\n' "$text" | awk '$1 == "ruby" { print $2; exit }') ;; | ||
| *) version=$(printf '%s\n' "$text" | awk '/^\[/ { tools = ($0 ~ /^\[tools\]/) } tools && match($0, /^[ \t]*"?ruby"?[ \t]*=[ \t]*[[{]?[ \t]*(version[ \t]*=[ \t]*)?"[^"]+"/) { s = substr($0, RSTART, RLENGTH); sub(/"$/, "", s); sub(/.*"/, "", s); print s; exit }') ;; | ||
| esac | ||
| if [ -n "$version" ]; then | ||
| echo "$version" | ||
| return | ||
| fi | ||
| done | ||
| } | ||
| ruby= | ||
| if [ -n "$base" ] && git cat-file -e "$base^{commit}" 2>/dev/null; then | ||
| ruby=$(ruby_of "$base") | ||
| [ "$ruby" = "$(ruby_of HEAD)" ] && ruby= | ||
| fi | ||
| echo "ruby=$ruby" >> "$GITHUB_OUTPUT" | ||
|
|
||
| - name: Setup the baseline's Ruby | ||
| id: baseline-ruby | ||
| if: steps.baseline.outputs.ruby != '' | ||
| uses: ruby/setup-ruby@7f797815d8adceb1c412e7abefdd1ab4abc47a5f # v1.335.0 | ||
| with: | ||
| ruby-version: ${{ steps.baseline.outputs.ruby }} | ||
| # The bundle for this Ruby too, which the baseline's workers load. | ||
| # (Gems installed for one Ruby version don't load on another.) | ||
| bundler-cache: true | ||
|
|
||
| - name: Link the baseline's Ruby where Duel finds it | ||
| if: steps.baseline.outputs.ruby != '' | ||
| env: | ||
| PREFIX: ${{ steps.baseline-ruby.outputs.ruby-prefix }} | ||
| run: | | ||
| mkdir -p ~/.rubies | ||
| ln -sfn "$PREFIX" ~/.rubies/ruby-"$("$PREFIX/bin/ruby" -e 'print RUBY_VERSION')" | ||
|
|
||
| # The candidate's Ruby, from its version file (the same one Duel reads) | ||
| - name: Setup Ruby & RubyGems | ||
| uses: ruby/setup-ruby@7f797815d8adceb1c412e7abefdd1ab4abc47a5f # v1.335.0 | ||
| with: | ||
| bundler-cache: true | ||
|
|
||
| # Against the main branch for a pull request, what was there before | ||
| # for a push, and the last release on a schedule (slow creep). Duel | ||
| # adds its report to the job's summary, writes it to duel.md for the | ||
| # comment, and sets the step's outputs to say whether to comment, as | ||
| # c.comment in .duel.rb says, whatever the outcome. | ||
| - name: Compare | ||
| id: compare | ||
| env: | ||
| DUEL_BASE: ${{ github.event_name == 'schedule' && 'last-release' || github.event_name == 'push' && github.event.before || '' }} | ||
| # (Run by hand, it compares anyway: with nothing changed, the code | ||
| # with itself, which checks the setup.) | ||
| DUEL_FORCE: ${{ github.event_name == 'workflow_dispatch' && '1' || '' }} | ||
| run: | | ||
| # (A branch's first push, or one that rewrote history, has no | ||
| # commit before it here to compare against.) | ||
| if [ "$GITHUB_EVENT_NAME" = push ] && ! git cat-file -e "$DUEL_BASE^{commit}" 2>/dev/null; then | ||
| echo "::notice::No commit before this push to compare against" | ||
| exit 0 | ||
| fi | ||
| bundle exec duel --markdown "$RUNNER_TEMP/duel.md" | ||
|
|
||
| # The report, for the Comment job | ||
| - name: Keep the report | ||
| if: ${{ !cancelled() && github.event_name == 'pull_request' }} | ||
| uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 | ||
| with: | ||
| name: duel-report | ||
| path: ${{ runner.temp }}/duel.md | ||
| if-no-files-found: ignore # none when the comparison didn't run | ||
| retention-days: 1 | ||
|
|
||
| # The report, as a new comment on the pull request, when Duel says to: | ||
| # always (post), or only when its last comment's status differs from | ||
| # this run's (change), so the latest comment is never stale. It never | ||
| # edits or deletes a comment. This job checks nothing out and runs none | ||
| # of the branch's code, so it alone can write. (A pull request from a | ||
| # fork gets no comment, as its token can't write one: the Duel job's | ||
| # summary has the report.) | ||
| comment: | ||
| name: Comment | ||
| needs: duel | ||
| if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && (needs.duel.outputs.comment == 'post' || needs.duel.outputs.comment == 'change') }} | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| pull-requests: write # to comment on the pull request with the report | ||
| steps: | ||
| - name: Get the report | ||
| id: report | ||
| continue-on-error: true # the Duel job keeps no report when it fails before comparing | ||
| uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 | ||
| with: | ||
| name: duel-report | ||
| path: ${{ runner.temp }}/report | ||
|
|
||
| - name: Comment on the pull request | ||
| if: steps.report.outcome == 'success' | ||
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
| PR: ${{ github.event.pull_request.number }} | ||
| COMMENT: ${{ needs.duel.outputs.comment }} | ||
| STATUS: ${{ needs.duel.outputs.status }} | ||
| run: | | ||
| marker="<!-- duel" | ||
| # The first line of the last comment Duel posted here: its marker, | ||
| # with its status (none in Duel 0.1.0's, which counts as another). | ||
| last=$(gh api --paginate "repos/$GITHUB_REPOSITORY/issues/$PR/comments" \ | ||
| --jq ".[] | select(.user.login == \"github-actions[bot]\") | .body | split(\"\n\")[0] | rtrimstr(\"\r\") | select(startswith(\"$marker status=\") or . == \"$marker -->\")" | tail -n 1) | ||
| if [ "$COMMENT" = post ] || { [ -n "$last" ] && [ "$last" != "$marker status=$STATUS -->" ]; }; then | ||
| { echo "$marker status=$STATUS -->"; cat "$RUNNER_TEMP/report/duel.md"; } > "$RUNNER_TEMP/comment.md" | ||
| gh api --method POST "repos/$GITHUB_REPOSITORY/issues/$PR/comments" -F body=@"$RUNNER_TEMP/comment.md" > /dev/null | ||
| fi | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.