Repository navigation
Triage: catalogue EvoRiskBench, PAGE and STEER-Bench (#196, #194, #198) - #222
Open
emmanuelgjr wants to merge 1 commit into
Open
emmanuelgjr wants to merge 1 commit into
emmanuelgjr wants to merge 1 commit into
Conversation
Three benchmark rows from the watcher triage of #196, #194 and #198. None of the three has released its benchmark, so each row records it as not released, quoting the paper or noting the arXiv record links none. PAGE and STEER-Bench are introduced by defense papers; each row names the authors' own defense evaluated on it, as the APort Vault row does for its disclosure. The OWASP entry column is DRAFT, as for every row in the catalogue. docs/TRIAGE_RULES.md rule 3 records the maintainer ruling that a defense paper introducing a named benchmark gets a row for the benchmark, and one with no named benchmark is noted and closed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 9, 2026
Open
Open
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three rows for
evals/EXTERNAL_BENCHMARKS.md, from the watcher triage of the issues below, plus a precedent note indocs/TRIAGE_RULES.mdrule 3.Closes #196
Closes #194
Closes #198
The rows
Every figure and quotation is taken from the paper's abstract, as the catalogue's header requires. Nothing has been re-run.
The precedent (maintainer ruling, 2026-10-09)
This PR adds the first catalogue rows for benchmarks that are a secondary contribution of a defense paper. Rule 3 now says:
C4
DRAFT — SME review required. The OWASP entry column is a reading of each benchmark's stated scope, not a reviewed mapping, under the catalogue's existing DRAFT banner.
Checks (local)
validate.js: 0 errors / 92 warnings / 329 passed, the same before and after.test:scripts: 137/137.stats:check: current.generate.js: no drift.The link check doesn't run on PRs. The three arXiv URLs resolved when checked today.
🤖 Generated with Claude Code