Repository navigation
feat(builder): draw the Benchmark step with the Benchmarks page's cards, as cards or a table - #267
Merged
harenome merged 8 commits intoNov 20, 2026
Conversation
recipe.benchmarks gives the catalogue's listed benchmarks with their datasets, tasks, machine needs and what they are run with (agents, or a kind of capability). The Benchmarks page draws its cards from it through catalogue.benchmark_cards, with the same output as before, so the builder's Benchmark step can draw the same cards.
A pick with one: true is a single choice: choosing its card lets the others go and choosing it again keeps it; in a table its box is a radio. Toggle picks are unchanged.
…ds, as cards or a table GET /new-run/benchmarks returns the step's grid: the Benchmarks page's cards as single-choice pick cards, or a table with ?view=table. The category pills stay; Run with (agents or capabilities) is a menu of its own beside Filter (Runner), Sort by and the cards/table toggle. The builder loads it once per view, switches the view in place and keeps it per viewer, and presses the recipe's benchmark, which stays in view under any filter, also after a recipe import. The step's own tiles, search and chips are gone.
A name column that takes the spare width broke a long name inside a word on a phone (LongMemEval as "LongMemEv al"). The name now wraps only between words, and the table scrolls sideways instead.
The benchmarks' listing carries each other version's tasks (OSWorld1's main pin: 369, its default tag: 368). The Benchmark step's card and row of a benchmark with versions hold them as data-tasks, and the builder shows the chosen benchmark's count for the version its recipe names, in place when the version changes, as the step's tiles did. The card toolkit takes data attributes for a card and its row.
…Evaluates, with a target icon "Run with" read oddly beside the runner, and the plug did not say it. The menu, its column and its field are now Evaluates (Agents, Capabilities), led by a new i-target icon in the sprite: three concentric circles in the sprite's line style. i-plug stays for Settings.
A chosen pick card takes --accent-soft as its background, so what is filled with it inside the card (a benchmark's category tag, an agent's initials) lost its fill, and in the dark theme an outlined tag (an agent's protocol) lost its outline. Inside a chosen card both are now a shade nearer the accent, so they stand out as on any card, on every pick card, in both themes. Unchosen cards do not change.
ThinkFlowLab#261 added the versions field to the catalogue's Benchmark without refreshing the pin; this records it, so its removal fails the check.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #266
What
The builder's Benchmark step now shows the same listing as the Benchmarks page, drawn with the shared card and table toolkits. Before,
command-builder.jsbuilt its tiles from three sources: the standard payload, the Harbor list fromlaunch.py, and catalogue lookups.recipe.benchmarks(cat)gives the listed benchmarks with their datasets, tasks, machine needs, what they evaluate (agents, or a kind of capability) and each other version's tasks.catalogue.benchmark_cardsmakes the cards from it for the Benchmarks page and the step. The Benchmarks page HTML does not change, except for one new sprite icon.GET /new-run/benchmarksreturns the step's grid (builder/benchmarks.html), with?view=tablefor the table. The builder loads it once per view, with the same loader as the Agent step's grid.one: trueis radio-like. A click on a card chooses it and lets the others go. A second click keeps it chosen. In the table, the narrow left column has a radio, and a row click chooses it. Toggle picks do not change.main).i-targeticon, a Filter menu (Runner), Sort by (Name, Tasks), and the cards/table toggle. The toggle loads the other view in place: the page does not reload, the choice and the draft stay, andlocalStoragekeeps the viewer's choice.--accent-softand a tag outlined with--bordertake a shade nearer the accent, so they keep their fill and outline in both themes.Benchmark.versionsis pinned. feat(catalogue): add OSWorld desktop-env-v1.0.3 as OSWorld1's default version #261 added the field without a pin refresh.Touches
backend (
nullius/standard/recipe.py) · routes (web/builder.py,web/catalogue.py) · templates and static (_cards.html,base.html,builder/benchmarks.html,tables.js,catalogue/catalogue.css,builder/command-builder.js,command-builder.css) · docs (builder.md) · tests (field_contract.json)Before and after
Verified
python:3.12, Node 20):python3 app/tests/parallel.py: 3108 tests in 159 modules, OK.pip install -r app/requirements.txt, thenpython3 app/tests/parallel.py app/ui_tests: 703 tests in 58 modules, OK.python3 app/tests/catalogue_docs.py --writeandpython3 app/tests/field_contract.py --writeleave the tree unchanged.app/ui_tests/test_recipe_execution_fixtures.pyand its fixtures are unchanged.test_benchmark_listing.py: The listing is the catalogue's listed benchmarks with their family. A benchmark with versions counts each version's tasks.test_builder_benchmarks.py: The step lists the Benchmarks page's benchmarks, each a single choice. A benchmark with versions carries each version's tasks. The controls are in place. The table chooses one with radios. The Benchmarks page stays a page of links.test_cards.py: A single choice is marked on its card. It lets the others go and stays chosen. A row click chooses its radio./benchmarksHTML of both servers differs only in thei-targetsymbol.main. A search for "gsm" left GSM8K and the chosen OSWorld1. A click on GSM8K left only GSM8K chosen.Not checked: a recipe import in the browser, and a real run from the step.
Risk