Skip to content

feat(compare): compare runs in columns, each a workspace and a run, under the table toolkit's controls - #207

Merged
harenome merged 21 commits into
ThinkFlowLab:mainfrom
harenome:feat/compare-columns
Nov 18, 2026
Merged

harenome merged 21 commits into
ThinkFlowLab:mainfrom
harenome:feat/compare-columns

Conversation

@harenome

@harenome harenome commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #206

What

Compare is one page of run columns. The separate picker is gone, and /compare/pick redirects to /compare with its picks.

Columns. The page opens with two columns. With one run or none, an empty column (pick=<workspace>:) waits beside the run. An empty column of two has no Remove. Workspace and Run are the first two rows of the table, labelled in the left column as the metrics are. The header row of each column holds the Baseline mark on the first column, and Move left, Move right and Remove. These are links, so the ordered pick= values stay the comparison. Add a run adds an empty column and stays in the current view.

Run list. A column's Run button opens a searchable list (static/compare/columns.js). The newest launch comes first, and the baseline's dataset comes before the other datasets. Checks are marked. A sharded run shows once, with its shard count. Search matches every word in any order and case, as the table toolkit's search does (NulliusTable.matches, now shared). The arrow keys and Enter pick a run. A list opens only on its button, or after a workspace change. A workspace picked lists its runs in place from a new JSON route, /compare/runs. A run picked opens the comparison with that run in its column. A run that another column holds is greyed, with the tooltip "Already in column N". A click or Enter on it does nothing, and the arrows skip it. The column's own run has a check mark.

Sharded runs. A sharded run is one column with the trials of every shard, folded as the Runs page folds it (runs.offered). A shard's id on a pick= or job= link stands for its whole run, so the Runs page's links keep working. Under the run, the column links each shard as "shard N · machine", from a macro that the Runs page now shares (shard_link).

Rows. A Benchmark row above Dataset gives the catalogue's title, or a dash when the catalogue does not know the dataset. Under each summary number of columns 2 to N, a Δ gives its difference from the baseline. A rate's Δ is in percentage points. Counts, tokens, cost and time use the row's own unit. It is green or red by the row's direction, and uncoloured when either run is a check. With one column, nothing is ranked.

Controls row. The table toolkit's controls row heads the table. Its views slot holds the cohort toggle (Whole runs, Matched tasks) and a view toggle (Side by side, Task by task). With more than one agent, it also holds an Order button with a new i-reorder icon. The Order menu holds Group by agent and Alternate agents. The actions slot holds Add a run. One view shows at a time: ?view=tasks for the task table, no parameter for side by side. Each view computes only its own rows. Every link the page builds keeps the view and the cohort, the judge form and its redirect too. Task by task opens with the same column rows (header, Workspace, Run) above its task header, which names each column by agent and model.

Removed text. The cohort explanation box, the "Side by side" heading, the "Order columns:" row and the Task by task explanation line are gone. So are the notes on a run with no recorded dataset and on repeated attempts: the subtitle and the task rows show those facts. The side table's headers no longer drag to reorder. The arrows replace them there, and the task table keeps dragging. Two warnings stay: no shared cohort (the shared notice) and datasets that differ (one line).

Table toolkit.

  • Sticky rows. A <tr data-sticky> stays in view while the page scrolls down, under the table's controls row, which now sticks under the top bar until the table ends. data-sticky="wide" does this only from 900 px wide. The table's box scrolls on both axes, so a CSS sticky row could pin only to the box. tables.js therefore moves the real rows into a fixed box under the controls row once they pass under it. The box keeps the table's column widths and sideways scroll. An inert, hidden clone holds each row's place. The moved rows keep their listeners. The box follows resizes, the 900 px limit and changes in the table. The table macro gets a sticky_head option that marks its header row.
  • One height. The controls row sets --tool-h, and every button, segmented toggle and filter pill in it takes that height.
  • .seg-toggle: a segmented toggle of links or buttons, the current one marked.
  • A toolbar menu also puts the focus on its first link. A menu ignores a modal over the page. The first-visit guide gives the focus back after its answer, so it no longer closes a list behind it.
  • controls.js adds no custom select to an inert clone.

Docs: the field guide defines the baseline difference and the one column of a sharded run. The results guide says how to pick each column. The CLI page map drops /compare/pick.

Touches

backend (nullius/compare/comparison.py, one parser option) · routes (web/compare.py, web/runs.py, web/judge.py) · templates and static (compare/compare.html, compare/_detail.html, compare/compare_task.html, compare/compare_picker.html removed, runs/_runtime.html, runs/index.html, _tables.html, base.html, compare/columns.js, compare/comparison.css, tables.js, controls.js, onboarding.js, style.css) · docs (docs/field-guide.md, docs/results.md, docs/cli/pages.md)

Before and after

Seeded data: a test profile with four workspaces of fixture runs (one empty), and a launch sharded over three machines. Before is main, after is this branch, same URL and width. Two runs of the same dataset unless named.

Before After
Side by side, 1280 px, light before-side-1280-light after-side-1280-light
Task by task, 1280 px, light (before: the task table under the side table) before-tasks-1280-light after-tasks-1280-light
Side by side, 390 px, dark before-side-390-dark after-side-390-dark
Task by task, 390 px, dark before-tasks-390-dark after-tasks-390-dark
Scrolled down, 1280 px: after, the identity rows float under the controls row before-side-scrolled-1280-light after-side-scrolled-1280-light
Choosing a run (before: the picker page. After: the second column's list, the baseline's run greyed, its own run checked) before-run-list-1280-light after-run-list-1280-light
Four runs, one sharded, one from another workspace before-four-1280-light after-four-1280-light

Verified

In a throwaway rootless python:3.12 container with Node.js and uv, on a copy of the branch:

  • python3 app/tests/parallel.py: 2249 tests in 127 modules, OK.
  • pip install -r app/requirements.txt, then python3 app/tests/parallel.py app/ui_tests: 637 tests in 53 modules, OK.
  • New: test_compare_kept.py pins every row and section the page showed before the change, by name. test_compare_columns.py runs columns.js under Node. It checks the list's order and separators, search and keys, a greyed run that a click and Enter skip, a modal over an open list, and a workspace change. test_comparison_routes.py covers the columns, the empty column, the Benchmark row, the deltas and the links. It also covers the controls row, the two views, the sticky rows, the warnings, /compare/runs and the /compare/pick redirect. test_launch_shards.py covers a sharded run as one column. test_tables.py covers the floating rows, a table without sticky rows and sticky_head.

Browser check, Chromium (Playwright 1.49.1), on the seeded data. No page JavaScript error, before or after.

  • The shots above.
  • At 1280 px and 390 px: /compare alone gives two empty columns with no list open. In Task by task, Add a run keeps the view and opens no list. The new column's list greys the two runs already picked. A run picked there, Move right and Remove keep the view. A shard's id gives one column with three shard links. The Order menu puts the focus on Group by agent.
  • At the end of the page, the floating box holds the header, Workspace, Run, Agent, Model and Benchmark rows at 1280 px. At 390 px it holds the header and Agent rows. Back at the top, the rows return to the table and no placeholder stays.

Not checked: the feel of real wheel and touch scrolling, dragging a task column header, other browsers and a screen reader. Below 640 px, the task table's first column still scrolls away sideways, as on main.

Risk

  • The sticky rows are generic toolkit code. Only compare/compare.html marks rows with data-sticky. No template passes sticky_head yet. Every table with sticky columns runs the changed NulliusTable.sticky: All runs (runs/index.html), Launches (runs/launches.html) and Recipes (builder/recipes.html). Without data-sticky rows it adds no box and no sticky controls row, but its observer now also watches the table's size. The one-height rule (--tool-h) applies to every controls row: All runs, Launches, Workspaces, Recipes and the four catalogue pages.
  • Limits of the floating rows. While they float, they are outside the <table>, so a listener on the table itself does not hear them. Listeners on the rows and their controls still work. The floating copy has no id, so a style keyed on the table id does not reach it, for example the table macro's expand column styles. A menu open in a floating row loses the focus when the page scrolls back and the row returns to the table.
  • Links. /compare/pick redirects, so old links still open. A shard's id now opens its whole run, not the shard alone.
  • To undo, revert the branch's commits. The change migrates nothing, and no stored data changes.

The run lists Compare adds search as the toolkit's tables do: every word, in
any order and case. NulliusTable.matches is that test, and the toolbar's
search now calls it.
The summary rows, conditional ones included, the detail rows, the cohort
switch, the cohort note, the judgement and both sections, so a redesign of
the columns cannot lose one.
The comparison shows from one run on, a column a run, the first the
baseline. Each column header picks its workspace and its run: the run list
is searchable, newest launch first, the baseline's dataset first, checks
marked, a sharded run once. A workspace picked lists its runs in place from
/compare/runs; a run picked opens the comparison with it. Add a run, remove
and reorder are links, so the ordered pick= values stay the comparison and
its cohort rides along. An empty column is pick=<workspace>:.

Under each summary number of columns 2..N, its difference from the
baseline in the row's unit, coloured by the row's direction. The best is
still marked; one column ranks nothing.

A shard's id stands for its whole run: Compare folds a participant's shards
as the Runs page does (runs.offered over _grouped and _fold), so the
Runs page's job= links keep working. The task table, its detail rows and the
per-task page follow the columns. /compare/pick redirects to /compare.
The field guide defines the difference from the baseline and the one column
a sharded run makes; Compare runs says how to pick each column; the page map
drops the old picker.
The guide's dialog took the focus and the answer's click, and the menu read both as leaving it, so on a first visit the list an empty column opens was closed behind it. A menu now ignores a modal over the page, and the guide gives the focus back when answered.
A single column filled the table's whole width. With one run or none, an
empty column now waits beside it, and an empty column of two offers no
remove, which would bring it straight back.
The workspace and run pickers sat in each column's header with no label.
They are now the table's first two rows, Workspace and Run, labelled in
the same left column as every metric below; the header keeps the
baseline mark and the column's tools.
A sharded run is one column, folded as Runs folds it, but the job id
under its run linked only the first shard: the folded run takes that
shard's job id. The column now links each shard as Runs lists them,
"shard N · machine", from one macro both pages share.
The side-by-side table showed a run's dataset but not the benchmark it
belongs to. A Benchmark row above Dataset gives the catalogue's title,
found the way each column's run list already names it, or a dash when
the catalogue does not know the dataset.
An empty column no longer opens its list when the page loads, nor after
"+" adds it. At 390 px the open list covered the table.
…t scrolls sideways

A row with data-sticky stays at the top of its table, at the summed heights
of the sticky rows above it; data-sticky="wide" pins it only from 900px wide,
where a tall row leaves room for the rest. The table's box, which already
scrolls sideways and so pins to itself, gets no taller than the window under
the top bar and the table's controls row, which sticks too. The offsets come
from the same place as the sticky columns', measured again as rows resize,
come in or the window crosses 900px. The table macro's sticky_head pins its
header row and keeps its controls row to the table.
The cohort switch and an Order menu are its views, Add a run its action,
in the table toolkit's controls row instead of a hand-written one and a
switch of its own above. Order opens, as Sort by does, on the two one-off
reorders, Group by agent and Alternate agents, still links; its icon is
Sort by's on its side. A toolbar menu puts the focus on its first link too.
The row heads the table in place of its heading; the table's box keeps
its name for a screen reader.
…in view

The side table opts into the toolkit's sticky rows: its header and the
Workspace, Run, Agent, Model and Benchmark rows stay at the top as it
scrolls down, under its controls row; below 900px wide, where the Run row
would cover the screen, only the header and the Agent row. An empty
column's grey is an image now, over the pinned rows' opaque colour.
The controls row's buttons, a text-only button, filter pills and Compare's
cohort toggle were four different heights. The row now sets --tool-h, a small
button's height with its icon, and every control in it takes that height.
The cohort toggle becomes the toolkit's segmented toggle, .seg-toggle.
…hort warning

The box under Compare's controls row explained each cohort, the adjusted
rate and repeated attempts. What it counted the page already shows: the
subtitle has the tasks run by every side, a task row its attempts and
"Dataset not recorded". Only the warning that a matched cohort is empty
stays, as the shared notice. compare_task.html still uses .cohort-note.
… run list

A run another column holds is greyed in the list, its tooltip naming that
column; a click or Enter does nothing on it and the arrows skip it, while
search still finds it. The page's picks already name a sharded run by its
whole run, so a shard picked counts as its run. The column's own run is
checked, and an empty column greys every run the others hold, also after a
workspace change refills its list.
…a time

A second toggle in the controls row, Side by side | Task by task, picks the
view (?view=tasks; side by side has no parameter). Every link the page builds
keeps it, as it keeps the cohort. The tasks view pins the task table's header
under the controls row and names each column by agent and model; the side view
is unchanged. Each view renders and computes only its own rows: the summary and
the run lists for side by side, the judgement for the tasks.
Each column's header (Baseline, move, remove), workspace and run rows are one
macro that both tables open with, pinned the same way, so a run is picked,
moved or removed in the tasks view too and Add a run fills its column there.
The tasks view embeds the run lists again.
…scrolling its box

A table with sticky rows had a wrapper no taller than the window, scrolled on both axes, so the page and the table each scrolled vertically. The wrapper now scrolls only sideways; once the sticky rows pass under the sticky controls row, the rows themselves move into a fixed box under it, kept at the table's sideways scroll and column widths, while a hidden, inert copy holds their place. Their controls keep their listeners, and the box follows resizes, the 900px rows and changes in the table.
@harenome
harenome merged commit be36d38 into ThinkFlowLab:main Nov 18, 2026
2 checks passed
@harenome
harenome deleted the feat/compare-columns branch November 18, 2026 14:09
harenome added a commit that referenced this pull request Nov 19, 2026
Closes #217

## What

Compare gets a third view, Chart (`?view=chart`), beside Side by side
and Task by task. It has the same columns, cohort and controls row. The
column rows (header, Workspace, Run) stay above the charts, so you
change the runs as in the other views.

**Cards.** Each chart is a card. Its header is one row of toolkit
buttons, all `--tool-h` high, with Remove at its right end:

- Style, with the chart icon. Its menu lists Bars, Dots and Scatter from
the registry of styles, and a check marks the current one.
- For Bars and Dots, a Data button with the sliders icon and the chosen
metric's name. Its menu lists the side table's summary rows, in the
table's groups, and a check marks the current one.
- For Scatter, an X button and a Y button. Each has its letter in the
icon slot and its metric's name, and opens the same menu for its axis.
- Remove, the same × as a column's Remove, with the name "Remove this
chart".

Every menu item is a link, and `NulliusTable.menu` opens the menus, so
the change adds no other script. A button never breaks its name over two
lines. When a row is too narrow, whole buttons move to a second row, and
Remove stays at the end of the first. At 1280 px, a Scatter card's
Style, X and Y fit on one row, also with long metric names. A style
change keeps the card's first metric. For Scatter, Y becomes the
headline rate, or the cost when the rate is X. Add chart, in the actions
slot beside Add a run, appends a Bars card of the headline rate.

**URL.** Each card is one `chart=` value, in order:
`chart=bars:adjusted_rate&chart=scatter:cost,adjusted_rate`. The metric
keys are stable names, not labels. Every link on the page keeps them,
with the view and the cohort. Without `chart=`, the page shows three
cards: Bars of the headline rate, Bars of the cost, and a Scatter of the
cost against the rate. The headline rate is the adjusted rate, or the
mean score for a benchmark that only scores. `chart=` alone shows one
Add chart button. A card whose metric this comparison does not have
keeps its controls and says "Not in this comparison."

**Styles.**

- Bars: one bar for each run, from zero, on the metric's own axis and
unit. A dashed mark in each row shows the baseline's value.
- Dots: the same layout with dots. The axis covers the values, not zero,
so close values stay apart.
- Scatter: one dot for each run, numbered by its column, with a key
under the plot. Runs at the same point sit in a ring around it, each
with its own number. A number goes on the first free side of its dot
inside the plot.

**Values.** Each value is the side table's own cell: the number its
difference uses, and the text it shows. A missing value has no mark,
only its dash. A check is hollow and named "(check)". A check's $0.00 or
0 tokens is a real zero, as in the table, so it has a mark. Nothing is
ranked. The tick labels use the row's unit: % for rates, $ for cost, k
and M for tokens, seconds or minutes for time. A tick label that would
touch the one before it is left out, but its gridline stays.

**Colours.** Each run has one colour in every chart: its column's place
on the page picks `--series-1` to `--series-8`, and the ninth column
starts again at the first. The drawing data puts the index on each mark
as a class (`s1` to `s8`), and the stylesheet maps each class to its
token, so the template has no colour logic. A check stays hollow, with
its outline in the run's colour. The baseline mark stays a dashed grey
line. The key under a Scatter gives each run a small swatch beside its
number and name, so a colour is never the only cue.

**The series palette.** Eight new tokens, `--series-1` to `--series-8`,
in `style.css`. They are muted blues, two teals, a brown and a plum, in
the app's existing lightness range. None of them reads as an outcome
colour for a pass (sage), a failure (red) or a warning (amber). The
nearest pair is the dark brown and the warning amber, at ΔE 9.5, and
every other pair is at 12.5 or more. The trace categories keep their own
`--cat-*` colours, so a colour has one meaning on each page. The dark
values serve the default dark theme and the charcoal, graphite and
midnight palettes, and they pass on the surface of each.

| Slot | Light | Dark |
| --- | --- | --- |
| 1 | `#215da5` | `#3a75bf` |
| 2 | `#179696` | `#16a8a8` |
| 3 | `#7d3f17` | `#ad6a44` |
| 4 | `#657dd1` | `#748ee3` |
| 5 | `#7e2e59` | `#aa5d83` |
| 6 | `#1587ba` | `#3aa0d4` |
| 7 | `#5e5197` | `#7768be` |
| 8 | `#2695ac` | `#38a3ba` |

The data-viz palette validator checks the colours against the card
surface (`--surface-2`). All checks pass in both modes:

| | Light (`#edf0f4`) | Dark (`#292b2f`, `#202024`, `#191f29`,
`#18263c`) |
| --- | --- | --- |
| Lowest contrast | 3.07:1 | 3.02:1 (default dark); 3.24:1 or more on
the others |
| Adjacent slots, protan and deutan ΔE (target 8) | 10.7 or more | 10.1
or more |
| Adjacent slots, normal vision ΔE (floor 15) | 15.2 or more | 15.2 or
more |
| First 4 slots, every pair, CVD ΔE | 10.4 or more | 9.5 or more |
| First 5 slots, every pair, CVD ΔE | 9.4 or more | 8.1 or more |
| Lightness band and chroma floor | pass | pass |

ΔE is the OKLab distance ×100, with protanopia and deuteranopia
simulated by Machado-Oliveira-Fernandes at full severity. Slots 6 to 8
repeat the blue and teal families, so a scatter of more than five runs
also relies on the numbers and the key. Another view can use the palette
with the class `s1` to `s8` on a mark, or `var(--series-n)` directly,
for any set of items with a stable order.

**Chart module.** `nullius/charts.py` computes the geometry in SVG
units. `templates/_charts.html` draws the marks and texts as it gets
them, with no arithmetic. Each chart is an SVG with `role="img"` and a
title. Each mark has a tooltip, and a table for a screen reader gives
every value. A registry, `STYLES`, gives each style its label, its
drawing function and the names of the metrics it takes (`("Metric",)` or
`("X", "Y")`). The card header reads it. A new style, for example Lines
for pass@k after #146, is one function and one entry. The trace
distributions (`traces/charts.py`) put durations in bins on a log axis.
They share no scale with these charts, so they keep their own code.

Docs: the results guide names the Chart view. `AGENTS.md` lists the
module.

## Touches

backend (`nullius/charts.py`, new) · routes (`web/compare.py`) ·
templates and static (`_charts.html` new, `compare/compare.html`,
`base.html` (an `i-chart` icon), `compare/columns.js`,
`compare/comparison.css`, `style.css`) · tests
(`test_categorical_charts.py` new, `test_comparison_routes.py`,
`field_contract.json`) · docs (`docs/results.md`, `AGENTS.md`)

## Before and after

Seeded data: a showcase profile of Terminal-Bench 2.0 runs, seven
compared (six agents and the oracle check). The view is new, so there is
no before. The nine-run shots add the `nop` check and a copy of the
first run.

| | After |
| --- | --- |
| The default cards, 1280 px, light | <img width="420"
alt="default-light-1280"
src="https://github.com/user-attachments/assets/8c5a551f-8969-4732-ac59-de5b2abd98be"/>
|
| The default cards, 390 px, dark | <img width="420"
alt="default-dark-390"
src="https://github.com/user-attachments/assets/7f4f9889-8ecd-440e-8e69-121b242e477e"/>
|
| Dots and a scatter, 1280 px, light | <img width="420"
alt="styles-light-1280"
src="https://github.com/user-attachments/assets/91ac0bbb-3990-44df-80c9-2348e9f56531"/>
|
| Dots and a scatter, 390 px, dark | <img width="420"
alt="styles-dark-390"
src="https://github.com/user-attachments/assets/f91782ed-b1d5-4309-9e26-e0c8cb1120a8"/>
|
| Two scatters with long metric names, 1280 px, light | <img width="420"
alt="long-names-light-1280"
src="https://github.com/user-attachments/assets/6db5d8cf-5977-4b29-9c49-02d613d10edf"/>
|
| Two scatters with long metric names, 390 px, dark | <img width="420"
alt="long-names-dark-390"
src="https://github.com/user-attachments/assets/4b55313a-3667-45f3-b4b1-1183fe40cad9"/>
|
| Seven runs at one point, 1280 px, light | <img width="420"
alt="ties-light-1280"
src="https://github.com/user-attachments/assets/5e8c8d60-b1a0-49f0-8f9a-65193d8fb294"/>
|
| Seven runs at one point, 390 px, dark | <img width="420"
alt="ties-dark-390"
src="https://github.com/user-attachments/assets/7600b4bf-3363-4a48-88b7-733a78f2b4ba"/>
|
| A card's Style menu, 1280 px, light | <img width="420"
alt="style-menu-light-1280"
src="https://github.com/user-attachments/assets/26d2e36f-be2c-4657-be4c-0281164fddfb"/>
|
| After Add chart, Remove and a style change, 1280 px, light | <img
width="420" alt="edited-light-1280"
src="https://github.com/user-attachments/assets/489bc6a5-dd7c-449e-9977-c4c306010194"/>
|
| After Add chart, Remove and a style change, 390 px, dark | <img
width="420" alt="edited-dark-390"
src="https://github.com/user-attachments/assets/e1f04391-eb6a-4d60-be19-105303cb976a"/>
|
| A Y menu opened with the keyboard, 1280 px, light | <img width="420"
alt="keyboard-menu-light-1280"
src="https://github.com/user-attachments/assets/74cd8b82-8f10-4256-8b86-ce78d4b24d03"/>
|
| A Y menu opened with the keyboard, 390 px, dark | <img width="420"
alt="keyboard-menu-dark-390"
src="https://github.com/user-attachments/assets/fa45a2be-f218-4055-90d1-e18577383289"/>
|
| Nine runs: eight colours, and the ninth back at the first, 1280 px,
light | <img width="420" alt="nine-light-1280"
src="https://github.com/user-attachments/assets/3be81d87-3f6e-4032-857d-29e24a80be06"/>
|
| Nine runs, 390 px, dark | <img width="420" alt="nine-dark-390"
src="https://github.com/user-attachments/assets/979167fb-990a-4d07-8450-b30fcd1241bc"/>
|
| No chart, 1280 px, light | <img width="420" alt="empty-light-1280"
src="https://github.com/user-attachments/assets/17d767e2-d83b-415a-abca-4789ed4de403"/>
|
| The trial page's distributions, unchanged, 1280 px, light | <img
width="420" alt="trial-distributions-light-1280"
src="https://github.com/user-attachments/assets/78c020b2-97cd-494c-b6dd-4ea338180e07"/>
|

## Verified

In a throwaway rootless `python:3.12` container with Node.js and uv, on
a copy of the branch:

- `python3 app/tests/parallel.py`: 3004 tests in 154 modules, OK.
- `pip install -r app/requirements.txt`, then `python3
app/tests/parallel.py app/ui_tests`: 655 tests in 54 modules, OK.
- New: `test_categorical_charts.py` covers the ticks and tick labels
that would touch, a gap for a missing value, a zero and a negative
value, a check, the baseline mark, a single run, a long name that wraps,
very different magnitudes, a scatter of runs at one point, the numbers
inside the plot, the registry, and each mark's series class, which
cycles after eight. `test_comparison_routes.py` covers the default
cards, the URL shape, Add chart, Remove, a style change that keeps the
metric, the Data, X and Y menus and the names on their buttons, no
chart, a metric the cohort does not have, a check run, one run and an
empty column, the column colours of eleven runs in every style, and the
`--series-*` tokens in the dark and the light theme.

Browser check, Chromium (Playwright 1.49.1), on the showcase data with
seven runs, at 1280 px light and 390 px dark:

- Every non-check mark of a card has its run's colour: six fills for the
six agents, with the oracle and `nop` hollow. With nine runs, the ninth
has the first run's colour.
- Every Scatter card shows a dot and a number for each run with both
values, and no number is outside its plot. That includes Trials against
Broke tests, where the seven runs share one point.
- Each card's header buttons are all 34 px high. At 1280 px every header
is one row, the scatters with long metric names too. At 390 px a Scatter
header takes two rows, with whole buttons on each.
- Add chart gives four cards. Remove on the second card gives three, and
the others stay in order. Dots in the first card's Style menu changes
only that card.
- Keyboard: Enter on the Y button opens its menu with the focus on the
first metric, and Tab and Enter pick the next one. An open menu does not
widen the page.
- No page JavaScript error. No sideways scroll on the Compare page.
- A trial page with Distributions: its HTML on `main` and on this branch
is the same, except the form tokens, the interface hash and the new icon
in the sprite.

Not checked: other browsers, a screen reader, touch.

## Risk

- The view toggle has a third link, and `?view=chart` is new. Every
other value of `view` still opens Side by side.
- `summary_rows` now gives each row a key, a group, its number and its
unit. The side table reads none of these, so its cells do not change.
- Every menu item is a full link, so a page with many cards carries many
long URLs (one for each metric of each axis).
- A menu choice reloads the page, so the focus goes back to the top.
- A value's text is the table's text. Where the table shows a dash for a
real 0 (Infra errors, Broke tests), the chart draws a zero mark beside
the dash.
- To undo, revert the two commits. No stored data changes.

## Related

- #206 and #207, the Compare redesign.
- #146, repeated attempts and pass@k, for a later Lines style.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Compare runs in columns, each column a workspace and a run, with the table toolkit's controls

1 participant