Measurement readiness: parity reads strings in arrays; amendment 2 adopted; busy docs made precise

[ryandmonk]

Readiness work the owner asked for before any shadcn/ui v3 authorization. No provider call was made to build or test it, and nothing paid, local or shadcn has started.

It has five commits, each reviewable alone:

1. 81203aa, documentation: no behavior changes.
2. a758c26, the parity tests, fail-first.
3. 299911d, the parity correction.
4. 1897740, documentation, from the review: no behavior changes.
5. 51cf86e, one more parity test, from the review: test only.

1. Documentation corrections

• docs/blocks/ADR-B3-AMENDMENT-2.md opened "Status: proposed, not adopted". It now records the adoption:

  • the owner adopted it on 2026-10-12, before any refinement cell ran live, at ADR-B3 amendment 2: corpus v2 and a binding add and remove check #100's head e129472, without D1 to D3;
  • that PR merged it as 3aae0ea, and corpus v2 is pinned at 0e4e4706…1ae9.

  The rest of the text stands as adopted. Two present-tense statements that were true at adoption, and are not now that Stage 3 has run, are marked "(at adoption)" (1897740).

• docs/BLOCKS.md still called amendment 2 "proposed, not adopted" in its ADR summary, and "proposed" in two later paragraphs. All three now say adopted (1897740).

• docs/deployment.md, the three precision points from the final review of Read a gateway 429 from its status, not only its message (Astryx request 539) #101. After this PR's review (1897740), the conditions for a busy refusal come right after the sentence that states it, and what each caller does follows them:

  • Only a paced bench run retries a busy refusal, and only until its backoff schedule is spent; then it reads a provider failure. A run without the pacer reads it as a provider failure at once (hosted.ts, gated on options.pacer).
  • The browser shows a generic headline for a busy generation call. It reads "The model provider could not produce a result — nothing was generated", with the Worker's busy message as the reason (composer-core/src/build.ts). A busy planning call falls back to deterministic routing, as any failed planning call does (apps/composer/app/planning.ts).
  • A model field counts as a model result, beside usage and token fields. A model's name alone in text does not count.

2. Parity reads strings in arrays

What was wrong.

• Parity is the committed per-prompt checklist of labels and components, scored for every passing cell and reported beside the ADR-B3 decision, never in it.
• It read a surface's text nodes and string props, but skipped every string an array holds.
• Astryx tables keep their column labels (columns) and row cells (data[].cells) exactly there. So a label such as rt-01's "priority" read as missing in every arm, though every arm's table listed it.

The correction (live/parity.ts, surfaceTexts):

• A string is read wherever it stands: a text node, a string prop, or an item of an array. An array item takes the key its array is under.
• Strings under the keys that name structure stay unread, arrays included: component, id, dspackSurface, system and intent.
• One function serves every arm and every condition, as before.

Untouched:

• the decision rule, pass rate, tokens per success, thresholds and minimum;
• the corpus and its digest, the prompts, and the operation scoring;
• request bytes.

The harness's code identity changes, as it must for any change under packages/block-bench/live: it is now 51c80cec…a229, against the Astryx run's fe2b8e84…cc86d. A shadcn run would record the new identity.

For shadcn/ui v3 the correction changes almost nothing.

• Its table is a set of primitives whose header labels are table-head text nodes, which parity always read.
• Its contract's only array-typed prop is a toast's hotkey. Its strings are now read, so a sonner with hotkey: ["altKey", "KeyT"] would satisfy gs-22's label "key". The reviewer re-scored the 20 passing shadcn cells from both smokes: none of their parity changes.
• The parity gap was Astryx's. An earlier handoff of mine said shadcn table labels would read as missing; that came from the emit profile, not from the surface parity reads, and was wrong.

The Astryx Stage-3 decision is unchanged

The check was run offline over the committed run, claude/blocks-stage3-astryx-evidence at 25630c3:

• The harness's own summarize(), over the committed runs.jsonl, reproduces the committed summary.json exactly.
• The recomputation is faithful. I rebuilt each record's checklist as arms.ts does and re-ran the old reading: it reproduces all 459 stored parity results.
• Corrected reading: 124 records' parity changes (block 30, block-positional 34, capped 30, production 30).
• 48 summary fields change, all of them parity. Decisions, the operation sensitivity, selection, the encoding comparison and totals are byte-identical.

Arm (Astryx, pooled)	Mean parity before	After	Fully satisfied before	After
block	0.828	0.963	75 of 113	105 of 113
block-positional	0.780	0.963	65 of 105	99 of 105
capped	0.818	0.959	75 of 114	105 of 114
production	0.808	0.948	70 of 108	96 of 108

The committed Astryx evidence is not changed by this PR. Its parity figures stay as measured; the corrected ones are this recomputation.

Tests

Fail-first. parity.test.ts was committed alone in a758c26 and run against main's parity: 3 of its 5 tests fail.

• An Astryx table's columns and cells: main scores 0.25 where 1 is due.
• Strings in nested arrays, and beside objects in a mixed array: unread.
• rt-01 through the real pipeline on the scripted provider, every arm and every condition: the block arm's fresh cell already misses label:priority. Every arm's fresh table lists the same columns, so after the fix every arm's fresh parity is the same, and complete.

The other 2 tests pin what must not change:

• strings under structure keys stay unread, arrays included;
• a shadcn table's header labels, which are text nodes, read as before.

From the review (51cf86e): a sixth test pins a component's name as structure. A shadcn-v3 component name would otherwise satisfy real labels: rt-12's "action" (alert-dialog-action), nb-26's "cancel" (alert-dialog-cancel) and nb-28's "message" (form-message). The test passes at this head and on main's parity, which never read a component's name. It fails when component is dropped from the structure keys, a change every earlier test let through.

At this head (51cf86e):

• packages/block-bench: 268 tests pass. They include the request goldens, drift, the corpus pins and every parity consumer (metrics, methodology, arms).
• The code identity is still 51c80cec…a229: the commits after 299911d change only documentation and a test.
• apps/composer: 280 tests pass at 299911d, including the Worker's pinned sha256 of env.AI.run's arguments. Nothing it builds or tests has changed since.
• pnpm typecheck: passes.
• Unchanged since main: the corpus files (v2 digest 0e4e4706…1ae9), the Worker, the app, the sidecars, composer-core and the bench's src.

Review

Round 1, at 299911d: no blocking findings. An independent reviewer recomputed the Astryx figures with its own script. It reproduced every number above except one rounding: block-positional's mean is 0.963492, which is 0.963, not 0.964. The table now says 0.963; the commit message of 299911d keeps the slip.

• Should-fix, both fixed:
  • docs/BLOCKS.md still called amendment 2 unadopted (1897740). Copilot found the same; its thread is answered and resolved.
  • Nothing pinned a component's name as structure: a reviewer's mutant dropping it passed all 267 tests (51cf86e).
• Nits:
  • "(at adoption)" markers in the ADR, and the busy paragraph's order (1897740).
  • Left as they are: an array under a text key is not read, and an array nested under a structure key inherits that key. Neither can occur under today's generation schemas, where text is a string and id and intent are plain strings.
• Confirmed:
  • The shadcn-v3 claim: its only array-typed prop is sonner.props.hotkey, and the accordion, whose value may be an array, is not generable.
  • Parity is read only for the report and the arm statistics, never by the decision.
  • Main's parity.ts restores the Stage-3 code identity fe2b8e84…cc86d.

Round 2, at 51cf86e: no blocking or should-fix findings. The same reviewer, in a fresh worktree at this head, found:

• Every edited doc statement matches the code. That covers the paced run's retries until the schedule is spent, the provider failure after it, the browser's headline for a generation call, and the fallback for a failed planning call. No other doc calls amendment 2 proposed or unadopted.
• The new test passes, and kills the mutant that drops component. It passes on main's parity, and 11 of 16 mutants are now killed. Each of the 5 survivors is unreachable or changes no result.
• The suites pass: block-bench 268 of 268, typecheck, and composer 280 of 280. The code identity is 51c80cec…a229, and the Astryx recomputation is identical to round 1.
• Optional nits, left as they are:
  • "adopted it whole" could add "without D1 to D3", which the ADR's status line already says;
  • this body said "three places" for a label only the summary carried exactly, now worded precisely here; the commit message of 1897740 keeps it.

CI (test) and Workers Builds are green on 51cf86e.

After merge

Workers Builds deploys main. One new zero-cost baseline follows the deployment, and no documentation-only deployment follows that baseline.

🤖 Generated with Claude Code

https://claude.ai/code/session_01R6Ax8qvfN3gYUC85oSGnoi

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Updated (UTC)
✅ Deployment successful! View logs	dspack-studio-composer	51cf86e	Oct 14 2026, 02:58 AM

[Copilot]

🟡 Changes recommended

The ADR’s adopted status conflicts with the still-current “proposed, not adopted” repository summary.

1 open finding

• Update BLOCKS.md to reflect Amendment 2 adoption · New

What changed in this PR

Corrects parity scoring for strings inside arrays and records measurement-readiness documentation updates.

Changes:

• Traverses array-held strings while excluding structural keys.
• Adds parity unit and pipeline coverage.
• Clarifies busy-refusal behavior and Amendment 2 adoption.

File	Description
packages/​block-bench/​live/​parity.ts	Adds array-string traversal.
packages/​block-bench/​live/​parity.test.ts	Tests parity behavior across arms and conditions.
docs/​deployment.md	Clarifies busy-refusal handling.
docs/​blocks/​ADR-B3-AMENDMENT-2.md	Records Amendment 2 adoption.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.