ADR-B3 amendment 2: corpus v2 and a binding add and remove check by ryandmonk · Pull Request #100 · aestheticfunction/dspack-studio

ryandmonk · 2026-10-12T20:40:09Z

Adopted by the owner on 2026-10-12, at this exact head (e129472), before any refinement cell ran on any provider. It is a pre-data amendment of ADR-B3, made before Stage 3; the 160-cell pilot stays skipped. No paid or provider call was made. The full record is docs/blocks/ADR-B3-AMENDMENT-2.md.

The owner's decisions:

Corpus v2 at sha256 0e4e4706e19ec92565fea2019258cd5d83f6bef0eca424a0634ccf4749b61ae9. Merging makes it the corpus a run measures by default.
The add and remove check binds both inputs: the pass-rate guardrail and tokens per success.
D1 to D3 excluded.
The open "such as" wording nit accepted without changing the reviewed head. The merged tree is the reviewed head unchanged, so the ADR's status line still reads "proposed, not adopted"; this description records the adoption.
v1 and both earlier smokes stay as they are, systems evidence only.

What it adopts

M: the add and remove check becomes binding under corpus v2

The rule. A refine-add or refine-remove that the pipeline passed, but whose operation was missed, counts as a failure. Missed means no list changed, or lists changed only the other way (live/operation.ts).
Where it applies: every arm alike, the comparative positional arm included, in both the pass rate and tokens per success, as the owner confirmed.
Edge cases:
- A mixed change passes, and its strict reading stays a sensitivity row.
- An add or remove the check never read fails closed, and the report counts it as unchecked.
- Summarizing takes the corpus's scoring with no default, and the report refuses a summary scored otherwise.
What stays: every call's tokens still count, and v1's scoring is unchanged.
Sound on net. The per-list changes sum to the net change in list elements, so an add that grew the surface, or a remove that shrank it, is never missed. 0 of 5,924 simulated single-node edits on 30 smoke surfaces read as missed.

C1 to C4: the template corrections

Found by a static audit of the corpus text against each design system's generation view (live/applicability.ts), the same for every arm:

	Template	Correction
C1	records-table add	A row any table can take, not a ticket's values. Only rt-01 is a ticket table.
C2	auth-form remove	The provider buttons, or the first field where the form has none (15 of 22 forms).
C3	guide-steps structural	Cards, a numbered badge and a comments field, in place of an accordion (a shadcn-v3 casualty, absent from Astryx), a timer and a photo.
C4	auth-form add	Apple where Microsoft is already offered (af-12). Without it, binding fails a correct no-op there in every arm, and the block's compact path has no compliant way to pass.

D1 to D3: offered, excluded by the owner

The same rules flag three more structural templates the owner did not name: records-table (tabs and an Open/Closed split), auth-form (a progress indicator and a checkbox) and null (a column layout and a sidebar).

v2 keeps v1's text for all three.
The ADR gives each replacement, the case for and against, and the digest if adopted.
Their effect on the block arm has an unknown sign.

Corpus v1 and v2 side by side

	File and pin	sha256
v1, unchanged byte for byte	`live/corpora/blocks.json`, `frozen.json`	`180f4480284b720e22a279a1147d95c6ae1132ff1fdcc03d8cddb1235a5e4231`
v2, frozen 2026-10-12	`live/corpora/blocks.v2.json`, `frozen.v2.json`	`0e4e4706e19ec92565fea2019258cd5d83f6bef0eca424a0634ccf4749b61ae9`
v2 with D1 to D3, if adopted		`8212e223c8fa9a2ac4223286b55eecba60f0df3c2560fbedbae9f1ac022138e0`

v2 against v1: only the version, freeze date, note, scoring and four instructions differ. Every prompt, label and checklist is v1's.
Running: a run measures v2 unless it passes --corpus 1. Each version is held to its own pin, and a v1 run is never resumed on v2.
The smokes: both measured v1 and remain systems evidence only.

What does not change

The 30% threshold, the 5-point margin and the minimum of 20 (DEFAULT_DECISION).
Call budgets, arms, conditions and the re-run policy.
Block contracts, model prompts, request construction and goldens, and production behavior.
git diff origin/main is empty for these paths:
- apps/, packages/composer-core, packages/contracts;
- packages/block-bench/src, goldens, results.json, RESULTS.md;
- live/arms.ts, live/providers, live/pacing.ts, live/parity.ts;
- live/corpora/blocks.json and frozen.json;
- the ADR-B3 addendum and BASELINE.
live/operation.ts changes only its doc comment; it is identical to main with comments stripped. live/types.ts adds the OperationScoring type.

Evidence

Fail-first, one commit of tests before each implementation:

Tests	Failing on	Fixed by
`90de74a`	`eee21c1`: 26 fail, in 5 files	`84eddda`
`8a7665b`	`84eddda`: 2 fail	`529c96c`
`df344c8`	`529c96c`: 5 fail	`ae2ab76`
`2cb9f0d`	`ae2ab76`: 2 fail	`d2dc8c8`
`e01e2cf`	`d2dc8c8`: 4 fail	`6b02374`
`96e2181`	`6b02374`: 1 fails	`e129472`

Tests: block-bench passes 258 of 258 in 17 files, and tsc is clean. The root pnpm typecheck is clean.
Request hashes are unchanged. hosted-build.test.ts (43) and the block ratchet (22) pass. The goldens use a fixed instruction, not the corpus.

Stage-3 plan and cost on v2 (offline, no provider)

Plan:
- The cell plans are identical to v1: pilot 160, Stage 3 960, full 3,600 cells, in the same order.
- The scripted estimate's paths, outcomes and passes are identical in all 3,600 cells.
- 660 cells change their instruction bytes.
- Stage-3 input tokens rise 0.006%.

Astryx, as its own run (ledger)	v1	v2
Central	$5.90, 684 requests	$5.90, 684
High	$11.65, 1,092	$11.66, 1,092
Ceiling to complete high	$16.12	$16.13
Per-window cap	825	825

The $17 cap still covers the high scenario.
shadcn-v3 as its own run: central $40.34, high $72.09, ceiling $76.56, the same as v1.
Stage 3 as one run: central $46.24, high $83.75 (v1: $46.24 and $83.74).
The Astryx-first command with --corpus 2 --max-requests 825 validates offline with the harness's own parser: 480 cells, v2 pinned, scoring binding.

Independent review

An independent reviewer checked each head offline, in fresh exports, for four things:

outcome-driven tuning leakage;
unequal difficulty between arms;
weakened or ambiguous scoring;
accidental changes outside the amendment.

Head	Verdict
`529c96c`	One material concern: the adoption clause allowed adoption after refinement data existed. Ten minors and nits, among them: move C4 into the core, and leave D1 to D3 out by default.
`d2dc8c8`	No unresolved material concern. Three minors (the bias wording, gaps in the adoption clause, a fail-open scoring default) and seven nits.
`6b02374`	No unresolved material concern. Four nits.
`e129472`	No unresolved material concern. One nit left open: the ADR names `decide` and `armStats` as helpers that read records as given. It should say "such as", since `compareEncodings` and `threadSums` do too.

Every earlier finding is addressed in df344c8 to e129472, fail-first wherever code changed.
The reviewer also confirmed:
- no refinement cell has run live;
- v1 is byte-identical to main;
- v2 differs from v1 only in its header and four instructions;
- the boundary against main is empty for every protected path;
- binding, recomputed independently, matches;
- v1 summaries match main's apart from the scoring field and zero unchecked counts.

🤖 Generated with Claude Code

https://claude.ai/code/session_01R6Ax8qvfN3gYUC85oSGnoi

Tests only; they fail on main (eee21c1), whose corpus is v1 alone and whose add and remove check stands beside the pass. - applicability.test.ts: a static audit of every refinement template finds the three defects named for correction (records-table add: a ticket's values in tables without those columns; auth-form remove: provider buttons the form does not have; guide-steps structural: an accordion, timer and photo no component renders), and the same rules' other findings; corpus v2 applies everywhere but those; an unaudited template is refused; the scripted block arm still falls back on every add, remove and structural template. - corpus.test.ts: v1 keeps its file, pin and digest; v2 has its own version, freeze date, pin and digest; each version is held to its own pin; v2 differs from v1 only in its header and the amended templates; each version declares its scoring. - methodology.test.ts: under v2 a passing add or remove that missed its operation is a failure in every arm alike, in the pass rate and in tokens per success; a mixed change passes; an unchecked one fails closed; the readings bracket the verdict; v1's scoring is unchanged. The v1 report tests name corpus v1 explicitly. - run.test.ts: --corpus 1 or 2, default 2, anything else refused; each version is held to its own pin; a corpus is never run under the other version's flag. - dry-run.test.ts: the v1 mechanics run on v1 explicitly; a v2 dry run records v2 and scores the check binding in every arm; a v1 run is never resumed on v2. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6Ax8qvfN3gYUC85oSGnoi

… remove check Proposed, not adopted (docs/blocks/ADR-B3-AMENDMENT-2.md). Corpus v1, its pin and its digest are unchanged; both smokes measured v1 and stay systems evidence only. - corpora/blocks.v2.json, frozen.v2.json: blocks-1b v2, frozen 2026-10-12, its own pin and digest. Every v1 prompt, intent, label and checklist, and every template that applies everywhere, is v1's. Three templates a static audit found cannot apply everywhere are corrected: - C1, records-table add: a row any table can take, not a ticket's values; - C2, auth-form remove: the provider buttons, or the first field where a form has none (never the password the block keeps); - C3, guide-steps structural: cards, a badge and a comments field, which every design system renders, in place of an accordion, a timer and a photo, which none does. - applicability.ts: the static audit, from the corpus text and each reference's generation view alone, the same for every arm. - corpus.ts, run.ts: versions side by side; a run measures v2 unless it passes --corpus 1, and each version is held to its own pin. - metrics.ts, report.ts, evaluate.ts: under v2 a passing add or remove that missed its operation is a failure in every arm, in the pass rate and in tokens per success; a mixed change passes and is read strictly as a sensitivity row; the readings bracket the verdict with the pipeline's pass alone. v1's scoring is unchanged. - docs: the amendment record, BLOCKS.md, the harness README. No threshold, budget, arm, condition, block contract, model prompt, request golden or production path changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6Ax8qvfN3gYUC85oSGnoi

Tests only; they fail on 84eddda. Outside the three corrections the owner named, the same two audit rules find four more templates that cannot apply everywhere; these tests ask v2 to have none left. - applicability.test.ts: v2 has no target or element issue at all. - corpus.test.ts: v2 also amends auth-form add (af-12 already offers Microsoft) and the records-table, auth-form and null structural templates (tabs, a checkbox, a progress indicator and a column layout, which Astryx, or neither design system, renders; and a split by ticket status in tables of other things). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6Ax8qvfN3gYUC85oSGnoi

Outside the three corrections the owner named, and kept separable: dropping this commit and its tests leaves v2 with C1 to C3 only. - C4, auth-form add: Apple where the form already offers Microsoft (af-12, outside Stage 3), so a binding check never fails a correct no-op. - D1, records-table structural: two tables in cards and a search field and filter button, in place of tabs (absent from Astryx) split by a ticket's status (only rt-01 has one). - D2, auth-form structural: two cards with step badges and an "I agree" button, in place of a progress indicator and a checkbox (both absent from Astryx). - D3, null structural: two cards and link buttons, in place of a column layout (no component in either design system) and a sidebar. - frozen.v2.json: v2 re-pinned; the amendment record gives both digests. Each structural change stays one the blocks cannot express, so the block arm still falls back. Since an impossible request weighs most on the block arm's shorter fallback, these can only help it, if they change anything; they follow from the static audit alone, before any refinement result. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6Ax8qvfN3gYUC85oSGnoi

cloudflare-workers-and-pages · 2026-10-12T20:41:48Z

Deploying with Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Updated (UTC)
✅ Deployment successful! View logs	dspack-studio-composer	`e129472`	Oct 12 2026, 09:50 PM

Tests only; 5 fail on 529c96c. - methodology.test.ts: an add or remove the check never read, which the binding rule fails closed, is counted and named in the decision's bullet and the refinement table, and the report never says the check failed none when it failed every one (review minor 6). A summary scored otherwise than its corpus's own rule is refused, so a v2 run is never reported with v1's scoring (review minor 7). - metrics.test.ts: the operation counts carry the unchecked ones. - applicability.test.ts: v1's Open and Closed split fits the tables whose items open and close, bugs and incidents as well as tickets (review minor 9). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6Ax8qvfN3gYUC85oSGnoi

…se a mis-scored summary - The report counts a passing add or remove that carries no operation record as unchecked: under binding it fails closed, and the counts, the by-block table and Changed now include it - renderReport refuses a summary scored otherwise than its corpus declares, so a v2 run is never reported with v1 scoring - The audit's split into Open and Closed fits tables of tickets, bugs and incidents (rt-01, rt-07, rt-21), and its header says each template's needs are recorded judgments - Text that described v1 scoring only now names both versions Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6Ax8qvfN3gYUC85oSGnoi

- v2 corrects every add and remove template (C1, C2, C4) and the guide-steps structural one (C3); its only remaining audit findings are v1's other structural templates - D1 to D3, if the owner adopts them, leave no finding and still send the scripted block arm to its fallback Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6Ax8qvfN3gYUC85oSGnoi

…wner - v2 keeps v1's records-table, auth-form and null structural templates; D1 to D3 stay audited and are offered for the owner's decision, with their digest - v2 is re-pinned: sha256 0e4e4706...61ae9 (with D1 to D3 it would be 8212e223...38e0); v1 is unchanged - The amendment is decided whole before any refinement cell runs live, and merging it adopts it - The ADR states the soundness on net, where the remaining error runs, that M also binds tokens per success for the owner to confirm, C2's uneven detection, and the audit's judgment calls Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6Ax8qvfN3gYUC85oSGnoi

- Summarizing without the corpus's scoring is refused: no default, in the summary or the readings - Under binding, the report defines unchecked as a failure in its scoring line, its bullet, its refinement note and its by-block note - Under corpus v1, the report never says every add and remove changed its lists while some were never read - Every test names the scoring it summarizes under Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6Ax8qvfN3gYUC85oSGnoi

…, the ADR's claims narrowed - summarize and the readings take the corpus's scoring with no default, and scoreRecords refuses any other value - Under binding, the report says the missed and unchecked ones fail, in its scoring line, bullet, refinement note and by-block note; under v1 it no longer says every add and remove changed its lists when some went unread - The ADR's adoption clause covers every provider and says the amendment lapses if a refinement cell runs first - The ADR names both ways a wrong change can pass, including the block's compact path (af-05), and corrects the field-order evidence, the fresh prompts the element rule would flag, the trap C4 closes and the reason for a new baseline Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6Ax8qvfN3gYUC85oSGnoi

- Corpus v1's decision is the one read on the records as they are, restored as an assertion - Under v1, the report says the check cannot confirm the adds and removes it never read Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6Ax8qvfN3gYUC85oSGnoi

- Under v1, the report says the check cannot confirm the adds and removes it never read - The ADR says an ignored instruction passes mostly, not only, on surfaces regenerated in full, since a compact refinement re-emits its whole call - The ADR says only the block's compact path has no compliant way to pass af-12 without C4, and that the scoring guarantee covers the summary and the report, not the lower-level helpers Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6Ax8qvfN3gYUC85oSGnoi

Copilot

🟢 Approval recommended

The implementation is coherent and well tested; the only finding is a non-blocking test-title grammar correction.

1 open finding

Use “corrects” in the test title · New

What changed in this PR

Introduces corpus v2 and binding add/remove operation scoring for ADR-B3 while preserving corpus v1 compatibility.

Changes:

Adds separately pinned corpus v2 with corrected refinement templates.
Applies binding operation scoring to pass rates and tokens per success.
Adds CLI selection, reporting, static applicability checks, and comprehensive tests.

File	Description
`packages/block-bench/live/types.ts`	Defines operation-scoring types.
`packages/block-bench/live/run.ts`	Adds version-aware corpus CLI handling.
`packages/block-bench/live/run.test.ts`	Tests corpus selection and pinning.
`packages/block-bench/live/report.ts`	Reports binding scoring and sensitivity.
`packages/block-bench/live/README.md`	Documents corpus selection.
`packages/block-bench/live/operation.ts`	Documents versioned scoring behavior.
`packages/block-bench/live/metrics.ts`	Implements binding scoring and unchecked counts.
`packages/block-bench/live/metrics.test.ts`	Updates metrics tests for explicit scoring.
`packages/block-bench/live/methodology.test.ts`	Tests binding methodology and reports.
`packages/block-bench/live/evaluate.ts`	Scores summaries according to their corpus.
`packages/block-bench/live/dry-run.test.ts`	Exercises v2 end-to-end.
`packages/block-bench/live/corpus.ts`	Supports versioned corpora and pins.
`packages/block-bench/live/corpus.test.ts`	Verifies v1 preservation and v2 integrity.
`packages/block-bench/live/corpora/frozen.v2.json`	Pins the v2 digest.
`packages/block-bench/live/corpora/blocks.v2.json`	Defines corpus v2.
`packages/block-bench/live/applicability.ts`	Implements static template applicability auditing.
`packages/block-bench/live/applicability.test.ts`	Tests applicability findings and corrections.
`docs/blocks/ADR-B3-AMENDMENT-2.md`	Records the amendment rationale and rules.
`docs/BLOCKS.md`	Documents versioned corpus behavior.

🧠 Review effort: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+};
+
+describe("corpus v2's refinement templates", () => {
+  it("correct every add and remove template, so the binding check never fails a correct no-op, and the guide-steps structural one; the rest are v1's", () => {


claude added 4 commits October 12, 2026 20:23

claude added 8 commits October 12, 2026 21:11

ryandmonk changed the title ~~Draft: ADR-B3 amendment 2: corpus v2 and a binding add and remove check~~ ADR-B3 amendment 2: corpus v2 and a binding add and remove check Oct 12, 2026

ryandmonk marked this pull request as ready for review October 12, 2026 22:23

Copilot AI balanced review requested due to automatic review settings October 12, 2026 22:23

ryandmonk merged commit 3aae0ea into main Oct 12, 2026
3 checks passed

Copilot started reviewing on behalf of ryandmonk October 12, 2026 22:24

View session

Copilot AI reviewed Oct 12, 2026

View reviewed changes

Comment thread packages/block-bench/live/applicability.test.ts

};

describe("corpus v2's refinement templates", () => {

it("correct every add and remove template, so the binding check never fails a correct no-op, and the guide-steps structural one; the rest are v1's", () => {

ryandmonk mentioned this pull request Oct 14, 2026

Measurement readiness: parity reads strings in arrays; amendment 2 adopted; busy docs made precise #102

Merged

Navigation Menu

Repository navigation

ADR-B3 amendment 2: corpus v2 and a binding add and remove check - #100

ADR-B3 amendment 2: corpus v2 and a binding add and remove check#100
ryandmonk merged 12 commits into
mainfrom
claude/blocks-corpus-v2-amendment

ryandmonk commented Oct 12, 2026 •

edited

Loading

Uh oh!

cloudflare-workers-and-pages Bot commented Oct 12, 2026 •

edited

Loading

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ryandmonk commented Oct 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What it adopts

M: the add and remove check becomes binding under corpus v2

C1 to C4: the template corrections

D1 to D3: offered, excluded by the owner

Corpus v1 and v2 side by side

What does not change

Evidence

Stage-3 plan and cost on v2 (offline, no provider)

Independent review

Uh oh!

cloudflare-workers-and-pages Bot commented Oct 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deploying with Cloudflare Workers

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟢 Approval recommended

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ryandmonk commented Oct 12, 2026 •

edited

Loading

cloudflare-workers-and-pages Bot commented Oct 12, 2026 •

edited

Loading