Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
99c7404
docs(p022): measure the ownir reference before any Rust exists (#261 …
claude Sep 8, 2026
6967963
test(cli): freeze the ownir CLI contract as a Python-authored fixture…
claude Sep 8, 2026
2900f60
fix(test): the cli_ownir writer never ends the aggregate runner (#261…
claude Sep 8, 2026
d471253
feat(own-cli): the production Rust OwnIR executable (#261 H.2)
claude Sep 8, 2026
9fa54a7
test(own-cli): replay the frozen CLI contract with zero Python (#261 …
claude Sep 8, 2026
2188eb9
ci(own-cli): replay the CLI contract on Linux and Windows (#261 H.4)
claude Sep 8, 2026
40cde85
fix(campaign): a binary crate's unit tests are addressable as catcher…
claude Sep 8, 2026
7bc33d2
evidence(own-cli): the #261 mutation campaign definition (#261 H.5)
claude Sep 8, 2026
78a0c6d
evidence(own-cli): run the #261 campaign and render its fragments (#2…
claude Sep 8, 2026
1a32185
docs(p022): record what #261 built, and the three findings it did not…
claude Sep 8, 2026
1fc55dc
fix(strict-door): Version byte parity, and CLI-B1 as a typed JSON bou…
claude Sep 8, 2026
5786e2d
evidence(own-cli): re-run the campaign over the repair surfaces (#261…
claude Sep 8, 2026
4db644f
docs(p022): repair the status contradiction and state the rulings as …
claude Sep 8, 2026
8b5cfb1
fix(strict-door): reproduce repr(json.loads(...)) for the Version fam…
claude Sep 8, 2026
c088dd8
test(cli-b1): flip eligibility on the facts bytes and nothing else (#…
claude Sep 8, 2026
8874a66
evidence(own-cli): extend the campaign over the R2b/R3b surfaces (#26…
claude Sep 8, 2026
f932374
docs(p022): record what the second measurement found, and name the tr…
claude Sep 8, 2026
a5a4312
evidence(own-cli): re-run the campaign over the R2b/R3b surfaces (#26…
claude Sep 8, 2026
51325a4
docs(p022): keep the status surfaces consistent with the second repai…
claude Sep 8, 2026
bc8176b
docs(p022): state V4 as a representation boundary, not an absolute (#…
claude Sep 8, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,42 @@ jobs:
- name: cargo test
run: cargo test

# P-022 step 7b (#261) — the production OwnIR executable's parity replay, on
# BOTH platforms. The Python half of this contract is gated by the `tests`
# matrix above (`tests/test_cli_ownir_fixtures.py` is auto-discovered like
# every other `test_*.py`, and re-verifies each `oracle: "python"` case
# against the reference on 3.11/3.12/3.13). This job is the other half and
# runs NO Python: it builds the binary and replays the frozen bytes.
#
# Windows is here because the fixture is authored on Linux and a byte
# contract that has only ever been replayed on its authoring platform has not
# been tested — path forms, line endings and the OS error text are exactly
# where a CLI port diverges. `rust-core` is deliberately NOT widened to
# Windows for every crate: that is a separate cost decision, and this is the
# crate whose contract is platform-shaped.
own-cli-parity:
name: own-cli (ownir parity replay)
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest]
runs-on: ${{ matrix.os }}
defaults:
run:
working-directory: rust
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- uses: dtolnay/rust-toolchain@fa04a1451ff1842e2626ccb99004d0195b455a88 # master, 2026-07-10
with:
toolchain: stable
- name: cargo test -p own-cli (the frozen CLI contract, zero Python)
run: cargo test -p own-cli
# The forced-panic and forced-death controls are OFF in every production
# build, so they need their own invocation. They are what makes #261's
# panic ruling a measurement rather than a design claim.
- name: cargo test -p own-cli --features fault-injection (the failure-mode controls)
run: cargo test -p own-cli --features fault-injection --test faults

# P-022 step 7a (#260) — COMPARE MODE over the committed corpus: the FAST half
# of #260's test matrix, and one leg of it. The five pinned OSS repositories,
# the large-solution controls and the examples tree are the scheduled/manual
Expand Down
352 changes: 352 additions & 0 deletions docs/evidence/p022-cli-1.json

Large diffs are not rendered by default.

248 changes: 248 additions & 0 deletions docs/evidence/p022-cli-1.result.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,248 @@
{
"schema": 1,
"comment": "Recorded mutation-campaign run (scripts/mutate_campaign.py --run). Raw facts only: outcomes, catchers, provenance. Counts are derived by scripts/render_checkpoint_status.py; regenerate this file by re-running the campaign, never by hand.",
"campaign": "p022-cli-1",
"definition": "docs/evidence/p022-cli-1.json",
"definition_sha256": "f86d83855bd7cd041cd7880370d3371142b9eff57b562061d44e9b2931fdb573",
"source_commit": "f932374ed8a5fc4731c37c3ce1cf5a071c1843a5",
"dirty": false,
"recorded_at": "2026-09-08T11:36:04Z",
"layers": [
"own-cli",
"own-cli-faults",
"own-ir",
"rust-rest"
],
"command": "every layer the definition declares, for every mutation",
"control": {
"id": "M00",
"outcome": "survived",
"catchers": [],
"elapsed_seconds": 5.3
},
"mutations": [
{
"id": "M01",
"outcome": "caught",
"catchers": [
"own-cli/src/main.rs::ownir::tests::an_advisory_alone_never_fails_the_run",
"own-cli/src/main.rs::ownir::tests::quiet_hides_the_advisory_and_leaves_the_exit_alone",
"own-cli/src/main.rs::ownir::tests::sarif_carries_shown_plus_suppressed",
"own-cli/tests/replay.rs::replays_the_whole_cli_contract_byte_for_byte"
],
"elapsed_seconds": 2.8
},
{
"id": "M02",
"outcome": "caught",
"catchers": [
"own-cli/src/main.rs::ownir::tests::a_suppression_is_counted_but_does_not_fail_the_run",
"own-cli/tests/replay.rs::replays_the_whole_cli_contract_byte_for_byte"
],
"elapsed_seconds": 5.6
},
{
"id": "M03",
"outcome": "caught",
"catchers": [
"own-cli/src/main.rs::ownir::tests::exit_is_independent_of_severity_and_verbosity",
"own-cli/tests/replay.rs::replays_the_whole_cli_contract_byte_for_byte"
],
"elapsed_seconds": 5.7
},
{
"id": "M04",
"outcome": "caught",
"catchers": [
"own-cli/src/main.rs::ownir::tests::sarif_carries_shown_plus_suppressed",
"own-cli/tests/replay.rs::replays_the_whole_cli_contract_byte_for_byte"
],
"elapsed_seconds": 5.7
},
{
"id": "M05",
"outcome": "caught",
"catchers": [
"own-cli-faults/tests/faults.rs::without_the_environment_variable_the_same_invocation_is_clean",
"own-cli/src/main.rs::ownir::tests::a_suppression_is_counted_but_does_not_fail_the_run",
"own-cli/src/main.rs::ownir::tests::the_stream_split_is_per_format",
"own-cli/src/main.rs::ownir::tests::the_summary_pluralizes_on_anything_but_one",
"own-cli/tests/replay.rs::replays_the_whole_cli_contract_byte_for_byte"
],
"elapsed_seconds": 6.0
},
{
"id": "M06",
"outcome": "caught",
"catchers": [
"own-cli/src/main.rs::ownir::tests::quiet_hides_the_advisory_and_leaves_the_exit_alone",
"own-cli/src/main.rs::ownir::tests::sarif_carries_shown_plus_suppressed",
"own-cli/tests/replay.rs::replays_the_whole_cli_contract_byte_for_byte"
],
"elapsed_seconds": 5.4
},
{
"id": "M07",
"outcome": "caught",
"catchers": [
"own-cli/src/main.rs::ownir::tests::the_stream_split_is_per_format",
"own-cli/tests/replay.rs::replays_the_whole_cli_contract_byte_for_byte"
],
"elapsed_seconds": 5.5
},
{
"id": "M08",
"outcome": "caught",
"catchers": [
"own-cli/tests/replay.rs::replays_the_whole_cli_contract_byte_for_byte"
],
"elapsed_seconds": 5.4
},
{
"id": "M09",
"outcome": "caught",
"catchers": [
"own-cli/src/main.rs::ownir::tests::the_verbose_breakdown_counts_suppressed_findings_too",
"own-cli/tests/replay.rs::replays_the_whole_cli_contract_byte_for_byte"
],
"elapsed_seconds": 5.8
},
{
"id": "M10",
"outcome": "caught",
"catchers": [
"own-cli/src/main.rs::sarif::tests::escapes_exactly_as_json_dumps_does",
"own-cli/src/main.rs::sarif::tests::hex_is_lowercase_and_padded",
"own-cli/tests/replay.rs::replays_the_whole_cli_contract_byte_for_byte"
],
"elapsed_seconds": 6.0
},
{
"id": "M11",
"outcome": "caught",
"catchers": [
"own-cli/src/main.rs::ownir::tests::a_missing_flag_value_is_its_own_message",
"own-cli/tests/replay.rs::replays_the_whole_cli_contract_byte_for_byte"
],
"elapsed_seconds": 5.8
},
{
"id": "M12",
"outcome": "caught",
"catchers": [
"own-cli-faults/tests/faults.rs::a_catchable_panic_is_one_diagnostic_and_exit_70",
"own-cli-faults/tests/faults.rs::debug_mode_shows_the_backtrace_and_still_exits_70"
],
"elapsed_seconds": 6.1
},
{
"id": "M13",
"outcome": "caught",
"catchers": [
"own-cli-faults/tests/faults.rs::a_catchable_panic_is_one_diagnostic_and_exit_70",
"own-cli-faults/tests/faults.rs::debug_mode_shows_the_backtrace_and_still_exits_70"
],
"elapsed_seconds": 6.0
},
{
"id": "M14",
"outcome": "caught",
"catchers": [
"own-cli/tests/replay.rs::replays_the_whole_cli_contract_byte_for_byte"
],
"elapsed_seconds": 5.8
},
{
"id": "M15",
"outcome": "caught",
"catchers": [
"own-cli/src/main.rs::ownir::tests::there_is_no_double_dash_separator",
"own-cli/tests/replay.rs::replays_the_whole_cli_contract_byte_for_byte"
],
"elapsed_seconds": 5.9
},
{
"id": "M16",
"outcome": "caught",
"catchers": [
"own-cli/tests/replay.rs::replays_the_whole_cli_contract_byte_for_byte",
"own-ir/tests/version_repr_census.rs::negative_zero_below_the_top_level_matches_the_reference",
"own-ir/tests/version_repr_census.rs::the_declared_divergences_are_exactly_the_declared_ones",
"own-ir/tests/version_repr_census.rs::the_version_family_is_byte_exact_with_the_reference"
],
"elapsed_seconds": 6.3
},
{
"id": "M17",
"outcome": "caught",
"catchers": [
"own-cli/tests/replay.rs::replays_the_whole_cli_contract_byte_for_byte",
"own-ir/tests/version_repr_census.rs::the_version_family_is_byte_exact_with_the_reference"
],
"elapsed_seconds": 7.7
},
{
"id": "M18",
"outcome": "caught",
"catchers": [
"own-cli/src/main.rs::ownir::tests::a_json_rejection_that_lost_its_prefix_is_an_internal_error_not_rc2",
"own-cli/tests/replay.rs::replays_the_whole_cli_contract_byte_for_byte"
],
"elapsed_seconds": 7.8
},
{
"id": "M19",
"outcome": "caught",
"catchers": [
"own-cli/src/main.rs::ownir::tests::a_json_rejection_that_lost_its_prefix_is_an_internal_error_not_rc2"
],
"elapsed_seconds": 5.6
},
{
"id": "M20",
"outcome": "caught",
"catchers": [
"own-ir/src/lib.rs::pyrepr::tests::a_repeated_key_is_rebound_in_place",
"own-ir/tests/version_repr_census.rs::the_version_family_is_byte_exact_with_the_reference"
],
"elapsed_seconds": 7.3
},
{
"id": "M21",
"outcome": "caught",
"catchers": [
"own-ir/src/lib.rs::pyrepr::tests::an_oversized_integer_keeps_its_digits",
"own-ir/src/lib.rs::pyrepr::tests::matches_cpython_float_repr",
"own-ir/src/lib.rs::pyrepr::tests::the_exponent_padding_is_the_defect_r2_left",
"own-ir/src/lib.rs::pyrepr::tests::the_literal_decides_int_or_float",
"own-ir/tests/version_repr_census.rs::the_version_family_is_byte_exact_with_the_reference"
],
"elapsed_seconds": 8.8
},
{
"id": "M22",
"outcome": "caught",
"catchers": [
"own-ir/tests/version_repr_census.rs::negative_zero_below_the_top_level_matches_the_reference",
"own-ir/tests/version_repr_census.rs::the_version_family_is_byte_exact_with_the_reference"
],
"elapsed_seconds": 9.6
},
{
"id": "M23",
"outcome": "caught",
"catchers": [
"own-ir/tests/version_repr_census.rs::the_version_family_is_byte_exact_with_the_reference"
],
"elapsed_seconds": 8.7
},
{
"id": "M24",
"outcome": "caught",
"catchers": [
"own-cli/tests/replay.rs::cli_b1_flips_on_the_facts_bytes_and_nothing_else"
],
"elapsed_seconds": 6.2
}
]
}
59 changes: 59 additions & 0 deletions docs/generated/p022-cli-census.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
<!-- GENERATED by scripts/render_checkpoint_status.py from tests/fixtures/cli_ownir/manifest.json. Do not edit: regenerate with `python scripts/render_checkpoint_status.py`; tests/run_tests.py fails while this file is stale. -->

# P-022 step 7b (#261) — the `own-cli ownir` contract, counted

`tests/fixtures/cli_ownir/` is the frozen CLI contract, authoritative via `python tests/test_cli_ownir_fixtures.py --write` on Linux and replayed against the built binary with **zero Python** by `own-cli/tests/replay.rs` on Linux and Windows CI. Every case names the rule it is the control for and the oracle that authored its bytes; both tables below are read from the manifest.

| measure | value |
|------------------------------------|------:|
| frozen cases | 88 |

## By oracle

The oracle boundary is #261's C-1, and it is the SURFACE rather than the reference's internal print branch.

| oracle | cases | what authored the bytes |
|---|------:|---|
| `owen-convention` | 7 | no Python byte oracle exists: the top-level shell, authored once from the `owen` convention and shared with the binary |
| `python` | 77 | an executed `python -m ownlang ownir` run |
| `python-docstring` | 4 | the same, where the bytes are the WHOLE module docstring on stdout — frozen as measured and flagged, so the owner can declare that class a defect knowing what was frozen |

## By rule

A case may be the control for more than one rule, so these do not sum to the case count — they say how much evidence each rule has, which is the question.

| rule | cases |
|---|------:|
| `advisory-never-fails-the-run` | 3 |
| `cli-b1-json-parser-detail` | 3 |
| `cli-b1-negative-control` | 1 |
| `debug-env-is-inert-when-nothing-crashes` | 1 |
| `declared-defect` | 1 |
| `docstring-on-stdout` | 4 |
| `duplicate-flag` | 1 |
| `empty-reason-never-suppresses` | 1 |
| `equals-spelling` | 3 |
| `exit-independent-of-severity` | 32 |
| `invalid-flag-value` | 6 |
| `machine-stdout-empty-when-clean` | 2 |
| `missing-flag-value` | 3 |
| `no-double-dash-separator` | 1 |
| `nonascii-in-file-field` | 3 |
| `nonascii-in-path` | 2 |
| `ok-line` | 4 |
| `ok-line-with-suppressed` | 3 |
| `os-error-placeholder` | 4 |
| `ownir-format-scope` | 1 |
| `path-echoed-as-given` | 1 |
| `quiet-hides-advisory` | 11 |
| `sarif-ascii-escape` | 2 |
| `sarif-carries-shown-plus-suppressed` | 7 |
| `stdin-out-of-contract` | 1 |
| `stream-split` | 36 |
| `strict-door` | 12 |
| `summary-suppressed-tail` | 26 |
| `unknown-flag-is-positional` | 2 |
| `usage-owen-shape` | 7 |
| `verbose-counts-every-finding` | 13 |
| `version-byte-parity` | 5 |
| `windows-path-form` | 2 |
Loading
Loading