Skip to content

test(metrics): kill the first wave of escaped mutants and fix two Display lines - #1881

Merged
dekobon merged 7 commits into
mainfrom
fix/issue-1736
Nov 12, 2026
Merged

dekobon merged 7 commits into
mainfrom
fix/issue-1736

Conversation

@dekobon

@dekobon dekobon commented Nov 12, 2026

Copy link
Copy Markdown
Owner

First wave of work on #1736. It kills the escaped mutants from the old run that are still live in the root crate's loc, abc, npa, npm, nom, wmc, cognitive, cyclomatic, nexits and tokens files, fixes two Display defects the new tests exposed, and adds four exclude_re entries with their proofs.

Refs #1736. This does not close it: the close condition is a mutation-test.yml run on main with every shard complete, zero missed and zero timed out, and later waves follow the fresh inventory from run 43278927651.

Commits

Commit What it does
fix(cognitive,nexits) cognitive::Stats printed the space's own score under sum and wrote min: with no space; nexits::Stats had no comma after average. Same family as #117. Changelog entry included
test(loc) four LineSet tests at non-zero word offsets
test(abc) four tests for the C impl's assignment, initialised-declarator, call and else arms, which had no fixture; one for abc::Stats's Display
test(npa) Objective-C protocol groups with several properties; @implementation instance variables (the class bucket)
test(metrics) per-space accessors of npm, npa, wmc, nom at values a constant cannot return; a class inside a function nested in a method for wmc::Stats::merge; Display for cyclomatic, tokens, loc
ci(mutation) four exclusions; two sections in docs/development/mutation_testing.md on replaying chosen mutants and writing an entry

Evidence

  • Every new test was replayed against its mutants with cargo mutants --in-place -f <file> -F <name> (cargo-mutants 27.1.0, rustc 1.99.0), and the failing assertion read from each mutant's log.
  • Whole-file replays: src/metrics/abc/c.rs 38 caught, 2 missed (the init_declarator guard, below); src/metrics/npa/objc.rs 43 caught, 0 missed.
  • Expected values are counted from the fixture, with an // expected: derivation beside each. A fresh-context review re-derived all of them and found no discrepancy.
  • make pre-commit: BCA_GATE: pass (gate=pre-commit). .cargo/mutants.toml comments and the doc sections were edited after that run; make markdown-lint, make toml-lint and cargo fmt --check were re-run and pass.

Exclusions

cargo mutants --list --no-config against cargo mutants --list over the workflow's selection differs by exactly these four:

Mutant Reason
LineSet::reserve, < to <= equivalent: at word == first_word both branches leave the set untouched
== to != in the init_declarator guard, C ABC equivalent while tree-sitter-c is pinned at 0.24.2: every such node has an = child and another child
the same in Objective-C ABC equivalent while tree-sitter-objc is pinned at 3.0.2; the pattern keeps the column to tell it from the live kind == MessageExpression site
delete ! in for_each_named_child non-terminating: a one-child list loops forever; reproduced as a 120 s timeout

Left open on purpose

  • The init_declarator guard replaced by true, in C and Objective-C. It changes no metric but skips the guard's child scan, which child_scan_cursors observes, so it is not excluded. The next wave drops the guard in those two impls instead.
  • fix(npm): a private method of an interface is counted as public #1880, found on the way: an interface's private method is counted as public by npm. Not fixed here.

`cognitive::Stats` printed the space's own score under `sum`, so a
file or a class read `sum: 0` whatever its functions scored; the JSON
`sum` field and every sibling metric's `Display` print the subtree's
sum. It also wrote `min:` with no space, and `nexits::Stats` had no
comma between `average` and `min`. The two earlier fixes of this kind
are nom's missing commas (#117) and nargs's headline that diverged from
its sums.

Found by writing the `Display` tests the mutation run asked for: no
test read either line, so replacing `fmt` with `Ok(())` passed. Each
test asserts the whole line against values counted from its fixture.

Refs #1736
Six mutants in `line_set.rs` survived the mutation run because every
test that reached them held a set starting at word 0, where adding an
offset and subtracting it compute the same index, or compared sets
whose last words coincided:

- `subtract` read the other set at `word + other.first_word`;
- `last_word` counted one or two words too many, so `union_with`
  reserved zero padding no row shows;
- `reserve` resized to `word + first_word + 1`;
- `eq` ended its comparison at `first_word * len`, short of a row past
  the other set's last word.

Four tests, each replayed against its mutants and failing at the
assertion written for them. The two over-reserve cases assert
`words.len()`, as `dense_rows_cost_one_word_per_sixty_four` does: row
equality ignores padding by design.

Refs #1736
`CCode::compute` shares its condition helpers with C++, and the
C-family parity tests pin those. Its assignment, initialised-declarator,
call and `else` arms had no fixture at all: deleting any of them, or
turning its `+=` into `-=`, passed the suite. Four tests, one per arm,
each on a fixture only that arm can score.

`abc::Stats`'s `Display` had no reader either. Its test uses counts of
2, 3 and 6, whose magnitude is 7, so a value under the wrong label
shows.

Each test was replayed against its mutants; the whole of `abc/c.rs` now
leaves two survivors, both on the `init_declarator` guard, which the
pinned C grammar makes always true.

Refs #1736
Two paths of `ObjcCode`'s `Npa` had no fixture that could tell them
from a constant:

- a `@optional` / `@required` group was only ever given one property,
  so `objc_count_group_properties` returning 1 passed;
- every fixture was an `@interface` or a `@protocol`, so the class
  bucket an `@implementation`'s instance variables feed was never
  read.

Both tests were replayed against `npa/objc.rs`, which now leaves no
survivor.

Refs #1736
… returns

The per-space accessors of `npm`, `npa` and `wmc` were asserted only on
an interface with one member, so `interface_npm`, `interface_nm`,
`interface_npa`, `interface_na` and `interface_wmc` returning 1 passed.
`nom`'s `closures()` had no reader, and `functions()` none that saw 0.
Each now has a fixture where it reads a value that is neither 0 nor 1:
a Java interface and class with several members, and an Objective-C
method holding two block literals, which open no space of their own.

`wmc::Stats::merge` passes a nested class's complexity up through a
function nested in a method; negating that term passed, because no
fixture put a class inside a nested function. A Python one does.

`cyclomatic`, `tokens` and `loc` had no `Display` test. The first two
assert the whole line against hand-counted values; `loc` has 24 fields,
so its test restates the label and accessor table and anchors it with
two literal fragments.

Refs #1736
Three are equivalent and one never terminates; each entry carries its
proof:

- `LineSet::reserve` with `<=`: at `word == first_word` both branches
  leave the set untouched.
- the `==` to `!=` flip inside the `init_declarator` guard of the C and
  Objective-C ABC impls: the pinned grammars give every such node an
  `=` child and another child, so one child scan finds a match either
  way.
- `for_each_named_child` with its `!` deleted: a one-child list loops
  forever, and the Go ABC tests that return one value hang on it.

The guard's replacement by `true` is not excluded. It skips the guard's
child scan, which the scan counter observes, so it is not equivalent in
the sense the file states.

The guide gains two short sections: how to replay chosen mutants and
read their logs, and how to write an entry that matches one mutant and
survives edits above it. Both note that cargo-mutants 27.1.0 applies no
name filter to `delete field` mutants.

Refs #1736
@codecov

codecov Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.49%. Comparing base (68b70b0) to head (e82c998).
⚠️ Report is 7 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1881      +/-   ##
==========================================
+ Coverage   98.47%   98.49%   +0.01%     
==========================================
  Files         403      403              
  Lines      124408   124680     +272     
  Branches   123973   124245     +272     
==========================================
+ Hits       122511   122803     +292     
+ Misses       1173     1153      -20     
  Partials      724      724              
Flag Coverage Δ
python 100.00% <ø> (ø)
rust 98.48% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/metrics/abc.rs 99.88% <100.00%> (+0.14%) ⬆️
src/metrics/cognitive.rs 99.91% <100.00%> (+<0.01%) ⬆️
src/metrics/cyclomatic.rs 99.57% <100.00%> (+<0.01%) ⬆️
src/metrics/loc.rs 99.71% <100.00%> (+<0.01%) ⬆️
src/metrics/loc/line_set.rs 100.00% <100.00%> (ø)
src/metrics/nexits.rs 100.00% <100.00%> (ø)
src/metrics/nom.rs 99.94% <100.00%> (+0.17%) ⬆️
src/metrics/npa.rs 100.00% <100.00%> (ø)
src/metrics/npm.rs 99.62% <100.00%> (+<0.01%) ⬆️
src/metrics/tokens.rs 99.42% <100.00%> (+0.01%) ⬆️
... and 1 more
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

CI's `clippy` job installs the `stable` tip, which is Rust 1.100 as of
today, and that clippy reports `fn _native` as a used
underscore-prefixed item: `#[pymodule]`'s expansion calls it. The
underscore is the Python module's name (`big_code_analysis._native`),
so the function keeps it and carries the allow with that reason.

The local gate could not see this on 1.99.0. Checked with a 1.100.0
toolchain: `cargo clippy --workspace --all-targets --all-features
--keep-going -- -D warnings` reported this one error before the change
and none after; `cargo fmt --check` and `cargo doc` with `-D warnings`
pass on it too.

Fixes #1882
@dekobon
dekobon merged commit e82c998 into main Nov 12, 2026
59 checks passed

This branch was previously deployed

1 inactive deployment
github-pages — e82c9985 Deployed Nov 12, 2026 by dekobon via deploy to GitHub Pages #1061
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant