Skip to content

fix: batch 2026-11-07 — GFM module, git fixture isolation, LOC literal rows and comment chains - #1856

Merged
dekobon merged 17 commits into
mainfrom
fix/batch-2026-11-07
Nov 9, 2026
Merged

dekobon merged 17 commits into
mainfrom
fix/batch-2026-11-07

Conversation

@dekobon

@dekobon dekobon commented Nov 8, 2026 •

Copy link
Copy Markdown
Owner

Batch fix of four issues, processed sequentially on one branch, each with its own regression tests, perturbation sweep and gate, then three refactor passes and six independent whole-branch reviews with remediation.

Fixes #1839
Fixes #1846
Fixes #1816
Fixes #1818

What changed

#1839 — refactor(cli): shared gfm.rs. escape_gfm_cell and escape_name move out of check_format.rs / markdown_report.rs into one module whose doc is the single statement of the two GFM policies. Byte-identical; no behaviour change; the report's escape_cell delegate is gone. Baseline: markdown_report.rs file PLOC 583 → 566.

#1846 — test(fixtures): scrub inherited GIT_*. All five Rust scratch-repo fixtures (library, CLI integration, CLI unit, web, Python-bindings unit) drop every inherited GIT_* before applying the caller's env, the fixture's identity/dates and the config pins. Reproduced before the fix: with GIT_DIR/GIT_INDEX_FILE naming a victim, the library's single_commit_one_file committed into it. Each boundary gains a child-process rerun against a victim repository (built through an independent oracle) and a separate GIT_OBJECT_DIRECTORY case; verified by revert at every boundary. Helpers are #[cfg(unix)] like their callers (the ungated form fails windows-latest under -D warnings; reproduced locally). Test-only.

#1816 — fix(loc): literal rows. Ruby %w/%W/%i/%I arrays, regex, :"…" symbols and backtick commands; Elixir strings, charlists, sigils, quoted atoms and quoted keywords (not in the issue; found in the grammar and confirmed by dump); C/C++/Mozcpp/ObjC char_literal — each lost a row to blank per spanning escape or empty separator row. Six LOC files; Elixir keeps its LLOC path. 14-case perturbation sweep, every route caught. Nested %W arrays measured linear in rows against the parent commit.

#1818 — fix(loc): C-family comment-only \ chains. GapScan::only_comments_before refused a continued row and ran out of its three-child window. The issue's forward-flag sketch cannot work (a node's gaps are scanned before its children's tokens), so the four C-family grammars now build a BareRowIndex once per walk — one cursor walk for comment spans, one byte pass settling each bare row's class at its logical line's end — and GapScan::classify reads it in O(1). The nine splicing languages declare their policy on Loc::bare_rows; the walk credits ahead of Loc::compute. New probes loc/c-comment-chain and loc/c-unclosed-comment-rows (1.02–1.05 clean; 1.91 / 1.70 with an O(rows) rescan reintroduced). only_comments_before removed as unreachable for every Gap grammar (checked against the five pinned grammars; a debug_assert! held over the full suite).

Plus: simplify-rust / rust-optimize / audit-tests passes, review remediation, lesson 117 and a testing.md rule.

Evidence (all at the tip, 678e3084, unless noted)

  • make pre-commit: BCA_GATE: pass (gate=pre-commit) — 7139 tests, no stage skipped; also passed at every one of the 12 commits
  • make bench-scaling: 121/121 within bound (two earlier runs under host load ~26 each flickered one unrelated probe; clean on re-run and at the freeze)
  • make chain-audit: 668 passed · make check-test-lang-gates-compare COMPARE_REF=0ac0fb4e: OK
  • Snapshot submodule: gitlink d411e605 unchanged — no corpus file holds any of these shapes, so nothing to publish
  • .bca-baseline.toml: four moved values, all in files this branch edits
  • Reviews: Astra at each checkpoint; whole-branch review, code-review --effort max, algorithmic performance, architecture, Astra ×2, less-reviewed-areas; one real defect found (the cfg gates) and fixed; Astra verified the remediation

Follow-ups, all now on this branch

  • Bindings' pytest fixture tests/test_vcs.py::_git_env inherited GIT_* — ca61b3c3
  • Perl qw() and the five pattern forms had the fix(metrics/loc): Ruby, Elixir and C literals lose a row #1816 shape — 980736ee
  • Elixir QuotedAtom/QuotedKeyword claimed by no role row — 8ab9d6b4 (StringLit; find, dump and Halstead change, CHANGELOG ### Fixed); src/metrics/loc/ being outside the role-bypass BOUNDARY is recorded as by-design in the roles module doc
  • vcs_command.rs reads tests/common/git_fixture.rs via #[path] — f11ab7b7

Filed from the review waves, not changed here

@codecov

codecov Bot commented Nov 8, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.03279% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 98.38%. Comparing base (731b8b7) to head (784045c).
⚠️ Report is 17 commits behind head on main.

Files with missing lines Patch % Lines
big-code-analysis-ast/src/roles/elixir.rs 87.80% 3 Missing and 2 partials ⚠️
big-code-analysis-cli/src/vcs_command.rs 94.44% 3 Missing and 2 partials ⚠️
src/metrics/loc.rs 99.33% 1 Missing ⚠️
src/metrics/loc/continuation.rs 98.85% 0 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1856      +/-   ##
==========================================
- Coverage   98.39%   98.38%   -0.02%     
==========================================
  Files         398      400       +2     
  Lines      121753   122045     +292     
  Branches   121318   121610     +292     
==========================================
+ Hits       119802   120074     +272     
- Misses       1227     1242      +15     
- Partials      724      729       +5     
Flag Coverage Δ
python 100.00% <ø> (ø)
rust 98.37% <98.03%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
big-code-analysis-ast/src/getter/elixir.rs 95.77% <ø> (ø)
big-code-analysis-ast/src/roles/mod.rs 97.43% <ø> (ø)
big-code-analysis-cli/src/check_format.rs 95.96% <ø> (-0.21%) ⬇️
big-code-analysis-cli/src/exemptions.rs 99.08% <ø> (ø)
big-code-analysis-cli/src/gfm.rs 100.00% <100.00%> (ø)
big-code-analysis-cli/src/lib.rs 95.43% <ø> (ø)
big-code-analysis-cli/src/markdown_report.rs 98.97% <100.00%> (-0.02%) ⬇️
...g-code-analysis-cli/src/markdown_report/hotspot.rs 97.80% <ø> (ø)
...-code-analysis-cli/src/markdown_report/sections.rs 100.00% <100.00%> (ø)
big-code-analysis-cli/src/provenance.rs 98.55% <100.00%> (-0.03%) ⬇️
... and 19 more
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@dekobon

dekobon commented Nov 8, 2026

Copy link
Copy Markdown
Owner Author

Pushed a28328ba for the Codecov patch gaps. Of the 10 reported lines, 6 were in the delta the local utils/patch-coverage.py run reproduces (the rest are Codecov's region-fraction scoring of the same lines):

Line Verdict
continuation.rs:156 — stray \ (even run) is code Covered: two new C-family rows; forcing the arm false fails with (4, 2, 2, 1). Also dropped a never-empty-slice arm by passing the byte the caller holds.
elixir.rs:131 — escape parent gate's false side Unreachable, removed: the grammar's node-types.json gives escape_sequence exactly the five literal parents (recovery keeps an unterminated string a string). New escape_sequence_parents_are_the_literals in the AST crate pins the five against a grammar bump. One Ancestors::parent read per escape leaf gone with it.
vcs_command.rs:1193 — cfg!(windows) branch Not coverable on Linux; the windows-latest leg runs its unit test.
vcs_command.rs:1445, :1473; loc.rs:9353 Panic-message arms of passing assert!s.

Local approximation went 98.84% → 99.27%; gate, bench-scaling (121/121) and chain-audit pass.

@dekobon
dekobon force-pushed the fix/batch-2026-11-07 branch from a28328b to 7a6525a Compare November 8, 2026 23:15
@dekobon

dekobon commented Nov 8, 2026

Copy link
Copy Markdown
Owner Author

Amended and force-pushed a28328ba → 7a6525a7 after an Astra review of the coverage commit. Three corrections:

  • The Elixir gate removal was a behaviour change under recovery, which the review flagged as unverified and I then reproduced: def f do "a\ + \ + end leaves the string's tokens directly in an ERROR with no string parent. The gated code read (3, 2, 0, 1); without the gate (3, 3, 0, 0) — the rows are the literal text the writer typed, so the new reading is the intended one. Pinned as a new row in the Elixir continuation table rather than left as an undocumented side effect.
  • escape_sequence_parents_are_the_literals now also follows any supertype whose subtypes list the leaf, so a grammar bump that reaches it indirectly cannot slip past (the pinned 0.3.5 grammar has no supertypes; the path is for the bump).
  • The C-family table's "every fixture passes gcc/g++/clang" doc now excepts the one deliberate stray-\ fixture.

Gate, check-test-lang-gates, and the full loc suite pass on 7a6525a7.

@dekobon

dekobon commented Nov 9, 2026

Copy link
Copy Markdown
Owner Author

Added 980736ee — fix(loc): credit every row of a Perl qw() list or pattern literal (Refs #1816). The whole-branch review found #1816's shape in Perl: qw() lists and the five pattern forms (qr//, m//, bare //, s///, tr///) each read one row of code short per uncovered interior row. Six kinds join the existing add_multiline_string_ploc arm in src/metrics/loc/perl.rs; the #1816 CHANGELOG entry is extended rather than duplicated. Evidence table, perturbation and gate results are on #1816. Astra review of the commit: no findings; make bench-scaling within bound.

@dekobon

dekobon commented Nov 9, 2026

Copy link
Copy Markdown
Owner Author

Added 0e2727df — test(py): drop inherited GIT_* from the pytest fixtures' git (Refs #1846), the follow-up the #1846 commit left noted in CHANGELOG. The pytest _git_env now scrubs inherited GIT_* before the caller's variables and the pins; two child-process regressions and a prefix test mirror the Rust ones, verified by revert. Evidence and the pre-change reproduction are on #1846. Astra review: no actionable findings.

@dekobon

dekobon commented Nov 9, 2026

Copy link
Copy Markdown
Owner Author

Amended and force-pushed 0e2727df → ca61b3c3 after four reviews of the pytest GIT_* commit (review skill, algorithmic-performance audit, architecture audit, Astra) and an Astra re-review of the fixes (approve, no findings).

Fixed:

  • The oracle shared the scrub under test. _clean_git reused _git_env(_ADA) while calling itself "the independent oracle"; Rust's clean_git deliberately shares only the is_git_var predicate with command, so a defect in the fixture's loop cannot reach it. _clean_git now has its own scrub (found by the review skill and the architecture audit).
  • The isascii() guard in _is_git_var's Windows branch is now pinned: not _is_git_var("g\u0131t_dir") (dotless i; Unicode upper-cases it to I, the Rust ASCII fold never does). Found by Astra.
  • The repository test reads back git ls-files == "work.rs\n" for the fixture's own repo instead of only .git existing.
  • Docstrings: Rust cross-references per the file's "Mirrors …" convention; _git_env states what a caller can (anything, scrubbed GIT_* included) and cannot (the seven pins) override; the NT fold is recorded as serving the predicate's contract, since Windows os.environ already upper-cases keys; the object-directory test's "kept apart" rationale corrected to localisation (the performance audit pointed out the child-passed assertion already rules out a vacuous pass); _victim_state docstring.
  • CHANGELOG rewrap.

Deliberately not taken: unix-gating the two child-process tests (Rust's #[cfg(unix)] has no stated Windows reason; nothing here is Unix-only; the Windows pytest leg will run them — documented in the helper), merging the two children into one (saves ~150–300 ms at the cost of localising which variable leaked and parity with the Rust pair), hoisting _git_env to import time (fragile against monkeypatch.setenv for a ~20 µs saving).

Revert check on the amended commit: replacing the fixture's scrub with dict(os.environ) (oracle scrub intact) fails exactly the two child-process tests. make pre-commit BCA_GATE: pass; ruff/mypy clean.

@dekobon

dekobon commented Nov 9, 2026

Copy link
Copy Markdown
Owner Author

Two more commits, closing out the PR body's follow-up list; the same four-reviewer wave (review skill, performance, architecture, Astra) ran against both, findings were fixed, and Astra re-reviewed the fixes (no actionable findings). Pushed ca61b3c3 → f11ab7b7.

8ab9d6b4 — fix(roles): Elixir quoted_atom / quoted_keyword on the StringLit row. Both were coverage debt; each is a quote-delimited body shaped as a string's, and Ruby's :"…" symbol is on its row. Consumers: find --type string/count report them; the dump flattens them; Halstead stops double-billing an interpolating :"q #{a}" (the atom AND the inner a — Ruby's #1353 shape). The Halstead getter move is forced by the bypass lint once the row claims them; NPA's defstruct field-count keeps QuotedAtom under a scoped allow. Perturbation: row revert fails the role test and both alterator rows; getter revert fails the Halstead test; removing the NPA allow fails role-bypass. make bench-scaling 121/121 (a Getter changed). No corpus file holds either form.

f11ab7b7 — refactor(cli): vcs_command.rs reads the shared git fixture through #[path] (html_report.rs precedent). 233 duplicated lines gone; the clean_git oracle and reruns stay local by design. Verified against the shared module: moving its pins ahead of the caller env fails the hostile-host test; removing its scrub fails both inherited-GIT_* reruns.

Review findings fixed before push: CHANGELOG overstated η2 (falls per distinct literal, not per occurrence) and implied full string-twin parity (the quoted atom's : still bills an operator — pre-existing, now stated and filed as #1858); the roles-doc sentence said the lint "reports every arm" (it ignores unclaimed kinds — reworded, and nargs.rs named alongside loc/ as also outside the boundary); row comment carried consumer policy (dropped per formatting.md); NPA allow reason restated as a naming rule; one overlong doc line.

Filed, not changed: #1858 (the : operator), #1859 (excludes for bare atom/keyword; BOUNDARY note), #1860 (test-support crate for the cli/web/py fixture copies).

`escape_gfm_cell`, the one GFM table-cell escaper, lived in
`check_format.rs`, the `check` command's own formatter, and the report,
exemptions and VCS emitters reached into it from there. `escape_name`,
the code-span escaper it is paired with, lived in `markdown_report.rs`
and was reached by the provenance footer the same way.

Both now live in `gfm.rs`, whose module doc is the single statement of
the crate's two GFM policies: what each escapes and why they must stay
separate (a code span takes its content literally, so the cell's
backslash escape would render doubled there), and that GFM's extended
autolinks are deliberately not addressed. Callers import directly; the
report's one-line `escape_cell` delegate is gone. The five focused unit
tests move with the functions; the renderer-level tests stay where their
callers are, and swapping the two policies at `render_cell_md` fails
eight of them.

No behaviour change. The moved test's comment claimed GFM consumes `\m`
in `src\main.rs`; GFM keeps a backslash before a non-punctuation
character, so the rationale now says what the escape is for.
`.bca-baseline.toml` records `markdown_report.rs`'s file PLOC, which the
move lowered.

Fixes #1839
…tory

Every git fixture in the Rust suites pinned the host's configuration
away (GIT_CONFIG_GLOBAL, GIT_CONFIG_NOSYSTEM, the ignore/attributes
overrides) but inherited everything else. Git exports GIT_DIR and
GIT_INDEX_FILE to its hooks, and GIT_DIR to `git rebase --exec` in a
linked worktree; inherited, they outrank `current_dir`, so a fixture's
`init`, `add` and `commit` wrote into the checkout running the test.
Reproduced on the library's `vcs` binary: with the pair naming a victim
repository, `single_commit_one_file` moved the victim's HEAD and left
`src/work.rs` in its index.

Each of the five boundaries now removes every inherited `GIT_*`
variable first, then applies the caller's environment, the fixture's
own identity and dates, and the configuration pins, in that order, so
what a caller means to set still reaches git. The prefix test reads the
encoded name bytes and is ASCII-case-insensitive on Windows, where
environment names are:

- tests/common/vcs_fixture.rs (`command`)
- big-code-analysis-cli/tests/common/git_fixture.rs (`command`)
- big-code-analysis-cli/src/vcs_command.rs (`fixture_git`, which the
  two read-backs there now share)
- big-code-analysis-web/src/web/server_tests.rs (`fixture_git`, shared
  by both repository builders and both read-backs)
- big-code-analysis-py/src/vcs.rs (`fixture_git`)

Each boundary gains a fixture-only target test and two child-process
regressions modelled on #1845's: one re-runs the target with GIT_DIR
and GIT_INDEX_FILE naming a victim built and read through an
independent clean git, and asserts the child passed and the victim's
HEAD and index are unchanged; the other re-runs it with
GIT_OBJECT_DIRECTORY naming a path whose parent does not exist (git
init creates one missing level, so one level is not enough). Verified
by revert at every boundary: without its scrub, both regressions fail.

Fixes #1846
A row inside a literal that reaches neither PLOC nor CLOC falls to
`blank = sloc - ploc - cloc`. #1260 fixed that for strings and heredocs
by crediting the whole literal's range; the same shape survived in the
literals no arm routed, each reading one row of code short and one row
blank per such row (measured on `main` 84a90e0, `bca metrics -O json`):

- Ruby `%w[]` / `%W[]` / `%i[]` / `%I[]`: an empty row between two
  words is covered by no child (the words are `bare_string` /
  `bare_symbol` on their own rows, the separators hidden), and in a
  non-interpolating array `\` + newline stays in the word's
  `string_content`, which starts on the row above. `regex`,
  `delimited_symbol` (`:"…"`) and `subshell` hold an `escape_sequence`
  spanning the bare row that likewise starts on the row above it.
- Elixir `string`, `charlist`, `sigil`, `quoted_atom` and — not in the
  issue, found in the grammar's `_quoted_i_*` rules and confirmed by
  dump — `quoted_keyword` (`["ab\` / `\` / `cd": 1]`): `\` + newline +
  `\` + newline is one `escape_sequence` leaf spanning two rows, which
  the `quoted_content` arm never sees and the leaf-gated catch-all
  credits at its start row only.
- C, C++, Mozcpp, Objective-C `char_literal`: spliced like a string
  literal, with the same spanning `escape_sequence`.

Ruby adds the five kinds to its `String | HeredocBody` range arm (the
uppercase spellings parse to the same kinds; every kind is a single
id). The four C-family arms add `CharLiteral` beside `StringLiteral`.
Elixir keeps its `quoted_content` arm and the LLOC catch-all intact —
routing `E::String` through the range arm would skip the statement
count the catch-all's comment warns about — and instead, inside the
leaf branch, after the unconditional start-row credit, adds the rows
an `escape_sequence` runs on to when its parent is one of the five
owners, through `add_string_interior_ploc`. A recovery leaf outside a
literal credits only its own row, as before. The semantic `StringLit`
roles are untouched: a C char literal and an Elixir atom are not
strings to Halstead, only to the row count.

Tests: the `FIXME(#1816)` rows in the C-family, Ruby and Elixir
continuation tables move to `(3, 3, 0, 0)`, with CRLF twins, two bare
rows, and a blank row after the literal (`blank()` saturates at zero,
so a double credit hides without one). New: every `%w`/`%W`/`%i`/`%I`
spelling in the three- and five-row shapes, alternate delimiters, EOF
with no newline, a one-row control and the string twin; the literal
inside a Ruby method and an Elixir `def`; the Ruby interpolation
comment row (code and comment, as the string twin reads) against
Elixir's token-level credit (comment-only); Elixir LLOC pinned at 1
for the assignment and the bare statement, DLOC 0 throughout. Each
route perturbed alone (`utils/perturb.py`, 14 cases): dropping any one
kind, or the Elixir interior credit, is caught by the named test.

Scaling: nested `%W[a…#{` arrays 200 deep with 64 / 256 / 1024 word
rows per level (tree-sitter flattens deeper nesting to an `ERROR`)
measured 0.02 / 0.08 / 0.31 s against 0.02 / 0.09 / 0.36 s on the
parent commit (release `bca metrics --metrics loc`): the added range
credits are a constant per literal over the walk's existing cost, so
no range witness was added.

No integration corpus holds any of these shapes, so no snapshot moves
and `.bca-baseline.toml` is unchanged.

Fixes #1816
Review of the #1846 regressions found two gaps in `clean_git`, the
`git` that builds and reads each victim repository. It kept the host's
global and system configuration away but not the per-user ignore and
attributes files, which git reads whatever those variables say: a
contributor's `XDG_CONFIG_HOME/git/ignore` of `*` failed the victim's
`git add keep.rs` before the child under test ever ran (reproduced with
`HOME` and `XDG_CONFIG_HOME` pointed at such a file: the CLI's
`fixture_git_ignores_an_inherited_repository` panicked on the victim's
`add`; with the pins it passes). And in the three unit-test modules it
was built through the `fixture_git` constructor it judges, so the
oracle shared whatever defect that constructor had.

Each oracle now pins `core.excludesFile` and `core.attributesFile` to
`/dev/null` with `-c`, and the three unit-test copies remove the
inherited `GIT_*` variables themselves. The web module's note on the
`#[core::prelude::v1::test]` spelling now names what a bare `#[test]`
would resolve to there: the `actix_web::test` attribute macro the file
imports, not a module.

Refs #1846
Translation phase 2 splices a comment row's trailing `\` too, so
`/* a */ \` + `/* b */ \` + `\` + an empty row is one comment line to
`gcc -E -C -P`. bca read the bare row as code — `(5, 2, 2, 1)` where
`(5, 1, 3, 1)` is right — in two cases: where the comment row before
it itself continued an earlier row (`GapScan::only_comments_before`
refused a continued row outright), and where that row held more than
three comments (it ran out of the three-child window the scan keeps).
Measured on the parent commit with a debug `bca metrics -O json`;
every fixture passes `gcc -fsyntax-only`.

The answer is a property of the whole logical line, which the gap
scan cannot read: a node's gaps are scanned before its descendants are
visited, so a flag carried through the walk sees a parent's trailing
rows before a child's tokens, and the tokens a bare row continues may
sit in nodes the scan never holds. Reading the line back from each
bare row, or forward to its end, is the chain's length per row.

The four C-family grammars now build a `BareRowIndex`
(`src/metrics/loc/continuation.rs`) once per walk: one cursor walk
collects the comment nodes' byte spans, one forward pass over the
bytes keeps `code_seen` / `comment_seen` and the bare rows pending for
the current logical line, and settles them at the line's end — code
anywhere makes them code, comments alone a comment, nothing blank.
Comment text is what the grammar's comment nodes cover, never a `/*`
inside a string or an unexpanded macro body. A newline inside a
comment ends the line whether or not a `\` precedes it, which keeps
the existing `\` + `/* a` + `b */ int z = 1;` = `(3, 1, 3, 0)`: the
bare row joins the comment's first physical row only, the policy
`only_comments_after` already applied to the row a bare run joins.
`GapScan::classify` reads the index where it read its neighbours; the
scan still finds the rows and credits each to the innermost space, so
ownership and the mixed code-and-comment rule are unchanged.

The nine languages that splice `\` + newline away declare their policy
on a new `Loc::bare_rows` (`BareRowPolicy::Gap` for Bash, Ruby,
Elixir, Tcl and iRules — their scan is unchanged — and `::Indexed`
for C, C++, Mozcpp and Objective-C) in place of each calling the scan
from `compute`. The walk builds `loc::BareRows` once before its first
node, only when `loc` is in the resolved selection and the file can
hold a bare row (#1821's hint), carries it in `NodeFacts`, and runs
the credit ahead of each `Loc::compute`. `init` writes only the SLOC
span and the credit only PLOC / CLOC rows, so the order is the one
each language had.

Tests: the three issue shapes under all four grammars, with CRLF,
longer chains, a six-comment row, an indented row, code before and
after the chain, a bare row between two comment rows, and the
comment-internal newline (`int a; /* x` + `y */ \` + `\`). The index
build count is pinned at exactly one per eligible walk — `loc`, and
`mi` which resolves to it — and zero under `cyclomatic` alone, a file
with no `\` row, or Ruby, with two walks building two; `scans > 0`
cannot see a build per node. Perturbations (`utils/perturb.py`, 8
cases): each of the settle rule's three edges, comments read as code,
code never seen, the index unread, and the credit dropped before
`compute` are caught by the named tables; the reseed of
`comment_seen` after a comment-internal newline was uncaught, because
the comment's next byte marks the line anyway, and was removed.

Scaling (`make bench-scaling`, 121 probes within bound): two new C
probes, `loc/c-comment-chain` (one logical line of `width` comment
rows and bare rows) and `loc/c-unclosed-comment-rows` (three comments
per row under a flat root `ERROR`), measured 1.01 and 1.07. Replacing
the O(1) lookup with a forward read over the rows after each bare row
moved them to 1.41 and 1.72 and `loc/c-unclosed-spliced-comments` to
1.73. `make chain-audit` passes.

`.bca-baseline.toml` records the moved `compute_per_node` and
`metrics_inner` Halstead effort, `loc.rs`'s file PLOC, and `GapScan`'s
WMC. No integration corpus holds a chain, so no snapshot moves. The
book's LOC chapter documents `Loc::bare_rows`.

Fixes #1818
Review of the #1818 probe found `loc/c-comment-chain` passing the
regression it claims to catch: a rescan of the logical line per bare
row fitted 1.41 on `LINEAR_WIDTHS`, under the 1.5 bound, because the
quadratic term is small against the walk at 2 000 rows — the shape
`SPACE_MERGE_WIDTHS` exists for. On that ladder the clean index fits
1.05 and the rescan 1.91 (`make bench-scaling`, the rescan reintroduced
in `BareRowIndex::class`); the rationale records both.

Refs #1818
The simplify-rust pass over the four fixes (#1839, #1846, #1816,

- `credit_bare_continuation_rows` drops its `bare_rows_possible` early
  return: the walk builds no `BareRows` for a file with no `\` row, so
  the function is never called where it would fire. The `Stats` field
  stays — removing a `Stats` field is held for the next major
  (`STABILITY.md`) and `PartialEq` reads it on purpose — and its doc
  now says the walk reads the hint once, ahead of every node.
- `BareRows::for_walk` takes the combined `loc`-selected-and-possible
  predicate, in place of a `.then(..).flatten()` around it.
- `BareRowIndex`, `RowClass` and `BareRowPolicy::comments` narrow to
  the visibility their one reader needs; `Indexer` and `RowShape`
  derive `Default`.
- Comments that still said each language calls the scan from
  `compute`, or that a bash row is credited "above", now name the
  walk's credit; `NodeFacts`' doc covers its non-`bool` member.
- `gfm.rs`'s moved `escape_cell_*` test, which repeated two of its
  neighbours' assertions under the removed name, folds its unique
  cases (`src\main.rs`, `dir\`, CRLF) into them.
- The victim-oracle docs in all five #1846 suites say what is
  independent of the fixture's `git` — the removal loop — rather than
  claiming the whole constructor is, since they share `is_git_var`.

`.bca-baseline.toml` records the moved Halstead effort of
`metrics_inner`.
The rust-optimize pass over the batch, behaviour unchanged:

- `BareRowIndex` stores `Vec<Option<BareRow>>` in place of a `RowClass`
  enum that was `BareRow` plus a `NotBare` variant and a map back;
  `Option<BareRow>` has a niche, so it is still one byte per row.
- `is_code` reads the byte at `at` through one slice pattern, which
  absorbs `at_splice` and drops the `byte` argument that always was
  `code[at]`; the comment spans iterate by value.
- `GapScan::classify` reads the index through one `and_then`.
- The web suite's `git_head` is `git_output` with its arguments.

`loc/c-comment-chain` and `loc/c-unclosed-comment-rows` still read
1.02–1.03 after the `is_code` change.
… test

The audit-tests pass over the batch found two hardenings in the #1846
regressions. The `GIT_OBJECT_DIRECTORY` rerun asserted only that the
child passed, which discriminates today because git 2.34 refuses to
create two missing levels of an object directory; a git that created
both, or a scrub that dropped the variable only after `init`, would
pass the rerun and leave the directory behind. Each of the five now
asserts the directory was never created. And `is_git_var`, in all five
copies, had no test of its own, so its Windows case-insensitive branch
ran nowhere (the child reruns are unix-only): each copy gains a
host-independent unit test over `GIT_DIR`, `GIT_`, `GITHUB_TOKEN`,
`GIT`, the empty name, and `git_dir` against `cfg!(windows)`.

Refs #1846
The six whole-branch reviews (review, code-review --effort max,
algorithmic performance, architecture, two Astra passes) found one
defect and several documentation slips.

The defect: the #1846 helpers (`clean_git`, `CLEAN_GIT_CONFIG`,
`victim_state`, `rerun_*_with`) in all five suites were ungated while
their only callers are `#[cfg(unix)]`, so the `windows-latest` CI leg,
which builds every test target under `RUSTFLAGS=-D warnings`, would
fail on `dead_code` (reproduced here by flipping the callers' gate to
`not(unix)`: four "is never used" errors in the library's `vcs`
binary, none with the helpers gated). Each helper now carries
`#[cfg(unix)]`, as the AST crate's `rerun_listing_test_with` does.

Documentation:
- The `GIT_OBJECT_DIRECTORY` doc comment in the two integration suites
  had been separated from its test by the `is_git_var` test added
  after it; it is back above the test it describes.
- `docs/development/benchmarking.md`'s probe table gains the two #1818
  probes.
- `vcs_command.rs` no longer claims a unit test "cannot reach"
  `tests/common/git_fixture.rs`; `html_report.rs` reaches a sibling
  through `#[path]`. Sharing the fixture that way is a follow-up.
- `loc::Stats::bare_rows_possible`'s doc cites the compute-side
  `PartialEq` contract (`STABILITY.md`, #1821) as the reason it stays.
- The #1846 CHANGELOG entry says the *Rust* fixtures, and names the
  bindings' pytest fixture, which still inherits `GIT_*`, as a
  follow-up.
- The C-family table's new rows record `blank` before #1801 in their
  third element, as the table's doc says every row does.
- `compute_per_node`'s comment on the credit's order no longer says it
  ran first before.

And one removal: `GapScan::only_comments_before` and the branch that
called it are gone. Every `Gap` grammar's comment token takes its row's
`\`, so a `\` in padding there always follows a non-comment token, and
the C family answers from the index first; a `debug_assert!` that the
branch was reached with `true` held over the full suite. Its WMC entry
leaves the baseline with it.

Refs #1846, #1818
Two comments in `python.rs` still named `GapScan::only_comments_before`,
removed in 9453fec; the C family's answer to the question they
describe is now `BareRowIndex`.

Refs #1818
Lesson 117, from #1818: a per-node walk that classifies the padding
between a node's children does so with those children as spans, not
tokens, with a child's own gaps still unvisited, and with any window
of neighbours a long enough logical line outruns. The per-line answer
is settled once, in a pass over the source before the first node, and
looked up — `BareRowIndex`. The sub-example is the issue's own chain
with its measured pre-fix tuples.

`testing.md` gains the forward trap beside the `#[cfg]` rule: a helper
called only from `#[cfg(unix)]` tests is dead code on every other
target, and the `windows-latest` leg builds the test targets under
`-D warnings`; gate the helper as the test is gated. Found in review
of #1846's regressions, where thirteen such helpers were ungated.
Codecov's patch report on the batch named two reachable production
lines no test executed.

`continuation.rs`: the arm that reads a `\` not followed by a newline
as code — the first of an even run, which phase 2 leaves as a stray
token the grammar recovers as an `ERROR`. Two C-family rows now reach
it: `/* c */ \\` + `\` + an empty row reads `(4, 3, 1, 1)` (the stray
makes the line code, so its bare row is code; forced to `false` it
reads `(4, 2, 2, 1)`), and `int a; \\ ` + `\` splices nothing. The
first is the table's one fixture the compilers reject ("stray '\' in
program"), and its doc says so. The function takes the byte its caller
already holds, which removes a never-empty-slice arm that existed only
for exhaustiveness.

`elixir.rs`: the `false` side of the escape's parent gate. For valid
input it is unreachable — the grammar's `node-types.json` gives
`escape_sequence` exactly the five literal parents — and
`escape_sequence_parents_are_the_literals` in the AST crate pins the
five against a grammar bump by reading the crate's `NODE_TYPES`,
through any supertype the leaf might one day be reached by. Under
recovery it was reachable: `def f do "a\` + `\` + `end` leaves the
string's tokens directly in an `ERROR` with no `string` parent, and
the gate left the bare row blank, `(3, 2, 0, 1)`. The gate is gone:
those rows are the literal text the writer typed and now read
`(3, 3, 0, 0)`, pinned by a new row; one `Ancestors::parent` read per
escape leaf goes with it.

The four lines still reported are the `cfg!(windows)` branch of
`is_git_var`, which the Windows leg covers, and three panic-message
arms of passing asserts.

Refs #1816, #1818
The branch's #1816 fix routed the Ruby, Elixir and C-family literals
whose interior rows reach no child through the range credit; the
whole-branch review found the same shape in Perl, which the issue never
named.

Six kinds have it. A `word_list_qw` holds `list_item` children on the
rows that carry a word, so an empty row between two words is covered by
nothing — the Perl shape of Ruby's `%w[]`, though narrower: `qw[a\` +
`\` + `b]` already read every row because each `\` is its own
`list_item`. The five pattern forms `regex_pattern_qr`,
`pattern_matcher_m`, the bare `pattern_matcher`, `substitution_pattern_s`
and `transliteration_tr_or_y` hold their text as one
`regex_pattern_content` leaf the catch-all credits at its start row, so
a `\` + newline + `\` + newline inside the pattern, or an empty row in a
replacement half, fell to `blank`. Each three-row literal read
`(3, 2, 0, 1)` and now `(3, 3, 0, 0)`; `s/ab\` + `\` + `cd/x` + `` +
`y/` read `(5, 3, 0, 2)` and now `(5, 5, 0, 0)`.

The six join the arm the string and heredoc kinds already use. Every
kind has an opening token (`qw`, `qr`, `m`, `/`, `s`, `tr`) the
catch-all credits on its row, so the helper's same-row skip holds; the
`regex_pattern_content` child stays on the catch-all, whose start-row
insert the range credit subsumes. No role change: Perl's `StringLit`
omits the pattern kinds on purpose, and row policy is the consumer's
(the #1816 precedent).

Pinned in `perl.rs`'s own module: the six kinds, the two-empty-row and
five-row `qw`, three delimiters, CRLF, EOF without a newline, the
replacement half alone and both halves, an empty row outside the
literal (the saturation guard: `blank()` floors at zero, so a credit
one row too far hides without it), one-row and `# word` controls, the
`q()` twin, and the literal inside a `sub` read through the nested
space. A `(?{ # c })` code block asserts PLOC and blank only: perl
reads a comment there and the grammar does not, so its `cloc` is the
grammar's gap, not this policy's. Dropping any one kind from the arm
fails its rows alone. An unterminated `qw(` at EOF is a flat `ERROR`
with no `word_list_qw` and stays as it was; documented, not pinned.

The one Perl corpus file holds no `qw` or pattern, so no snapshot
moves; the baseline is unchanged.

Refs #1816
#1846's sweep scrubbed every Rust boundary, including the bindings'
Rust unit-test `fixture_git`, and left the bindings' pytest fixture as
a follow-up: `_git_env` in `tests/test_vcs.py` spread `os.environ`
under the caller's variables and the configuration pins, so an
inherited `GIT_DIR` / `GIT_INDEX_FILE` outranked `cwd` exactly as it
did for the Rust fixtures. Reproduced before the change: `_build_repo`
run with the pair naming a victim repository moved the victim's HEAD
and left `work.rs` in its index.

`_git_env` now builds the same three layers as the Rust `command`s:
`os.environ` with every `GIT_*` dropped, then the caller's `extra`,
then the pins. `_is_git_var` reads the first four characters, exactly
on POSIX and ASCII-case-insensitively on Windows, as the Rust
`is_git_var` does; a non-ASCII prefix is never folded, since Unicode
upper-cases the dotless i to `I` and the Rust fold does not.

Two child-process regressions modelled on the Rust ones re-run
`_build_repo` in a fresh interpreter with the variables exported: with
`GIT_DIR` and `GIT_INDEX_FILE` naming a victim built and read through
an oracle with its own scrub — sharing only `_is_git_var` with the
fixture, as the Rust `clean_git` shares only the predicate with
`command` — the child must pass, the victim's HEAD and index must not
move, and the fixture's own repository must list `work.rs`; with
`GIT_OBJECT_DIRECTORY` naming a path whose parent does not exist, the
child must pass and the directory must not appear. Verified by revert:
with the scrub removed, both fail. A prefix test pins `_is_git_var`'s
edges, the non-ASCII one included. The reruns are not gated to POSIX
as the Rust ones are: nothing in them is Unix-only, so the Windows
pytest leg runs them too. The #1846 CHANGELOG entry drops its
follow-up note and names the pytest fixture.

Refs #1846
…iterals

No Elixir role row named `quoted_atom` (`:"a b"`) or `quoted_keyword`
(the `"a b":` key of a pair); both sat in `.role-coverage-baseline.txt`
as debt. Each is a quote-delimited body of `quoted_content`,
`escape_sequence` and `interpolation` shaped as a `string`'s, and
Ruby's `:"a b"` symbol has been `StringLit`'s since #1698, so the
Elixir row claims both; the bare `atom` and `keyword` tokens stay
plain operands.

Three consumers read the row. `find --type string` and `count` now
report the two forms. The AST dump flattens each to one verbatim leaf
as it does a `string` — the `quoted_keyword` token runs through its
`: ` separator, so the leaf's text does too. Halstead's getter moves
the two kinds from the plain operand arm into the `string_operand_type`
arm beside `String | Charlist | Sigil`, which the bypass lint forces
once the row claims them: `x = :"q #{a}"` billed the atom and the `a`
inside it (N2 3) while `x = "q #{a}"` billed only the `a` (N2 2) —
Ruby's `BareSymbol` shape from #1353. The plain forms are unchanged.
NPA's `count_field_entries` keeps `QuotedAtom` as a `defstruct` field
name under a `role-bypass: allow(StringLit)`, since the row's other
owners are not field names; removing the marker fails `role-bypass`.

Pinned: the Elixir role test's fixture gains both forms and its
`StringLit` expectation names them (fails with the row reverted); two
alterator rows flatten each (fail with the row reverted); a Halstead
test reads the interpolating atom as its string twin, 4 / 5, the
keyword 5 / 6 for its pair's `1`, and the plain forms 4 / 4 and 5 / 5
(fails with the getter move reverted). `--update` removed exactly the
two coverage entries. Neither Elixir corpus file holds either form, so
no snapshot moves.

The quoted atom's `:` still bills as a Halstead operator, where the
bare atom's is inside its token and Ruby's `:"` is one token with its
symbol; a pre-existing spelling dependence, left for its own change.

Also recorded in the roles module doc, under "What the bypass lint
does not scan": `src/metrics/loc/` and `nargs.rs` are outside the
consumer boundary by design — an arm there names a kind to say which
rows it credits, a per-kind policy no role expresses, so a claimed
kind is not membership restated and the lint would flag it for
nothing — at the cost that an arm there can name a kind no row claims
unseen, as `loc/elixir.rs` named these two while they were debt. The
sentences move `roles/mod.rs` past its `sloc` baseline, refreshed
here.

Refs #1816
`vcs_command.rs`'s unit tests carried their own copy of
`tests/common/git_fixture.rs` — the inherited `GIT_*` scrub and its
`is_git_var`, the seven host-configuration pins, the fixed identity
and dates, `HostileGitHost` with its hooks and config-path helpers, and
`assert_no_user_name` — on the reading that a unit test cannot reach
an integration-test module. `html_report.rs` has reached its
validators through a module-scope `#[cfg(test)] #[path]` since its
hotspot report (87392bd),
and 9453fec named sharing the fixture the same way as a follow-up.

The test module now declares `git_fixture_for_tests` at module scope
(the attribute resolves relative to `src/`; nested under `mod tests`
it would look under a phantom `src/vcs_command/tests/`) and reads
`FIXED_NOW`, `DAY`, `git_at_with_env`, `git_output`,
`assert_no_user_name`, `HostileGitHost` and `is_git_var` from it. The
blame fixture's closure was `git_at_with_env` line for line: the
caller's environment, then Ada's identity and both dates, then the
pins. The two read-backs are `git_output`. The oracle `clean_git`, the
child reruns and the victim tests stay local by design — the oracle's
scrub is its own so a defect in the fixture's cannot reach it — and the
prefix test stays, now also pinning the `#[path]` wiring.

Verified against the shared module: moving its pins ahead of the
caller's environment fails `blame_fixture_ignores_the_hosts_global_
and_system_config`; removing its scrub fails both
`blame_fixture_ignores_an_inherited_*` reruns. The shared host's doc
counts two byte-identical copies now (web, Python), the CLI unit tests
no longer among them. Test-only; 233 lines removed.

Refs #1846
@dekobon
dekobon force-pushed the fix/batch-2026-11-07 branch from f11ab7b to 784045c Compare November 9, 2026 02:55
@dekobon

dekobon commented Nov 9, 2026

Copy link
Copy Markdown
Owner Author

Rebased onto origin/main (731b8b7e, 10 commits: the #1805 of_children chain borrow, the should_panic testing note, and eight dependency/actions bumps); force-pushed f11ab7b7 → 784045cb with a lease on the replaced tip. Still 17 commits, no fixups or squashes.

Conflicts and how each was resolved:

Tree check: the rebased tree differs from the pre-rebase tip by exactly main's own diff on the 24 files only main touched (1197/−192 both ways), and both sides' hunks survive in the six files both touched (CHANGELOG, shapes.rs, benchmarking.md, lessons_learned.md — lesson 116 is main's, 117 ours — testing.md, the baseline). make pre-commit BCA_GATE: pass on 784045cb; snapshot submodule clean; mergeable.

@dekobon
dekobon merged commit 784045c into main Nov 9, 2026
64 checks passed
@dekobon

dekobon commented Nov 9, 2026

Copy link
Copy Markdown
Owner Author

Merged by fast-forwarding main to 784045cb (git push origin 784045cb:main) once all four runs on the head were green (CI, CodeQL, Fuzz, Pages; 54 checks pass, 10 skipped). The branch was already rebased onto 731b8b7e, so this is the rebase merge without rewriting: the 17 signed commits land as they were reviewed, which the main ruleset (required_signatures + required_linear_history) requires and GitHub's server-side rebase cannot provide. GitHub detected the fast-forward and marked the PR merged at 784045cb; #1816, #1818, #1839 and #1846 closed.

@dekobon
dekobon deleted the fix/batch-2026-11-07 branch November 9, 2026 03:19
dekobon added a commit that referenced this pull request Nov 9, 2026
PR #1843 (3e1c7e1, 6b15a74, ac7cff4) landed on main at 23:33 on
2026-11-07. PR #1844 (docs/agent-instructions) was cut from b85a3cb,
before #1843, never rebased, and squash-merged at 23:50 as ca56098.
A squash writes the PR's final tree over main as one commit, so
ca56098's diff against its own parent silently reverted every file
#1843 touched -- the role rows, checker, getter and metric changes,
their tests, the #1828 CHANGELOG and book entries, lesson 117, the
role baselines, and the big-code-analysis-output gitlink (5fbe03bf
back to d411e605) -- under a title that names only documentation.
On main at 784045c, `record R(int A, int B);` and
`data class D(val a: Int, ...)` again report nargs 0 with no function
space, the bug #1828 was closed for.

Re-apply `git diff b85a3cb ac7cff4` onto main. Every code, test
and snapshot hunk applied cleanly; CHANGELOG.md and
lessons_learned.md conflicted with the entries #1856 added since and
were resolved by keeping both (the #1828 lesson is 118, after
#1818's 117). The grammar-dispatch hunk is ported to its new home,
docs/development/grammar_dispatch.md section 7. .bca-baseline.toml
is regenerated with `make self-scan-write-baseline-headroom`; the
gitlink returns to 5fbe03bf, which is already on the submodule's
origin/main.

Verified: `bca check --threshold nargs=5` flags `D::D: nargs = 6`;
`Total(params int[] values)` reads 1; `make pre-commit` passes.

Fixes #1828
dekobon added a commit that referenced this pull request Nov 10, 2026
#1856 claimed `quoted_atom` and `quoted_keyword` on the Elixir
`StringLit` row and rejected the bare `atom` / `keyword` tokens, but
the rejection lived only in the row comment; the four kinds — `atom`
(14), `keyword` (34), `_atom` (131), `_keyword` (160) — stayed as
unexplained debt in `.role-coverage-baseline.txt`.

The row now records the decision where the coverage gate reads it:
`excludes [Atom, Keyword, Atom2, Keyword2]`, each with its reason (a
single token with no quoted body that the getter bills as an operand on
its own; the two hidden `_atom` / `_keyword` choices reach no node).
The row comment keeps only the Ruby `simple_symbol` contrast the
strings do not say. `check-role-coverage.py --update` retires exactly
the four lines; the census and bypass baselines do not move, and no
emitted predicate changes.

Also a note beside `BOUNDARY` in `utils/check-role-bypass.py` naming
`src/metrics/loc/` and `src/metrics/nargs.rs` as outside it by design,
pointing at the fuller statement in `roles/mod.rs`.

Fixes #1859

This branch was previously deployed

1 inactive deployment
github-pages — 784045cb Deployed Nov 9, 2026 by dekobon via deploy to GitHub Pages #1036
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant