Repository navigation
Description
Objective
Exercise direct AST documentation operations in the existing parse-once fuzz architecture before adapters close.
Umbrella: #1654 · Planning ID: D26
Completion prerequisites: D05: #1661, D06: #1660
Shared architecture and constraints
Follow the umbrella's shared contracts. AST recognition serves two consumers: exact legacy convention/coverage streaming for root LOC, and richer source documentation for owner lookup and local text. Source attachment, profile eligibility and physical metric ownership are separate. Preserve the legacy candidate view before run selection, original emission timing, and metrics-only early exits.
Documentation fragments, logical documents and convention-qualified owner edges have separate identities. Source offsets refer to the actual parsed buffer and included ranges. The index is caller-owned, immutable and source-bound; lookup validates tree/index identity. Complete text means local decoded/cleaned content with forward source mapping, not inherited or generator-expanded documentation.
Implementation touchpoints
fuzz/Cargo.tomlfuzz/src/lib.rsfuzz/fuzz_targets/fuzz/corpus/
Implementation steps
-
Integrate current recovery-driver seeds and role documentation fixtures with the new extraction workload, retaining independent mapping/owner invariants.
-
Add the AST path dependency and call its operation using the existing parsed tree/source. Preserve one-parse fanout; no stable facade dependency until D23 adds facade coverage.
-
Exercise collection, lookup, raw access, local text availability and forward mapping, including segmented/unknown content without unconditional flattening.
-
Add language-tagged coverage or targets/seeds for every LANG variant, since current per-language targets cover only a subset. Helper outcomes are explicit; source languages require exercised positive seed cases.
-
Close infrastructure on an actual D05+D06 comment path, dispatch coverage and initial seeds; adapter issues own their additional cases.
-
Check local range/identity/order/mapping properties and unrelated valid docs surviving local errors. Preserve included-range behavior in synthetic adopted-tree tests where relevant.
Tests and edge cases
-
Included-range gaps versus lexical continuation gaps, equal-shaped nodes with different context, partial comment/string delimiters, and cache-preserving recovery traversal in all language dispatches.
-
Source ranges in bounds and inside included segments; output mapping origins correct by category and index identity enforced.
-
Repeated query determinism, enumeration/lookup agreement and no cross-index owner leakage.
-
Seeds for EOF docs, concatenation/parentheses, data POD, conditional/gathered docs, attributes, invalid bytes and unknown text.
-
Verify language-tagged dispatcher reaches every declared variant; capabilities alone are not positive support.
-
Run make fuzz-check and seed replay; bound exploratory runs with -runs=N. Promote minimized failures into normal regression tests.
Use independently specified expected owners, fragment counts/order, byte ranges, text and availability. Pair them with separate legacy metric assertions where relevant. Do not sort/trim observed results, rely only on facade/delegate equality, or accept generated snapshots without an expected-value anchor. Use verbatim source for EOF/CRLF cases. Perturb the decision-bearing expression and confirm the intended assertion fails; use the maintained utils/perturb.py only on tracked, clean targets, recording or stashing real work first. Multi-target cargo runs require --no-fail-fast; include the mutated crate and dependent crates before claiming a unique guard. Keep irreproducible fixtures/manifests in tests, maintained utils or an issue attachment, following .claude/rules/temp-files.md.
Acceptance criteria
- The workspace-excluded fuzz crate builds and replays its committed corpus.
- One parse exercises the real AST collector/text code and all-language seed coverage is explicit.
- Adapter-owned seeds and D23 facade additions have clear completion ownership.
Verification and completion evidence
-
Follow current role-census/role-coverage/role-bypass requirements and applicable role-parity-corpus checks. Preserve independent semantic fixtures; refresh
.role-coverage-observed.txtwhen any sampled corpus gitlink changes. -
Put revisions, exact commands, tool versions and durable input links beside parity/coverage claims; host-local evidence paths are not citations.
-
Recheck current source and the prerequisite contracts/baseline applicable to this issue; record intentional changes to the premises.
-
Run focused production-path tests and affected language siblings. Use package-isolated
-p big-code-analysis-ast/-p big-code-analysisfeature builds; workspace feature unification is not isolation. -
Run the repository's required checks for the touched surfaces. For implementation changes:
make worktree-setup,make pre-commitread from itsBCA_GATE:verdict, and relevant test-language membership comparison. Add scaling and chain-audit checks for traversal changes; runmake fuzz-checkand seed replay for fuzz changes. -
Record exact test/perturbation/probe evidence in a completion comment and update this body. Review unexpected metric drift before snapshot acceptance; follow baseline and integration-submodule discipline if behavior deliberately changes.
-
Land public API documentation, re-export/stability review and changelog entries with the originating API change. Keep unrelated CLI/web/Python surfaces outside this issue unless explicitly listed.
Revalidation against main (2026-11-04)
Source baseline inspected: 07822c69211fe851c452ac6c0ebbd99cf92589a5. Relevant landed work: PR #1687, PR #1689, PR #1772, PR #1790, PR #1804. These are implementation updates to the existing acceptance criteria; this review does not mark extraction work complete.
-
Reuse utils/recovery-oracle.py for deterministic line/word/operator truncation sweeps and existing inline documentation/control-flow inputs. This supplements, not replaces, the planned all-LANG parse-once fuzz fanout.
-
Fuzz and replay the current role/context composition and for_walk cache protocol, including secondary queries, included ranges and wide ERROR nodes. ERROR flatness is about how recovery assembled the node, not its position.
-
Add continued-comment/CRLF shapes from fix(tcl): a
\-continued comment parses its next row as code #1802 and attribute-value nested declarations from fix(ast): exclude_tests reach keeps an item nested in an attribute value #1786; preserve unrelated valid documentation and avoid fabricated attachment to parser-invented commands. -
Seed preservation follows the new evidence policy: committed seeds/tests, reusable driver manifests attached or in maintained utils, never only transient scratch.
Activity
dekobon commentedon Nov 4, 2026
Plan refreshed against current main
Updated this issue body after reviewing changes since #1654 was filed, through
07822c69211fe851c452ac6c0ebbd99cf92589a5. Relevant PRs: PR #1687, PR #1689, PR #1772, PR #1790, PR #1804.\-continued comment parses its next row as code #1802 and attribute-value nested declarations from fix(ast): exclude_tests reach keeps an item nested in an attribute value #1786; preserve unrelated valid documentation and avoid fabricated attachment to parser-invented commands.The original source-documentation scope remains. New tests and current integration touchpoints are in the body; acceptance checkboxes remain open. This is source/history analysis, not a claim that the feature or new validation has run.