Skip to content

fix(extraction): a build folder inside src/ is indexed for its TypeScript, JSX, Vue, Svelte and Astro files - #2681

Merged
colbymchenry merged 1 commit into
mainfrom
claude/funny-hodgkin-87b459
Oct 19, 2026
Merged

colbymchenry merged 1 commit into
mainfrom
claude/funny-hodgkin-87b459

Conversation

@colbymchenry

Copy link
Copy Markdown
Owner

Problem

The default ignore list skips every folder named build, at any depth, as build output (DEFAULT_IGNORE_DIRS → build/). JVM source roots were already exempt (#1642). JS/TS apps keep real source there too. luci-go's milo/ui keeps its whole LUCI "build" feature in src/build/ (src/build/legacy/build_page/build_page.tsx, src/build/pages/builder_page/…). A fresh index of it had 0 files under src/build/, so its routes' 19 lazy loads into @/build/... could never link, and none of that feature was visible to codegraph_explore.

The rule

Inside a src/ folder, a folder named build (any case) is walked. Only the files no compiler or bundler writes are indexed from it:

  • Indexed: TypeScript (.ts, .tsx, .mts, .cts, ArkTS .ets), .jsx, .vue, .svelte and .astro.
  • Still left out: .d.ts declarations, JavaScript (.js, .mjs, .cjs) and every other type, since a build may have written them. Every build/ folder outside src/ also stays out, whatever it holds.

Unchanged:

Implementation, all in src/extraction/index.ts:

Why this rule: measured first

  1. Sourcegraph regex search, file:(^|/)src/(.+/)?build/, exhaustive: 200,725 files in 3,572 repos, grouped into 8,147 build/ folders. Leaving out folders inside another default-ignored folder (a committed node_modules etc.) and JVM roots:
    • 741 folders (585 repos) hold TypeScript, JSX, Vue, Svelte or Astro files. They are source: packages/next/src/build, astro's src/core/build, docusaurus, nx executors, twenty, payload, mastra, pnpm, backstage, kibana, shopware, stencil, nitro, vitepress, RuoYi's src/views/tool/build/*.vue, milo. Six also hold output markers. These are mixed folders (a build script beside its output), and one of them only has TypeScript inside its own committed node_modules/, which stays excluded.
    • 568 folders (423 repos) hold only JavaScript. About half are output or vendored distributions: pdf.js and its pdf.worker.js, three.js, toastr, bootstrap plugins, YUI, CRA and Vite builds of apps nested under src/, Chrome devtools extension builds, Symfony Encore and Laravel public/build, WordPress wp-includes/build, Moodle amd/build, Vaadin bundles, Sphinx HTML. The rest is real JavaScript (SvelteKit's src/exports/vite/build/*.js, three-mesh-bvh, Yoast).
    • No output folder holds TypeScript or components. That is the signal the rule uses.
  2. Path-only vetoes don't separate the two. Excluding static/, public/, assets/, resources/, webapp/ or wwwroot/ ancestors would drop source (astro's src/assets/build, primate's src/public/build, komodo's ui/src/resources/build, shopware) and still keep 60+ output folders.
  3. A folder beside a package.json under src/ is either output (only JavaScript) or build tooling (kibana's src/dev/build, shopware's Vite plugins, both TypeScript). The per-file rule handles both without reading the disk.
  4. GitHub code search, which only matches path prefixes, so root src/build/ only. 4,648 .ts, 151 .tsx, 117 .vue files against 2,176 .js. 479 folders were classified by listing each repo's tree:
    • 229 hold hand-written files and no output markers;
    • every output folder is JS-only, among them Flutter web builds, Sphinx HTML and cx_Freeze or py2app bundles. The bundles hold copied .py, which is why Python stays out of the rule.
  5. The old kits (cgjsx, cgroute, cglzb; 50 JS/TS repos): milo's three folders were the only build/ folders below src/. All three are TypeScript.

Considered and not done:

  • "Imported from indexed source" as the signal. Tests and servers import build output (require('../build/main.js'), a SvelteKit adapter's build/handler.js). The scope would also depend on file contents, so the watcher and sync would need persisted state to agree with the indexer.
  • Reading each folder's contents. That needs the disk, so the decision could drift between the indexer, a long-lived watcher and scoped sync.

Verification

  • __tests__/src-build-folders.test.ts, 6 tests:
    • scope parity for filesystem, untracked and tracked enumeration (buildDefaultIgnore, buildScopeIgnore, scanDirectory, scanDirectoryAsync, every folder on the way walkable);
    • explicit-rule precedence and the !src/build/ opt-in;
    • a milo-shaped React Router app: its lazy routes into @/build/... link, and an edit, a new page and new output under src/build/ go through getGitChangedFiles and cg.sync();
    • the watcher: a source edit there is pending and syncs scoped, output and outside-src events are dropped, and a removed subfolder schedules the full reconcile.
  • Red on main, green here. All 6 fail against the src/extraction/index.ts of origin/main 24ea3c5 and pass with the change.
  • The react-router-lazy-barrel.test.ts header said its fixture had to avoid build/. That comment is updated.
  • tsc --noEmit is clean.
  • Full suite on origin/main f43432c plus this change (kernel built, CODEGRAPH_KERNEL_EXPECT=1, --maxWorkers=4 --minWorkers=1): 7,595 tests, 7,515 passed, 0 failed, 80 skipped.
  • On 6e68a11, the head this PR sits on (one Go-only commit later, fix(go): a constant in a const ( … ) group keeps its own doc comment #2680, which changed the kernel): rebuilt the kernel and reran tsc plus the new and touched test files, fix(go): a constant in a const ( … ) group keeps its own doc comment #2680's own test, extraction.test.ts (including the codegraph explore "RealtimePlusService" No relevant code found for "RealtimePlusService" #1642 suite), watcher.test.ts and every kernel parity suite: 19 files, 1,073 tests, all passed, none skipped.

A/B on real repos

Arms: origin/main 24ea3c5, and this branch on it. They differ only in dist/extraction/index.js, and both carry the same kernel. Each repo was indexed fresh with codegraph init -y and dumped with scripts/dump-graph.mjs.

Repo Files Edges touching new files Elsewhere
luci-go milo/ui (git) 2,499 → 2,694 (+190 src/build/, +5 generated protos in src/proto/**/build/) +4,966 −15 / +5 edges, −62 failed refs
milo/ui, plain folder (filesystem walk) identical to the git checkout, in both arms
nitro 800 → 870 +2,329 −18 / +9 edges, −529 failed refs
astro (packages/astro) 3,070 → 3,119 +3,117 −19 / +9 edges, −581 failed refs
inventree frontend 571 → 580 +893 −3 edges, −10 failed refs
RuoYi-Vue3 161 → 167 (.vue) +299 −1 edge, +1 unresolved
vitepress 275 → 280 +261 −8 failed refs
docusaurus (packages/docusaurus) 269 → 272 +130 −13 failed refs

milo: every one of the 19 lazy-route refs into @/build/... now links.

22 controls byte-identical between arms:

  • Real build output under src/:
    • kkFileView's vendored pdf.js builds in server/src/main/resources/static/;
    • a CRA app's build/ nested in Games-and-Go's src/, and Mimi.jl's src/explorer/mimiexplorer-app/build;
    • calm-hub's Vite output in src/main/webapp/build;
    • phpip's Laravel src/public/build;
    • Reactime's devtools src/extension/build;
    • ziya's hashed src/build.
  • Root CRA builds beside package.json: supertokens/dashboard, mihailgaberov/chat.
  • Other cases:
    • vue-element-admin's root build/ tooling;
    • pypa/build's Python src/build/ package, which stays out because it isn't JS/TS;
    • SvelteKit's JS-only src/exports/vite/build.
  • No build folder below src/: excalidraw, bulletproof-react, react-redux-realworld, proshop-v2, takenote, create-t3-turbo, matx, mantis, Folo, ghostfolio.

Triage of every edge that changed outside the newly indexed files:

  • Wrong links on main, now right (20).
    • milo: 8 useBuildsClient calls and imports from @/build/hooks/prpc_clients had been name-matched to a namesake in src/common/hooks/.
    • nitro (8): imports of ./build/assets.ts, ../../src/build/virtual/{app,cache,public-assets}.ts and ../../src/build/unbundled/plugins.ts had been guessed onto other files and now resolve as imports.
    • astro (2): remote.test.ts imports dist/assets/build/remote.js and now reaches src/assets/build/remote.ts.
    • inventree (2): two routes now reach their page components instead of the lazy declarations in router.tsx.
  • Same edge, lower confidence (23): these keep their target and drop from 0.9 to 0.7, because the new files add namesakes.
  • Wrong before and after (9). All target outside libraries or dist builds, so name matching guessed one project namesake on main and another, at lower confidence, here:
    • astro: Rollup's plugin-context this.getModuleInfo() (6 calls), import build from '../dist/core/build/index.js' and zod's z.output;
    • nitro: a chained }).start().
  • Wrong link now unresolved (1, RuoYi). import download from './download' passed as $download had been linked to a download function in src/utils/request.js. With a third namesake it is left unresolved, and silent beats wrong.
  • Displaced by the JSX child cap (1, inventree). PartDetail renders more than MAX_JSX_CHILDREN (30) components. BuildOrderTable from src/tables/build/ now takes the 30th slot, and the later PageDetail tag falls past the cap. It has 30 children in both arms.
  • Regression (2, milo), from an existing resolver gap.
    • src/gitiles/components/commit_table/context/index.ts writes export { DefaultExpandedProvider, … } from './context'. The named re-exports resolve by name match, not through ./context, on main too.
    • On main the only candidate was the right one. Now src/build/components/build_table/context.ts declares the same two constants, and the match moves there.
    • A five-file fixture reproduces the wrong link on main with no build folder involved: export { Provider } from './context' links to another file's Provider. I'll fix that separately (see Follow-ups).

Cost: the extra files only. Index times are within noise (milo 35.5 s → 32.0 s, nitro 7.6 s → 8.3 s, astro 17.8 s → 16.0 s).

Upgrading: codegraph sync with this build adds the src/build/ files to an index main built, on the git path (docusaurus) and in a plain folder (milo), so no re-index is needed.

Follow-ups (separate PRs)

  1. Named re-exports (export { X } from './m') resolve through the module they name, not by name.
  2. The same rule for Go, Rust and C#, whose toolchains never write source into a build folder. The survey has 87 such folders in 64 repos for Go (golang/go's src/go/build), 294 in 222 for Rust (turbopack, tauri, dioxus) and 130 in 77 for C# (roslyn, dotnet/wpf). PHP (WordPress writes build/*.php) and Python (frozen apps copy .py into build/) are not safe.

Not done:

  • JavaScript-only source folders named build (SvelteKit's src/exports/vite/build). In the survey they are as often output as source, and no path-only signal tells them apart.
  • Build folders outside src/ (config/build/*.ts webpack configs, VS Code's build/ tooling).

🤖 Generated with Claude Code

…ript, JSX, Vue, Svelte and Astro files

The default ignore list skipped every folder named build as build output,
so a JS/TS feature kept in one, like luci-go milo's src/build/, was never
indexed and the routes that lazily load its pages linked nothing.

A build folder below src/ is now walked, and its hand-written source
(.ts .tsx .mts .cts .ets, .jsx, .vue, .svelte, .astro, never .d.ts) is
indexed. What a build step may have written there, JavaScript and every
other type, stays out, as does every build folder outside src/. JVM
packages named build keep every file (#1642). The project's own root
rules still decide first: an exclude excludes the folder, and re-including
it (!src/build/) keeps every type, as before.

The rule reads paths only and lives in the two default matchers every
scope consumer builds on, so the full index, the git-status sync, scoped
sync and the watcher share it (#1728).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant