Repository navigation
fix(vba): read the Access ERD export with Access encoding rules - #323
Conversation
#53 gave the VBA family an encoding-robust read because Access writes Windows-1252 and accented identifiers are everywhere in Spanish-language projects, but it scoped that to extensions. The Access structure export is ERD/*.md, so it stayed on the plain utf-8 path. The damage was not cosmetic. A .sql naming a table DID get the CP1252 fallback and emitted a placeholder named TbSituación; the ERD declaring that same table did NOT and declared a replacement-character spelling. The two never matched, so the declaration never joined the placeholder and the table's field list hung off an orphan node — for exactly the tables whose names carry accents. Widen isVbaFamilyFile with the ERD path shape, reusing isAccessErdFile from grammars so the shape has one owner. Matching on shape rather than extension is load-bearing: .md alone would pull every markdown file in a repo through the CP1252 fallback. Unlike the JSON artifacts in #316, the fallback is correct here — the ERD document comes out of the same Access tooling as the rest of the export and shares its encoding provenance, where JSON is UTF-8 by specification and must fail loudly instead. Refs #322
The first attempt put the ERD path shape inside isVbaFamilyFile by importing isAccessErdFile into vba-source.ts. That turned a leaf module into a grammars dependant and broke index-orphan-watchdog on CI: that suite mocks grammars with a from-scratch object, so the newly imported helper was undefined and the orchestrator's own index reported success: false — a failure that looked unrelated to encoding. Combine the two predicates at the read site instead, as usesAccessEncoding in extraction/index.ts, the only place that needs both. isVbaFamilyFile goes back to owning extensions only, grammars keeps owning the ERD path shape, and vba-source.ts stays a leaf. Also spread the real module in that suite's grammars mock instead of rebuilding it. A from-scratch mock silently returns undefined for every helper it forgot, so any future import from grammars fails there as an unrelated-looking index error; the intentional stubs still override. Refs #322
|
Update — the first push failed CI on Linux, and it was this PR's fault.
Cause. The first attempt put the ERD path shape inside Fix. The two predicates are combined at the read site instead, as That suite's What I should have done the first time: I ran the Windows CI leg locally ( Local full suite on Windows: 21 failed | 3775 passed. Those 21 are the documented Windows-only failures in |
Closes #322
What was wrong
#53 introduced
readVbaSource()because Access writes Windows-1252 and accented identifiers are everywhere in Spanish-language projects — but it scoped the routing to extensions (isVbaFamilyFile). The Access structure export isERD/*.md, so it stayed on the plainfs.readFile(path, 'utf-8')path.Reproduced against the built
dist/, decoding the same document from UTF-8 bytes and from CP1252 bytes exactly asreadForExtractionwould:The damage is the asymmetry, not the ugly names. A
.sqlor.basnaming that same table does get the CP1252 fallback, so it emits a placeholder spelledTbSituación. The ERD declaredTbSituaci<fffd>n. The two never matched, so:type_member+contains) hung off an orphan node nobody references,AccessErdExtractoralready carried its ownstripBom(), so the encoding problem was half-solved: BOM handled, encoding not.What changed
isVbaFamilyFilenow also accepts the ERD path shape, reusingisAccessErdFilefromgrammars.tsso the shape keeps one owner and the two cannot drift.Matching on shape, not extension is load-bearing: adding
mdto the extension regex would pull every markdown file in every repo through the CP1252 fallback.And unlike the JSON artifacts in #316, the fallback is correct here. The ERD document comes out of the same Access tooling as the rest of the export and shares its encoding provenance; JSON is UTF-8 by specification and must fail loudly instead. Same reasoning, opposite conclusion, because the inputs differ.
Verification
src/fails the routing truth table and the join test, the latter withexpected [ 'Estructura_Datos.md', …(3) ] to include 'TbSituación'.readVbaSourcedecode tests pass either way by design —readVbaSourcealways did CP1252; what this PR changes is which files reach it. They are guards against a future regression in the decoder, not the regression proof. The routing and join tests are.classnode, and no node in the ERD file carries a replacement character.pnpm exec vitest run vba extraction-sql-query sql-query-discovery— 74 files, 1220 passed, 1 skipped (the Windows CI leg, run locally on Windows).npx tsc --noEmit— clean.