Ingest and sources (workstream)

[Seungpyo1007]

Purpose

Workstream tracker for ingest and sources, under the roadmap #97. PRs in this area link here with Closes #<this>.

Scope

• app/ingest collectors and parsers (Wikipedia CPU/GPU/smartphone tables, PassMark, notebookcheck, topcpu, SPEC, technical.city, cgdirector)
• app/ingest/enrich.py benchmark backfill runner (fill-only-nulls)
• weekly-ingest workflow (Monday, opens ingest/* PRs to TechAPI develop)

Rules

• Never invent values. A parser that can't read a field leaves it null; e.g. a bare "Cores" column doesn't imply a thread count (Parse multi-row headers in the Wikipedia GPU ingest; stop guessing threads #93).
• Dedup new slugs against existing ones in both directions and ignoring punctuation.
• Source policy follows TechAPI SPEC.md: GSMArena and PhoneDB only for corrections; Namuwiki banned; no bypassing anti-bot protection (Cloudflare, WAF).

Open items

• Wikipedia AMD Ryzen/EPYC list parsers (models live in column 1 under a family column 0); the importers currently live outside the engine.
• Transposed spec tables (MacBook/Surface-style "model per column") as a reusable parser.
• DX12 guard for topcpu GPU scores by architecture, not year (Time Spy estimates on TeraScale/Tesla-era cards).
• Xeon thread counts for about 390 rows (Intel spec pages; the tables list cores only).

[Seungpyo1007]

New open item (2026-09-27): AMD legacy CPU list pages need a dedicated parser. A dry run of WikipediaCpuIngest on 15 AMD list pages (Athlon/Athlon 64/X2/XP, Duron, FX, K5/K6, Opteron, Phenom, Sempron, Turion, mobile, APUs) yields ~87 "complete" rows, but the values are wrong:

• Voltage cells are read as core/thread counts (1.25 V → cores 1, threads 25).
• Bulldozer/Piledriver modules (cores) is read as cores (threads) (FX-8350 → 4C/8T).
• Most rows on Opteron, Phenom, Turion, Duron, K6 and the APU list miss cores entirely (~1,100 rows).

Nothing was imported. Needed: per-page column mapping (Cores/Modules/Threads headers), a voltage-column exclusion, and a module-aware core count.