fix(data): merge 18 duplicate Intel Atom CPU records - #321
Merged
Merged
Conversation
The strict integrity gate (integrity_check.py --strict) has been hard-blocking weekly-refresh for at least the last 5 runs: 18 Intel Atom CPUs were each imported twice under two slugs, once without an "intel-" prefix (Wikipedia- sourced) and once with it (a later Kaggle-dataset import). These were not harmless duplicates - the two sides disagreed on release_date, threads and other fields for most pairs. Cross-checked every pair against https://en.wikipedia.org/wiki/List_of_Intel_Atom_processors: - release_date: the "atom-X" (Wikipedia) record matches the table almost exactly, often to the day. The "intel-atom-X" record's dates are frequently wrong, in two cases (Z3530, Z3560) off by a full 8 years. - threads: Wikipedia lists Hyper-Threading support for 17 of these 18 SKUs (only D2500 does not). The "intel-atom-X" record already encoded threads = 2x cores; the "atom-X" record wrongly had threads = cores for 16 of them. - clock/tdp/cache/socket/process_node: spot-checked the same way; kept whichever side matched Wikipedia (mostly the "atom-X" side, e.g. Z3560's "intel-atom-z3560" record had clock_ghz=1.0 vs Wikipedia's stated 1.83). Each merged record keeps the union of both sides' source_urls and survives under the shorter "atom-X" slug/path. `integrity_check.py --strict` now reports 0 hard anomalies. Closes #296
4 tasks
This was referenced Sep 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
TechEngine's
integrity_check.py --strictgate has been hard-blockingweekly-refreshfor at least the last 5-6 runs (all failures/cancellations since 2026-08-31), previously misdiagnosed as CI infra flakiness. The real cause: 18 Intel Atom CPUs were each imported twice under two slugs — once without anintel-prefix (Wikipedia-sourced, older) and once with it (a later Kaggle-dataset import) — and the strict gate treats same-name CPU records as a hard anomaly.These were not harmless exact duplicates; the two sides disagreed on
release_date,threads, and other fields for most pairs.How each pair was resolved
Cross-checked every pair against https://en.wikipedia.org/wiki/List_of_Intel_Atom_processors:
atom-X(Wikipedia-cited) record matches the table almost exactly, often to the exact day. Theintel-atom-Xrecord's dates are frequently wrong — in two cases (Z3530, Z3560) off by a full 8 years (claiming 2022 releases for 2014 chips).intel-atom-Xrecord already correctly encodedthreads = 2× cores; theatom-Xrecord wrongly hadthreads = coresfor 16 of them (missing HT).atom-Xside (e.g. theintel-atom-z3560record hadbase_clock_ghz=1.0vs Wikipedia's stated 1.83 GHz).Each merged record keeps the union of both sides'
source_urls, survives under the shorteratom-Xslug/file path, and staysverified: true. The corresponding stale dump pages for the 18 deletedintel-atom-Xslugs are removed too (the dump generator doesn't prune records that no longer exist, so those pages would otherwise have stayed orphaned).Verification
python integrity_check.py <data> --strict(the exact commandweekly-refreshruns) now reports 0 hard anomalies, down from 18.Closes #296