Skip to content

perf(dump): dump only the collections a change can affect - #117

Merged
Seungpyo1007 merged 1 commit into
mainfrom
perf/dump-scoped
Sep 30, 2026
Merged

Seungpyo1007 merged 1 commit into
mainfrom
perf/dump-scoped

Conversation

@Seungpyo1007

Copy link
Copy Markdown
Member

Step 5 of GetTechAPI/TechAPI#350. Stacked on #116 (uses its changed_paths); merge that first.

  • app.dump --only COLLECTION (repeatable) and --changed-since BASE.
  • Partial runs merge into the existing v1/index.json manifest instead of overwriting it.
  • Dependency map: brand -> everything; soc -> socs/smartphones/tablets/watches/pdas; cpu/gpu -> own + laptops; _verify/ -> nothing.
  • Scored collections (smartphones/cpus/gpus/socs) stay whole because scores are partly population-relative, so per-record dumping would be wrong there.
  • Not changed: the DB seed step still loads every record, and the dump-refresh workflow / the mandatory-dump rule are untouched (that decision is separate).
  • Tests: partial manifest merge, --only, change->collection mapping.

@Seungpyo1007 Seungpyo1007 added the ci Continuous integration changes label Sep 30, 2026
@Seungpyo1007 Seungpyo1007 self-assigned this Sep 30, 2026
`app.dump --only COLLECTION` (repeatable) and `--changed-since BASE` restrict
the replay to the affected collections and keep the other manifest entries
instead of overwriting them. brand changes still mean everything; soc/cpu/gpu
add the collections that embed them; scored collections are always dumped
whole because scores are partly population-relative.

Refs GetTechAPI/TechAPI#350
@Seungpyo1007
Seungpyo1007 merged commit bebe9fc into main Sep 30, 2026
1 check passed
@Seungpyo1007
Seungpyo1007 deleted the perf/dump-scoped branch September 30, 2026 08:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci Continuous integration changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant