Skip to content

data(smartphone): import PhoneDB raw device variants (batch 1) - #27

Merged
Seungpyo1007 merged 9 commits into
mainfrom
data/phonedb-import
Jun 19, 2026
Merged

Seungpyo1007 merged 9 commits into
mainfrom
data/phonedb-import

Conversation

@Seungpyo1007

@Seungpyo1007 Seungpyo1007 commented Jun 19, 2026 •

Copy link
Copy Markdown
Member

Summary

Adds 2,661 variant-level smartphone records + 127 SoC seeds scraped from PhoneDB (phonedb.net), continuing the raw-import campaign. Each PhoneDB device id is one regional/storage variant, so these are variant-level records (distinct from the model-level GSMArena import). All records are raw seed (verified: false) with source_urls pointing at the exact device page.

Source & method

  • Source: https://phonedb.net device pages (robots.txt permits — only /ad/ and one path are disallowed). Attribution via per-record source_urls.
  • Throttled concurrent fetch; no CAPTCHA/Cloudflare bypass.
  • Coverage skews recent — 2023: 768, 2024: 993, 2025: 501 — filling the post-2020 gap left by the older GSMArena source.
  • Top brands: Samsung 629, Xiaomi 383, Oppo 256, Motorola 179, Apple 164, Huawei 158, Vivo 124, ZTE 86, …

Notes

  • SoCs auto-seeded where missing (verified: false).
  • Existing brands only; devices with brands not yet in data/brand were skipped (follow-up).
  • Local validator passes (python -m app.validate).
  • Data-only: the static site/public/v1 dump is left to the weekly-refresh pipeline.
  • First batch of an ongoing PhoneDB import (resumable; ~22k device ids remain).

Relates to #1.

Closes #1

@github-actions github-actions Bot added data Dataset changes enhancement New feature or request labels Jun 19, 2026
@Seungpyo1007 Seungpyo1007 moved this from Todo to In Progress in TechAPI-Project Jun 19, 2026
@Seungpyo1007
Seungpyo1007 merged commit ffe83b6 into main Jun 19, 2026
4 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in TechAPI-Project Jun 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data Dataset changes enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

TechAPI dataset roadmap and status: all categories (1989-2026)

1 participant