Skip to content

plan: 32-photo-trace-ai — optional AI silhouettes for photo cutouts + PoC - #4

Merged
YokiToki merged 39 commits into
mainfrom
claude/last-visible-commit-36705c
Oct 25, 2026
Merged

YokiToki merged 39 commits into
mainfrom
claude/last-visible-commit-36705c

Conversation

@YokiToki

@YokiToki YokiToki commented Oct 17, 2026 •

Copy link
Copy Markdown
Owner

What

  • New OpenSpec change 32-photo-trace-ai (planning only, no app code) and its row in openspec/ROADMAP.md. Target version 1.3.0, depends on 25a and 29.
  • A standalone PoC in poc/ai/ (npm run poc:ai, not part of the app) that runs background-removal models on the straightened sheet in the browser (ONNX Runtime Web) and compares them with the classic segmenter. Results: poc/ai/FINDINGS.md. Adds onnxruntime-web as a dev dependency.

Why

The colour segmenter for photo cutouts is a set of heuristics with known weak spots. A neural engine, run in the browser, is added as an alternative. The classic path stays the default and is the baseline.

Plan highlights

  • Engine switch Classic | AI; both produce the same Segmentation, so everything downstream is shared. segment.ts, colour.ts, outline.ts stay untouched (guarded by a frozen-output test).
  • Hybrid edge: the network says what is the object, the classic paper-colour test places the exact edge (0.5 mm target).
  • Model manager window: list, download, cancel, delete, choose the active model; weights stay in the browser (Cache API), several models side by side.
  • Licence classes (permissive / non-commercial / copyleft) with per-class consent; non-permissive models (e.g. RMBG-1.4) are optional third-party integrations, never default, never bundled or hosted by the project.
  • Backend is chosen per model (WebGPU / WASM), with capability detection.
  • Task 1 is a gate: candidates are measured on the reference photos before the rest is built.

PoC findings so far (owner's photos, one thread)

  • BiRefNet-lite at 1024 fails with std::bad_alloc on both backends: dropped.
  • InSPyReNet fp16 (MIT): good, ~2 s on WebGPU, empty mask on WASM.
  • ormbg (Apache-2.0): good, WASM only, 15–20 s.
  • ISNet imgly fp16 (MIT): 0.25 s on WebGPU, misses part of the cutters.
  • RMBG-1.4 (non-commercial): best quality, ~6 s on WASM.
  • MVANet and BEN2 mark the whole sheet: dropped. Most networks lose part of the cutters that the classic path finds, so AI is an alternative for hard photos, not a better default.
  • Still to measure: the no-flash photo on the shortlist, BiRefNet-lite 512, multi-threaded WASM.

For reviewers

Open questions are in design.md: huggingface.co vs self-hosted weights, minimum target device. Click-to-select (SAM) is deliberately out of scope.

🤖 Generated with Claude Code

skarakovski and others added 7 commits October 17, 2026 20:45
…stays baseline, v1.3.0)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… gain)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…yReNet

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…tcome

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@YokiToki YokiToki changed the title plan: 32-photo-trace-ai — optional AI silhouettes for photo cutouts plan: 32-photo-trace-ai — optional AI silhouettes for photo cutouts + PoC Oct 18, 2026
skarakovski and others added 22 commits October 18, 2026 11:13
…ommit-36705c

# Conflicts:
#	openspec/ROADMAP.md
…plan updated

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…k tuning requirement

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…C reference); mask tuning makes them sufficient

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ults

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…age, v1.4.0); poc: whole-image mode

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…oc: depth models and cat sample

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…with crop and whole-frame plaque; poc: shape combiner, experiment images

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…del, detail, mix, 3D relief); cutout PoC moved to cutout.html

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…etail), backends per model from measurements

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…nal smoothing, bilinear nodes)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… presets, working raster size

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ult resolution

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… contrast, solo layer view, whole-animal preset with less detail

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ine requirement and task

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…use the cutout outline smoothing

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…awing preset auto-applied; plan 34 updated

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…soft engrave threshold, auto-preset checkbox

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…r outline, bin fit, compare)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…oughness metric); plan 32: smooth outline requirement and task

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…no grid ribbing

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
skarakovski and others added 10 commits October 22, 2026 09:48
…(shared outline stage, v1.6.0)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…lts tolerance 0.25 / step 1 mm; plan 35 updated

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… silent download, disabled run without a model, AI activity line and full log (32, 33, 34)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…eport AI events to it, model manager links to the log; versions shifted (32 1.4.0 … 35 1.7.0)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…d art, Rack cards at 1 U, margin check; v1.7.1)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…and AI runs, diagnostics mode, no FPS; v1.8.0)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…s, model summary, consent-based export, log and undo; v1.9.0)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…g, slope limit and final smoothing, subject presets incl. Drawing/logo, Advanced outline group); 35 now depends on 34

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…nity blue, Keychain green, Rack yellow; interface colours unchanged)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@YokiToki
YokiToki merged commit 03b0220 into main Oct 25, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant